Value evaluation method

Abstract

This value evaluation method is a software value evaluation method, and includes: determining a collection item that is an item of metadata collected from each component constituting software; collecting evaluation target data that is data of the collection item for each component; determining a policy that is a reference for value evaluation; and evaluating the evaluation target data on the basis of the policy to obtain an evaluation result.

Description

Valuation Method

[0001] The present invention relates to a value assessment method.

[0002] There are various evaluation indices for software, such as processing speed, memory usage, reliability, etc. Patent Literature 1 discloses a method including: detecting whether a data element includes an existing data domain tag, processing the data element into a transformed data element, generating a data domain tag, the data domain tag including at least a data domain identifier and a timestamp, adding the data domain tag to the transformed data element, creating a signature of the transformed data element and the added data domain tag using a private key, and creating another signature of the data domain tag using the private key.

[0003] US Patent Application Publication No. 2022-0300617

[0004] The invention described in Patent Document 1 cannot evaluate the reliability of software from various viewpoints.

[0005] A value evaluation method according to a first aspect of the present invention is a software value evaluation method, which includes determining collection items, which are metadata items to be collected from each component that constitutes the software, collecting evaluation target data, which is data of the collection items, for each of the components, determining a policy that is the basis for value evaluation, and evaluating the evaluation target data based on the policy to obtain an evaluation result.

[0006] According to the present invention, the reliability of software can be evaluated from various viewpoints.

[0007] Overall configuration diagram of the trust evaluation system Diagram showing an example of an evaluation target Hardware configuration diagram of a general-purpose terminal Configuration diagram of trusted party evidence Configuration diagram of certifier policy Diagram showing an example of a certifier policy Operational flow diagram of the trust evaluation system focusing on the operation of the trusted party device Operational flow diagram of the trust evaluation system focusing on the operation of the certifier device Diagram showing an example of a user interface Diagram showing an example of a visualization diagram

[0008] -Embodiment- Hereinafter, an embodiment of a reliability evaluation system will be described with reference to FIGS.

[0009] 1 is an overall configuration diagram of a trust evaluation system 1. The trust evaluation system 1 includes a trusted party device 10, a certifier device 20, a wide area storage 30, a child component manufacturer device 40, a third-party certification authority device 50, and a common policy creator device 60. The trusted party device 10, the certifier device 20, the wide area storage 30, the child component manufacturer device 40, the third-party certification authority device 50, and the common policy creator device 60 are connected by a network 70. The network 70 may be a wide area network such as the Internet, or may be a small-scale LAN.

[0010] The trusted party device 10 is a terminal used by the trusted party 11, who is a person (trustee) trusted by the certifier described below. The trusted party manufactures certain components. The trusted party device 10 includes a trusted party storage 12, which is a non-volatile storage device. The certifier device 20 is a terminal used by the certifier 21, who is a person (trustor) who trusts the trusted party. The certifier device 20 includes a certifier storage 22, which is a non-volatile storage device. The wide-area storage 30 is a non-volatile storage device that can be read and written via the network 70. It is desirable that the data stored in the wide-area storage 30 cannot be changed.

[0011] The child component manufacturer device 40 is a terminal used by a child component manufacturer 41 that manufactures child components that become parts of components manufactured by a trusted party. The third-party certification authority device 50 is a terminal installed at a third-party authority, which will be described later. The common policy creator device 60 is a terminal used by a common policy creator 61 that creates a common policy. A common policy is, for example, a law or an industrial standard.

[0012] 2 is a diagram showing an example of an evaluation target by the reliability evaluation system 1. Software product P1 can be created using component C4 or component C5. Component C4 can be created using components C1 and C2. Component C5 can be created using components C2 and C3. Furthermore, software product P1 can also be created using an alternative component X1 that is produced in-house without using components C4 or C5. In such cases, reliability evaluation can be used to determine which component to use in creating software product P1 or whether to create an alternative component X1.

[0013] 3 is a hardware configuration diagram of a general-purpose terminal 80 representing the trusted party device 10, the certifier device 20, the child component manufacturer device 40, the third-party certification authority device 50, and the common policy creator device 60. The general-purpose terminal 80 includes a central processing unit (CPU) 81, a read-only storage device (ROM) 82, a read / write storage device (RAM) 83, an input / output device 88 serving as a user interface, and a communication device 85. The CPU 81 executes various calculations by loading programs stored in the ROM 82 into the RAM 83. The general-purpose terminal 80 may be realized by a field programmable gate array (FPGA), a rewritable logic circuit, or an application-specific integrated circuit (ASIC), instead of the combination of the CPU 81, the ROM 82, and the RAM 83. The general-purpose terminal 80 may also be realized by a different combination of configurations, such as a combination of the CPU 81, the ROM 82, the RAM 83, and an FPGA.

[0014] The input / output device 84 is, for example, a mouse, a keyboard, and a liquid crystal display. The trusted party 11 uses the input / output device 84 to input data to the trusted party device 10. The certifier 21 uses the input / output device 84 to input data to the certifier device 20 and to check data output by the certifier device 20. The communication device 85 realizes communication between the general-purpose terminals 80 via the network 70.

[0015] 4 is a diagram illustrating the configuration of the trusted party evidence 13. The trusted party evidence 13 includes metadata 131, detailed associations 132, a self-signed certificate 133, training data 134, behavioral data 135, extended data 136, child element evidence 137, and a third-party certificate 138. In FIG. 4, solid lines indicate required configurations, and dashed lines indicate non-required configurations. In other words, the training data 134, behavioral data 135, extended data 136, child element evidence 137, and third-party certificate 138 may not be included in the trusted party evidence 13.

[0016] Metadata 131 is metadata of the component. Detail association 132 is data on the component details and related elements. Self-signed certificate 133 is a certificate by the trusted party 11. The self-signed certificate 133 is used to verify the integrity of the data structure. Training data 134 is data on additional information, data history, and data processing of the software. Operational data 135 is data on operational constraints and the operating environment of the component. Extension data 136 is additional data that can be customized for the component and its related elements. Child element evidence 137 is evidence by the trusted party 11 regarding the child element. Third-party certificate 138 is a certificate created by the third-party certification authority device 50.

[0017] 5 is a configuration diagram of the certifier policy 24. The certifier policy 24 is generated based on the trusted party evidence 13, general criteria, and special criteria. The certifier policy 24 includes a rule-based specification 241, a policy document 242, a self-signed certificate 243, an inferred specification 244, and additional policies 245. In FIG. 5, solid lines indicate mandatory configurations, and dashed lines indicate non-mandatory configurations. In other words, the inferred specification 244 and additional policies 245 may not be included in the certifier policy 24.

[0018] An example of the rule-based specification 241 is "If X is x1 or x2, then a certain threshold value is set to Y." This rule-based specification 241 takes precedence over the inferred specification 244. For example, if there is a contradiction between the rule-based specification 241 and the inferred specification 244, the description of the rule-based specification 241 is adopted and the description of the inferred specification 244 is ignored. The self-signed certificate 243 is used to verify the integrity of the data structure. The inferred specification 244 is a predictor or classifier using a machine learning model such as XG Boost. Additional policies 245 are provided for future expansion.

[0019] FIG. 6 is a diagram showing an example of a certifier policy. The certifier policy is composed of a rule-based specification and an inferred specification. The rule-based specification includes one or more namespaces. Each namespace includes people, data, etc., which are classifications of dimensions of the CPSF (Cyber-Physical Security Framework). Each of people, data, etc. includes items such as quality, cost, and delivery date, which are classifications of dimensions of QCDFESG (Quality, Cost, Delivery Date, Flexibility, Environmental, Social, and Governance). The inferred specification includes one or more classifications and one or more predictions. Each classification and prediction includes one or more classes.

[0020] 7 is an operational flow diagram of the trustworthiness evaluation system 1, focusing on the operation of the trusted party device 10. First, the trusted party device 10 creates or updates a metadata format of a component based on input from the trusted party 11 (S801), and stores it in the trusted party storage 12. In other words, in S801, metadata items and parameters to be collected as evidence for each component are set.

[0021] Next, the trusted party device 10 requests metadata about the child component and evidence of elements related to the child component from the child component manufacturer device 40 (S802). Note that although only one child component manufacturer device 40 is shown in this diagram, if there are multiple child components, the trusted party device 10 will make similar requests to each child component manufacturer device 40. The trusted party device 10 also saves data about the components and related elements that it has created (S803).

[0022] Upon receiving this request, the child component manufacturer device 40 provides the trusted party device 10 with pre-created, approved evidence (S804). Next, the trusted party device 10 combines the self-certified data with the child component data received from the child component manufacturer device 40 (S805) and stores the combined data in the trusted party storage 12. Next, the trusted party device 10 requests the third-party certification authority device 50 to provide third-party certified non-confidential information (S806). Upon receiving this request, the third-party certification authority device 50 checks and authenticates the provided data (S807). The authenticated data is stored in the trusted party device 10 and the wide-area storage 30 (S808). The trusted party device 10 receives the data from the third-party certification authority device 50 and stores it in the trusted party storage 12 (S809).

[0023] 8 is an operational flow diagram of the trustworthiness evaluation system 1, focusing on the operation of the certifier device 20. First, the certifier device 20 receives common criteria from the common policy creator device 60 (S811). Next, the certifier device 20 determines special criteria, a storage policy, and a threshold based on input from the certifier 21 (S812), and stores them in the certifier storage 22. Note that the threshold may be the value entered by the certifier 21 as is, or the value entered by the certifier 21 may be input to a pre-trained machine learning model, and the resulting inferred value may be used.

[0024] Next, the certifier device 20 requests evidence of the components and related elements from the trusted party device 10 (S813). Upon receiving this request, the trusted party device 10 provides the certifier device 20 with approved evidence of the trusted party device 10, child components, and related elements (S814). The certifier device 20 then obtains third-party certification from the wide-area storage 30 (S815). Next, the certifier device 20 applies a policy to the received data and generates metrics (S816). Finally, the certifier device 20 visualizes the reliability metrics and provides them to the certifier 21. The certifier device 20 stores the reliability metrics and the judgment results in the certifier storage 22 (S817).

[0025] 9 is a diagram showing an example of a user interface 900 of the certifier device 20. The upper part of the user interface 900 is an input field 910 for the certifier 21, and the lower part is an evaluation result display field 920. The input field 910 includes an object parameter setting field 911, a display parameter search field 912, a certifier parameter setting field 913, a new evaluation button 914, and an evaluation update button 915.

[0026] The object parameter setting field 911 allows the certifier 21 to set various parameters related to the object. The display parameter search field 912 has a field in which a character string can be input. When the certifier 21 inputs the name of a parameter in this field, details of the input parameter are displayed in a visualization result field 922 described below. The certifier parameter setting field 913 allows the certifier 21 to set various parameters related to the object. The new evaluation button 914 is a button for resetting the current input in the input field 910 and making a new evaluation. The update evaluation button 915 is a button for reflecting the input of the certifier 21 in the input field 910 in the evaluation.

[0027] The evaluation result display field 920 includes a result details field 921, a visualization result field 922, a summary field 923, a load button 924, and a save button 925. The result details field 921 displays details of the evaluation results in text. The visualization result field 922 displays a chart visualizing the evaluation results. However, since it is difficult to display all evaluation results at once in the visualization result field 922, only the evaluation results of the parameters entered by the certifier 21 in the display parameter search field 912 and parameters related to the entered parameters are displayed in the visualization result field 922. The summary field 923 displays a summary of the evaluation results. The summary field 923 may include settings for changing the perspective of the summary to be displayed. The load button 924 is a button for loading past evaluation results and displaying them in the evaluation result display field 920. The save button 925 is a button for saving the currently displayed evaluation results.

[0028] FIG. 10 is a diagram showing an example of a visualization diagram displayed in the visualization result field 922 of the user interface 900 shown in FIG. 9. The radar chart shown in this figure shows the evaluation values ​​of "item 1" and "item 2" to be evaluated. In this figure, the evaluation value of "item 1" is shown with a solid line, and the evaluation value of "item 2" is shown with a dashed line. The thick lines in this figure indicate the standard values. This figure shows the evaluation values ​​for each item (QCDFESG), that is, quality, cost, delivery date, flexibility, environment, governance, and sociality. In this figure, it can be seen that "item 1" is below the standard values ​​in all items, and "item 2" is above the standard values ​​except for "governance."

[0029] By looking at this diagram, the certifier 21 can determine that if he or she has to choose between "Item 1" and "Item 2," he or she should select "Item 2," which exceeds all items. Also, if it is a requirement that all items exceed the standard value, the certifier 21 may consider not selecting either item.

[0030] According to the first embodiment described above, the following advantageous effects can be obtained. (1) The software value evaluation method using the reliability evaluation system 1 includes determining collection items, which are metadata items to be collected from each component constituting the software (S801 in FIG. 7), collecting evaluation target data, which is data on the collection items, for each component (S813 in FIG. 8), determining a policy, which is the basis for the value evaluation (S812 in FIG. 8), and evaluating the evaluation target data based on the policy to obtain an evaluation result (S816 in FIG. 8). Therefore, the reliability of software can be evaluated from various perspectives.

[0031] (2) The software value evaluation method using the reliability evaluation system 1 includes creating a visualization diagram that visualizes the evaluation results (S817 in FIG. 8), which allows the certifier 21 to easily understand the evaluation results.

[0032] (3) The software value evaluation method by the reliability evaluation system 1 includes generating a user interface that allows policy setting and displays a visualization diagram (S817 in FIG. 8). This allows the certifier 21 to easily set policies and make judgments.

[0033] (4) The policy includes rule-based specifications defined by humans and inference specifications generated using machine learning models, and the rule-based specifications take precedence over the inference specifications. Therefore, the rules explicitly defined by the certifier 21 can be applied with priority.

[0034] (Variation 1) In the above-described embodiment, the reliability assessment system 1 is used to assess the reliability of software. However, the reliability assessment system 1 may also be used for managing software vulnerabilities and audit records. When used for vulnerability management, the system can be used to discover vulnerabilities at any level of the supply chain and to analyze the risks of affected components.

[0035] In the above-described embodiments and modifications, the functional block configurations are merely examples. Some functional configurations shown as separate functional blocks may be integrated, or a configuration shown in a single functional block diagram may be divided into two or more functions. Furthermore, some of the functions of each functional block may be provided by other functional blocks.

[0036] The above-described embodiments and modifications may be combined with each other. Although various embodiments and modifications have been described above, the present invention is not limited to these. Other embodiments conceivable within the scope of the technical concept of the present invention are also included within the scope of the present invention.

[0037] 1: Trustworthiness evaluation system 10: Trustee device 11: Trustee 12: Trustee storage 13: Trustee evidence 20: Certifier device 21: Certifier 24: Certifier policy

Claims

1. A method for evaluating the value of software, comprising: determining collection items, which are metadata items to be collected from each component constituting the software; collecting evaluation target data, which is data of the collection items, for each of the components; determining a policy that is the basis for value evaluation; and evaluating the evaluation target data based on the policy to obtain an evaluation result.

2. A value assessment method according to claim 1, further comprising creating a visualization diagram that visualizes the assessment results.

3. A value assessment method according to claim 2, further comprising generating a user interface capable of setting the policy and displaying the visualization diagram.

4. A value assessment method as described in claim 1, wherein the policy includes a rule-based specification defined by a human and an inference specification generated using a machine learning model, and the rule-based specification takes precedence over the inference specification.

Images