Evaluation device, evaluation method, and evaluation program

The evaluation system addresses the subjectivity in software reliability assessment by using transparency-based scoring, enabling objective evaluation of software reliability through visualization information, thus providing a more accurate and transparent assessment of software trustworthiness.

WO2024247014A1PCT designated stage expired Publication Date: 2024-12-05NIPPON TELEGRAPH & TELEPHONE CORP
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
PCT/JP2023/019803
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-26
Publication Date
2024-12-05

AI Technical Summary

Technical Problem

Conventional methods for evaluating software reliability lack objectivity, particularly in assessing transparency, which is crucial for determining the reliability of software, and often rely on subjective measures.

Method used

An evaluation system comprising an information collection unit that gathers visualization information about software configuration and state, and a software evaluation unit that calculates a reliability score based on transparency, enabling objective evaluation of software reliability.

Benefits of technology

The system allows for the objective quantification of software reliability by directly correlating transparency with reliability, providing a more accurate and transparent assessment of software trustworthiness.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure JP2023019803_05122024_PF_FP_ABST
    Figure JP2023019803_05122024_PF_FP_ABST
Patent Text Reader

Abstract

An evaluation device (10) comprises an information collection unit (121) that collects visualization information in which information relating to software is visualized, and a software evaluation unit (122) that calculates an evaluation score for the reliability of the software on the basis of the transparency of the visualization information collected by the information collection unit (121).
Need to check novelty before this filing date? Find Prior Art

Description

Evaluation device, evaluation method, and evaluation program

[0001] The present invention relates to an evaluation device, an evaluation method, and an evaluation program.

[0002] Software is used not only in IT (Information Technology) systems, but also in controlling automobiles, aircraft, and other machines that affect human life, as well as critical infrastructure such as electricity and water, which are essential to daily life. To accommodate a wide range of applications and requirements, software is required to have an increasing number of functions and become more complex. For example, when using software created by third parties (suppliers), including OSS (Open Source Software), users are exposed to security risks. Therefore, it is necessary to evaluate the reliability of software security before using it, and techniques for evaluating software reliability are known.

[0003] Known technologies for evaluating software reliability include the OpenSSF (Source Security Foundation) scorecard, which measures metrics (binary provision, branch protection, CI (Continuous Integration) testing, dependency, fuzzing, maintenance, and number of vulnerabilities) of an OSS repository from a security perspective and calculates a score that increases with the reliability of the software's security (see, for example, Non-Patent Document 1). The OpenSSF best practice standard is also known, which measures and evaluates metrics other than security (project description, presence or absence of documentation, license notation, reporting process, and software quality) (see, for example, Non-Patent Document 2). Another known project is the CHAOSS, which quantitatively measures the soundness of a project using numerical values.

[0004] OpenSSF Scorecard, [online], [Retrieved May 11, 2023], Internet <https: / / github.com / ossf / scorecard#scorecard-checks> FLOSS Best Practices Criteria, [online], [Retrieved May 11, 2023], Internet <https: / / bestpractices.coreinfrastructure.org / ja / criteria / 0>

[0005] However, in the prior art, it may not be possible to objectively evaluate the reliability of software. For example, in the prior art, the reliability of software is evaluated based on information about the software, focusing on metrics such as the number of vulnerabilities or results such as whether maintenance is performed. As such, the prior art does not describe a method for evaluating software from the perspective of transparency, such as a method for evaluating software reliability based on transparent software information, and therefore may not be able to objectively evaluate the reliability of software.

[0006] For example, the OpenSSF scorecard and the OpenSSF Best Practices Standards do not mention that the transparency of information about software is related to software reliability, and because the OpenSSF Best Practices Standards involve subjective self-assessment, it may not be possible to objectively evaluate the reliability of software.

[0007] As another example, the CHAOSS project discusses the concept of software information transparency, but does not provide information on specific evaluation methods based on this information transparency. Furthermore, the CHAOSS project not only uses easily quantifiable metrics such as code size, number of commits, number of stars, and time to issue addition and closure, but also experimental indicators of the relationships in the development community, such as the elephant factor, which indicates how widely involved members are in activities. For example, based on these experimental indicators, the CHAOSS project subjectively assesses a project where most of the activities are not carried out by core company employees or other members as being largely shared among specific members and therefore unattractive to new contributors and other members. Therefore, the CHAOSS project, which subjectively evaluates software reliability based on experimental indicators, may not be able to objectively evaluate software reliability. The present invention aims to solve this problem and objectively evaluate software reliability.

[0008] In order to solve the above-mentioned problems, the present invention is characterized by having an information collection unit that collects visualized information in which information about software is visualized, and a software evaluation unit that calculates an evaluation score of the reliability of the software based on the transparency of the visualized information collected by the information collection unit.

[0009] According to the present invention, the reliability of software can be objectively evaluated.

[0010] FIG. 1 is a diagram showing an example of the configuration of an evaluation system. FIG. 2 is a diagram for explaining an example of the evaluation system. FIG. 3 is a diagram for explaining an example of the evaluation system. FIG. 4 is a diagram for explaining an example of the configuration of the evaluation system. FIG. 5 is a diagram for explaining an example of visualized information. FIG. 6 is a diagram for explaining an example of software evaluation. FIG. 7 is a diagram for explaining an example of software evaluation. FIG. 8 is a diagram for explaining an example of evaluation results of software and a supplier. FIG. 9 is a flow diagram showing an example of a processing procedure executed by the evaluation device. FIG. 10 is a flow diagram showing an example of a processing procedure executed by the evaluation device. FIG. 11 is a diagram showing an example of the configuration of a computer that executes an evaluation program.

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings, but the present invention is not limited to the embodiment.

[0012] [Overview] An overview of the evaluation system 1 will be described using Figures 1 to 3. Figure 1 is a diagram showing an example of the configuration of the evaluation system. Figures 2 and 3 are diagrams for explaining an example of the evaluation system. In the example of Figure 1, the evaluation system 1 has an evaluation device 10 and a supplier 20. In the example of Figure 2, the evaluation device 10 of the evaluation system 1 collects "visualization information (configuration)" regarding the configuration of software and "visualization information (status)" regarding the status of the software, which are provided by multiple suppliers 20X and 20Y, respectively, and calculates an evaluation score of the reliability of the software based on the transparency of this visualization information.

[0013] When the evaluation device 10 receives an inquiry from a user regarding the evaluation result of software, the evaluation device 10 estimates the content of the response to the inquiry by referring to the reliability of the software, which is the evaluation result, and responds to the user with the evaluation result. In the example of Fig. 1, the evaluation system 1 has only one supplier 20, but as in the example of Fig. 2, the evaluation system 1 may have multiple suppliers 20X and 20Y.

[0014] Here, visualization information refers to visualized information about software. For example, visualization information about at least one of the software configuration and state, such as the "Visualization Information (Configuration)" and "Visualization Information (State)" in FIG. 2 . The visualization information is similar to the concept of the Software Bill of Materials (SBOM), but it may also include information about software operation states, such as operation logs, and communication states, such as communication logs, which are not yet clearly defined in the current SBOM. Furthermore, the visualization information about software configuration may have the same amount of information as the SBOM, but it does not depend on the specific implementation of the SBOM (Software Package Data Exchange (SPDX) or Cyclone DX). Transparency is one of the properties of visualization information, and refers to, for example, the degree of transparency. Software reliability refers to the degree to which software can be trusted, such as the degree of reliability quantitatively evaluated for the reliability of software security.

[0015] The transparency of the visualized information is directly proportional to the reliability of the software, and can be said to be essentially synonymous with it. In other words, when the transparency of the visualized information is high, the reliability of the software is high, and when the transparency of the visualized information is low, the reliability of the software is low. When the transparency of the visualized information is high, it indicates that the visualized information provided by the supplier 20 contains a lot of information about the software and that it is accurate. Therefore, when the transparency of the visualized information is high, the evaluation device 10 evaluates the reliability of the software positively, and when the transparency of the visualized information is low, the evaluation device 10 evaluates the reliability of the software negatively.

[0016] In the example of FIG. 3 , the evaluation device 10 calculates an evaluation score, which is the sum of the basic scores, by adding or subtracting a base score that is directly proportional to the reliability of the software for each evaluation rule related to the transparency of the visualization information based on the transparency of the visualization information for software A. Specifically, the evaluation device 10 adds or subtracts a base score according to each evaluation rule when each evaluation rule is satisfied, such as by adding "+1" to the base score when evaluation rule (1) is satisfied and by subtracting "-1" from the base score when evaluation rule (2) is satisfied. The evaluation device 10 does not add or subtract a base score when an evaluation rule is not satisfied. Similarly, the evaluation device 10 calculates an evaluation score by adding or subtracting a base score for each evaluation rule related to the visualization information for software B based on the transparency of the visualization information for this software. In the example of FIG. 3 , the evaluation device 10 calculates the evaluation scores for both software A and software B as "+10," but the evaluation scores may be different values ​​or may be values ​​other than "+10."

[0017] Unlike conventional techniques that cannot objectively evaluate software reliability because the evaluation is subjective even when software reliability is quantitatively evaluated, the evaluation device 10 quantitatively evaluates software reliability from the perspective of transparency not available in conventional techniques, as described above. This allows the evaluation device 10 to objectively evaluate software reliability.

[0018] [Example of the configuration of the evaluation system] Next, an example of the configuration of the evaluation system 1 will be described in detail with reference to Fig. 4. Fig. 4 is a diagram showing an example of the configuration of the evaluation system. In the example of Fig. 4, the evaluation system 1 has a supplier 20 and an evaluation device 10.

[0019] [Example of Supplier Configuration] The supplier 20 is a providing device that provides visualization information to the evaluation device 10. An example of the supplier 20 is a client terminal. In the example of FIG. 4 , the supplier 20 includes a storage unit 21, a control unit 22, and a communication unit 23.

[0020] [Storage Unit] The storage unit 21 stores various information such as visualization information, an OS (Operating System) and various programs executed by the supplier 20. Examples of the storage unit 21 include storage devices such as HDD (Hard Disk Drive), SSD (Solid State Drive), and optical disks, and information-rewritable semiconductor memories such as RAM (Random Access Memory), flash memory, and NVSRAM (Non Volatile Static Random Access Memory).

[0021] [Control Unit] The control unit 22 controls the entire supplier 20. For example, the control unit 22 is configured with one or more processors having an internal memory in which programs defining various processing procedures and control information are stored, and the processor executes each process using the programs and the internal memory. Examples of the control unit 22 include electronic circuits such as a CPU (Central Processing Unit), an MPU (Micro Processing Unit), and a GPU (Graphics Processing Unit), and integrated circuits such as an ASIC (Application Specific Integrated Circuit) and an FPGA (Field Programmable Gate Array).

[0022] [Communication Unit] The communication unit 23 transmits the visualization information to the evaluation device 10. For example, the communication unit 23 transmits the visualization information to the communication unit 11 of the evaluation device 10 via a network such as the Internet. An example of the communication unit 23 is a network interface card (NIC).

[0023] [Example of Configuration of Evaluation Device] The evaluation device 10 collects visualization information and estimates the reliability of software based on the transparency of the visualization information. The evaluation device 10 may be, for example, a server. In the example of FIG. 4 , the evaluation device 10 includes a communication unit 11, a control unit 12, and a storage unit 13.

[0024] [Communication Unit] The communication unit 11 receives visualization information from the supplier 20. For example, the communication unit 11 receives visualization information from the communication unit 23 of the supplier 20 via a network such as the Internet. The communication unit 11 may transmit a response having response content estimated by the evaluation result reference unit 124 (described later) to the user's terminal. An example of the communication unit 11 is a NIC.

[0025] [Control Unit] The control unit 12 controls the entire evaluation device 10. For example, the control unit 12 is configured with one or more processors having an internal memory in which programs defining various processing procedures and control information are stored, and the processor executes each process using the programs and the internal memory. Examples of the control unit 12 include electronic circuits such as a CPU, MPU, and GPU, and integrated circuits such as an ASIC and FPGA. In the example of FIG. 4, the control unit 12 has an information collection unit 121, a software evaluation unit 122, a supplier evaluation unit 123, and an evaluation result reference unit 124.

[0026] [Information Collecting Unit] The information collecting unit 121 collects visualization information in which information related to software is visualized. In the example of FIG. 4 , the information collecting unit 121 collects visualization information from the supplier 20. The information collecting unit 121 may collect visualization information related to at least one of the software configuration and the software state. For example, the information collecting unit 121 collects visualization information related to the software configuration, and if visualization information related to the software state exists, the information collecting unit 121 further collects visualization information related to the software state.

[0027] The information collecting unit 121 may collect visualization information regarding at least one of the software configuration regarding at least one of the software components and dependencies, and the software state regarding at least one of the software operation state and communication state. For example, the information collecting unit 121 collects the software configuration regarding at least one of the software components and dependencies, and if visualization information regarding the software state regarding at least one of the software operation state and communication state exists, the information collecting unit 121 further collects visualization information regarding the software state regarding at least one of the software operation state and communication state.

[0028] An example of information collection by the information collecting unit 121 will be described below with reference to FIG. 5 . FIG. 5 is a diagram illustrating an example of visualization information. FIG. 5 also illustrates an example of a visualization information database (DB) 131, which is a DB in which visualization information collected by the information collecting unit 121 is registered. In the example of “Visualization Information (Configuration)” in FIG. 5 , the information collecting unit 121 collects visualization information (configuration) related to the configuration of software, including components that are the components of the software and items of dependency relationships. In the example of “Visualization Information (Operation Status)” in FIG. 5 , the information collecting unit 121 collects visualization information (operation status) related to the communication status of software, including time and messages related to the operation status of software. In the example of “Visualization Information (Communication Status)” in FIG. 5 , the information collecting unit 121 collects visualization information (communication status) related to the communication status of software, including the source, destination, and protocol of the software.

[0029] [Software Evaluation Unit] The software evaluation unit 122 calculates an evaluation score of the reliability of the software based on the transparency of the visualization information collected by the information collection unit 121. For example, the software evaluation unit 122 calculates an evaluation score of the reliability of the security of the software. In this way, the software evaluation unit 122 quantitatively evaluates the reliability of the software based on the transparency of the visualization information, which is not mentioned in conventional techniques, and therefore can objectively evaluate the reliability of the software.

[0030] The software evaluation unit 122 may calculate an evaluation score of the reliability of the software based on the transparency of visualized information regarding at least one of the software configuration and the software state collected by the information collection unit 121. Because this visualized information has higher transparency, the software evaluation unit 122 can evaluate the reliability of the software more objectively.

[0031] The software evaluation unit 122 may calculate an evaluation score of the reliability of the software based on visualized information regarding at least one of the software configuration, with respect to at least one of the software components and dependencies, and the software status, with respect to at least one of the operating state and communication state, collected by the information collection unit 121. Because this visualized information has even greater transparency, the software evaluation unit 122 can evaluate the reliability of the software more objectively. The software evaluation unit 122 may calculate the evaluation score of the software based further on at least one of the strictness of the visualized information and the presence or absence of additional information. Because this visualized information has even greater transparency, the software evaluation unit 122 can evaluate the reliability of the software more objectively.

[0032] An example of software reliability evaluation by the software evaluation unit 122 will be described below with reference to Figures 6 and 7. Figures 6 and 7 are diagrams for explaining an example of software reliability evaluation. In the example of Figures 6 and 7, the software evaluation unit 122 adds or subtracts a basic score of the software reliability each time a plurality of evaluation rules classified from a plurality of evaluation perspectives related to the transparency of visualized information is satisfied, and does not add or subtract the basic score when an evaluation rule is not satisfied, and calculates the sum of the basic scores as the evaluation score.

[0033] Specifically, the software evaluation unit 122 calculates the evaluation score by adding or subtracting a basic score each time each of multiple evaluation rules broadly classified according to the perspective of "Is visualization information provided?" in Figure 6, the perspective of "Is the visualization information strict?" in Figure 6, the perspective of "Is additional information provided to the visualization information in addition to the basic information?" in Figure 7, and the perspective of "Is the visualization information modified?" in Figure 7 is satisfied.

[0034] The above-described viewpoints, basic scores, and evaluation scores are merely examples and can be changed as appropriate depending on user settings, etc. For example, the basic score may be multiplied by a weight W (i.e., weights w1, w2, ... corresponding to each viewpoint), and the sum of the basic scores multiplied by each weight may be calculated as the evaluation score. In the examples of FIGS. 6 and 7 , when "+x" in FIG. 6 is "+1," the software evaluation unit 122 calculates the evaluation score as the sum of the basic scores for viewpoints with a basic score of "+1" multiplied by each weight, and the sum of the basic scores for viewpoints with a basic score of "-1" multiplied by each weight. That is, the software evaluation unit 122 calculates the evaluation score as (+1) × sum of the weights + (-1) × sum of the weights.

[0035] When the weights of the basic scores in Figures 6 and 7 are w1, w2, ..., from top to bottom in Figure 6, the software evaluation unit 122 calculates the sum of (+1) x w1, (+1) x w2, (+1) x w4 + (+1), (+1) x w5, (+1) x w9, (+1) x w10, (+1) x w11, and (+1) x w12 as (+1) x the sum of each weight. The software evaluation unit 122 calculates the sum of (-1) x w3, (-1) x w6, (-1) x w7, (-1) x w8, and (-1) x w13 as (-1) x the sum of each weight. Here, the perspective of "whether visualization information is provided" in Figure 6 has a greater impact on the reliability of the software than the other perspectives. For this reason, the software evaluation unit 122 may, for example, set the weight w1 of the basic score corresponding to the perspective of "whether visualization information is provided" to "10" and set each of the other weights to "1", thereby setting the weight w1 to a value greater than each of the other weights.

[0036] [Supplier Evaluation Unit] The supplier evaluation unit 123 is a providing device evaluation unit that calculates an evaluation score of the reliability of the supplier 20 based on the reliability of the software evaluated by the software evaluation unit 122. For example, the supplier evaluation unit 123 calculates an evaluation score of the reliability of the security of the supplier 20 by comprehensively taking into account the evaluation scores of the reliability of the software security. As an example, the supplier evaluation unit 123 calculates the average or total value of the evaluation scores of the reliability of the software for which visualization information has been provided by the same supplier 20 as the evaluation score of the reliability of the supplier 20.

[0037] An example of supplier evaluation by the supplier evaluation unit 123 will be described below with reference to Figures 4 and 8. Figure 8 is a diagram showing an example of evaluation results of software and suppliers. Figure 8 is also a diagram showing an example of the software evaluation DB 132 and supplier evaluation DB 133 of Figure 4. The software evaluation DB 132 refers to a DB in which the reliability of software evaluated by the software evaluation unit 122 is registered as the evaluation result of the software. The supplier evaluation DB 133 refers to a DB in which the reliability of suppliers 20 evaluated by the supplier evaluation unit 123 is registered as the evaluation result of the supplier 20.

[0038] In the example of the "software evaluation DB" in FIG. 8 , the evaluation score of the reliability of software A, for which visualization information has been provided by supplier 20X, is "+2," and the evaluation score of the reliability of software B, for which visualization information has been provided by supplier 20X, is "+1." In this case, as in the "supplier evaluation DB" in FIG. 8 , the supplier evaluation unit 123 calculates "+1.5," which is the average value of the evaluation scores of the reliability of these pieces of software for which visualization information has been provided by the same supplier 20X, as the evaluation score of the reliability of the supplier 20. As in the "software evaluation DB" in FIG. 8 , if the software for which visualization information has been provided by supplier 20Y is a single piece of software, such as software C, the supplier evaluation unit 123 calculates "-1," which is the same as the evaluation score of the reliability of software C, as the evaluation score of the reliability of the supplier 20, as in the "supplier evaluation DB" in FIG. 8 .

[0039] [Evaluation Result Reference Unit] When the evaluation result reference unit 124 receives an inquiry about the evaluation result of software, it estimates the content of a response to the inquiry by referring to the reliability of the software evaluated by the software evaluation unit 122. For example, when the evaluation result reference unit 124 receives an inquiry about the evaluation result of software from a user's terminal or the like via the communication unit 11, it estimates the content of a response to the inquiry by referring to the reliability of the software evaluated by the software evaluation unit 122. In this way, the evaluation result reference unit 124 can estimate the content of a response according to the inquiry.

[0040] When receiving an inquiry regarding the evaluation result of the supplier 20, the evaluation result reference unit 124 may estimate the content of a response to the inquiry by referring to the reliability of the supplier 20 evaluated by the supplier evaluation unit 123. This also allows the evaluation result reference unit 124 to estimate the content of a response in response to the inquiry. The evaluation result reference unit 124 may estimate the content of a response by referring to at least one of the reliability of the software and the reliability of the supplier 20 stored in the storage unit 13. This allows the evaluation result reference unit 124 to estimate the content of a response in response to the inquiry.

[0041] 4 and 8, an example of response content estimation by the evaluation result reference unit 124 will be described. In the examples of Fig. 4 and 8, the evaluation result reference unit 124 estimates the response content by referring to the evaluation scores registered in the software evaluation DB 132 and the supplier evaluation DB 133 using a software name or a supplier name as a query such as a search keyword.

[0042] For example, when the evaluation result reference unit 124 receives an inquiry about the evaluation result of software A from a user of the evaluation system 1, it references the "software evaluation DB" in Fig. 8, which corresponds to the software evaluation DB 132 in Fig. 4, and estimates the evaluation score of the reliability of software A, "+2", as the response content. Also, when the evaluation result reference unit 124 receives an inquiry about the evaluation result of supplier 20X from a user of the evaluation system 1, it references the "supplier evaluation DB" in Fig. 8, which corresponds to the supplier evaluation DB 133 in Fig. 4, and estimates the evaluation score of the reliability of supplier 20X, "+1.5", as the response content.

[0043] [Storage Unit] The storage unit 13 stores various types of information such as a visualization information DB 131, a software evaluation DB 132, and a supplier evaluation DB 133, as well as an OS and various programs executed by the evaluation device 10. Examples of the storage unit 13 include storage devices such as HDDs, SSDs, and optical disks, and semiconductor memories that allow rewriting of information such as RAMs, flash memories, and NVSRAMs.

[0044] [Example of Processing Procedure] An example of processing procedures executed by the evaluation device 10 will be described below with reference to Figures 9 and 10. Figures 9 and 10 are flow diagrams showing an example of processing procedures executed by the evaluation device. First, an example of processing procedures of information collection processing, evaluation processing, and registration processing executed by the evaluation device 10 will be described with reference to Figure 9. In step S1, the information collection unit 121 collects visualized information in which information related to software is visualized. For example, when software whose reliability is to be evaluated by the evaluation device 10 is determined, the information collection unit 121 collects visualized information in which information related to this software is visualized from the supplier 20.

[0045] In step S2, the software evaluation unit 122 calculates an evaluation score of the reliability of the software based on the transparency of the visualization information collected by the information collection unit 121. For example, the software evaluation unit 122 calculates an evaluation score of the reliability of the software based on an evaluation rule regarding the transparency of the visualization information.

[0046] In step S3, the software evaluation unit 122 registers the reliability of the software evaluated in step S2 in the software evaluation DB 132. For example, the software evaluation unit 122 registers the calculated evaluation score in the software evaluation DB 132 every time an evaluation score of the reliability of the software is calculated in step S2.

[0047] Next, an example of the processing procedure of the estimation process and the response process executed by the evaluation device 10 will be described with reference to Figure 10. In step S11, when the evaluation result reference unit 124 receives an inquiry about the evaluation result of software, it references the reliability of the software evaluated by the software evaluation unit 122 and estimates the content of the response to the inquiry. For example, when the evaluation result reference unit 124 receives an inquiry about the evaluation result of software from a user, it references the evaluation score registered in the software evaluation DB 132 in the storage unit 13 using the software name as a query such as a search keyword, and estimates this evaluation score as the content of the response to the inquiry about the evaluation result of the software.

[0048] In step S12, the evaluation result reference unit 124 causes the communication unit 11 to respond with the response content estimated in step S11. For example, the evaluation result reference unit 124 causes the communication unit 11 to transmit the response content estimated in step S11 to the user's terminal.

[0049] [Modifications] The components of each unit shown in the figure are conceptual functional components and do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of the device is not limited to that shown in the figure, and all or part of it can be functionally or physically distributed or integrated in any unit depending on various loads, usage conditions, etc. Furthermore, all or any part of the processing functions performed by the device can be realized by a CPU and a program executed by the CPU, or can be realized as hardware using wired logic.

[0050] Furthermore, among the above processes, all or part of the processes described as being performed automatically can be performed manually, and all or part of the processes described as being performed manually can be performed automatically using known methods. In addition, the processing procedures, specific names, various information, parameters, etc. shown in the above documents and drawings can be changed as desired unless otherwise specified.

[0051] [Program] The evaluation device 10 can be implemented by installing an estimation program as package software or online software on a computer. For example, the computer can function as the evaluation device 10 by executing the estimation program.

[0052] Fig. 11 is a diagram showing an example of a computer that executes an evaluation program. In the example of Fig. 11, the computer 1000 has a memory 1010, a CPU 1020, a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0053] The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM 1012. The ROM 1011 stores a boot program such as a BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to a hard disk drive 1090. The disk drive interface 1040 is connected to a disk drive 1100. For example, a removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to, for example, a mouse 1110 and a keyboard 1120. The video adapter 1060 is connected to, for example, a display 1130.

[0054] The hard disk drive 1090 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. In other words, a program that defines each process executed by the evaluation device 10 is implemented as a program module 1093 in which computer-executable code is written. The program module 1093 is stored, for example, in the hard disk drive 1090. For example, a program module 1093 for executing processes similar to those of the functional configuration of the evaluation device 10 is stored in the hard disk drive 1090. The hard disk drive 1090 may be replaced by an SSD.

[0055] Data used in the processing of the above-described embodiment is stored as program data 1094, for example, in the memory 1010 or the hard disk drive 1090. The CPU 1020 then reads the program module 1093 or the program data 1094 stored in the memory 1010 or the hard disk drive 1090 into the RAM 1012 as necessary and executes them.

[0056] The program module 1093 and program data 1094 are not limited to being stored in the hard disk drive 1090, but may be stored in a removable storage medium and read by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and program data 1094 may be stored in another computer connected via a network (such as a local area network (LAN) or a wide area network (WAN)). The program module 1093 and program data 1094 may then be read by the CPU 1020 from the other computer via the network interface 1070.

[0057] REFERENCE SIGNS LIST 1 Evaluation system 10 Evaluation device 11 Communication unit 12 Control unit 13 Storage unit 20 Supplier 21 Storage unit 22 Control unit 23 Communication unit 121 Information collection unit 122 Software evaluation unit 123 Supplier evaluation unit 124 Evaluation result reference unit 131 Visualization information DB 132 Software evaluation DB 133 Supplier evaluation DB

Claims

1. An evaluation device comprising: an information collection unit that collects visualization information in which information about software is visualized; and a software evaluation unit that calculates an evaluation score of the reliability of the software based on the transparency of the visualization information collected by the information collection unit.

2. The evaluation device described in claim 1, characterized in that the information collecting unit collects the visualization information regarding at least one of the software configuration and the software state, and the software evaluation unit calculates an evaluation score of the reliability of the software based on the transparency of the visualization information regarding at least one of the software configuration and the software state collected by the information collecting unit.

3. The evaluation device described in claim 2, characterized in that the information collecting unit collects the visualization information regarding at least one of the software configuration for at least one of the software's components and dependencies, and the software state for at least one of the software's operational state and communication state, and the software evaluation unit calculates an evaluation score of the software's reliability based on the visualization information regarding at least one of the software configuration for at least one of the software's components and dependencies, and the software state for at least one of the software's operational state and communication state, collected by the information collecting unit.

4. The evaluation device according to claim 1, characterized in that the software evaluation unit calculates an evaluation score of the reliability of the software based further on at least one of the strictness of the visualization information and the presence or absence of additional information.

5. The evaluation device described in claim 1, characterized in that the information collecting unit further has a providing device evaluation unit that collects the visualization information from the providing device and calculates an evaluation score of the reliability of the providing device based on the reliability of the software evaluated by the software evaluation unit.

6. The evaluation device according to any one of claims 1 to 5, further comprising an evaluation result reference unit which, when receiving an inquiry about the evaluation result of the software, estimates the content of a response to the inquiry by referring to the reliability of the software evaluated by the software evaluation unit.

7. An evaluation method executed by an evaluation device, comprising: an information collection step of collecting visualization information in which information relating to software is visualized; and a software evaluation step of calculating an evaluation score of the reliability of the software based on the transparency of the visualization information collected by the information collection step.

8. An evaluation program that causes a computer to execute an information gathering step of gathering visualization information in which information about software is visualized, and a software evaluation step of calculating an evaluation score of the reliability of the software based on the transparency of the visualization information gathered by the information gathering step.

Citation Information

Patent Citations

  • System and method for providing virus protection

    US20130055238A1

  • Evaluation device, system, control method, and program

    WO2020240766A1