Verification device, verification method, and verification program

Abstract

A verification device 1 for verifying an output of artificial intelligence (AI) comprises: an acquisition unit 11 that acquires a context related to the output of AI according to a predetermined item; a search unit 13 that searches, from a precedent context acquired in the past according to the predetermined item, for a precedent context similar to the context within a predetermined range; a verification unit 14 that verifies the validity of the output of AI on the basis of whether or not the context is an outlier with respect to the searched precedent context; and an output unit 15 that outputs a context related to the output of AI, the similar precedent context, and a verification result regarding the validity of the output of AI.

Description

Verification device, verification method, and verification program

[0001] The present disclosure relates to a verification device, a verification method, and a verification program.

[0002] With the advancement of AI (Artificial Intelligence) technology, automation using AI is being promoted even in operations that require reliability, such as network operations. In operations that require reliability, errors in judgment can have a significant impact on social functions, so in order to prepare for the possibility of AI making incorrect judgments, it is necessary to be able to verify AI output and detect errors.

[0003] Therefore, a method has been proposed for verifying AI output based on data related to the accuracy of the AI ​​output (for example, the presence or absence of a ping response) and a probability value calculated using a Bayesian network (see Non-Patent Document 1).

[0004] Sato, et al., "Consideration and Proposal of a Workflow Engine for Automation Using AI," Institute of Electronics, Information and Communication Engineers, IEICE Technical Report, 120.433, 2021, pp. 92-97

[0005] However, since it is difficult for maintenance personnel engaged in network operations to interpret the probability values ​​calculated by Bayesian networks and the process of those calculations, there was a problem in that humans could not verify the verification results of the AI ​​output or the verification process.

[0006] The present disclosure has been made in consideration of the above circumstances, and a purpose of the present disclosure is to provide a technology that allows humans to verify the verification results of AI output.

[0007] A verification device according to one aspect of the present disclosure is a verification device that verifies the output of AI (Artificial Intelligence), and includes: an acquisition unit that acquires a context related to the AI ​​output according to specified items; a search unit that searches for a precedent context that is similar to the context within a specified range from precedent contexts previously acquired according to the specified items; a verification unit that verifies the validity of the AI ​​output based on whether the context is an outlier with respect to the searched precedent context; and an output unit that outputs the context related to the AI ​​output, the similar precedent context, and a verification result of the validity of the AI ​​output.

[0008] A verification method according to one aspect of the present disclosure is a verification method for verifying the output of AI (Artificial Intelligence), in which a verification device acquires a context related to the output of the AI ​​according to specified items, searches for a precedent context similar to the context within a specified range from precedent contexts acquired in the past according to the specified items, verifies the validity of the output of the AI ​​based on whether the context is an outlier with respect to the searched precedent context, and outputs the context related to the output of the AI, the similar precedent context, and a verification result of the validity of the output of the AI.

[0009] A verification program according to an aspect of the present disclosure causes a computer to function as the verification device.

[0010] According to the present disclosure, a technology can be provided that allows humans to verify the verification results of AI output.

[0011] FIG. 1 is a diagram showing the functional block configuration of a verification device. FIG. 2 is a diagram showing the processing flow of a verification method performed by the verification device. FIG. 3 is a diagram showing an example of a context related to the output of AI. FIG. 4 is a diagram showing an example of a context of a previous example similar to the context related to the output of AI. FIG. 5 is a diagram showing an image of comparative verification between a context related to the output of AI and a similar context of a previous example. FIG. 6 is a diagram showing the hardware configuration of the verification device.

[0012] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In the description of the drawings, the same parts are designated by the same reference numerals and the description thereof will be omitted.

[0013] [Summary of the present disclosure] In work that requires reliability, whether there are precedents for similar situations is an important factor in determining reliability. In other words, whether there are sufficiently similar precedents in the past is an important factor in the judgment and verification of maintainers.

[0014] Therefore, this disclosure discloses a method for verifying AI output by obtaining data (context) related to the AI ​​output, searching for precedent contexts similar to that context, and checking whether there are sufficiently similar precedent contexts among those precedent contexts.

[0015] At this time, the verification result of the AI ​​output is output together with the context related to the AI ​​output and the context of similar precedents. By having a human view these contexts and compare them with each other, the verification result of the AI ​​output can be interpreted (verified) by a human. Furthermore, the verification process of examining the context of precedents with high similarity can be said to be highly interpretable in that it is easy for a human to evaluate the similarity.

[0016] In particular, the context is selected based on domain knowledge to reflect output errors. For example, in the case of fault cause estimation AI, this would be the presence or absence of a ping response. The context is organized in a text format that is easy for humans to interpret, and then converted into a vector using a pre-trained machine learning model.

[0017] In this way, the context related to the AI ​​output is compared with similar contexts from previous examples, and these contexts are output along with the verification results, allowing humans to interpret the verification results. Also, the context related to the AI ​​output is checked to see if it is similar to the contexts from previous examples, allowing maintainers to interpret the verification process.

[0018] 1 is a diagram showing the functional block configuration of a verification device 1 according to this embodiment. The verification device 1 is a device that verifies the output of AI, and includes an acquisition unit 11, a conversion unit 12, a search unit 13, a verification unit 14, an output unit 15, and a storage unit 16.

[0019] The acquisition unit 11 has the function of inputting the AI ​​output output from the AI ​​model and acquiring data related to the AI ​​output (context related to items related to the accuracy of the AI ​​output) from the NW (Network) 2 and the OpS (Operation System) 3 in accordance with items that have been set in advance based on domain knowledge related to the AI ​​output.

[0020] The conversion unit 12 has the function of converting a context related to the output of the AI ​​into a vector using a pre-trained machine learning model.

[0021] The search unit 13 has a function of searching for a precedent context that is similar within a specified range to a context related to the output of the AI ​​from among the precedent contexts stored in the memory unit 16, based on the magnitude and degree of the distance between the vectors of multiple contexts.

[0022] The verification unit 14 has the function of determining whether a context related to the AI ​​output is an outlier with respect to similar previous contexts, and verifying the validity of the input AI output based on the determination result.

[0023] The output unit 15 has a function of outputting from the verification device 1 to the outside the context related to the AI ​​output, the context of similar precedents, and the verification results of the validity of the AI ​​output.

[0024] The memory unit 16 has a function of storing the context of precedents acquired (collected) in the past according to items set in advance based on domain knowledge related to the output of the AI.

[0025] The items set in advance based on domain knowledge are examples of items.

[0026] 2 is a diagram showing the processing flow of the verification method performed by the verification device 1. Here, a specific example will be described in which the verification device 1 is applied to a failure cause estimation AI model that is used in network operation and maintenance and that estimates the device causing the failure.

[0027] Step S1: First, the acquisition unit 11 inputs the AI ​​output from the failure cause estimation AI model, and acquires data (context) related to the AI ​​output from the OpS 3 that performs network maintenance and operation, according to items set in advance based on domain knowledge related to the AI ​​output. Then, the acquisition unit 11 saves the acquired context in text format so that maintenance personnel can understand it.

[0028] Examples of items that are set in advance based on domain knowledge include ping response results, construction information, time zone of failure occurrence, device name, device type, and device status (e.g., CPU usage rate, memory usage rate, and traffic volume). If the output of the AI ​​is "faulty device = device A," the acquisition unit 11 acquires data (context) corresponding to each item as shown in FIG. 3 from the OpS 3 as data related to "device A."

[0029] Step S2: Next, the conversion unit 12 converts the context related to the AI ​​output into a vector using a pre-trained machine learning model.

[0030] Step S3: Next, the search unit 13 searches the storage unit 16 for one or more precedent contexts that are similar to the context related to the output of the AI ​​within a predetermined range.

[0031] For example, the conversion unit 12 uses a pre-trained machine learning model to convert each precedent context stored in the storage unit 16 into a vector. The search unit 13 calculates the distance between each precedent context vector and a context vector related to the AI ​​output, and extracts a top predetermined number of precedent contexts ( FIG. 4 ) with small distances (high similarities).

[0032] Step S4: Next, the verification unit 14 uses outlier detection to determine whether the context related to the AI ​​output is an outlier with respect to similar precedent contexts. That is, the verification unit 14 determines whether the similar precedent contexts include a precedent context that is sufficiently similar to the context related to the AI ​​output. Thereafter, the verification unit 14 verifies the validity of the AI ​​output output from the failure cause estimation AI model based on the determination result.

[0033] Specific methods for determining whether an outlier is present include, for example, a method of setting a threshold for similarity and determining whether there is one or more precedent contexts with high similarity, and a method of using the local outlier factor method to determine whether the density around a vector embedded with a context related to the AI ​​output is higher than the density around a vector embedded with the context obtained in the search results.

[0034] FIG. 5 is a diagram showing an image of comparative verification between a context related to the output of AI and a context of a previous example.

[0035] As shown in Figure 5(a), if the distance between the context vector related to the AI ​​output and the context vector of a similar previous example in the vector space is sufficiently small (sufficiently similar), the verification unit 14 determines that it is not an outlier and that the AI ​​output is correct.

[0036] On the other hand, as shown in Figure 5(b), if the distance between the context vector related to the AI ​​output and the context vector of a similar previous example in the vector space is large (not sufficiently similar), the verification unit 14 determines that it is an outlier and that the AI ​​output is erroneous.

[0037] 3 and 4, since there are large differences in items (1) to (3), the verification unit 14 determines that the context related to the AI ​​output is an outlier, and determines that the AI ​​output is incorrect. Note that the verification result may be a probability value of correctness (for example, probability of correctness = 95%) according to the magnitude of the distance (degree of similarity) between the vector of the preceding context.

[0038] Step S5: Finally, the output unit 15 outputs to an external device, etc., the context related to the AI ​​output obtained in step S1, the context of one or more precedents searched in step S3, and the verification result of the correctness of the AI ​​output verified in step S4.

[0039] For example, the output unit 15 displays the context in text format on a display device of the OpS 3. A maintainer can compare the context related to the AI ​​output displayed on the screen with a similar preceding context, thereby interpreting (verifying) the verification results and verification process of the AI ​​output.

[0040] Effect of this embodiment According to this embodiment, the verification device 1 includes an acquisition unit 11 that acquires a context related to the AI ​​output in accordance with items set in advance based on domain knowledge related to the AI ​​output, a memory unit 16 that stores precedent contexts acquired in the past in accordance with the items, a search unit 13 that searches the memory unit 16 for precedent contexts that are similar to the context related to the AI ​​output within a predetermined range, a verification unit 14 that verifies the validity of the AI ​​output based on whether the context related to the AI ​​output is an outlier with respect to the searched precedent contexts, and an output unit 15 that outputs the precedent context similar to the context related to the AI ​​output and the verification result of the validity of the AI ​​output, thereby making it possible to provide a technology that enables humans to verify the verification results and verification process of AI output.

[0041] [Others] The present disclosure is not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of the present disclosure.

[0042] The verification device 1 of the present embodiment described above can be realized, for example, by using a general-purpose computer system including a CPU 901, a memory 902, a storage 903, a communication device 904, an input device 905, and an output device 906, as shown in Fig. 6. The memory 902 and the storage 903 are storage devices. In this computer system, the CPU 901 executes a predetermined program loaded onto the memory 902, thereby realizing each function of the verification device 1.

[0043] The verification device 1 may be implemented by one computer, or by multiple computers, or may be a virtual machine implemented on a computer.

[0044] The program for the verification device 1 can be stored in a computer-readable recording medium such as a HDD, SSD, USB memory, CD, or DVD. The computer-readable recording medium is, for example, a non-transitory recording medium. The program for the verification device 1 can also be distributed via a communication network.

[0045] REFERENCE SIGNS LIST 1 Verification device 11 Acquisition unit 12 Conversion unit 13 Search unit 14 Verification unit 15 Output unit 16 Storage unit 2 NW 3 OpS 901 CPU 902 Memory 903 Storage 904 Communication device 905 Input device 906 Output device

Claims

1. A verification device for verifying the output of AI (Artificial Intelligence), comprising: an acquisition unit that acquires a context related to the output of the AI ​​in accordance with specified items; a search unit that searches for a precedent context that is similar to the context within a specified range from precedent contexts acquired in the past in accordance with the specified items; a verification unit that verifies the validity of the output of the AI ​​based on whether the context is an outlier with respect to the searched precedent context; and an output unit that outputs the context related to the output of the AI, the similar precedent context, and the verification results of the validity of the output of the AI.

2. The verification device according to claim 1, wherein the predetermined items are items that are set in advance based on domain knowledge regarding the output of the AI.

3. The verification device according to claim 1, wherein the AI ​​is AI used in network operation and maintenance.

4. A verification method for verifying the output of AI (Artificial Intelligence), comprising: a verification device acquiring a context related to the AI ​​output according to specified items; searching for a precedent context similar to the context within a specified range from precedent contexts acquired in the past according to the specified items; verifying the validity of the AI ​​output based on whether the context is an outlier with respect to the searched precedent context; and outputting the context related to the AI ​​output, the similar precedent context, and the verification result of the validity of the AI ​​output.

5. A verification program that causes a computer to function as the verification device according to any one of claims 1 to 3.

Images