Cause estimation method, cause estimation program, and information processing device.

Abstract

Improve the accuracy of answers to questions. [Solution] The information processing device 10 selects first nodes 4a to 4d from the nodes of the knowledge graph 3 that are similar to the first phenomenon to be estimated as a cause. Starting from the first nodes 4a to 4d, the information processing device 10 traces the causal relationship to the terminal second nodes 5a to 5c and generates subknowledge graphs 3a to 3c that include the first nodes 4a to 4d and the second nodes 5a to 5c. The information processing device 10 determines the reliability of the subknowledge graphs 3a to 3c based on the similarity of the first nodes 4a to 4d included in the subknowledge graphs 3a to 3c to the first phenomenon. Then, based on the reliability of the subknowledge graphs 3a to 3c, the information processing device 10 decides whether or not to include the third phenomenon shown in the second nodes 5a to 5c of the subknowledge graphs 3a to 3c as a candidate cause 8 of the first phenomenon.

Description

[Technical Field]

[0001] The present invention relates to a cause estimation method, a cause estimation program, and an information processing device. [Background technology]

[0002] Computer-based reasoning techniques can generate answers to questions by referencing multiple documents. The knowledge presented in these documents can be represented, for example, in a knowledge graph. A knowledge graph systematically links various pieces of knowledge and represents them in a graph structure.

[0003] Regarding techniques related to information inference, for example, a technique has been proposed to estimate the problem graph based on temporal patterns from past state change data, without the need to prepare heuristic rules. Methods for generating knowledge graphs and subgraph clusters for root cause analysis have also been proposed. Knowledge graph embedding representations have been proposed to implement semantic extension of entities, improve the expressive power of complex relationships between entities in the knowledge graph, and enhance the accuracy and comprehensiveness of knowledge graph completion. A device for training knowledge graph embedding models of ontology-enhanced knowledge graphs has also been proposed. Furthermore, a technique for inferring the causal relationships of failures in microservices where trace data cannot be collected has also been proposed. [Prior art documents] [Patent Documents]

[0004] [Patent Document 1] Special Publication No. 2024-517562 [Patent Document 2] U.S. Patent Application Publication No. 2023 / 0050889 [Patent Document 3] U.S. Patent Application Publication No. 2022 / 0121966 [Patent Document 4] Japanese Patent Publication No. 2023-4969 [Patent Document 5] International Publication No. 2024 / 142312 [Overview of the project] [Problems that the invention aims to solve]

[0005] Traditional techniques that infer answers to questions by referring to multiple documents only refer to documents individually. As a result, the causal relationships between various phenomena such as failures remain unclear, and the accuracy of estimating the causes of phenomena is insufficient.

[0006] In one respect, this project aims to improve the accuracy of estimating the causes of phenomena. [Means for solving the problem]

[0007] One proposal provides a cause estimation method in which a computer performs the following processes: The computer accepts input in the form of a sentence describing the first phenomenon to be investigated for cause estimation. The computer selects one or more first nodes from a knowledge graph composed of multiple nodes representing multiple phenomena and edges representing the causal relationships between multiple phenomena, which represent a second phenomenon similar to the first phenomenon. Starting from the first node, the computer traverses the causal relationships indicated by the edges to the terminal second node, generating a subknowledge graph containing the first and second nodes. Based on the similarity of the first nodes in the subknowledge graph to the first phenomenon, the computer determines the confidence that the third phenomenon, indicated by the second node in the subknowledge graph, is the cause of the first phenomenon. Then, based on the confidence of the subknowledge graph, the computer decides whether or not to include the third phenomenon, indicated by the second node in the subknowledge graph, as a candidate cause of the first phenomenon. [Effects of the Invention]

[0008] According to one embodiment, the accuracy of estimating the cause of the phenomenon can be improved.

Brief Description of Drawings

[0009] [Figure 1] It is a diagram showing an example of a cause estimation method according to the first embodiment. [Figure 2] It is a diagram showing an example of the system configuration of the second embodiment. [Figure 3] It is a diagram showing an example of the hardware of the server. [Figure 4] It is a diagram showing an example of estimating the cause of an IT system failure. [Figure 5] It is a diagram showing an example of estimating the cause of a failure using an LLM. [Figure 6] It is a diagram showing an example of KG inference. [Figure 7] It is a diagram showing an example of estimating the cause of a failure using a causal KG. [Figure 8] It is a block diagram showing the functions for estimating the cause of a failure in a server. [Figure 9] It is a diagram showing an example of a causal KG. [Figure 10] It is a flowchart (1 / 2) showing an example of the procedure of KG inference processing. [Figure 11] It is a diagram showing an example of the splitting process of an input query. [Figure 12] It is a diagram showing an example of a subquery obtained by decomposing an input query. [Figure 13] It is a diagram showing an example of a method for calculating similarity. [Figure 14] It is a diagram showing an example of the result of selecting similar entities. [Figure 15] It is a flowchart (2 / 2) showing an example of the procedure of KG inference processing. [Figure 16] It is a diagram showing an example of the process of generating a path starting from a similar entity. [Figure 17] It is a diagram showing an example of a method for extracting a subKG. [Figure 18] It is a diagram showing an example of the calculation example of the reliability score for each subKG. [Figure 19]This figure shows an example of how to calculate a confidence score. [Figure 20] This figure shows an example of the results of the fault cause estimation. [Figure 21] This figure shows an example of a response display screen. [Figure 22] This figure shows an example of the process for generating estimated results of the cause of a failure. [Figure 23] This figure shows an example of the display content in the detailed information display section for potential causes of failure. [Figure 24] This figure shows an example of the screen used to calculate the confidence score. [Figure 25] This figure shows an example of a method for correcting confidence scores. [Figure 26] This figure shows an example of the corrected result of the confidence score. [Modes for carrying out the invention]

[0010] The following description of this embodiment will be made with reference to the drawings. Note that each embodiment can be implemented by combining multiple embodiments within a reasonable scope. [First Embodiment] The first embodiment is a cause estimation method that can improve the accuracy of estimating the cause of a phenomenon. When estimating the cause of a failure, the phenomenon includes the failure that occurred, the cause of the failure, and other phenomena other than the failure and its cause.

[0011] Figure 1 shows an example of a cause estimation method according to the first embodiment. Figure 1 shows an information processing device 10 for implementing the cause estimation method. The information processing device 10 can implement the cause estimation method, for example, by executing a cause estimation program.

[0012] The information processing device 10 includes a storage unit 11 and a processing unit 12. The storage unit 11 is, for example, a memory or storage device of the information processing device 10. The processing unit 12 is, for example, a processor of the information processing device 10. The information processing device 10 may have multiple processors. The processor that executes one of the multiple processes performed by the information processing device 10 may be different from the processor that executes a different process.

[0013] The memory unit 11 stores, for example, a cause estimation program. The memory unit 11 also stores a pre-prepared knowledge graph 3. The knowledge graph 3 consists of multiple nodes representing multiple phenomena and edges representing the causal relationships between multiple phenomena. The edges representing causal relationships are, for example, arrows from a node representing a phenomenon that leads to the cause of a causal relationship to a node representing a phenomenon that leads to the result of a causal relationship. Since the knowledge graph 3 represents the causal relationships between phenomena, it can also be called a causal knowledge graph.

[0014] The processing unit 12 executes the cause estimation method according to the processing procedure shown in, for example, the cause estimation program. The processing procedure of the cause estimation method executed by the processing unit 12 is as follows: The processing unit 12 receives input of a sentence 1 that describes the first phenomenon to be investigated for cause estimation. Sentence 1 is, for example, a natural language description of a failure that occurred in a computer system. The input sentence 1 may contain descriptions of multiple phenomena. Therefore, the processing unit 12 may divide the input sentence 1 into sentences for each phenomenon. For example, the processing unit 12 divides sentence 1 that describes the first phenomenon into a first sentence 2a that describes the phenomenon that appeared as a failure and a second sentence 2b that describes the phenomena that occurred in conjunction with that failure.

[0015] In the example in Figure 1, sentence 1 is written as "XXX after YYY". This indicates that phenomenon "XXX" appeared after phenomenon "YYY". If sentence 1 is a description of a failure, then "XXX" is a description of the failure and "YYY" is a description of a phenomenon other than the failure. In this case, the processing unit 12 divides sentence 1 into a first sentence 2a, which describes the failure phenomenon "XXX", and a second sentence 2b, which describes the phenomenon other than the failure "YYY".

[0016] The processing unit 12 selects one or more first nodes 4a to 4d from among multiple nodes in the knowledge graph 3 that represent a second phenomenon similar to the first phenomenon. If statement 1 is split into a first statement 2a and a second statement 2b, the processing unit 12 selects the node representing the phenomenon similar to the first statement 2a and the node representing the phenomenon similar to the second statement 2b as the first nodes 4a to 4d.

[0017] In the example in Figure 1, the similarity of the first node 4a to the first sentence 2a is "0.92". The similarity of the first node 4b to the first sentence 2a is "0.87". The similarity of the first node 4c to the second sentence 2b is "1.0". The similarity of the first node 4d to the second sentence 2b is "0.89". Therefore, the first nodes 4a and 4b are selected because they are similar to the first sentence 2a, and the first nodes 4c and 4d are selected because they are similar to the second sentence 2b.

[0018] The processing unit 12 starts from the first nodes 4a to 4d and traces the causal relationship indicated by the edge to the terminal second nodes 5a to 5c, generating subknowledge graphs 3a to 3c that include the first nodes 4a to 4d and the second nodes 5a to 5c. Subknowledge graphs 3a to 3c also include the nodes that were traversed when tracing the causal relationship. For example, subknowledge graph 3a is generated by tracing the causal relationship starting from the first node 4a to the terminal second node 5a. Similarly, subknowledge graph 3b is generated by tracing the causal relationship starting from the first node 4a to the terminal second node 5b. Subknowledge graph 3c is generated by tracing the causal relationship starting from the first node 4b to the terminal second node 5c. Although omitted in Figure 1, subknowledge graphs starting from the first nodes 4c and 4d can also be generated.

[0019] The processing unit 12 determines the confidence level that the third phenomenon, shown in the second nodes 5a to 5c included in the subknowledge graphs 3a to 3c, is the cause of the first phenomenon. For example, the processing unit 12 determines the confidence level based on the similarity of the first nodes 4a to 4d included in the subknowledge graphs 3a to 3c to the first phenomenon. In this case, the higher the similarity of the first nodes 4a to 4d included in the subknowledge graphs 3a to 3c, the higher the confidence level of the subknowledge graphs 3a to 3c. This yields a confidence level determination result 7 indicating the confidence level of the subknowledge graphs 3a to 3c.

[0020] Subknowledge graph 3b contains multiple first nodes 4a and 4d. Similarly, subknowledge graph 3c also contains multiple first nodes 4b and 4c. Thus, the confidence level of subknowledge graphs 3b and 3c, which contain multiple first nodes, is higher than that of subknowledge graphs that contain only one of the first nodes.

[0021] The processing unit 12 may also set similarity weights for each of the first nodes 4a to 4d. For example, the processing unit 12 has a weight management table 6. The weight management table 6 has weights set for nodes similar to a sentence depending on whether the phenomenon described in the sentence is a problem or not. For example, the similarity weight for a node with a similar phenomenon to a sentence that indicates a problem is higher than the similarity weight for a node with a similar phenomenon to a sentence that indicates a phenomenon other than a problem.

[0022] Furthermore, the weight management table 6 contains weights for the first nodes 4a to 4d, which are ordered from highest to lowest similarity to the sentences that describe the phenomenon (sentence 1, the first sentence 2a, or the second sentence 2b). In this case, the smaller the value indicating the order based on similarity, the higher the weight.

[0023] When weights are used, the processing unit 12 determines the confidence level of subknowledge graphs 3a to 3c based on the similarity of the first nodes 4a to 4d and the weights of the first nodes 4a to 4d. For example, for each subknowledge graph 3a to 3c, the processing unit 12 calculates a weighted sum of the similarity of the first nodes 4a to 4d that it contains, and assigns a higher confidence level the higher the result of this calculation.

[0024] In the example in Figure 1, subknowledge graph 3a contains the first node 4a, and subknowledge graph 3b contains both the first node 4a and the first node 4d. In this case, the confidence level "b" of subknowledge graph 3b is higher than the confidence level "a" of subknowledge graph 3a because it includes the first node 4d.

[0025] Subknowledge graph 3c contains a first node 4b that is similar to a failure and a first node 4c that is similar to a phenomenon other than a failure. The first node 4b that is similar to the first sentence 2a that indicates a failure is ranked second in similarity. Therefore, the confidence level "c" of subknowledge graph 3c is smaller than the confidence levels "a,b" of subknowledge graphs 3a and 3b, respectively, which contain the first node 4a that is most similar to the first sentence 2a that indicates a failure.

[0026] The processing unit 12 determines whether to include the third cause shown in the second nodes 5a to 5c of the subknowledge graphs 3a to 3c as a candidate cause 8 of the first phenomenon, based on the confidence level of the subknowledge graphs 3a to 3c. For example, the processing unit 12 compares the confidence level of the subknowledge graphs 3a to 3c with a pre-set candidate threshold "T". The processing unit 12 decides to include the third cause corresponding to the subknowledge graph whose confidence level is greater than the candidate threshold "T" as a candidate cause 8.

[0027] In the example in Figure 1, the confidence levels "a" and "b" of subknowledge graphs 3a and 3b are greater than the candidate threshold "T", but the confidence level "c" of subknowledge graph 3c is less than the candidate threshold "T". Therefore, the candidate cause 8 shows phenomenon "A" shown at the second node 5a included in subknowledge graph 3a, and phenomenon "B" shown at the second node 5b included in subknowledge graph 3b, as the causes of the failure.

[0028] In this way, the processing unit 12 can use the knowledge graph 3, which is a causal KG, to obtain candidate other phenomena that may be the cause of the phenomenon shown in the input sentence 1. For example, by creating the knowledge graph 3 based on correct past knowledge, the accurate causal relationships between phenomena are correctly represented in the knowledge graph 3. Therefore, the processing unit 12 can estimate the cause of the phenomenon with high accuracy by estimating the cause of the phenomenon based on the knowledge graph 3. Moreover, because it uses the knowledge graph 3 which accurately represents the causal relationships between phenomena, the occurrence of hallucination is suppressed.

[0029] Estimating the cause of such phenomena can be used, for example, to estimate the cause of system failures in computer systems. For example, a knowledge graph 3 can be created that shows the causal relationship between the cause and effect of a failure, based on information such as the history of responses to system failures in computer systems. By using such a knowledge graph 3, the processing unit 12 can estimate the cause of the failure and obtain appropriate candidates for the cause of the failure.

[0030] Furthermore, the processing unit 12 can divide statement 1 into a first statement 2a indicating a failure and a second statement 2b indicating a phenomenon that occurred as a result of the failure. This allows for highly accurate inference of the cause, taking into account each of the phenomena, even if multiple phenomena are described in statement 1.

[0031] For example, the processing unit 12 can assign different weights to the first nodes 4a and 4b, which are similar to the first sentence 2a that indicates a failure, and to the first nodes 4c and 4d, which are similar to the second sentence 2b that indicates phenomena other than failures. Phenomena other than failures include those that have little relation to the root cause of the failure. Therefore, the processing unit 12 can improve the accuracy of the confidence score by increasing the similarity weight of the first nodes 4a to 4d that are similar to failures.

[0032] The processing unit 12 can also set similarity weights based on the order of similarity. For example, the first node that ranks higher in similarity to the first sentence 2a or the second sentence 2b will be assigned a higher weight. This improves the accuracy of the confidence score calculation.

[0033] Furthermore, the processing unit 12 can include in the subknowledge graphs 3a to 3c nodes along the path traced from the first starting node to the second ending node, as well as nodes reachable by tracing predetermined causal relationships from the nodes along the path. For example, if a phenomenon occurs at a node already traced along the path, the processing unit 12 will include in the subknowledge graph nodes corresponding to other phenomena occurring simultaneously with the phenomenon at the node already traced along the path. The processing unit 12 may also include in the subknowledge graph nodes that show logs of when the phenomenon at the node already traced along the path occurred. By expanding the subknowledge graph in this way, the processing unit 12 can comprehensively include information on phenomena related to the occurrence of failures in any of the subknowledge graphs 3a to 3c. As a result, the accuracy of the confidence score calculation is improved.

[0034] When the processing unit 12 outputs the candidate cause 8, it may output a candidate cause 8 in which the information indicating the causes that it has decided to include in the candidate cause 8 is arranged according to its confidence level. In the example in Figure 1, the confidence level "b" of subknowledge graph 3b is greater than the confidence level "a" of subknowledge graph 3a. Therefore, in the candidate cause 8, the processing unit 12 places phenomenon "B" shown in the second node 5b included in subknowledge graph 3b higher in the ranking, and phenomenon "A" shown in the second node 5a included in subknowledge graph 3a lower in the ranking. This allows the user to check the contents of the candidate causes in order of highest confidence level, for example, when a large number of failure causes are obtained, making the work of identifying the failure cause more efficient.

[0035] Furthermore, the processing unit 12 may output information about the phenomena shown at each node in the sub-knowledge graphs 3a and 3b, which include the second nodes 5a and 5b corresponding to the causes decided to be included in the candidate causes 8, along with information indicating the cause. This allows the user to, for example, learn the procedure for confirming whether the identified fault cause is actually occurring. The user can also obtain information about the path by which the fault occurred from the identified fault cause. As a result, the user can efficiently identify the cause of the fault.

[0036] Furthermore, the processing unit 12 may use a trained language model (e.g., LLM (Large Language Model)) to correct the confidence levels of sub-knowledge graphs 3a to 3c based on the information used to generate knowledge graph 3. The information used to generate knowledge graph 3 includes knowledge not included in knowledge graph 3. By using a language model, for example, the confidence levels of sub-knowledge graphs 3a to 3c can be corrected so as not to contradict the knowledge in the information used to generate knowledge graph 3. As a result, the accuracy of the confidence levels is improved.

[0037] [Second Embodiment] The second embodiment is a computer system that achieves highly accurate estimation of the cause of failure in IT (Information Technology) systems by utilizing a knowledge graph.

[0038] Figure 2 shows an example of the system configuration of the second embodiment. The server 100 and the terminal device 30 are connected via the network 20. The server 100 is a computer that estimates the cause of a failure. The terminal device 30 is a computer used by a user who requests the estimation of the cause of a failure.

[0039] Server 100 has a causal KG, for example, which represents the causal relationships of failures using a knowledge graph (hereinafter referred to as KG). When Server 100 obtains a query from the terminal device 30 indicating the details of the failure that occurred, it estimates the cause of the failure using the causal KG based on that query.

[0040] Figure 3 shows an example of server hardware. Server 100 is controlled as a whole by a processor 101. The processor 101 is connected to memory 102 and several peripheral devices via a bus 109.

[0041] Server 100 may be a multiprocessor system having multiple processors. A collection of multiple processors in a multiprocessor system can be called a processor 101. A processor 101 may also be called a processor circuitry. Each of the multiple processors can execute some or all of the multiple processes executed by Server 100. When there are multiple related processes, the processor executing one process may be different from the processor executing a different process.

[0042] The processor 101 is, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or a DSP (Digital Signal Processor). At least some of the functions that the processor 101 implements by executing a program may be implemented by electronic circuits such as an ASIC (Application Specific Integrated Circuit) or a PLD (Programmable Logic Device).

[0043] Memory 102 is used as the main memory of the server 100. Memory 102 temporarily stores at least a portion of the OS (Operating System) program and application programs that are to be executed by the processor 101. Memory 102 also stores various data used for processing by the processor 101. For memory 102, a volatile semiconductor memory device such as RAM (Random Access Memory) is used.

[0044] Peripheral devices connected to bus 109 include a storage device 103, a graphics controller 104, an input interface 105, an optical drive device 106, a device connection interface 107, and a network interface 108.

[0045] The storage device 103 electrically or magnetically writes and reads data from its built-in recording medium. The storage device 103 is used as an auxiliary storage device for the server 100. The storage device 103 stores the OS program, application programs, and various data. For example, the storage device 103 can be an HDD (Hard Disk Drive) or an SSD (Solid State Drive).

[0046] The graphics controller 104 is an arithmetic unit that performs image processing. The graphics controller 104 is, for example, a GPU (Graphics Processing Unit). A monitor 21 is connected to the graphics controller 104. The graphics controller 104 displays images on the screen of the monitor 21 according to instructions from the processor 101. The monitor 21 can be an OLED (Electroluminescence) display device or a liquid crystal display device. If a GPU is used as the graphics controller 104, the graphics controller 104 can also perform complex numerical calculations such as matrix calculations.

[0047] The input interface 105 is connected to a keyboard 22 and a mouse 23. The input interface 105 transmits signals from the keyboard 22 and mouse 23 to the processor 101. Note that the mouse 23 is just one example of a pointing device; other pointing devices can also be used. Other pointing devices include touch panels, tablets, touchpads, and trackballs.

[0048] The optical drive device 106 uses laser light or the like to read data recorded on the optical disc 24 or write data to the optical disc 24. The optical disc 24 is a portable recording medium on which data is recorded in a way that makes it readable by the reflection of light. Examples of optical discs 24 include DVD (Digital Versatile Disc), DVD-RAM, CD-ROM (Compact Disc Read Only Memory), and CD-R (Recordable) / RW (ReWritable).

[0049] The device connection interface 107 is a communication interface for connecting peripheral devices to the server 100. For example, a memory device 25 and a memory reader / writer 26 can be connected to the device connection interface 107. The memory device 25 is a recording medium equipped with a communication function with the device connection interface 107. The memory reader / writer 26 is a device that writes data to or reads data from the memory card 27. The memory card 27 is a card-type recording medium.

[0050] The network interface 108 is connected to the network 20. The network interface 108 transmits and receives data to and from other computers or communication devices via the network 20. The network interface 108 is a wired communication interface, for example, connected by cable to a wired communication device such as a switch or router. Alternatively, the network interface 108 may be a wireless communication interface, connected by radio waves to a wireless communication device such as a base station or access point.

[0051] The server 100 can implement the processing functions of the second embodiment using the hardware described above. The information processing device 10 shown in the first embodiment can also be implemented using hardware similar to that of the server 100 shown in Figure 3.

[0052] The server 100 implements the processing functions of the second embodiment by executing a program recorded on, for example, a computer-readable recording medium. The program describing the processing to be executed by the server 100 can be recorded on various recording media. For example, the program to be executed by the server 100 can be stored in the storage device 103. The processor 101 loads at least a portion of the program in the storage device 103 into the memory 102 and executes the program. Alternatively, the program to be executed by the server 100 can be recorded on a portable recording medium such as an optical disc 24, a memory device 25, or a memory card 27. The program stored on the portable recording medium becomes executable after being installed in the storage device 103, for example, under control from the processor 101. The processor 101 can also directly read and execute the program from the portable recording medium.

[0053] This section explains the importance and difficulty of fault root cause estimation in IT systems. For the stable operation of IT systems and networks, fault root cause estimation technology that can expedite fault recovery is crucial. In recent years, in particular, configurations have become more complex due to virtualization and multi-vendor support, and the causes of faults have become more diverse. Therefore, accurately estimating fault root causes has become increasingly difficult.

[0054] Figure 4 shows an example of fault cause estimation in an IT system. Figure 4 illustrates an example of fault cause estimation in a multi-vendor antenna base station (RU: Radio Unit) and base station control unit (DU: Distributed Unit) connection. For example, consider a system operating with RU31 and DU33, where RU32 is added. RU31 and DU33 are manufactured by the same vendor (Vendor A), but RU32 is manufactured by a different vendor (Vendor B). In such a case, the addition of RU32 may cause a system failure.

[0055] When a failure occurs, information related to the failure is input to server 100, and server 100 estimates the cause of the failure. For example, server 100 analyzes the likelihood of each possible cause of the failure. From the analysis results, server 100 estimates the phenomenon that is the root cause of the failure.

[0056] Possible causes of communication failures include hardware (HW) failures / compatibility issues, compatibility issues between RU-SW (switch) and DU-SW, and incorrect settings for XX (configuration item).

[0057] As such, there are numerous causes of failures. Furthermore, in recent years, the complexity of configurations due to virtualization and multi-vendor support has led to a diversification of failure causes. As a result, the difficulty in estimating the cause of failures has increased.

[0058] To estimate the cause of a failure, documentation detailing past failures and their causes can be used. For example, using LLM (Limited Licensing Management) makes it possible to estimate the cause of a failure based on a large amount of documentation.

[0059] Figure 5 shows an example of fault cause estimation using LLM. For example, a set of documents 35 is prepared in advance that shows past faults of the managed IT system, other phenomena that occurred in conjunction with those faults, and the causes of those faults. The set of documents 35 includes documents 35a and 35b for each fault.

[0060] Server 900 has LLM901 implemented. Server 900 can use LLM901 to engage in natural language dialogue with terminal device 30 and provide answers regarding the cause of a problem. For example, user 34 inputs the question "What causes the router to become overloaded?" to terminal device 30. Terminal device 30 sends a string representing the question to server 900. Server 900 performs inference using the question as input to LLM901. LLM901 is a trained model that performs inference by referring to document set 35. When server 900 receives the answer output from LLM901, it sends the outputted answer to terminal device 30.

[0061] When server 900 performs inference by referring to document set 35 without using KG, for example, multiple documents 35a and 35b are each referred to individually. In the example in Figure 5, the fault in document 35a is "Router A becomes overloaded." This is similar to the fault described in user 34's question. Therefore, based on document 35a, server 900 sends an answer to terminal device 30, for example, "The following causes are possible: - Cable degradation (see document A)."

[0062] In reality, there isn't always just one reason why a router becomes overloaded. For example, a router can become overloaded due to a misconfiguration of network equipment. Therefore, the ideal response would be one that lists possible causes, such as "The following are possible causes: - Cable degradation (see Document A) - XX configuration error (see Document B)."

[0063] In this way, in reasoning using LLM901, since each of the multiple documents 35a and 35b is only referenced individually, the server 900 cannot recognize the relationship between documents 35a and 35b. In the example in Figure 5, the server cannot recognize that the "XX configuration error" shown in document 35b is also related to the failure. Therefore, it is difficult to obtain an ideal answer using only LLM901 for reasoning.

[0064] KG represents the relationships between pieces of knowledge. Therefore, it is conceivable to use KG-based inference techniques for estimating the cause of failure. Figure 6 shows an example of KG inference. For example, let's assume that KG37 about the presidents of the United States has been generated in advance. KG37 consists of multiple nodes 37a to 37e that represent entities and edges 37g to 37j that represent relationships. Entities are information that represents things, concepts, events, etc. in the world. Nodes 37a to 37e are represented by shapes such as ellipses, and edges 37g to 37j are represented by arrows that connect nodes 37a to 37e.

[0065] In KG inference, structured data within KG37 is used to infer the appropriate answer to the input query 36. For example, in the case of input query 36 which inquires about the birthplace of former President Obama, an entity representing the president's name is identified. Among the edges 37g to 37i connected to node 37a of that entity, edge 37g of the relation related to birthplace is identified. Then, the entity represented by node 37b connected to the identified edge 37g is output as the answer 38. In the example in Figure 6, "Hawaii" is obtained as the answer 38.

[0066] Applying this KG inference to fault cause estimation enables inference that takes into account the relationships between documents. Therefore, the server 100 in the second embodiment refers to a KG (causal KG) that shows the causal relationship of the fault and responds with an appropriate fault cause and its estimation result corresponding to the fault case shown in the input query.

[0067] Figure 7 shows an example of failure cause estimation using a causal KG. A causal KG 39 can be generated based on a set of documents 35, which includes multiple documents 35a and 35b. The causal KG 39 is represented by nodes 39a to 39e corresponding to entities that show phenomena related to the failure, and edges 39f to 39i that show the causal relationships between those entities. The phenomena shown by the entities include phenomena that are recognized as failures, phenomena that may be the root cause of the failure, or other phenomena.

[0068] For example, the entity representing the failure "Router A becomes overloaded" in document 35a is represented by node 39d. The entity representing the phenomenon "Frequent retransmissions between Router A and Device B" in document 35a is represented by node 39c. The entity representing the cause "Cable degradation" in document 35a is represented by node 39a.

[0069] The entity indicating the failure "Device B is unable to send data" in Document 35b is represented by node 39e. The entity indicating the phenomenon "Frequent retransmissions between Router A and Device B" in Document 35b is represented by node 39c. The entity indicating the cause of Document 35b "Configuration error of XX" is represented by node 39b.

[0070] Edges 39f and 39g, indicating a causal relationship, are set from the node corresponding to the entity causing the problem in documents 35a and 35b to the node corresponding to the entity causing the phenomenon. Edges 39h and 39i, indicating a causal relationship, are set from the node corresponding to the entity causing the phenomenon in documents 35a and 35b to the node corresponding to the entity causing the failure.

[0071] Documents 35a and 35b share the same phenomenon. Therefore, the entities representing the respective phenomena in documents 35a and 35b are represented by the same node 39c. The generated causal KG39 is stored, for example, on server 100.

[0072] When user 34 enters a question into terminal device 30, that question is sent from terminal device 30 to server 100 as an input query. For example, if the input query is "What causes the router to become overloaded?", server 100 inputs the input query into LLM111. Server 100 then refers to causal KG39, performs information processing according to LLM111, and obtains the output of LLM111.

[0073] For example, server 100 identifies nodes 39a and 39b, which are reached by tracing the edges in the reverse direction of the arrow, starting from node 39d, which corresponds to the entity "Router A becomes overloaded" in causal KG39. Server 100 then outputs the cause of the failure indicated for each entity corresponding to the reached nodes 39a and 39b as an answer. This results in an answer that shows multiple possible causes of the failure. Furthermore, by using LLM111, the content of the answer can be represented as a string in natural language.

[0074] In the example in Figure 7, when there are multiple entities that could be causing the failure, these entities are presented to user 34 as answers, but it is not indicated which entity is most likely to be the cause. By effectively utilizing the causal KG39, it is possible to evaluate the likelihood that each entity is the cause of the failure. Therefore, server 100 evaluates the likelihood of each potentially causing the failure using a confidence score, and presents the entity with the highest confidence score to user 34. This allows for the presentation of an appropriate cause of the failure to user 34.

[0075] Figure 8 is a block diagram showing the functions for estimating the cause of failure in a server. Server 100 has a storage unit 110, a KG inference unit 120, and an interaction system 130. Storage unit 110 stores LLM 111, an embedding model 112, and a causal KG 113.

[0076] LLM111 is a pre-trained neural network language model. LLM111 takes natural language text (or coded data of that text) as input and outputs answers in natural language to the questions presented in that text.

[0077] The embedding model 112 is a pre-trained model for converting natural language sentences into numerical vectors based on the terms they contain. Causal KG113 is a KG generated based on documents describing the cause of the failure. Causal KG113 is a graph that shows the relationship between the cause of the failure, the phenomenon that occurs as a result of that cause, and the resulting failure.

[0078] The KG inference unit 120 works in conjunction with the dialogue system 130, which utilizes the LLM 111, to estimate the cause of a failure based on the causal KG 113. For example, when the KG inference unit 120 obtains an input query containing a question string from the terminal device 30, it decomposes the input query into a description of the failure and a description of other phenomena. The decomposition of the input query can be performed using the dialogue system 130. For example, the KG inference unit 120 inputs a prompt to the dialogue system 130 indicating the decomposition of the input query. The dialogue system 130 then uses the LLM 111 to decompose the text shown in the input query into subqueries.

[0079] The KG inference unit 120 also searches for multiple entities similar to each decomposed subquery (similar entities) within the causal KG. For example, the KG inference unit 120 uses the embedded model 112 to convert the subqueries into numerical vectors.

[0080] The KG inference unit 120 calculates a confidence score for the root cause of the failure indicated by the root cause path starting from each similar entity found. For example, the KG inference unit 120 generates a subKG containing the relevant path. The KG inference unit 120 then calculates a confidence score for each subKG based on the combination of similar entities contained within the subKG and the similarity between those similar entities and the subquery.

[0081] The KG inference unit 120 sorts the fault causes and their estimated results, whose calculated confidence scores are above a threshold, by confidence score and sends them to the terminal device 30 as a response. The dialogue system 130 uses LLM 111 to engage in natural language dialogue. For example, when the dialogue system 130 receives a prompt from the KG inference unit 120 instructing it to split an input query, it decomposes the input query into, for example, a description of a failure and a description of other phenomena. The dialogue system 130 then sends each of the descriptions obtained through the decomposition as a subquery to the KG inference unit 120.

[0082] The functions of each element shown in Figure 8 can be realized, for example, by having the processor 101 execute the program module corresponding to that element. Next, I will explain the causal relationship KG113 in detail.

[0083] Figure 9 shows an example of a causal KG. Figure 9 shows a portion of the causal KG40. The causal KG40 includes nodes 41a to 41j corresponding to entities. Nodes 41a to 41j are connected by edges 42a to 42i that indicate the relationships between entities. These relationships include "cause" relationships and "indicate" relationships.

[0084] Edges indicating a "cause" relationship are arrows connecting a node of an entity that indicates the cause of a phenomenon to a node of another entity that arises as a result of the occurrence of that entity. Edges indicating an "indicate" relationship are arrows connecting a node of an entity that indicates information about the observed phenomenon (e.g., a log of the observed phenomenon) to a node of an entity that indicates the observed phenomenon.

[0085] Node 41a of entity "A" is connected to node 41e of entity "E" by edge 42a, indicating an "indicate" relationship. Node 41b of entity "B" is connected to node 41e of entity "E" by edge 42b, indicating a "cause" relationship. Node 41c of entity "C" is connected to node 41f of entity "F" by edge 42c, indicating a "cause" relationship. Node 41d of entity "D" is connected to node 41g of entity "G" by edge 42d, indicating an "indicate" relationship. Node 41e of entity "E" is connected to node 41g of entity "G" by edge 42e, indicating a "cause" relationship. Node 41e of entity "E" is connected to node 41h of entity "H" by edge 42f, indicating a "cause" relationship. Node 41f of entity "F" is connected to node 41h of entity "H" by edge 42g, which indicates a "cause" relationship. Node 41f of entity "F" is connected to node 41i of entity "I" by edge 42h, which indicates a "cause" relationship. Node 41h of entity "H" is connected to node 41j of entity "J" by edge 42i, which indicates a "cause" relationship.

[0086] The double-line edges 42e and 42f, connected from node 41e to nodes 41g and 41h respectively in causal KG40, represent an AND relationship. An AND relationship indicates that when the entity at the beginning of the relationship occurs, multiple entities at the end of the relationship may occur simultaneously. Here, "simultaneously" means that multiple entities occur triggered by the occurrence of the same entity, although the timing of their occurrence may be staggered.

[0087] Next, we will explain the procedure for KG inference processing using causal KG40. Figure 10 is a flowchart (1 / 2) showing an example of the KG inference process. The process shown in Figure 10 will be explained below according to the step numbers.

[0088] [Step S101] The KG inference unit 120 obtains an input query from the terminal device 30. The input query includes descriptions of failures, descriptions of other phenomena, etc. [Step S102] The KG inference unit 120 decomposes the input query into subqueries. For example, the KG inference unit 120 inputs a prompt to the dialogue system 130 instructing it to decompose the input query and sends the input query to the dialogue system 130. The dialogue system 130 interprets the meaning of the input query text using the LLM 111 and decomposes the input query into a description of the failure and a description of other phenomena. The dialogue system 130 sends each of the decomposed descriptions to the KG inference unit 120 as a subquery.

[0089] [Step S103] The KG inference unit 120 acquires information within the causal KG40. For example, the KG inference unit 120 acquires information about entities corresponding to each node in the causal KG40 and information about relationships corresponding to each edge.

[0090] [Step S104] The KG inference unit 120 searches for entities within the causal KG 40 and selects entities similar to each subquery. For example, the KG inference unit 120 calculates the similarity between each subquery and each entity. Then, for each subquery, the KG inference unit 120 identifies a predetermined number of entities, starting with those with the highest similarity.

[0091] Figure 11 shows an example of the input query splitting process. Prompt 51 is input from the KG inference unit 120 to the dialogue system 130 using LLM 111. Prompt 51 is, for example, the string "Separate incidents and events occurred at the same time written in the sentence in the QUERY section if possible. Do NOT include other information in the answer." Prompt 51 instructs that the sentence of the input query 52 be split into "description of the problem" and "description of other phenomena," and that no other information should be included.

[0092] The KG inference unit 120 inputs the input query 52 to the dialogue system 130 following the prompt 51. The dialogue system 130 uses the LLM 111 to decompose the sentence shown in the input query 52 into multiple descriptions. The dialogue system 130 then sends the answer 53, which includes subqueries 53a, 53b, ... for each of the decomposed descriptions, back to the KG inference unit 120.

[0093] Figure 12 shows an example of a subquery obtained by breaking down an input query. For example, when the input query 54a, "The FCC TB radio is experiencing a VSWR alarm when the DU transmits packets to the radio for an extended period," is entered into the dialogue system 130, the dialogue system 130 sends a response 55a. Response 55a includes the statements "The FCC TB radio is experiencing a VSWR alarm." and "The DU is transmitting packets to the radio for an extended period." as subqueries.

[0094] When the input query 54b, "The L1app is shutting down after a "Too little space in output mbuf" message with n71," is entered into the dialogue system 130, the dialogue system 130 sends a response 55b. The response 55b includes the statements "The L1app is shutting down." and "A "Too little space in output mbuf" message is received with n71." as subqueries.

[0095] When the input query 54c, "An Rx Gain alarm is observed as soon as the DU starts exchanging UL / DL packets, and the radio is unable to radiate afterward," is entered into the dialogue system 130, the dialogue system 130 sends a response 55c. The response 55c includes the following subqueries: "An Rx Gain alarm is observed," "The DU is exchanging UL / DL packets," and "The radio is unable to radiate."

[0096] When the input query 54d, "EVM failures are occurring on antenna ports with an RCT setup," is entered into the dialogue system 130, the dialogue system 130 sends a response 55d. The response 55d includes the statements "EVM failures are occurring." and "This is happening on antenna ports with an RCT setup." as subqueries.

[0097] In this way, by using LLM111, the various statements shown in the input query are decomposed into "descriptions about the failure" and "descriptions about other phenomena." The KG inference unit 120 calculates the similarity between the subquery represented by the decomposed description and the entity represented by the node in the causal KG, based on the subquery.

[0098] Figure 13 shows an example of how similarity is calculated. Assume that two subqueries, 56a and 56b, are generated by decomposing the input query 54e. Subquery 56a contains a "description of the problem," and subquery 56b contains a "description of other phenomena."

[0099] The KG inference unit 120 uses, for example, the embedding model 112 to convert subqueries 56a and 56b into numerical vectors. By using the embedding model 112, the descriptions within subqueries 56a and 56b are mathematically embedded from a space with dimensions equal to the number of words to a vector space with lower dimensions. The embedding model 112 is, for example, a deep learning model called "Transformer".

[0100] The KG inference unit 120 also uses the embedded model 112 to create numerical vectors for the entities represented by each node 57a to 57e included in the causal KG 57. The KG inference unit 120 then calculates the similarity between subqueries 56a and 56b and each entity represented by nodes 57a to 57e in the causal KG 57 by comparing the numerical vectors. For example, the KG inference unit 120 uses the cosine similarity between the numerical vectors as the similarity between subqueries 56a and 56b and the entities represented by nodes 57a to 57e.

[0101] In the example in Figure 13, the similarity between subquery 56a and the entity shown in node 57a is "0.92". The similarity between subquery 56a and the entity shown in node 57b is "0.51". The similarity between subquery 56a and the entity shown in node 57c is "0.87". The similarity between subquery 56a and the entity shown in node 57d is "0.28". The similarity between subquery 56a and the entity shown in node 57e is "0.40".

[0102] Furthermore, the similarity between subquery 56b and the entity shown in node 57a is "0.82". The entities shown in subquery 56b and node 57b have the same description "YYY", and the similarity is "1.0". The similarity between subquery 56b and the entity shown in node 57c is "0.38". The similarity between subquery 56b and the entity shown in node 57d is "0.48". The similarity between subquery 56b and the entity shown in node 57e is "0.89".

[0103] The KG inference unit 120 searches for the top n entities (where n is a natural number) in terms of similarity from the causal KG 57 for each of the subqueries 56a and 56b. The KG inference unit 120 selects the relevant entities and ranks the selected entities in descending order of similarity.

[0104] In the example in Figure 13, the top two entities are selected for subqueries 56a and 56b, respectively. For subquery 56a, the entity with the highest similarity is the one shown at node 57a, and the entity with the second highest similarity is the one shown at node 57c. For subquery 56b, the entity with the highest similarity is the one shown at node 57b, and the entity with the second highest similarity is the one shown at node 57e.

[0105] Here, the entity most similar to subquery 56a is designated as "1'", and the entity second most similar to subquery 56a is designated as "1''". If a third most similar entity to subquery 56a is selected, that entity is designated as "1'''".

[0106] Additionally, the entity most similar to subquery 56b is designated as "2'", and the entity second most similar to subquery 56b is designated as "2''". If the third most similar entity to subquery 56b is selected, that entity is designated as "2'''".

[0107] Figure 14 shows an example of similar entity selection results. For example, when the statement "The RU was not transmitting after configuring the carriers in 2T2R mode." shown in input query 54f is decomposed by LLM111, the answer 55f includes subqueries 56c and 56d. Subquery 56c is "The RU was not transmitting." and subquery 56d is "Carriers were configured in 2T2R mode."

[0108] The causal KG43 contains nodes 43a to 43w, each corresponding to a different entity. The entity most similar to subquery 56c is the entity corresponding to node 43d. The entity second most similar to subquery 56c is the entity corresponding to node 43t. The entity third most similar to subquery 56c is the entity corresponding to node 43n. The entity most similar to subquery 56d is the entity corresponding to node 43r. The entity second most similar to subquery 56d is the entity corresponding to node 43c. The entity third most similar to subquery 56d is the entity corresponding to node 43p.

[0109] Of the nodes 43a to 43w included in the causal KG43 shown in Figure 14, nodes 43a, 43b, 43f, 43i, 43l, 43m, 43q, and 43v correspond to entities that indicate the cause of the failure.

[0110] Once the search for similar entities (similar entities) for each subquery is complete, the KG inference process generates subKGs based on the similar entities, and calculates a confidence score for each subKG.

[0111] Figure 15 is a flowchart (2 / 2) showing an example of the KG inference process. The process shown in Figure 15 will be explained below according to the step numbers. [Step S201] The KG inference unit 120 extracts subKGs that include the root cause path from each similar entity to the cause entity. Here, the total number of extracted subKGs is "N" (where N is a natural number).

[0112] [Step S202] The KG inference unit 120 assigns subKG1, subKG2, ..., subKG to each extracted subKG. N Label it as such. [Step S203] The KG inference unit 120 initializes the value of index i to "1" (i=1).

[0113] [Step S204] The KG inference unit 120 is subKG i Depending on the similarity between the similar entities contained within and the subqueries similar to those similar entities, subKG i The system calculates the failure cause and the confidence score of the estimated result.

[0114] [Step S205] The KG inference unit 120 updates index i (i = i + 1). [Step S206] The KG inference unit 120 determines whether the value of index i is greater than "N". If the value of index i is greater than "N", the KG inference unit 120 proceeds to step S207. If the value of index i is less than or equal to "N", the KG inference unit 120 proceeds to step S204.

[0115] [Step S207] The KG inference unit 120 sorts the failure causes and their estimated results obtained from each extracted subKG in descending order of confidence score. [Step S208] The KG inference unit 120 responds to the terminal device 30 with the cause of the failure and its estimated result, for which the confidence score is above the threshold.

[0116] In this way, highly reliable failure causes and estimated results of those failure causes can be provided. Moreover, the causal KG is generated based on information (documentary) about failures that have occurred in the past. Therefore, because the confidence score is calculated based on the causal KG, the occurrence of hallucination is suppressed.

[0117] Next, we will specifically explain the process of generating a root cause path starting from a similar entity. Figure 16 shows an example of a path generation process starting from a similar entity. For example, let's assume that entity "J" corresponding to node 41j is the similar entity that is most similar to the subquery "Description of the problem".

[0118] The KG inference unit 120 traces the nodes connected by causal relationship edges ("cause" relation) from node 41j of the causal KG40 similar entity "J" upwards (in the reverse direction of the arrow) to the terminal node. If the path that can be traced to the nodes branches along the way (there are two or more higher-level nodes connected by the "cause" relation), the KG inference unit 120 generates a new separate path that includes the path before the branch. In the example of causal KG40, the path branches at node 41h. Therefore, two paths 61 and 62 are generated starting from node 41j.

[0119] Path 61 is a path that traverses the nodes in the following order: node 41j (entity "J"), node 41h (entity "H"), node 41e (entity "E"), and node 41b (entity "B"), all of which are similar entities "1'". Entity "B" is the entity that indicates the cause of the failure.

[0120] Path 62 is a path that traverses the nodes in the following order: node 41j (entity "J"), node 41h (entity "H"), node 41f (entity "F"), and node 41c (entity "C"), all of which are similar entities "1'". Entity "C" is the entity that indicates the cause of the failure.

[0121] Once a root cause path is generated, the KG inference unit 120 extracts a subKG containing that path from the causal KG 40 for each path. Figure 17 shows an example of a method for extracting subKGs. The KG inference unit 120 extracts a subgraph from the causal KG 40 consisting of entities that satisfy any of the following conditions and the relationships between them, and uses it as a subKG. Condition 1: Entities contained within the path (root cause path) Condition 2: Entities connected to the root cause path by an AND relationship (the related entity occurs simultaneously) Condition 3: Entities that meet conditions 1 and 2 are connected by a relationship ("indicate") that indicates the occurrence of the phenomenon. In the example in Figure 17, first, under condition 1, entities "J, H, E, B" included in path 61 are extracted. Next, under condition 2, entity "G" connected to entity "E" in an AND relationship is extracted. Furthermore, under condition 3, entities "A" and "D" connected to entities "E" and "G" respectively in an "indicate" relationship are extracted. Finally, subKG63, which shows the extracted entities and the relationships connecting them, is extracted.

[0122] Subkey values ​​(subKGs) are extracted starting from each similar entity that is similar to any of the subqueries, resulting in, for example, the extraction of multiple subkey values. A confidence score is then calculated for each of these multiple subkey values.

[0123] Figure 18 shows an example of calculating the confidence score for each subKG. For example, subKG64a and 64b are extracted based on the similar entity "1'" which is similar to "subquery 1". SubKG64c is extracted based on the similar entity "1''". SubKG64d is extracted based on the similar entity "1''''".

[0124] Similarly, subKGs are extracted starting from similar entities that are similar to "subquery 2". For example, subKG64N is extracted starting from the similar entity "2'''".

[0125] Then, a confidence score is calculated for each of the extracted subKG64a, 64b, 64c, 64d, ..., 64N. The calculated confidence scores are associated with the subKG64a, 64b, 64c, 64d, ..., 64N and stored as the confidence score calculation result 67, for example, in memory 102.

[0126] In the calculation of the reliability score, for example, the similarity of similar entities included in the subKG to be calculated with respect to the subquery is used. In this case, the higher the similarity of the similar entities included in the subKG, the higher the reliability score of that subKG.

[0127] FIG. 19 is a diagram showing an example of a method for calculating a reliability score. For example, the KG inference unit 120 has a weight management table 65. In the weight management table 65, weights for similar entities are set. For example, in the weight management table 65, for each subquery, weights of similar entities corresponding to the order of similarity to that subquery are set.

[0128] For example, the weight of the similar entity with the highest similarity (similarity order 1) with respect to the subquery of "description related to failure" (subquery 1) is "w 1' =1". The weight of the similar entity with the second highest similarity (similarity order 2) with respect to that subquery is "w 1'' =0.9". The weight of the similar entity with the nth highest similarity (similarity order n) (n is a natural number) with respect to that subquery is "w 1(n) =0.9 n-1 " ((n) is a superscript of 1).

[0129] The weight of the similar entity with the highest similarity (similarity order 1) with respect to the subquery of "description related to other phenomena" (subquery 2) is "w 2' =10 -1 ". The weight of the similar entity with the second highest similarity (similarity order 2) with respect to that subquery is "w 2'' =10 -1 ·0.9". The weight of the similar entity with the nth highest similarity (similarity order n) (n is a natural number) with respect to that subquery is "w 2(n) =10 -1 ·0.9 n-1 " ((n) is a superscript of 2).

[0130] The weights of similar entities to the subqueries (subquery 3, ...) that are similar to "descriptions of other phenomena" other than "subquery 2" are the same as those for "subquery 2". For example, suppose three subKGs, subKG66a to subKG66c, are extracted from a causal KG. The label for subKG66a is "1", the label for subKG66b is "2", and the label for subKG66c is "3".

[0131] In subKG66a~66c, the top two entities similar to "Subquery 1" and "Subquery 2" are identified as similar entities. The similarity of similar entity "1'" to "Subquery 1" is "0.94". The similarity of similar entity "1''" to "Subquery 1" is "0.88". The similarity of similar entity "2'" to "Subquery 2" is "0.90". The similarity of similar entity "2''" to "Subquery 2" is "0.85".

[0132] The KG inference unit 120 calculates a weighted sum of similarities among the similar entities contained in subKG66a to 66c for each subquery, using the similarity of the entity with the highest similarity. For example, subKG66c contains two similar entities "2'" and "2''" for "subquery 2". In this case, the similarity of the more similar entity "2'" is used in the calculation of the weighted sum. The weighted sums for each subKG66a to 66c are as follows: • subKG1:S1=0.94×w 1' =0.94 • subKG2:S2=0.94×w 1' +0.85 × w 2'' =1.0165 • subKG3:S3=0.88×w 1'' +0.90 × w 2' =0.882 Next, the KG inference unit 120 multiplies the weighted sum by a coefficient k (k=1 / K) based on the number of subqueries K (K is a natural number) obtained by dividing the input query. In the example in Figure 19, the number of subqueries is "K=2" (k=1 / 2). Therefore, the weighted sums of each subKG66a~66c are modified as follows. subKG1:S1'=0.94×k=0.47 ·subKG2:S2'=1.0165×k=0.50825 • subKG3:S3'=0.882×k=0.441 Furthermore, the KG inference unit 120 normalizes the weighted sum after correction by coefficient k to a value between 0 and 1. For example, the KG inference unit 120 calculates "maximum similarity of similar entities in subKG + (1 - maximum similarity) × corrected weighted sum". The maximum similarity of similar entities in subKG66a is "0.94". The maximum similarity of similar entities in subKG66b is "0.94". The maximum similarity of similar entities in subKG66c is "0.90". Therefore, the normalized value of the weighted sum of subKG66a to 66c is as follows. ·subKG1:S1”=0.94+(1-0.94)×0.47≒0.968 ·subKG2:S2”=0.94+(1-0.94)×0.50825≒0.970 ·subKG3:S3”=0.90+(1-0.90)×0.441≒0.944 The normalized values ​​of subKG66a to 66c become the confidence scores for each of subKG66a to 66c. Specifically, the confidence score for subKG66a is "S1" = 0.968", the confidence score for subKG66b is "S2" = 0.970", and the confidence score for subKG66c is "S3" = 0.944".

[0133] In this case, if we rank the causes of the failures in descending order of their confidence scores, the order would be "B" (subKG2), "A" (subKG1), and "C" (subKG3). The KG inference unit 120 sorts the failure causes and estimated results for each subKG in descending order of confidence score and transmits them to the terminal device 30 as failure cause estimation results.

[0134] Figure 20 shows an example of the failure cause estimation results. In the example in Figure 20, subKG64b has the highest confidence score, subKG64a has the second highest, subKG64c has the third highest, and subKG64N has the fourth highest. In this case, the failure cause estimation result 70 shows the failure cause and estimation result for each failure in the following order: failure cause "B" for subKG64b, failure cause "A" for subKG64a, failure cause "G" for subKG64c, and failure cause "E" for subKG64N.

[0135] The fault cause estimation result 70 is sent to the terminal device 30 as a response to the input query received from the terminal device 30. The estimated cause of the failure 70 is displayed, for example, on the response display screen of the terminal device 30.

[0136] Figure 21 shows an example of the answer display screen. The answer display screen 71 includes a fault cause candidate display unit 71a and detailed information display units 71b, 71c, ... for each fault cause candidate.

[0137] The fault cause candidate display unit 71a displays an overview of the estimated fault cause candidates, sorted by their confidence score. Each estimated fault cause candidate overview includes information on items such as "Incident registered in DB," "Root Cause(s)," "Conditions," "Intermediate event," and "Confidence score," associated with an ID assigned according to the candidate's order. The "Incident registered in DB" item shows the details of the fault. The "Root Cause(s)" item shows the root cause of the fault. The "Conditions" item shows the state at the time of the fault as shown in logs, etc. The "Intermediate event" item shows information about the phenomena that occurred from the time the fault cause occurred until the fault occurred. The "Confidence score" item shows the confidence score of the fault cause and its estimated result.

[0138] Detailed information display sections 71b, 71c, ... show detailed results of the estimated candidate causes of the failure. The estimated causes of the failure are generated, for example, based on a subKG that includes the path to the entity causing the failure.

[0139] Figure 22 shows an example of the process for generating the estimated cause of a failure. For example, the KG inference unit 120 creates a response document 72 that indicates the estimated cause of the failure is entity "B" included in subKG63, based on subKG63.

[0140] The similar entity display section 72a of response text 72 shows, for example, that the similar entity to the failure shown in the input query is "J". Entity "J" is the entity that serves as the starting point for path 61 (root cause path) of subKG63. The display of similar entities allows the user to confirm that the failure case they intended has been correctly retrieved from causal KG40.

[0141] The fault cause display section 72b shows the root cause of the fault and the location of the reference document related to that cause (for example, the document name and its position within the document). The fault cause is entity "B" which corresponds to the top-level node 41b of path 61. By displaying the reference document in addition to the fault cause, the user can easily understand the content of the fault cause.

[0142] The reference information display unit 72c shows reference information useful for the user to confirm whether the presented candidate cause of failure is the true cause of the failure that needs to be resolved. The reference information consists of entities "G", "A", and "D" that are included in subKG63 but not in path 61. The display of reference information makes it easier for the user to confirm whether the candidate cause of failure is the true cause of the failure.

[0143] The related information display unit 72d shows related information that occurs from the time the cause of the failure occurs until the failure occurs. The related information consists of entities "E" and "H" in path 61 other than the starting entity and the top-level entity. By displaying the related information, the user can easily recognize the chain of events from the occurrence of the presented cause of the failure until the failure is triggered.

[0144] The answer text 72 is displayed, for example, in the detailed information display sections 71b, 71c, ... on the answer display screen 71. Figure 23 shows an example of the display content of the detailed information display section for potential fault causes. The detailed information display section 71b shows a response text indicating the estimated cause of the fault. In the example in Figure 23, the detailed information display section 71b includes confidence score information 73. The confidence score information 73 includes the confidence score of the displayed potential fault cause, as well as a link 73a to the basis for calculating the confidence score. When the user selects link 73a, the confidence score calculation basis screen is displayed in a separate tab or pop-up.

[0145] Figure 24 shows an example of the confidence score calculation basis screen. The confidence score calculation basis screen 74 shows the basis for calculating the confidence score. For example, the confidence score calculation basis screen 74 includes similar entities included in the subKG corresponding to the cause of the failure being displayed, and the similarity of those similar entities to the subqueries.

[0146] As described above, highly accurate KG inference (failure cause estimation) is achieved using KG, which indicates the causal relationship of failures, for the input query. Moreover, the number of estimated results (failure causes) returned as failure cause estimation results is reduced, and the answers are displayed in order of confidence score. This reduces the amount of time the user has to spend checking each estimated result.

[0147] [Other embodiments] The KG inference unit 120 can also use LLM to correct the confidence score during or after its calculation by comparing it with the contents of the documents and specifications used when generating the causal KG.

[0148] Figure 25 shows an example of a method for correcting the confidence score. For example, the KG inference unit 120 inputs a prompt 81 to the dialogue system 130 using LLM 111, instructing it to update the confidence score. The content of the prompt 81 is, for example, as follows. "After reading input query, its root cause candidates and their reference documents, update confidence scores of each root cause candidate between 0 and 1 if necessary." Subsequently, the KG inference unit 120 inputs the input query 82, the confidence score calculation result 83, and the reference document group 84 to the dialogue system 130. The content of the input query 82 is, for example, the sentence "The RU was not transmitting after configuring the carriers in 2T2R mode." The reference document group 84 includes, for example, the specifications used to generate the causal KG, and log data from system operation. The dialogue system 130 corrects the confidence score according to the instructions of prompt 81.

[0149] For example, a document showing the history of past failure resolutions may contain a resolved failure that matches the failure shown in input query 82, and the cause of that failure may be clearly indicated in the document. In that case, the confidence score correction using LLM111 will be adjusted to increase the confidence score of the subKG corresponding to the cause indicated in the document.

[0150] The dialogue system 130 corrects the confidence score of any of the subKGs shown in the confidence score calculation result 83 and returns the corrected confidence score calculation result 85 to the KG inference unit 120.

[0151] Figure 26 shows an example of the confidence score correction result. In the example in Figure 26, the confidence score of subKG with ID "3" is corrected by comparing the confidence score calculation result 83 with the contents of each document included in the reference document group 84 using LLM111. As a result, the corrected confidence score calculation result 69 is output.

[0152] By correcting the confidence score using LLM111 in this way, it is possible to generate a confidence score that more accurately reflects the causal relationship between cause and effect. In the second embodiment, the KG inference unit 120 selects a predetermined number of entities with high similarity as similar entities, but for example, entities with a similarity of a predetermined threshold or higher can also be selected as similar entities.

[0153] Furthermore, in the second embodiment, the KG inference unit 120 includes failure causes with a confidence score above a threshold in its response, but for example, it may include a predetermined number of failure causes in the response, starting with those with the highest confidence scores.

[0154] Although embodiments have been illustrated above, the configurations of each part shown in the embodiments can be replaced with others having similar functions. Furthermore, other arbitrary components or processes may be added. Moreover, any two or more configurations (features) from the embodiments described above may be combined. [Explanation of Symbols]

[0155] 1 sentence 2a First sentence 2b The second sentence 3. Knowledge Graph 3a-3c Sub-knowledge graph 4a~4d First node 5a~5c Second node 6. Weight Management Table 7. Confidence Determination Results 8. Possible causes 10 Information Processing Devices 11 Storage section 12 Processing Units

Claims

1. The system accepts input of a sentence describing the first phenomenon that is the subject of cause estimation. From the multiple nodes of the knowledge graph, which is composed of multiple nodes representing multiple phenomena and edges representing the causal relationships between the multiple phenomena, one or more first nodes representing a second phenomenon similar to the first phenomenon are selected. Starting from the first node, trace the causal relationship indicated by the edge to the terminal second node, and generate a subknowledge graph including the first node and the second node. Based on the similarity of the first node included in the subknowledge graph to the first phenomenon, the confidence that the third phenomenon shown in the second node included in the subknowledge graph is the cause of the first phenomenon is determined. Based on the confidence level of the subknowledge graph, a decision is made as to whether or not to include the third phenomenon shown in the second node included in the subknowledge graph as a candidate cause of the first phenomenon. A method for estimating the cause of a computer's execution of a process.

2. The computer further performs a process of splitting the sentence into a first sentence indicating a failure and a second sentence indicating a phenomenon that occurred as a result of the failure. In the process of selecting the first node, a node similar to the first statement and a node similar to the second statement are selected as the first node. The method for estimating the cause according to claim 1.

3. In the process of determining the confidence level, different weights are assigned to the first node that is similar to the first sentence and the first node that is similar to the second sentence, and the confidence level is determined based on the similarity of the first node and the weight of the first node. The method for estimating the cause according to claim 2.

4. In the process of determining the confidence level, a weight is set for the first node according to the order of similarity to the sentences from highest to lowest, and the confidence level is determined based on the similarity of the first node and the weight of the first node. The method for estimating the cause according to claim 1.

5. In the process of generating the subknowledge graph, the subknowledge graph is generated which includes nodes along the path taken from the first node, which is the starting point, to the second node, which is the ending point, and nodes that can be reached by following predetermined causal relationships from the nodes along the path. The method for estimating the cause according to claim 1.

6. The system outputs the candidate causes, which are the information indicating the causes that have been decided to be included in the candidate causes, arranged according to their reliability. The cause estimation method according to claim 1, wherein the computer further performs the processing.

7. Information regarding the phenomenon shown in each node included in the subknowledge graph, including the second node corresponding to the cause that has been decided to be included as a candidate cause, is output along with information indicating the cause. The cause estimation method according to claim 1, wherein the computer further performs the processing.

8. Using a trained language model, the confidence level of the sub-knowledge graph is corrected based on the information used to generate the knowledge graph. The cause estimation method according to claim 1, wherein the computer further performs the processing.

9. The system accepts input of a sentence describing the first phenomenon that is the subject of cause estimation. From the multiple nodes of the knowledge graph, which is composed of multiple nodes representing multiple phenomena and edges representing the causal relationships between the multiple phenomena, one or more first nodes representing a second phenomenon similar to the first phenomenon are selected. Starting from the first node, trace the causal relationship indicated by the edge to the terminal second node, and generate a subknowledge graph including the first node and the second node. Based on the similarity of the first node included in the subknowledge graph to the first phenomenon, the confidence that the third phenomenon shown in the second node included in the subknowledge graph is the cause of the first phenomenon is determined. Based on the confidence level of the subknowledge graph, a decision is made as to whether or not to include the third phenomenon shown in the second node included in the subknowledge graph as a candidate cause of the first phenomenon. A cause estimation program that causes a computer to perform a process.

10. A processing unit that receives input of a sentence describing a first phenomenon to be estimated as the cause, selects one or more first nodes describing a second phenomenon similar to the first phenomenon from the nodes of a knowledge graph composed of multiple nodes describing multiple phenomena and edges describing causal relationships between the multiple phenomena, traces from the first node to the second node at the end of the causal relationship indicated by the edges, generates a subknowledge graph including the first node and the second node, determines the confidence that the third phenomenon described by the second node in the subknowledge graph is the cause of the first phenomenon based on the similarity of the first node in the subknowledge graph to the first phenomenon, and determines whether or not to include the third phenomenon described by the second node in the subknowledge graph as a candidate cause of the first phenomenon based on the confidence of the subknowledge graph. An information processing device having

Images