Circuit Fault Identification Method and System Integrating Multimodal Information and Knowledge Graph
By constructing a circuit fault knowledge graph and parsing a multimodal large model, a joint feature representation of circuit faults is generated, which solves the problem of low efficiency in existing circuit fault diagnosis methods and realizes accurate fault identification and comprehensive solutions for complex circuit systems.
Patent Information
- Application Number
- CN202611578232.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-10-13
- Publication Date
- 2026-11-10
AI Technical Summary
Existing circuit fault diagnosis methods are inefficient and lack accuracy, especially in complex circuit systems or complex faults. They are unable to fully analyze the fault correlations and provide accurate comprehensive solutions.
A circuit fault identification method integrating multimodal information and knowledge graphs is proposed. By constructing a knowledge graph of circuit faults, combining multimodal large-scale model parsing of natural language descriptions and circuit images, joint feature representations are generated. Subgraph matching and path search are performed to generate a candidate fault set. The circuit fault identification result is determined by semantic verification and confidence calculation.
It enables comprehensive and accurate identification of circuit faults, breaks through the limitations of traditional methods, and ensures the rigor and accuracy of fault reasoning.
Smart Images

Figure CN123068656A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of circuit fault identification technology, and in particular to a circuit fault identification method and system that integrates multimodal information and knowledge graph. Background Technology
[0002] In the operation and maintenance of circuit systems, fault diagnosis is a crucial step in ensuring the stable and reliable operation of the system. Traditional circuit fault diagnosis methods often rely on manual experience, which is inefficient and difficult to guarantee accuracy, especially when dealing with complex circuit systems or compound faults, where diagnosis becomes extremely difficult.
[0003] With the development of artificial intelligence technology, some model- or data-driven fault diagnosis methods have emerged, but these methods still have many shortcomings. For example, while pure knowledge graph-based fault diagnosis methods possess structured reasoning capabilities, they lack flexibility and struggle to handle unstructured information such as images and natural language descriptions. Conversely, diagnostic methods relying solely on large models lack accuracy in their domain-specific knowledge and rigor in their reasoning, and are prone to reasoning biases when faced with specialized circuit terminology and complex fault rules. Furthermore, existing methods often fail to comprehensively analyze fault relationships when dealing with complex faults, making it difficult to provide accurate and comprehensive solutions. Summary of the Invention
[0004] The purpose of this invention is to overcome the shortcomings and defects of the prior art and to provide a circuit fault identification method and system that integrates multimodal information and knowledge graph.
[0005] According to a first aspect of this application, a circuit fault identification method integrating multimodal information and knowledge graph is provided, comprising:
[0006] S1. Transform the entities and relations in the circuit domain into triples of head entity-relation-tail entity, construct a knowledge graph of circuit faults based on the triples, and represent it in a quantitative manner; the knowledge graph includes at least a structural entity layer, a functional entity layer, and a fault knowledge layer;
[0007] S2. Employ a multimodal large model to perform multimodal analysis on the user-input natural language description of fault phenomena text and / or circuit images, forming a joint feature representation that integrates the feature vectors of fault phenomenon text and circuit images;
[0008] S3. Based on the joint feature representation, search for relevant triples from the knowledge graph; perform subgraph matching based on the relevant triples to determine the target subgraph; perform path search and association rule mining based on the target subgraph to generate candidate fault hypotheses and propagation paths, and obtain a candidate fault set;
[0009] S4. Perform semantic verification on the candidate fault hypotheses and calculate the comprehensive confidence level, determine the circuit fault identification result based on the comprehensive confidence level; retrieve fault-related knowledge from the knowledge graph based on the circuit fault identification result;
[0010] S5. After calling the large language model to generate a structured fault diagnosis report based on the circuit fault identification results and fault-related knowledge, output the report.
[0011] According to a second aspect of this application, a circuit fault identification system integrating multimodal information and knowledge graphs is provided, used to implement the circuit fault identification method integrating multimodal information and knowledge graphs described in the first aspect of this application, comprising:
[0012] The knowledge graph module includes a knowledge graph of circuit faults and its vectorized representation, which is constructed based on the triples of head entity-relationship-tail entity formed by the transformation of entities and relations in the circuit domain. The knowledge graph includes at least a structural entity layer, a functional entity layer and a fault knowledge layer.
[0013] The joint feature generation module is used to perform multimodal parsing on the user-input natural language description of fault phenomena text and / or circuit images using a multimodal large model, forming a joint feature representation that fuses the feature vectors of the fault phenomenon text and circuit images;
[0014] The fault identification and reasoning module is used to find relevant triples from the knowledge graph based on the joint feature representation; perform subgraph matching based on the relevant triples to determine the target subgraph; and perform path search and association rule mining based on the target subgraph to generate candidate fault hypotheses and propagation paths, thereby obtaining a candidate fault set.
[0015] The fault result determination module is used to perform semantic verification on the candidate fault hypotheses and calculate the comprehensive confidence score, determine the circuit fault identification result based on the comprehensive confidence score, and retrieve fault-related knowledge from the knowledge graph based on the circuit fault identification result.
[0016] The fault diagnosis report output module is used to call the large language model to generate a structured fault diagnosis report based on the circuit fault identification results and fault-related knowledge, and then output it.
[0017] This invention, by analyzing circuit images and / or fault phenomenon texts and combining them with multimodal large model processing of natural language descriptions, can comprehensively process multimodal unstructured information related to circuit faults, breaking the limitation of traditional methods that can only process structured data. Based on knowledge graphs, this invention provides a structured circuit knowledge system, ensuring the rigor and accuracy of fault reasoning. Attached Figure Description
[0018] Figure 1This is a flowchart of the circuit fault identification method that integrates multimodal information and knowledge graph according to the present invention.
[0019] Figure 2 This is a schematic diagram of the circuit fault identification system that integrates multimodal information and knowledge graph according to the present invention. Detailed Implementation
[0020] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
[0021] See Figure 1 As shown in the exemplary embodiment of this application, the circuit fault identification method integrating multimodal information and knowledge graph includes the following steps:
[0022] S1. Transform the entities and relations in the circuit domain into triples of head entity-relation-tail entity, construct a knowledge graph of circuit faults based on the triples, and represent it in a quantitative manner; the knowledge graph includes at least a structural entity layer, a functional entity layer, and a fault knowledge layer;
[0023] S2. Employ a multimodal large model to perform multimodal analysis on the user-input natural language description of fault phenomena text and / or circuit images, forming a joint feature representation that integrates the feature vectors of fault phenomenon text and circuit images;
[0024] S3. Based on the joint feature representation, search for relevant triples from the knowledge graph; perform subgraph matching based on the relevant triples to determine the target subgraph; perform path search and association rule mining based on the target subgraph to generate candidate fault hypotheses and propagation paths, and obtain a candidate fault set;
[0025] S4. Perform semantic verification on the candidate fault hypotheses and calculate the comprehensive confidence level, determine the circuit fault identification result based on the comprehensive confidence level; retrieve fault-related knowledge from the knowledge graph based on the circuit fault identification result;
[0026] S5. After calling the large language model to generate a structured fault diagnosis report based on the circuit fault identification results and fault-related knowledge, output the report.
[0027] Specifically, in step S1, entities and relationships in the circuit domain are automatically extracted from multi-source heterogeneous data, including circuit schematics, PCB design files, component datasheets, historical repair records, and industry standard documents, using rule matching, named entity recognition, and relationship extraction techniques. The physical structure, component attributes, fault rules, historical fault cases, and industry standards in the circuit diagram are transformed into triples of head entity-relationship-tail entity. After conflict resolution, consistency verification, and expert review of the triples of head entity-relationship-tail entity, they are vectorized and then stored and indexed using a graph database to build a knowledge graph of circuit faults that supports dynamic updates.
[0028] The entities extracted in the knowledge graph construction can include circuit elements, signal nodes, fault types, test points, and solution components, and the relationships include physical connection relationships, electrical attribute relationships, causal relationships, and solution relationships.
[0029] In one implementation, the vectorized representation of the knowledge graph includes:
[0030] Based on the knowledge graph, a knowledge representation learning algorithm is used to initialize the head entity, relation, and tail entity in the triples as low-dimensional vectors. The distances of positive sample triples are calculated, and negative sample triples are constructed and their distances are calculated. With the training objective of minimizing the positive sample distance and maximizing the negative sample distance, a loss function is constructed and iteratively optimized multiple times to obtain low-dimensional dense vectors for each entity and relation in the knowledge graph. The distance representation of the positive sample triples adopts the TransE model as shown below:
[0031] ;
[0032] The loss function is as follows:
[0033] ;
[0034] in, This represents the distance between positive sample triples. For the head entity, For the relationship, For tail entities, For the positive sample set, For the negative sample set, For the interval hyperparameter, , For the head and tail entities of the negative triple, This represents the L2 norm.
[0035] The knowledge graph includes at least a structural entity layer, a functional entity layer, and a fault knowledge layer.
[0036] The structural entity layer represents the physical composition of the circuit. Its entities include specific components, network nodes, and functional modules, and their relationships include electrical connections, physical inclusion, and signal flow.
[0037] The functional entity layer represents the logical function of the circuit. Its entities include power modules, amplification units, and filter circuits. Their relationships include function implementation, signal transformation, and performance parameter association.
[0038] The fault knowledge layer represents the domain knowledge of faults. Its entities include fault modes, failure phenomena, root causes, detection methods, and maintenance solutions. Its relationships include causal initiation, symptom correspondence, solution resolution, and condition triggering.
[0039] Specifically, in step S2, the circuit image includes a circuit image of the faulty device and an image of the actual faulty device; the circuit image of the faulty device includes a circuit schematic diagram and a PCB layout diagram of the faulty device, and the image of the actual faulty device includes a photograph or video of the actual faulty device.
[0040] The multimodal large model can be a visual large language model, which includes a multimodal information parsing unit. This multimodal information parsing unit includes a visual encoding parsing module and a large language model parsing module. In some preferred embodiments, the multimodal information parsing unit of the multimodal large model can be used to perform multimodal parsing on the user-input natural language description of the fault phenomenon text and / or circuit images, forming a joint feature representation that fuses the feature vectors of the fault phenomenon text and the circuit images. For example, for fault phenomenon texts described in natural language, the large language model parsing module of the multimodal large model performs intent recognition, extraction and structuring of key entity and state information to form the feature vector of the fault phenomenon text; for circuit images, the visual encoding parsing module and the large language model parsing module perform component recognition, network annotation, and topology understanding on the circuit images and convert them into structured netlist descriptions or text summaries as the feature vectors of the circuit images; for physical photos / videos, the visual encoding parsing module and the large language model parsing module perform image feature analysis to identify physical anomalies such as burnout, bulging, discoloration, and cold solder joints, compare them with standard states, generate anomaly description text as the feature vector of the circuit images, and fuse the feature vectors of the fault phenomenon text and the feature vectors of the circuit images to form the joint feature representation.
[0041] In some embodiments, the joint feature representation of this application is obtained by extracting feature vectors from the user-input fault phenomenon text and circuit image respectively, and then fusing them through a cross-modal attention mechanism:
[0042] ;
[0043] in, This represents the joint feature representation. and These are feature vectors extracted from fault phenomenon text and circuit images through encoding, respectively. Let be the attention function. , , These are the learnable weight matrix and the learnable projection matrix of the attention mechanism, respectively. , , These refer to the query, key, and value in the attention mechanism.
[0044] After obtaining the joint feature representation, the joint feature representation is associated with the knowledge graph.
[0045] Based on the joint feature representation, relevant triples are searched from the knowledge graph; subgraph matching is performed based on the relevant triples to determine the target subgraph; path search and association rule mining are performed based on the target subgraph to generate candidate fault hypotheses and propagation paths, resulting in a candidate fault set.
[0046] The method described in this application for searching relevant triples from the knowledge graph based on the joint feature representation is a retrieval process based on semantic similarity calculation, retrieving and fusing features from the knowledge graph. The K most relevant triples By calculating triples With joint feature representation The semantic similarity is determined, where the similarity scoring function is:
[0047] ;
[0048] in, For triples The vectorized representation of, This represents the similarity score.
[0049] In other implementations, when searching for relevant triples from the knowledge graph, the query vector can be based on key information entities (such as component numbers, fault phenomenon terms, and signal names). Multi-hop retrieval can be performed in the knowledge graph, and relevant triples can be determined by similarity. Subgraphs that are directly or indirectly related to the query entity can be recalled based on the relevant triples as target subgraphs for matching. The subgraphs contain triples of relevant components, historical cases, fault rules, and solutions.
[0050] In some implementations, the retrieved relevant subgraph information can be combined with the structured parsing information obtained after parsing the user's original input (natural language description of the fault phenomenon text and fault image) in the form of structured text as "contextual knowledge" to form prompt words, which are then input into the large model. The large model then performs fusion reasoning based on the injected circuit domain-related knowledge and general logic capabilities to generate a candidate fault set, which may include fault hypotheses and corresponding propagation paths.
[0051] In other implementations, path search and association rule mining are performed based on the target subgraph to generate candidate failure hypotheses and propagation paths, including:
[0052] The target subgraph is inferred based on a graph neural network. The node feature representation of the target subgraph is updated by the graph neural network, and the failure probability of the graph node is output by the classification layer. Nodes with failure probabilities greater than a preset threshold are selected as candidate failure nodes and included in the candidate failure node set. The candidate failure hypothesis and its propagation path are generated by combining the path search results in the target subgraph with predefined association rules, thereby obtaining the candidate failure set.
[0053] Specifically, regarding the graph structure of the target subgraph of a knowledge graph. ,in, For a set of nodes, Given the set of edges, when using the graph neural network to reason about the target subgraph, the first edge of the graph structure... Layer nodes Features The update method is as follows:
[0054] ;
[0055] in, Represents a node The neighborhood group, It is an aggregate function. For trainable weight matrix, For activation function, For the first Layer nodes neighboring nodes eigenvectors, The first graph neural network layer;
[0056] The node The failure probability is calculated through a classification layer:
[0057] ;
[0058] in, Indicates the probability of failure. , The weight and bias parameters for the classification layer. It is a node The label variable, For nodes Marked as faulty state For normalized exponential functions, For nodes The final feature vector, The total number of floors indicates that the probability of failure exceeds a preset threshold. The node was included in the candidate fault node set. .
[0059] In the process of rule-based reasoning, it can be based on the predefined fault propagation rules in the knowledge graph. Starting with the corresponding entities and fault hypotheses, logical chain deduction is performed to obtain the propagation path of candidate fault hypotheses, determine the fault causes and corresponding solutions, thereby realizing fault cause analysis and solution provision.
[0060] Furthermore, in this application, the graph neural network can be trained using labeled historical fault cases. During relevant inference based on the trained graph neural network, it can also classify nodes or predict links in the current fault subgraph (target subgraph) according to node characteristics, outputting a candidate fault set and corresponding confidence levels, and generating a visualized fault propagation path diagram. The candidate fault set includes candidate fault hypotheses, and the candidate fault hypotheses contain candidate fault nodes.
[0061] In this application, step S4, which involves performing semantic verification on the candidate fault hypotheses and calculating the comprehensive confidence score, and determining the circuit fault identification result based on the comprehensive confidence score, includes calculating the comprehensive confidence score of the candidate fault hypotheses based on the different evidence strengths of the candidate fault hypotheses after passing the semantic verification and the fault probability inferred by the graph neural network, ranking the candidate fault hypotheses according to the comprehensive confidence score, and selecting the candidate fault hypotheses with the highest comprehensive confidence score as the circuit fault identification result; the structured fault diagnosis report includes fault hypothesis, fault location, fault root cause chain analysis, fault solution, and fault prevention recommendations.
[0062] Specifically, the overall confidence score is calculated using the following formula:
[0063] ;
[0064] in, This represents the overall confidence score. For the first Multimodal analytical evidence strength of candidate failure hypotheses For the first Strength of evidence for rule-based reasoning of candidate failure hypotheses For the first A graph neural network infers the failure probability from 10 candidate failure hypotheses. For activation function, , , The weighting coefficients are adjustable and satisfy the following conditions: , It is a pre-defined, extremely small positive number.
[0065] Once the circuit fault identification result is determined, the final result is output, which may include fault location, fault cause analysis, and fault solution.
[0066] It should be noted that the semantic verification described in this application involves combining knowledge of circuit principles and descriptions of historical cases to perform semantic analysis on candidate fault hypotheses, eliminating candidate fault hypotheses with low probability or contradictions, and then calculating the comprehensive confidence level.
[0067] This application comprehensively evaluates the evidence strength from multimodal parsing information, knowledge graph rule reasoning, and graph neural network prediction for each semantically verified candidate fault hypothesis, performs weighted ranking, obtains a comprehensive confidence score, generates a list of diagnostic conclusions arranged in descending order of probability, and determines the candidate fault hypothesis with the highest ranking (i.e., the highest score) as the fault identification result output based on the confidence score.
[0068] Furthermore, in this application, for the candidate fault hypothesis with the highest final score, fault-related knowledge corresponding to the candidate fault is retrieved from the knowledge graph, such as the standard maintenance process, replacement component list, precautions, etc., and the natural language generation capability of the large language model is invoked to integrate fault location, root cause chain analysis, specific solutions, and preventive suggestions to form a structured fault diagnosis report, which is output to the user through an interactive interface. The large language model can be the multimodal large model or any other available large language model with natural language generation capability.
[0069] According to embodiments of this application, a circuit fault identification system integrating multimodal information and knowledge graphs is further provided, used to implement the circuit fault identification method integrating multimodal information and knowledge graphs described in the first aspect of this application, including:
[0070] The knowledge graph module includes a knowledge graph of circuit faults and its vectorized representation, which is constructed based on the triples of head entity-relationship-tail entity formed by the transformation of entities and relations in the circuit domain. The knowledge graph includes at least a structural entity layer, a functional entity layer and a fault knowledge layer.
[0071] The joint feature generation module is used to perform multimodal parsing on the user-input natural language description of fault phenomena text and / or circuit images using a multimodal large model, forming a joint feature representation that fuses the feature vectors of the fault phenomenon text and circuit images;
[0072] The fault identification and reasoning module is used to find relevant triples from the knowledge graph based on the joint feature representation; perform subgraph matching based on the relevant triples to determine the target subgraph; and perform path search and association rule mining based on the target subgraph to generate candidate fault hypotheses and propagation paths, thereby obtaining a candidate fault set.
[0073] The fault result determination module is used to perform semantic verification on the candidate fault hypotheses and calculate the comprehensive confidence score, determine the circuit fault identification result based on the comprehensive confidence score, and retrieve fault-related knowledge from the knowledge graph based on the circuit fault identification result.
[0074] The fault diagnosis report output module is used to call the large language model to generate a structured fault diagnosis report based on the circuit fault identification results and fault-related knowledge, and then output it.
[0075] The knowledge graph module of this application can be implemented by a graph database and a vector index component. The circuit fault identification system of this application that integrates multimodal information and knowledge graph has each module corresponding to a step of the circuit fault identification method of this application that integrates multimodal information and knowledge graph. The knowledge graph module is constructed through step S1 of the circuit fault identification method of this application that integrates multimodal information and knowledge graph. For details regarding the technology of the circuit fault identification system of this application that integrates multimodal information and knowledge graph, please refer to the relevant description of the circuit fault identification method of this application that integrates multimodal information and knowledge graph.
[0076] This invention, by analyzing circuit images and / or fault phenomenon texts and combining them with multimodal large model processing of natural language descriptions, can comprehensively process multimodal unstructured information related to circuit faults, breaking the limitation of traditional methods that can only process structured data. Based on knowledge graphs, this invention provides a structured circuit knowledge system, ensuring the rigor and accuracy of fault reasoning.
[0077] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. It will be apparent to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or basic features of the present invention.
[0078] Therefore, the embodiments should be regarded as exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, it is intended that all variations falling within the meaning and scope of the equivalents of the claims be included within the invention.
[0079] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.
Claims
1. A circuit fault identification method integrating multimodal information and knowledge graph, characterized in that, include: S1. Transform the entities and relations in the circuit domain into triples of head entity-relation-tail entity, construct a knowledge graph of circuit faults based on the triples, and represent it in a quantitative manner; the knowledge graph includes at least a structural entity layer, a functional entity layer, and a fault knowledge layer; S2. Employ a multimodal large model to perform multimodal analysis on the user-input natural language description of fault phenomena text and / or circuit images, forming a joint feature representation that integrates the feature vectors of fault phenomenon text and circuit images; S3. Based on the joint feature representation, search for relevant triples from the knowledge graph; perform subgraph matching based on the relevant triples to determine the target subgraph; perform path search and association rule mining based on the target subgraph to generate candidate fault hypotheses and propagation paths, and obtain a candidate fault set; S4. Perform semantic verification on the candidate fault hypotheses and calculate the comprehensive confidence level, and determine the circuit fault identification result based on the comprehensive confidence level; Based on the circuit fault identification results, fault-related knowledge is retrieved from the knowledge graph; S5. After calling the large language model to generate a structured fault diagnosis report based on the circuit fault identification results and fault-related knowledge, output the report.
2. The circuit fault identification method integrating multimodal information and knowledge graph according to claim 1, characterized in that, The structural entity layer represents the physical composition of the circuit, including structural entities and the relationships between them; the functional entity layer represents the logical function of the circuit, including functional entities and the relationships between them; the fault knowledge layer represents the domain knowledge of the fault, including fault entities and the relationships between them.
3. The circuit fault identification method integrating multimodal information and knowledge graph according to claim 1, characterized in that, The circuit images include circuit images of the faulty device and physical images of the faulty device; the circuit images of the faulty device include circuit schematics and PCB layout diagrams of the faulty device, and the physical images of the faulty device include photos or videos of the physical device.
4. The circuit fault identification method integrating multimodal information and knowledge graph according to claim 1, characterized in that, The joint feature representation is obtained by extracting feature vectors from the user-input fault phenomenon text and circuit image respectively, and then fusing them through a cross-modal attention mechanism: ; in, This represents the joint feature representation. and These are feature vectors extracted from fault phenomenon text and circuit images through encoding, respectively. Let be the attention function. , , These are the learnable weight matrices.
5. The circuit fault identification method integrating multimodal information and knowledge graph according to claim 1, characterized in that, The vectorized representation of the knowledge graph includes: Based on the knowledge graph, a knowledge representation learning algorithm is used to initialize the head entity, relation, and tail entity in the triples as low-dimensional vectors. The distances of positive sample triples are calculated, and negative sample triples are constructed and their distances are calculated. With the training objective of minimizing the positive sample distance and maximizing the negative sample distance, a loss function is constructed and iteratively optimized multiple times to obtain low-dimensional dense vectors for each entity and relation in the knowledge graph. The distances of the positive sample triples are represented as follows: ; The loss function is as follows: ; in, This represents the distance between positive sample triples. For the head entity, For the relationship, For tail entities, For the positive sample set, For the negative sample set, For the interval hyperparameter, For the head and tail entities of the negative triple, This represents the L2 norm.
6. The circuit fault identification method integrating multimodal information and knowledge graph according to claim 1, characterized in that, The step of performing path search and association rule mining based on the target subgraph to generate candidate fault hypotheses and propagation paths includes: The target subgraph is inferred based on a graph neural network. The node feature representation of the target subgraph is updated by the graph neural network, and the failure probability of the graph node is output by the classification layer. Nodes with failure probabilities greater than a preset threshold are selected as candidate failure nodes and included in the candidate failure node set. The candidate failure hypothesis and its propagation path are generated by combining the path search results in the target subgraph with predefined association rules, thereby obtaining the candidate failure set.
7. The circuit fault identification method integrating multimodal information and knowledge graph according to claim 6, characterized in that, When the graph neural network infers the target subgraph, the first... Layer nodes Features The update method is as follows: ; in, Represents a node The neighborhood group, It is an aggregate function. For trainable weight matrix, For activation function, For the first Layer nodes neighboring nodes eigenvectors, The first graph neural network layer; The node The failure probability is calculated through a classification layer: ; in, Indicates the probability of failure. , The weight and bias parameters for the classification layer. It is a node The label variable, For nodes Marked as faulty state For normalized exponential functions, For nodes The final feature vector, The total number of layers; nodes with a failure probability exceeding a preset threshold are included in the candidate failure node set. , This indicates a preset threshold.
8. The circuit fault identification method integrating multimodal information and knowledge graph according to claim 6, characterized in that, The process of performing semantic verification on the candidate fault hypotheses and calculating the comprehensive confidence score, and determining the circuit fault identification result based on the comprehensive confidence score, includes calculating the comprehensive confidence score of the candidate fault hypotheses based on the different evidence strengths of the candidate fault hypotheses after passing the semantic verification and the fault probability inferred by the graph neural network, ranking the candidate fault hypotheses according to the comprehensive confidence score, and selecting the candidate fault hypotheses with the highest comprehensive confidence score as the circuit fault identification result. The structured fault diagnosis report includes fault assumptions, fault location, fault root cause chain analysis, fault solutions, and fault prevention recommendations.
9. The circuit fault identification method integrating multimodal information and knowledge graph according to claim 8, characterized in that, The overall confidence score is calculated using the following formula: ; in, This represents the overall confidence score. For the first Multimodal analytical evidence strength of candidate failure hypotheses For the first Strength of evidence for rule-based reasoning of candidate failure hypotheses For the first A graph neural network infers the failure probability from 10 candidate failure hypotheses. For activation function, , , The weighting coefficients are adjustable and satisfy the following conditions: , It is a pre-defined, extremely small positive number.
10. A circuit fault identification system integrating multimodal information and knowledge graph, characterized in that, The circuit fault identification method for fusing multimodal information and knowledge graphs as described in any one of claims 1-9 includes: The knowledge graph module includes a knowledge graph of circuit faults and its vectorized representation, which is constructed based on the triples of head entity-relationship-tail entity formed by the transformation of entities and relations in the circuit domain. The knowledge graph includes at least a structural entity layer, a functional entity layer and a fault knowledge layer. The joint feature generation module is used to perform multimodal parsing on the user-input natural language description of fault phenomena text and / or circuit images using a multimodal large model, forming a joint feature representation that fuses the feature vectors of the fault phenomenon text and circuit images; The fault identification and reasoning module is used to find relevant triples from the knowledge graph based on the joint feature representation; perform subgraph matching based on the relevant triples to determine the target subgraph; and perform path search and association rule mining based on the target subgraph to generate candidate fault hypotheses and propagation paths, thereby obtaining a candidate fault set. The fault result determination module is used to perform semantic verification on the candidate fault hypotheses and calculate the comprehensive confidence score, determine the circuit fault identification result based on the comprehensive confidence score, and retrieve fault-related knowledge from the knowledge graph based on the circuit fault identification result. The fault diagnosis report output module is used to call the large language model to generate a structured fault diagnosis report based on the circuit fault identification results and fault-related knowledge, and then output it.
