Knowledge question and answer dynamic graph construction method, knowledge question and answer method, storage medium, program product and electronic equipment

Abstract

The invention provides a knowledge question and answer dynamic graph construction method, a knowledge question and answer method, a storage medium, a program product and electronic equipment, the knowledge question and answer dynamic graph construction method comprises the following steps: collecting multi-modal data at least comprising time sequence image data to capture text semantics and locate a position; then semantic alignment is carried out through a self-adaptive gating fusion network of a multi-layer perceptron, and a unified text vector is generated; performing cross fusion on entities, relationships and global features split from the unified text vector and visual features of image projection by using an improved multi-modal partition fusion network to obtain multi-modal deep features; and finally, deep features are enhanced and then input into a decoder, and a multi-element set with a timestamp is generated and merged into a graph database to construct a knowledge question and answer dynamic graph. Through multi-source data fusion and PFKAN knowledge graph construction, semantic retrieval and reasoning robustness is improved, and dynamic adaptation and efficient decision in a multi-role scene are realized.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence and natural language processing technology, and in particular relates to a method for constructing a dynamic knowledge question-answering graph, a knowledge question-answering method, a storage medium, a program product, and an electronic device. Background Technology

[0002] In the operation and maintenance of modern power systems, substation operation logs, relay protection setting sheets, manufacturer technical manuals, PMS equipment ledgers, SCADA timing data, and infrared thermal imaging images from UAV inspections are core information sources for ensuring the safe and stable operation of the power grid. This information covers key aspects such as equipment condition monitoring, emergency fault handling, and maintenance plan development, and has irreplaceable value for decision-making by dispatchers, operation and maintenance specialists, and on-site maintenance workers.

[0003] The existing methods for obtaining power operation and maintenance information mainly rely on manual cross-system queries. On the one hand, unstructured texts such as regulations and drawings are stored in different systems, while structured or visual data such as SCADA time-series data and infrared thermal images are stored in different systems, forming serious data silos. On the other hand, power equipment fault diagnosis usually requires complex causal reasoning by combining "phenomenon-principle-regulation", but traditional keyword matching-based retrieval systems lack logical reasoning capabilities and are unable to cope with complex fault tracing problems.

[0004] Although large language models have performed well in the general question-answering field in recent years, they have significant shortcomings when directly applied to power operation and maintenance scenarios: First, they lack the ability to perceive visual modalities such as infrared images and cannot use temperature field features to assist in diagnosis; second, they lack rigorous domain logic constraints, which can easily lead to "illusions" and give incorrect suggestions that do not comply with safety procedures, posing a great safety hazard; and third, they lack a feedback loop of expert knowledge, making it difficult for the system to continuously evolve in practical applications. Summary of the Invention

[0005] In view of the shortcomings of the prior art described above, the purpose of this application is to provide a method for constructing a dynamic knowledge question-answering graph, a knowledge question-answering method, a storage medium, a program product, and an electronic device, to solve the problems of lack of multimodal perception, imprecise domain logic constraints, and security risks in the application of large language models in the prior art.

[0006] Firstly, this application provides a method for constructing a dynamic knowledge question-answering graph, the method comprising:

[0007] Collect multimodal data, wherein the multimodal data includes at least image data with temporal attributes;

[0008] Based on the multimodal data, at least encoding mapping, semantic alignment, feature fusion, and enhanced decoding processes are performed to construct a dynamic knowledge-question-answering graph, wherein...

[0009] During encoding mapping, high-dimensional content vectors are obtained by encoding multimodal data. These high-dimensional content vectors capture the semantic features of the text and perform positional mapping on the text content; and

[0010] When performing semantic alignment, an adaptive gated fusion network based on a multilayer perceptron generates a unified text vector; and

[0011] During feature fusion, an improved multimodal partitioning fusion network is used to cross-fuse custom combined features to obtain multimodal deep features. These combined features include at least entity features, relation features, and globally shared features obtained from the unified text vector splitting, as well as visual features obtained from the image data projection.

[0012] During enhanced decoding, the multimodal deep features are enhanced, and the enhanced features are input into a preset decoder to incorporate the generated set of tuples into the graph database to obtain the knowledge question-answering dynamic graph. The set of tuples includes timestamps associated with temporal attributes.

[0013] In some embodiments of the first aspect of this application, during encoding mapping, a high-dimensional content vector is obtained by encoding based on multimodal data. This high-dimensional content vector captures text semantic features and performs positional mapping on the text content, specifically including:

[0014] Based on the multimodal data, a procedure hierarchy structure tree is constructed to obtain a procedure hierarchy index sequence, wherein the procedure hierarchy index sequence includes at least a chapter path sequence;

[0015] An adaptive sliding window is used to segment long texts in the procedure hierarchy structure tree into procedure content blocks, and each procedure content block is encoded to obtain a high-dimensional content vector.

[0016] The chapter path sequence is mapped to a path vector using a sine / cosine position encoding function, calculated as follows:

[0017] ;

[0018] ;

[0019] in, For position encoding functions, This represents the depth position of the current text content within the procedure hierarchy structure tree. For vector dimensions, For an even-numbered dimension in a vector, For odd-numbered dimensions in a vector, This is the index of the vector dimension.

[0020] In some embodiments of the first aspect of this application, a unified text vector is generated based on an adaptive gating fusion network using a multilayer perceptron during semantic alignment, specifically including:

[0021] The high-dimensional content vector and the path vector are concatenated, and the gating coefficient is calculated through a fully connected layer, as shown in the following formula:

[0022] ;

[0023] in, The gating coefficient, For the Sigmoid activation function, For the gated weight matrix, It is a high-dimensional content vector. This represents a vector concatenation operation. For path vectors, This is a gated bias term;

[0024] The high-dimensional content vector and the path vector are weighted and fused based on the gating coefficient to generate the unified text vector, and the calculation formula is as follows:

[0025] ;

[0026] in, To unify text vectors, The gating coefficient, This represents element-wise multiplication. It is a high-dimensional content vector. This is the path vector.

[0027] In some embodiments of the first aspect of this application, when performing feature fusion, an improved multimodal partitioning fusion network is used to cross-fuse custom combined features to obtain multimodal deep features, specifically including:

[0028] The unified text vector is semantically structured to obtain entity feature region, relation feature region, and globally shared context region, calculated as follows:

[0029] ;

[0030] ;

[0031] ;

[0032] in, For entity feature regions, It is the ReLU activation function. This is the entity feature region weight matrix. To unify text vectors, For entity feature region bias term, For relational feature regions, This is the weight matrix of the relation feature region. For the relation feature region bias term, For globally shared context area, For the globally shared context region weight matrix, For globally shared context area bias items;

[0033] The image data includes infrared images. The infrared image features are projected into visually aligned feature regions in the same dimension as the unified text vector, calculated as follows:

[0034] ;

[0035] in, For visual feature alignment area, For normalization layer, This is the weight matrix for visually aligned feature regions. Infrared image features, For visual alignment feature region bias term;

[0036] The combined feature is obtained by concatenating the entity feature region, the relation feature region, the global shared context region, and the visual alignment feature region, and the calculation formula is as follows:

[0037] ;

[0038] in, As a combination feature, This indicates a feature concatenation operation. This is the entity feature region weight matrix. For entity feature regions, This is the weight matrix of the relation feature region. For relational feature regions, For the globally shared context region weight matrix, For globally shared context area, This is the weight matrix for visually aligned feature regions. For visual feature alignment area;

[0039] An improved multimodal partitioning fusion network is used to cross-fuse the combined features to obtain multimodal deep features. Specifically, each feature component of the combined features is independently calculated nonlinearly using an element-level expansion method before aggregation. The calculation formula is as follows:

[0040] ;

[0041] ;

[0042] in, For multimodal deep features, For multimodal deep features The total number of dimensions; For multimodal deep features The Each component is output; Combination features of input The Each scalar feature component; For scalar feature components of B-order spline basis functions; In order to be with the first The scalar characteristic component and the first Trainable control coefficients corresponding to the first-order basis functions; Let be the total order of the B-spline basis functions.

[0043] In some embodiments of the first aspect of this application, the multimodal deep features are enhanced during enhanced decoding, and the enhanced features are input into a preset decoder to incorporate the generated tuple set into the graph database to obtain the knowledge question-answering dynamic graph, specifically including:

[0044] The feature weights are adjusted based on the channel convolutional attention module, and the calculation formula is as follows:

[0045] ;

[0046] ;

[0047] in, For multimodal deep features, This refers to the multimodal deep features after the feature weights have been adjusted by the channel convolutional attention module. This represents element-wise multiplication. For the Sigmoid activation function, For multilayer perceptrons with shared parameters, This represents the average value of the aggregated global semantics. Represents the extreme value of aggregated global semantics;

[0048] The spatial convolutional attention module focuses on key information regions, and the calculation formula is as follows:

[0049] ;

[0050] ;

[0051] in, These are the enhanced features corresponding to the key information regions after being focused by the spatial convolutional attention module. This refers to the multimodal deep features after the feature weights have been adjusted by the channel convolutional attention module. This represents element-wise multiplication. For the Sigmoid activation function, To adopt Two-dimensional convolutional layers with large kernels, This represents the average value of the aggregated global semantics. This represents a vector concatenation operation. Represents the extreme value of aggregated global semantics;

[0052] The enhanced features are input into a preset decoder to incorporate the generated tuple set into the graph database to obtain the knowledge question-answering dynamic graph, wherein the tuple set expression is as follows:

[0053] ;

[0054] in, For a set of tuples, For the head entity, For header entity type, This refers to the relationship between the head and tail entities. For tail entities, For tail entity type, This indicates the current operating or defective status of the equipment. For timestamps.

[0055] To achieve the above and other related objectives, a second aspect of this application provides a knowledge question-answering method applied to a knowledge question-answering dynamic graph constructed by any of the aforementioned knowledge question-answering dynamic graph construction methods, wherein the method includes:

[0056] Receive natural language questions input from the user and identify the user's query intent;

[0057] Based on the user's query intent, the constructed knowledge question-answering dynamic graph is used to process the query to obtain a set of candidate answers. The knowledge question-answering dynamic graph, during processing, includes at least constraint reasoning based on fault trees.

[0058] The candidate answer set is input into a preset large language model to obtain the question-and-answer results, and the question-and-answer results are then visualized and displayed to the user.

[0059] In some embodiments of the first aspect of this application, the constructed knowledge question-answering dynamic graph is used to process the user's query intent to obtain a set of candidate answers, specifically including:

[0060] The user query intent is obtained using a confidence-based weighted voting strategy, calculated as follows:

[0061] ;

[0062] in, For the user's query intent, This is a function for finding the maximum value. The discrete values ​​output by rule matching. To supervise the classification confidence of the learning model output, The semantic probabilities output by the large language model. The speech weights corresponding to different judgment paths; and

[0063] Constraint reasoning is performed based on fault tree, wherein the user's query intent is extracted as the root node and combined with the fault tree to perform reverse reasoning to obtain a set of fault causes.

[0064] Based on the set of fault causes, multi-hop reasoning is performed in the knowledge question-answering dynamic graph to generate structured reasoning results, and user questions are encoded into vectors for vector retrieval to identify matching procedure content blocks. The candidate answer set is obtained by combining the fusion weights, and the calculation formula is as follows:

[0065] ;

[0066] in, For the candidate answer set, To integrate weights, For the reasoning result, To match the procedure content block, Represent and manipulate.

[0067] To achieve the above and other related objectives, a third aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the knowledge question-answering dynamic graph construction method described in any of the preceding claims.

[0068] To achieve the above and other related objectives, a fourth aspect of this application provides a computer program product, which includes computer program code that, when executed on a computer, enables the computer to implement the knowledge question-answering dynamic graph construction method described in any of the above claims.

[0069] To achieve the above and other related objectives, a fifth aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory; the processor executes the computer program to implement the knowledge question-answering dynamic graph construction method described in any of the preceding claims.

[0070] As described above, the knowledge question-answering dynamic graph construction method, knowledge question-answering method, storage medium, program product, and electronic device described in this application achieve high-precision, traceable, and personalized intelligent question answering for power equipment operation and maintenance knowledge management. Through multi-source data fusion and PFKAN knowledge graph construction, the robustness of semantic retrieval and reasoning is improved. Combined with hybrid retrieval and tracing mechanisms, the accuracy and reliability of answers are ensured. With preference learning, dynamic adaptation and efficient decision-making in multi-role scenarios are achieved. Specific effects are as follows:

[0071] 1. Comprehensive knowledge integration: By integrating multi-source heterogeneous data such as power grid operation and maintenance site logs, drawings, and material lists from substations, and by using OCR technology to digitize paper documents, a unified and complementary semantic data foundation has been built, eliminating information silos.

[0072] 2. High retrieval accuracy: It adopts document hierarchical parsing and vectorization fusion technology to preserve the chapter hierarchy, which significantly improves the accuracy of semantic retrieval across chapters and documents;

[0073] 3. Strong reasoning ability: It utilizes the PFKAN network and attention mechanism to construct a knowledge graph, which enhances the collaborative modeling ability of entity relationships and demonstrates stronger robustness in multi-hop reasoning and complex semantic association queries.

[0074] 4. The questions and answers are accurate and traceable. By combining intent parsing, hybrid retrieval (knowledge graph multi-hop reasoning + RAG) and answer generation, and attaching source information, the answers are ensured to be accurate, verifiable and traceable.

[0075] 5. Personalized adaptation: Through user role modeling and incremental preference learning, the search ranking and answer presentation are dynamically optimized to achieve personalized intelligent interaction in multi-role scenarios;

[0076] 6. The system boasts superior performance, employing a layered architecture design with modules working collaboratively to ensure rapid response and stable operation under high concurrency environments, effectively improving information acquisition efficiency and management decision-making capabilities at power grid operation and maintenance sites and substation sites. Attached Figure Description

[0077] Figure 1 The diagram shows the steps of the knowledge question-answering dynamic graph construction method of this application in one embodiment;

[0078] Figure 2The diagram shows a scenario flow diagram of the knowledge question-answering dynamic graph construction method of this application in one embodiment;

[0079] Figure 3 The diagram shows a data processing flow of a knowledge question-answering dynamic graph construction method in one embodiment of this application.

[0080] Figure 4 The diagram shows a multimodal deep feature acquisition process in one embodiment of the knowledge question answering dynamic graph construction method of this application.

[0081] Figure 5 The diagram shows a schematic flow of constructing a knowledge question-answering dynamic graph in one embodiment of the knowledge graph construction method of this application.

[0082] Figure 6 The diagram shows the steps of the knowledge-based question-answering method of this application in one embodiment.

[0083] Figure 7 The diagram shown is a structural schematic of the electronic device of this application in one embodiment.

[0084] Component designation explanation S102~S112 step S602~S606 step 700 Electronic terminal 701 processor 702 memory 7021 operating system 7022 app 703 Network interface 704 bus system 705 User Interface Detailed Implementation

[0085] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0086] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. Therefore, the drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0087] Furthermore, the use of terms such as "first" and "second" in this application is for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of the various embodiments can be combined with each other, but only on the basis of being achievable by those skilled in the art. If the combination of technical solutions is contradictory or impossible to implement, such a combination of technical solutions should be considered non-existent and not within the scope of protection claimed in this application.

[0088] With the in-depth development of large-scale model technology in the question-answering field, a number of key technologies for knowledge question answering using large language models have emerged in the power industry. However, the existing problems mentioned in the background technology still exist. Therefore, in view of the technical challenges of traditional methods and the limitations of large language models in the power industry, this application proposes a dynamic knowledge graph construction method, a knowledge question answering method, a storage medium, a program product, and electronic equipment applicable to knowledge question answering in the power industry. By constructing a panoramic knowledge graph of power equipment covering multi-source data such as operation logs, technical specifications, and infrared thermal images, it innovatively introduces a multimodal partitioned fusion Kolmogorov-Arnold network to achieve deep collaborative modeling of text and visual features. In the question-answering reasoning stage, it combines the built-in fault tree prior knowledge to perform logical pruning on the retrieval space, realizing constrained multi-hop reasoning guided by "fault chain" and accurate matching with the original text. At the same time, it introduces a human-machine collaborative feedback mechanism, allowing experts to correct the diagnostic results and update the graph weights in reverse. Finally, it provides a high-precision, safety-compliant, and self-evolving power operation and maintenance knowledge question answering service, significantly improving fault handling efficiency and power grid safety management level. The technical solutions in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0089] like Figure 1 As shown, in one embodiment of the invention, the knowledge question-answering dynamic graph construction method of this application includes the following steps:

[0090] Step S102: Collect multimodal data, wherein the multimodal data includes at least image data with temporal attributes;

[0091] Step S104: Based on the multimodal data, at least encoding mapping, semantic alignment, feature fusion, and enhanced decoding processing are performed to construct a dynamic knowledge question-answering graph;

[0092] In step S106, during the encoding mapping, a high-dimensional content vector is obtained by encoding based on multimodal data. The high-dimensional content vector captures the semantic features of the text and performs position mapping on the text content.

[0093] Step S108: When performing semantic alignment, an adaptive gated fusion network based on a multilayer perceptron generates a unified text vector.

[0094] In step S110, when performing feature fusion, the improved multimodal partitioning fusion network is used to cross-fuse the custom combined features to obtain multimodal deep features;

[0095] In step S112, the multimodal deep features are enhanced during the enhanced decoding process, and the enhanced features are input into a preset decoder to incorporate the generated tuple set into the graph database to obtain the knowledge question-answering dynamic graph.

[0096] It should be noted that, in this embodiment, as Figure 2 The diagram shows a scenario application flowchart of the knowledge question-answering dynamic graph construction method of this application. Specifically, firstly, multimodal data is collected. The multimodal data includes at least image data with time-series attributes. For example, original operation and maintenance information is obtained from multi-source operation and maintenance datasets such as substation operation logs, relay protection setting sheets, manufacturer technical manuals, PMS equipment ledgers, SCADA time-series data, and UAV inspection infrared thermal imaging images. Then, optical character recognition processing is performed on the scanned parts, and temperature rise area feature extraction is performed on the infrared images to obtain the corresponding multimodal data.

[0097] Specifically, in this embodiment, the substation operation log records time-series events such as equipment switching operations, inspection records, and historical anomalies; the relay protection setting sheet and manufacturer's technical manual provide the boundary parameters and standard specifications for equipment operation; the PMS equipment ledger establishes the equipment's asset ID and model attributes; SCADA time-series data provides real-time telemetry information such as voltage, current, and oil temperature; and the UAV infrared thermal imaging provides an intuitive thermodynamic representation of the equipment's operating status. After uniform formatting, noise removal, and multimodal alignment, all data forms a high-quality dataset that can be directly entered into the knowledge processing layer, providing a multi-dimensional information foundation of text, numerical values, and visual data for subsequent models. Among them, the multi-source operation and maintenance dataset refers to a unified data set that combines unstructured procedures, semi-structured ledgers, structured time-series data, and unstructured images to comprehensively support the fault diagnosis needs in power operation and maintenance scenarios. In order to support the subsequent construction of a time-sensitive power panoramic knowledge graph, this step specifically strengthens the collection and alignment of time-stamped multi-source time-series data.

[0098] Specifically, in this embodiment, SCADA time-series data provides real-time telemetry information such as voltage, current, and oil temperature with precise timestamps from milliseconds to seconds; UAV infrared thermal imaging images and substation operation logs extract equipment status inspection snapshots with timestamps from minutes to hours; and PMS equipment ledgers provide static reference information with timestamps at the day level, including commissioning date and scheduled maintenance cycle. After all data is uniformly formatted, noise-removed, and multimodal aligned based on the time axis, a high-quality dataset that can be directly entered into the knowledge processing layer is formed.

[0099] Furthermore, in this embodiment, as Figure 3 The diagram illustrates how to construct a dynamic knowledge-answering graph, which involves at least encoding mapping, semantic alignment, feature fusion, and enhanced decoding. Specifically, encoding mapping involves encoding high-dimensional content vectors based on multimodal data, which capture text semantic features and map the text content to their positions. Semantic alignment involves generating unified text vectors using an adaptive gating fusion network based on a multilayer perceptron. Feature fusion involves cross-fusion of custom combined features using an improved multimodal partitioning fusion network to obtain multimodal deep features. These combined features include at least entity features, relational features, and globally shared features obtained by splitting the unified text vectors, as well as visual features obtained by projecting the image data. Enhanced decoding involves enhancing these multimodal deep features and inputting them into a preset decoder to incorporate the generated tuple set into the graph database to obtain the dynamic knowledge-answering graph. The tuple set includes timestamps associated with temporal attributes. The processing steps during the construction process will be described in detail in the subsequent description.

[0100] Furthermore, in one embodiment of the invention, during the encoding mapping, a high-dimensional content vector is obtained by encoding based on multimodal data. This high-dimensional content vector captures the semantic features of the text and performs positional mapping on the text content, specifically including:

[0101] Based on the multimodal data, a procedure hierarchy structure tree is constructed to obtain a procedure hierarchy index sequence, wherein the procedure hierarchy index sequence includes at least a chapter path sequence;

[0102] An adaptive sliding window is used to segment long texts in the procedure hierarchy structure tree into procedure content blocks, and each procedure content block is encoded to obtain a high-dimensional content vector.

[0103] The chapter path sequence is mapped to a path vector using a sine / cosine position encoding function, calculated as follows:

[0104] ;

[0105] ;

[0106] in, For position encoding functions, This represents the depth position of the current text content within the procedure hierarchy structure tree. For vector dimensions, For an even-numbered dimension in a vector, For odd-numbered dimensions in a vector, This is the index of the vector dimension.

[0107] It should be noted that, in this embodiment, a hierarchical structure tree of procedures is constructed based on the multimodal data. Specifically, the document structure tree of the procedure hierarchy is constructed by performing structured parsing on the input power maintenance procedures, accident prevention measures, and technical guidelines documents, using regular expression matching to identify standard clause numbers (such as "1.1.2", "3.a"), and combining this with a BERT-based semantic classifier to identify chapter titles and body paragraphs, thereby constructing a hierarchical index sequence of documents. ,in Representing the The hierarchical node information; while maintaining the integrity of the procedure hierarchy structure tree, a method combining sliding window and semantic boundary detection is used to segment long texts into procedure content blocks with independent semantics. And attach a complete source path tag to each content block in the tree structure. This ensures that the fault handling provisions after slicing still retain the contextual constraints of the applicable voltage level and equipment type.

[0108] Furthermore, in this embodiment, a bidirectional global embedding model fine-tuned for the power sector is used to encode each procedure content block into a high-dimensional content vector. ,in, For vector dimensions, the bidirectional global embedding model here can use the Transformer-based BGE-large-zh (BAAI General Embedding) model or the RoBERTa bidirectional pre-trained model as a base. In this context, power regulations contain numerous complex context-dependent technical terms. A bidirectional attention mechanism can simultaneously capture the left and right contextual information of words in the text, thereby generating high-dimensional feature representations with global semantic awareness. Simultaneously, the chapter path sequence... Input position encoding function Perform topology mapping to generate path vectors containing hierarchical structure information. The specific location code calculation is as follows:

[0109] ;

[0110] ;

[0111] in, For position encoding functions, This represents the depth position of the current text content within the procedure hierarchy structure tree. For vector dimensions, For an even-numbered dimension in a vector, For odd-numbered dimensions in a vector, This is the index of the vector dimension.

[0112] Furthermore, in one embodiment of the invention, the generation of a unified text vector based on an adaptive gating fusion network using a multilayer perceptron during semantic alignment specifically includes:

[0113] The high-dimensional content vector and the path vector are concatenated, and the gating coefficient is calculated through a fully connected layer, as shown in the following formula:

[0114] ;

[0115] in, The gating coefficient, For the Sigmoid activation function, For the gated weight matrix, It is a high-dimensional content vector. This represents a vector concatenation operation. For path vectors, This is a gated bias term;

[0116] The high-dimensional content vector and the path vector are weighted and fused based on the gating coefficient to generate the unified text vector, and the calculation formula is as follows:

[0117] ;

[0118] in, To unify text vectors, The gating coefficient, This represents element-wise multiplication. It is a high-dimensional content vector. This is the path vector.

[0119] It should be noted that, in this embodiment, an adaptive gating mechanism based on a multilayer perceptron (MLP) is used to dynamically calculate the fusion weights of content information and structural path information in order to generate a unified semantic index vector. The specific fusion computing strategy is shown in the following formula:

[0120] ;

[0121] ;

[0122] This involves concatenating the high-dimensional content vector and the path vector, and then calculating the gating coefficients using a fully connected layer. The gating coefficient, For the Sigmoid activation function, For the gated weight matrix, It is a high-dimensional content vector. For path vectors, For the gating bias term, specifically, when When the value approaches "1", it indicates that the system considers the literal meaning of the content block to be more important in solving the current problem, thus amplifying its value. The weight; when When the value approaches "0", it indicates that the system considers the chapter position of the content block to be more important, thus amplifying the path vector. The weights are used to achieve adaptive context awareness; and the high-dimensional content vector and the path vector are weighted and fused based on the gating coefficients to generate the unified text vector, wherein, To unify text vectors, The gating coefficient, This represents element-wise multiplication. It is a high-dimensional content vector. The path vector will eventually be unified into a text vector. Stored in a vector database to support high-precision retrieval that takes into account both semantic matching and procedural hierarchy constraints.

[0123] Furthermore, in one embodiment of the invention, during feature fusion, an improved multimodal partitioning fusion network is used to cross-fuse custom combined features to obtain multimodal deep features, specifically including:

[0124] The unified text vector is semantically structured to obtain entity feature region, relation feature region, and globally shared context region, calculated as follows:

[0125] ;

[0126] ;

[0127] ;

[0128] in, For entity feature regions, It is the ReLU activation function. This is the entity feature region weight matrix. To unify text vectors, For entity feature region bias term, For relational feature regions, This is the weight matrix of the relation feature region. For the relation feature region bias term, For globally shared context area, For the globally shared context region weight matrix, For globally shared context area bias items;

[0129] The image data includes infrared images. The infrared image features are projected into visually aligned feature regions in the same dimension as the unified text vector, calculated as follows:

[0130] ;

[0131] in, For visual feature alignment area, For normalization layer, This is the weight matrix for visually aligned feature regions. Infrared image features, For visual alignment feature region bias term;

[0132] The combined feature is obtained by concatenating the entity feature region, the relation feature region, the global shared context region, and the visual alignment feature region, and the calculation formula is as follows:

[0133] ;

[0134] in, As a combination feature, This indicates a feature concatenation operation. This is the entity feature region weight matrix. For entity feature regions, This is the weight matrix of the relation feature region. For relational feature regions, For the globally shared context region weight matrix, For globally shared context area, This is the weight matrix for visually aligned feature regions. For visual feature alignment area;

[0135] An improved multimodal partitioning fusion network is used to cross-fuse the combined features to obtain multimodal deep features. Specifically, each feature component of the combined features is independently calculated nonlinearly using an element-level expansion method before aggregation. The calculation formula is as follows:

[0136] ;

[0137] ;

[0138] in, For multimodal deep features, The total number of dimensions of the multimodal deep features; For multimodal deep features The Each component is output; Combination features of input The Each scalar feature component (e.g., a specific temperature mapping value extracted from an infrared image, or a weight of a certain dimension of text semantic features). For scalar feature components of B-order spline basis functions; In order to be with the first The scalar characteristic component and the first Trainable control coefficients corresponding to the first-order basis functions; Let be the total order of the B-spline basis functions.

[0139] It should be noted that, in this embodiment, as Figure 4 As shown, the unified text vector Semantic structural decomposition is performed to obtain text entity feature regions. Textual Relationship Feature Area Shared context area with the global The calculation formula is as follows:

[0140] ;

[0141] ;

[0142] ;

[0143] in, For entity feature regions, It is the ReLU activation function. This is the entity feature region weight matrix. To unify text vectors, For entity feature region bias term, For relational feature regions, This is the weight matrix of the relation feature region. For the relation feature region bias term, For globally shared context area, For the globally shared context region weight matrix, This is a globally shared context region bias term; simultaneously, the extracted infrared image features... Mapped to visual alignment feature regions through a linear projection layer. The calculation formula is as follows:

[0144] ;

[0145] in, For visual feature alignment area, For normalization layer, This is the weight matrix for visually aligned feature regions. Infrared image features, For the visual alignment feature region bias term, further, Used to identify the power equipment itself. Used to characterize topological connections and causal logic Used to characterize the thermal defect state of equipment.

[0146] Furthermore, in this embodiment, the combined feature is obtained by concatenating the entity feature region, the relationship feature region, the global shared context region, and the visual alignment feature region, and the calculation formula is as follows:

[0147] ;

[0148] in, As a combination feature, This indicates a feature concatenation operation. This is the entity feature region weight matrix. For entity feature regions, This is the weight matrix of the relation feature region. For relational feature regions, For the globally shared context region weight matrix, For globally shared context area, This is the weight matrix for visually aligned feature regions. The visual feature alignment region is defined, and the combined features are cross-fused using an improved multimodal partitioning fusion network to obtain multimodal deep features. Specifically, an improved Kolmogorov-Arnold network is used to perform nonlinear mapping and cross-fusion between the aforementioned multi-source partitioning features. A learnable spline activation function is used to capture the complex correlation between textual semantics and visual thermal imagery. The calculation formula is as follows:

[0149] ;

[0150] ;

[0151] in, For multimodal deep features, For multimodal deep features The total number of dimensions; For multimodal deep features The Each component is output; Combination features of input The Each scalar feature component (e.g., a specific temperature mapping value extracted from an infrared image, or a weight of a certain dimension of text semantic features). For input scalar components of B-order spline basis functions; In order to be with the first The eigencomponent and the th characteristic component Trainable control coefficients corresponding to the first-order basis functions; Let be the total order of the B-spline basis functions.

[0152] Specifically, in this embodiment, the improved Kolmogorov-Arnold network includes Layer, number The first layer The output of each neuron The calculation formula is as follows:

[0153] ;

[0154] in, For the first The first layer The output of each neuron It is a learnable univariate nonlinear activation function. For the first The first layer The output of each neuron, in this embodiment, is parameterized as a set of learnable nonlinear activation functions composed of B-spline basis functions, the physical source of which is the spliced ​​multimodal features. By taking a specific numerical value in a certain dimension, such as a temperature mapping value of infrared features or a semantic weight of text features, the model can adaptively learn complex nonlinear mappings between text and images. For example, it can learn a strong correlation between the visual feature of "temperature gradient > 50℃" and the textual concept of "critical defect", which significantly improves the model's generalization ability in power fault diagnosis.

[0155] Furthermore, in one embodiment of the invention, the multimodal deep features are enhanced during enhanced decoding, and the enhanced features are input into a preset decoder to incorporate the generated tuple set into the graph database to obtain the knowledge question-answering dynamic graph, specifically including:

[0156] The feature weights are adjusted based on the channel convolutional attention module, and the calculation formula is as follows:

[0157] ;

[0158] ;

[0159] in, For multimodal deep features, This refers to the multimodal deep features after the feature weights have been adjusted by the channel convolutional attention module. This represents element-wise multiplication. For the Sigmoid activation function, For multilayer perceptrons with shared parameters, This represents the average value of the aggregated global semantics. Represents the extreme value of aggregated global semantics;

[0160] The spatial convolutional attention module focuses on key information regions, and the calculation formula is as follows:

[0161] ;

[0162] ;

[0163] in, These are the enhanced features corresponding to the key information regions after being focused by the spatial convolutional attention module. This refers to the multimodal deep features after the feature weights have been adjusted by the channel convolutional attention module. This represents element-wise multiplication. For the Sigmoid activation function, To adopt Two-dimensional convolutional layers with large kernels, This represents the average value of the aggregated global semantics. This represents a vector concatenation operation. Represents the extreme value of aggregated global semantics;

[0164] The enhanced features are input into a preset decoder to incorporate the generated tuple set into the graph database to obtain the knowledge question-answering dynamic graph, wherein the tuple set expression is as follows:

[0165] ;

[0166] in, For a set of tuples, For the head entity, For header entity type, This refers to the relationship between the head and tail entities. For tail entities, For tail entity type, This indicates the current operating or defective status of the equipment. For timestamps.

[0167] It should be noted that, in this embodiment, as Figure 5 As shown, in the fused multimodal deep features An improved convolutional block attention module is applied to the module, which sequentially filters key fault feature dimensions through channel attention mechanism and focuses on abnormal temperature rise areas in infrared images and key fault verbs in text through spatial attention mechanism, dynamically adjusting feature weights to suppress background noise interference.

[0168] Specifically, in this embodiment, the improved convolutional block attention module includes a channel convolutional attention module and a spatial convolutional attention module. The feature weights are adjusted based on the channel convolutional attention module, meaning the channel convolutional attention module automatically assigns higher weights to key feature channels. The calculation formula is as follows:

[0169] ;

[0170] ;

[0171] in, For multimodal deep features, This refers to the multimodal deep features after the feature weights have been adjusted by the channel convolutional attention module. This represents element-wise multiplication. For the Sigmoid activation function, For multilayer perceptrons with shared parameters, This represents the average value of the aggregated global semantics. This represents the extreme value of the aggregated global semantics, where, and The model aggregates the average and extreme values ​​of global semantics, enabling it to focus on both the overall and local aspects. Its physical significance lies in the fact that different feature channels contribute significantly to fault diagnosis. For example, the visual channel representing the "infrared high temperature gradient" is obviously more important than the channel representing the "background sky," which is equivalent to adding a "feature filter" to the model and automatically amplifying the fault signal of the key channel.

[0172] Furthermore, in this embodiment, the spatial convolutional attention module focuses on key information regions, and the calculation formula is as follows:

[0173] ;

[0174] ;

[0175] in, These are the enhanced features corresponding to the key information regions after being focused by the spatial convolutional attention module. This refers to the multimodal deep features after the feature weights have been adjusted by the channel convolutional attention module. This represents element-wise multiplication. For the Sigmoid activation function, To adopt Two-dimensional convolutional layers with large kernels are used to capture a wider spatial contextual awareness field. This represents the average value of the aggregated global semantics. This represents a vector concatenation operation. This represents the extreme value of the aggregated global semantics. After the operation of the spatial convolution attention module, the originally diffuse features are sharpened. For example, it guides the precise locking onto the boundary of the abnormal temperature rise area in the infrared image and the core verbs such as "trip" and "breakdown" in the text description, thereby automatically ignoring the interference of environmental background and irrelevant text embellishment, which greatly improves the fault location accuracy of the system in complex field environments.

[0176] Furthermore, in this embodiment, the enhanced features are input into a preset decoder to incorporate the generated tuple set into the graph database to obtain the knowledge question-answering dynamic graph. Specifically, this embodiment employs a bilinear decoder for relation prediction and introduces a timestamp mechanism. First, entity recognition is performed, and then a conditional random field layer is used to predict entity boundaries and extract the head entity. Tail-end entity Next, a relationship score is performed to calculate whether a specific relationship exists between the two entities. probability fraction ,in, For the head entity, For tail entities, These are the enhanced features corresponding to the key information regions after being focused by the spatial convolutional attention module. For head entity transpose, For a specific relationship, For a specific relationship The system uses a bilinear weight matrix; finally, it binds the state to time. For identified state-type entities, such as "overheating," it automatically binds the current SCADA timestamp. To generate a set of tuples with extended triples. The expression is as follows:

[0177] ;

[0178] in, For a set of tuples, For the head entity, For header entity type, This refers to the relationship between the head and tail entities. For tail entities, For tail entity type, This indicates the current operating or defective status of the equipment. For timestamps.

[0179] Specifically, in this embodiment, taking the storage of a tuple as an example: <Entity: No. 2 main transformer, Type: Equipment, Relationship: Association, Entity: Phase A bushing, Type: Component, Status: Overheated (75℃), Time: 2026-02-06 10:00>, this allows subsequent queries to accurately retrieve "the specific status of a component at a certain moment." In practical applications, if maintenance personnel ask, "Is there a recent deterioration trend in the Phase A bushing of No. 2 main transformer?", it will no longer only return a static "main transformer includes bushing" structural relationship as in traditional diagrams, but will instead be able to... By filtering attributes through time windows, the historical trajectory of the bushing having experienced "overheating at 75℃" on "2026-02-06" can be accurately extracted. Then, combined with SCADA time-series data, its possible insulation aging trend can be predicted, making the knowledge graph leap from a static knowledge encyclopedia to a dynamic equipment medical record, which greatly improves the system's ability to handle complex, time-varying power fault-related questions and answers.

[0180] To achieve the above and other related objectives, a second aspect of this application provides a knowledge question-answering method applied to a knowledge question-answering dynamic graph constructed by any of the aforementioned knowledge question-answering dynamic graph construction methods, wherein, as Figure 6 As shown, the method includes the following steps:

[0181] Step S602: Receive the natural language question input by the user and identify the user's query intent;

[0182] Step S604: Based on the user's query intent, the constructed knowledge question-answering dynamic graph is used to process the query to obtain a set of candidate answers. The knowledge question-answering dynamic graph is processed by at least constrained reasoning based on a fault tree.

[0183] Step S606: Input the candidate answer set into a preset large language model to obtain the question-and-answer results, and visualize the question-and-answer results to the user terminal.

[0184] It should be noted that, in this embodiment, the user inputs a natural language question, and the user's query intent is identified. Based on the user's query intent, the constructed knowledge question-answering dynamic graph is used for processing to obtain a candidate answer set. The knowledge question-answering dynamic graph processing includes at least constrained reasoning based on a fault tree. Innovatively, fault tree prior knowledge is introduced, and the knowledge graph is pruned using a fault logic tree before retrieval to locate the root cause node of the fault. This is combined with RAG (Retrieval-Augmented) algorithm. The generation (retrieval-enhanced generation) method accurately matches the original text of the procedure in a vector database to obtain a set of candidate answers. The set of candidate answers is then input into a preset large language model to obtain the question-and-answer results. This corresponds to inputting the structured diagnostic path derived from reasoning and the original text of the procedure into the large language model to generate natural language answers that meet safety regulations and standards, along with dual traceability information, namely the graph node ID and the procedure clause number. The question-and-answer results are then visualized and displayed to the user. This application ensures that the system possesses both the rigor of logical reasoning and the accuracy of the original text of the procedure when answering highly specialized power fault questions.

[0185] Specifically, in this embodiment, the candidate answer set is input into a large language model, and a power field knowledge template containing a standard terminology library and safe operating procedures is injected during the generation process. The natural language suggestions generated by the model not only include fault characterization and handling steps, but also come with dual traceability tags, pointing to the specific clause number of the referenced procedure and the device node ID in the knowledge graph, respectively. In addition, the output results also support multimodal display and automatically attach links to secondary wiring diagrams related to the faulty equipment and summaries of historical similar defect work orders, thereby providing operation and maintenance personnel with comprehensive information support from theoretical basis to field drawings.

[0186] Furthermore, in one embodiment of the invention, the constructed knowledge question-answering dynamic graph is used to process the user's query intent to obtain a candidate answer set, specifically including:

[0187] The user query intent is obtained using a confidence-based weighted voting strategy, calculated as follows:

[0188] ;

[0189] in, For the user's query intent, This is a function for finding the maximum value. The discrete values ​​output by rule matching. To supervise the classification confidence of the learning model output, The semantic probabilities output by the large language model. The speech weights corresponding to different judgment paths; and

[0190] Constraint reasoning is performed based on fault tree, wherein the user's query intent is extracted as the root node and combined with the fault tree to perform reverse reasoning to obtain a set of fault causes.

[0191] Based on the set of fault causes, multi-hop reasoning is performed in the knowledge question-answering dynamic graph to generate structured reasoning results, and user questions are encoded into vectors for vector retrieval to identify matching procedure content blocks. The candidate answer set is obtained by combining the fusion weights, and the calculation formula is as follows:

[0192] ;

[0193] in, For the candidate answer set, To integrate weights, For the reasoning result, To match the procedure content block, Represent and manipulate.

[0194] It should be noted that, in this embodiment, after the question-answering engine receives a user's natural language question, it first performs rule matching using a built-in power-specific dictionary tree to quickly locate industry-related terms such as "heavy gas" and "differential protection" to prevent subsequent model segmentation errors of technical terms. Then, it uses a sequence labeling model based on the BERT-BiLSTM-CRF architecture to perform deep analysis of the question, accurately extracting key slot information such as equipment name, fault location, and fault phenomenon. Simultaneously, it predicts the user's intent category. To further improve the robustness of intent recognition, the system also calls a large language model in parallel for semantic reasoning, and finally adopts a confidence-based weighted voting strategy.

[0195] ;

[0196] in, For the user's query intent, This is a function for finding the maximum value. The discrete values ​​output by rule matching. To supervise the classification confidence of the learning model output, The semantic probabilities output by the large language model, in high-risk power scenarios, The weights of statements represent different decision paths, where rule matching is typically assigned. The highest weight ensures the system can rigidly identify critical keywords such as circuit breakers and heavy gas leaks with zero tolerance; while for colloquial long-tail questions, it relies on... and The system uses a flexible probability-based fallback mechanism to balance the professional rigor and interactive flexibility of the consultation system, determine the final intent instruction, and ensure that it can accurately identify user needs even when faced with colloquial or vague expressions.

[0197] Furthermore, in this embodiment, for the identified fault diagnosis intent, this application no longer blindly searches the entire knowledge base, but loads a pre-built power equipment fault tree model, wherein the extracted fault phenomenon serves as the root node of the fault tree, and the model is deduced backwards along logic gates (AND gates / OR gates), traversing all possible bottom events, thereby generating a set containing potential fault causes. For example, by inferring "high main transformer oil temperature" to "cooler completely stopped", "abnormal oil level" or "internal short circuit", the resulting set of fault causes will serve as a strong constraint condition, generating a semantic mask to logically prune the search space of the knowledge graph.

[0198] Specifically, in this embodiment, the physical meaning of the pruning process is illustrated using a real-world fault scenario: When the system parses a user problem containing the fault phenomenon of "main transformer differential protection tripping," it reverse-engineers the preset differential protection fault tree to obtain... The set is strictly limited to electrical root causes such as "interphase short circuit within the main transformer", "inter-turn short circuit in the winding", or "disconnection in the secondary circuit of the CT". During subsequent multi-hop searches, the system will directly use semantic masks to prune irrelevant interference nodes in the graph, such as "damaged cooler oil pump" or "paint peeling on the transformer exterior", which belong to mechanical or non-electrical faults. Through this prior pruning mechanism, the search attention of the large language model is forcibly constrained to the core logical chain of electrical faults, instantly reducing the graph search space, which originally contained millions of nodes, to hundreds of strongly related nodes. This not only multiplies the retrieval response speed in high-concurrency scenarios, but also fundamentally eliminates the logical illusions caused by the large model's association with professional fields, ensuring the safety and reliability of maintenance suggestions.

[0199] Furthermore, in this embodiment, within the pruned search space, a dual-channel retrieval is performed in parallel: on the one hand, multi-hop reasoning is performed in the knowledge graph based on the fault cause set to locate relevant historical defect case nodes and generate structured knowledge fragments; on the other hand, user questions are encoded as vectors, and the cosine similarity formula is used to retrieve the procedure content block that best matches the fault description from the vector database, and then the fusion weights are dynamically adjusted according to the fault urgency. The structured reasoning results are combined with the unstructured original text to form a candidate answer set, calculated as follows:

[0200] ;

[0201] in, For the candidate answer set, To integrate weights, For the reasoning result, To match the procedure content block, The system indicates and operates the circuit, automatically adjusting the power level when the intent involves emergency high-risk scenarios such as "power outage" or "fire". For queries, priority is given to recommending definitive historical cases and standard procedures from the graph to quickly assist in decision-making; while for general queries, the time required is reduced. Value, showcasing more details of the procedures and terms.

[0202] Furthermore, in this embodiment, the method also includes constructing a human-machine collaborative feedback and expert correction module to achieve system self-evolution based on operation and maintenance roles and expert experience. This module displays differentiated views according to different roles such as dispatchers, operation and maintenance specialists, and field maintenance workers, and provides an "expert correction interface". When a diagnosis is incorrect, experts are allowed to directly correct the answer. The module uses an incremental learning mechanism to record correction feedback and updates the connection weights and retrieval strategies of the knowledge graph in reverse, thereby achieving continuous growth and optimization of the knowledge base.

[0203] Specifically, in this embodiment, a refined operation and maintenance role modeling and permission view allocation are first performed. When a user accesses the system, the module first constructs a high-dimensional role feature vector based on the job information in the user's login credentials, such as dispatcher, operation and maintenance specialist, and field maintenance worker. Based on this vector, a differentiated information presentation view is dynamically rendered: for dispatchers, the potential impact of faulty equipment on the power grid topology and the overall network risk level are prioritized to assist them in making load transfer decisions; for operation and maintenance specialists, the focus is on displaying historical defect statistics curves of equipment, manufacturer technical parameters, and early warnings of family-related defects to support them in formulating medium- and long-term maintenance strategies; and for on-site maintenance workers, specific fault isolation steps, safety fence setting specifications, and lists of required spare parts are directly pushed to ensure the standardization and safety of on-site operations. This role-based adaptive view generation mechanism ensures that operation and maintenance personnel at different levels can obtain the most critical decision-making information in the first instance.

[0204] Furthermore, in this embodiment, multi-dimensional interactive feedback and online expert correction operations are recorded. During the user's browsing of diagnostic results, not only are explicit feedback behaviors such as clicks and favorites recorded, but an innovative expert correction interface is also introduced. When a high-privilege expert user, such as the chief engineer, discovers a deviation in the generated fault cause or handling suggestion, they can directly annotate or rewrite the text answer through the interface. The difference between the content before and after correction is automatically captured, and this operation is recorded as a high-confidence positive-negative sample pair. ,in This indicates the correct answer as revised by experts. This indicates incorrect answers generated by the system. This expert-in-the-loop mechanism effectively solves the problem of insufficient samples in pure data-driven models under rare failures (long tail effect).

[0205] Furthermore, in this embodiment, a preference weight update based on incremental learning is then performed. To adapt to the expert's business logic in real time, the module employs an online stochastic gradient descent (SGD) strategy to update the role preference weights. Incremental updates are performed, and the calculation formula is as follows: ,in, For a specific relationship The bilinear weight matrix, For learning rate, This is the feature vector of the current answer. This represents the discrepancy between expert feedback and system prediction. The physical meaning of this formula is that if the recommended answer is rejected by the expert, the algorithm will automatically reduce the projection value of that type of feature, such as a certain erroneous reasoning path, in the weight vector; conversely, if the expert's correction verifies a certain new fault association, the algorithm will strengthen the weight of that path, thereby achieving online fine-tuning of the algorithm parameters.

[0206] Furthermore, in this embodiment, adaptive retrieval rearrangement and dynamic graph evolution are finally implemented. After the weight update is completed, the new weights are not only utilized in subsequent searches... The candidate answer set is weighted and ranked to achieve personalized recommendations. Furthermore, it triggers the dynamic update mechanism of the knowledge graph. For high-confidence data corrected by experts, the entity relation extraction model is used to analyze the corrected text and automatically extract new causal relationship edges. For example, adding a new edge "CT secondary open path". The edges that cause "differential protection malfunction" are identified and written into the Neo4j graph database. Simultaneously, the weight coefficients of the hybrid retrieval are dynamically adjusted. Based on feedback history, the system automatically balances the dependence ratio between graph reasoning and original text retrieval. Through this closed-loop process, a qualitative change from passive querying to active learning is achieved. As usage time increases, the power panoramic knowledge graph will continuously improve itself and evolve into an industry knowledge brain that encompasses implicit expert experience.

[0207] It should be understood that the disclosed apparatus or method can be implemented in other ways, given the several embodiments provided in this application. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules / units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or units may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules or units may be electrical, mechanical, or other forms.

[0208] The modules / units described as separate components may or may not be physically separate. The components shown as modules / units may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules / units can be selected to achieve the objectives of the embodiments of this application, depending on actual needs. For example, the functional modules / units in the various embodiments of this application may be integrated into one processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into one module / unit.

[0209] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0210] This application also provides an electronic device, such as... Figure 7 As shown, the electronic device includes a processor and a memory.

[0211] like Figure 7 As shown, the electronic device includes: at least one processor 701, a memory 702, at least one network interface 703, and a user interface 705. The various components in the device are coupled together via a bus system 704. It is understood that the bus system 704 is used to implement communication between these components. In addition to a data bus, the bus system 704 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 7 The general will label all buses as bus systems.

[0212] The user interface 705 may include a monitor, keyboard, mouse, trackball, clicker, button, touchpad, or touch screen.

[0213] It is understood that memory 702 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM) or programmable read-only memory (PROM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM) and synchronous static random access memory (SSRAM). The memories described in the embodiments of this application are intended to include, but are not limited to, these and any other suitable categories of memory.

[0214] In this embodiment, the memory 702 is used to store various types of data to support the operation of the electronic terminal 700. Examples of this data include any executable program for operation on the electronic terminal 700, such as the operating system 7021 and application programs 7022. The operating system 7021 contains various system programs, such as the framework layer, core library layer, and driver layer, for implementing various basic services and handling hardware-based tasks. The application program 7022 may contain various applications, such as a media player and a browser, for implementing various application services. The methods provided in this embodiment can be included in the application program 7022.

[0215] The methods disclosed in the embodiments of this application can be applied to processor 701, or implemented by processor 701. Processor 701 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 701 or by instructions in the form of software. The processor 701 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 701 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. Processor 701 may be a microprocessor or any conventional processor, etc. The steps of the accessory optimization method provided in the embodiments of this application can be directly reflected as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium, which is located in memory. The processor reads the information in the memory and combines it with its hardware to complete the steps of the aforementioned method.

[0216] In an exemplary embodiment, the electronic terminal 700 may be used by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), or complex programmable logic devices (CPLDs) to execute the aforementioned method.

[0217] According to the method provided in the embodiments of this application, this application also provides a computer program product, which includes: computer program code, which, when run on a computer, causes the computer to execute the method of any embodiment of the above-described knowledge question answering dynamic graph construction method and / or any of the knowledge question answering methods described in the embodiments.

[0218] According to the method provided in the embodiments of this application, this application also provides a computer-readable storage medium storing program code, which, when run on a computer, causes the computer to execute the method of any embodiment of the above-described knowledge question answering dynamic graph construction method and / or any of the knowledge question answering methods described in the embodiments.

[0219] The terms “component,” “module,” “system,” etc., used in this specification are used to refer to computer-related entities, hardware, firmware, combinations of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or a computer. As illustrated, applications running on computing devices and computing devices can both be components. One or more components may reside in a process and / or an execution thread, and components may be located on a single computer and / or distributed among two or more computers. Furthermore, these components can be executed from various computer-readable media on which various data structures are stored. Components can communicate, for example, via local and / or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system, and / or a network, such as the Internet interacting with other systems via signals).

[0220] Those skilled in the art will recognize that the various illustrative logical blocks and steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0221] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0222] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0223] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0224] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0225] In the above embodiments, the functions of each functional unit can be implemented entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. A computer program product includes one or more computer instructions (programs). When the computer program instructions (programs) are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., high-density digital video discs (DVDs), or semiconductor media (e.g., solid-state disks (SSDs)).

[0226] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0227] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0228] In summary, this application provides a method for constructing a dynamic knowledge question-answering graph, a knowledge question-answering method, a storage medium, a program product, and an electronic device. It achieves high-precision, traceable, and personalized intelligent question answering for knowledge management of power equipment operation and maintenance. Through multi-source data fusion and PFKAN knowledge graph construction, it improves the robustness of semantic retrieval and reasoning. Combined with hybrid retrieval and tracing mechanisms, it ensures the accuracy and reliability of answers. With the help of preference learning, it achieves dynamic adaptation and efficient decision-making in multi-role scenarios.

[0229] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A method for constructing a dynamic knowledge question-answering graph, characterized in that, The method includes: Collect multimodal data, wherein the multimodal data includes at least image data with temporal attributes; Based on the multimodal data, at least encoding mapping, semantic alignment, feature fusion, and enhanced decoding processes are performed to construct a dynamic knowledge-question-answering graph, wherein... During encoding mapping, high-dimensional content vectors are obtained by encoding multimodal data. These high-dimensional content vectors capture the semantic features of the text and perform positional mapping on the text content; and When performing semantic alignment, an adaptive gated fusion network based on a multilayer perceptron generates a unified text vector; and During feature fusion, an improved multimodal partitioning fusion network is used to cross-fuse custom combined features to obtain multimodal deep features. These combined features include at least entity features, relation features, and globally shared features obtained from the unified text vector splitting, as well as visual features obtained from the image data projection. During enhanced decoding, the multimodal deep features are enhanced, and the enhanced features are input into a preset decoder to incorporate the generated set of tuples into the graph database to obtain the knowledge question-answering dynamic graph. The set of tuples includes timestamps associated with temporal attributes.

2. The method for constructing a dynamic knowledge question-answering graph according to claim 1, characterized in that, During encoding mapping, high-dimensional content vectors are obtained by encoding based on multimodal data. These high-dimensional content vectors capture the semantic features of the text and perform positional mapping on the text content, specifically including: Based on the multimodal data, a procedure hierarchy structure tree is constructed to obtain a procedure hierarchy index sequence, wherein the procedure hierarchy index sequence includes at least a chapter path sequence; An adaptive sliding window is used to segment long texts in the procedure hierarchy structure tree into procedure content blocks, and each procedure content block is encoded to obtain a high-dimensional content vector. The chapter path sequence is mapped to a path vector using a sine / cosine position encoding function, calculated as follows: ； ； in, For position encoding functions, This represents the depth position of the current text content within the procedure hierarchy structure tree. For vector dimensions, For an even-numbered dimension in a vector, For odd-numbered dimensions in a vector, This is the index of the vector dimension.

3. The method for constructing a dynamic knowledge question-answering graph according to claim 2, characterized in that, When performing semantic alignment, an adaptive gated fusion network based on a multilayer perceptron generates a unified text vector, specifically including: The high-dimensional content vector and the path vector are concatenated, and the gating coefficient is calculated through a fully connected layer, as shown in the following formula: ； in, The gating coefficient, For the Sigmoid activation function, For the gated weight matrix, It is a high-dimensional content vector. This represents a vector concatenation operation. For path vectors, This is a gated bias term; The high-dimensional content vector and the path vector are weighted and fused based on the gating coefficient to generate the unified text vector, and the calculation formula is as follows: ； in, To unify text vectors, The gating coefficient, This represents element-wise multiplication. It is a high-dimensional content vector. This is the path vector.

4. The method for constructing a dynamic knowledge question-answering graph according to claim 1, characterized in that, When performing feature fusion, an improved multimodal partitioning fusion network is used to cross-fuse custom combined features to obtain multimodal deep features, specifically including: The unified text vector is semantically structured to obtain entity feature region, relation feature region, and globally shared context region, calculated as follows: ； ； ； in, For entity feature regions, It is the ReLU activation function. This is the entity feature region weight matrix. To unify text vectors, For entity feature region bias term, For relational feature regions, This is the weight matrix of the relation feature region. For the relation feature region bias term, For globally shared context area, For the globally shared context region weight matrix, For globally shared context area bias items; The image data includes infrared images. The infrared image features are projected into visually aligned feature regions in the same dimension as the unified text vector, calculated as follows: ； in, For visual feature alignment area, For normalization layer, This is the weight matrix for visually aligned feature regions. Infrared image features, For visual alignment feature region bias term; The combined feature is obtained by concatenating the entity feature region, the relation feature region, the global shared context region, and the visual alignment feature region, and the calculation formula is as follows: ； in, As a combination feature, This indicates a feature concatenation operation. This is the entity feature region weight matrix. For entity feature regions, This is the weight matrix of the relation feature region. For relational feature regions, For the globally shared context region weight matrix, For globally shared context area, This is the weight matrix for visually aligned feature regions. For visual feature alignment area; An improved multimodal partitioning fusion network is used to cross-fuse the combined features to obtain multimodal deep features. Specifically, each feature component of the combined features is independently calculated nonlinearly using an element-level expansion method before aggregation. The calculation formula is as follows: ； ； in, For multimodal deep features, For multimodal deep features The total number of dimensions; For multimodal deep features The Each component is output; Combination features of input The Each scalar feature component; For scalar feature components of B-order spline basis functions; In order to be with the first The scalar characteristic component and the first Trainable control coefficients corresponding to the first-order basis functions; Let be the total order of the B-spline basis functions.

5. The method for constructing a dynamic knowledge question-answering graph according to claim 1, characterized in that, During enhanced decoding, the multimodal deep features are enhanced, and the enhanced features are input into a preset decoder to incorporate the generated tuple set into the graph database to obtain the knowledge question-answering dynamic graph. Specifically, this includes: The feature weights are adjusted based on the channel convolutional attention module, and the calculation formula is as follows: ； in, For multimodal deep features, This refers to the multimodal deep features after the feature weights have been adjusted by the channel convolutional attention module. This represents element-wise multiplication. For the Sigmoid activation function, For multilayer perceptrons with shared parameters, This represents the average value of the aggregated global semantics. Represents the extreme value of aggregated global semantics; The spatial convolutional attention module focuses on key information regions, and the calculation formula is as follows: ； ； in, These are the enhanced features corresponding to the key information regions after being focused by the spatial convolutional attention module. This refers to the multimodal deep features after the feature weights have been adjusted by the channel convolutional attention module. This represents element-wise multiplication. For the Sigmoid activation function, To adopt Two-dimensional convolutional layers with large kernels, This represents the average value of the aggregated global semantics. This represents a vector concatenation operation. Represents the extreme value of aggregated global semantics; The enhanced features are input into a preset decoder to incorporate the generated tuple set into the graph database to obtain the knowledge question-answering dynamic graph, wherein the tuple set expression is as follows: ； in, For a set of tuples, For the head entity, For header entity type, This refers to the relationship between the head and tail entities. For tail entities, For tail entity type, This indicates the current operating or defective status of the equipment. For timestamps.

6. A knowledge-based question-and-answer method, characterized in that, The method for constructing a dynamic knowledge-answering graph according to any one of claims 1-5 is applied to the dynamic knowledge-answering graph constructed by the method described in any one of claims 1-5, wherein the method includes: Receive natural language questions input from the user and identify the user's query intent; Based on the user's query intent, the constructed knowledge question-answering dynamic graph is used to process the query to obtain a set of candidate answers. The knowledge question-answering dynamic graph, during processing, includes at least constraint reasoning based on fault trees. The candidate answer set is input into a preset large language model to obtain the question and answer results, and the question and answer results are visualized and displayed to the user terminal.

7. The knowledge question-answering method according to claim 6, characterized in that, Based on the user's query intent, the constructed knowledge question-answering dynamic graph is used to process the query and obtain a set of candidate answers, specifically including: The user query intent is obtained using a confidence-based weighted voting strategy, calculated as follows: ； in, For the user's query intent, This is a function for finding the maximum value. The discrete values ​​output by rule matching. To supervise the classification confidence of the learning model output, The semantic probabilities output by the large language model. The speech weights corresponding to different judgment paths; and Constraint reasoning is performed based on fault tree, wherein the user's query intent is extracted as the root node and combined with the fault tree to perform reverse reasoning to obtain a set of fault causes. Based on the set of fault causes, multi-hop reasoning is performed in the knowledge question-answering dynamic graph to generate structured reasoning results, and user questions are encoded into vectors for vector retrieval to identify matching procedure content blocks. The candidate answer set is obtained by combining the fusion weights, and the calculation formula is as follows: ； in, For the candidate answer set, To integrate weights, For the reasoning result, To match the procedure content block, Represent and manipulate.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the knowledge question answering dynamic graph construction method according to any one of claims 1 to 5 and / or the knowledge question answering method according to any one of claims 6 to 7.

9. A computer program product, characterized in that, The computer program product includes computer program code, which, when run on a computer, causes the computer to implement the knowledge question answering dynamic graph construction method according to any one of claims 1 to 5 and / or the knowledge question answering method according to any one of claims 6 to 7.

10. An electronic device, characterized in that, The electronic device includes a processor and a memory; wherein the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device performs the knowledge question answering dynamic graph construction method as described in any one of claims 1 to 5 and / or the knowledge question answering method as described in any one of claims 6 to 7.

Images