Electric power cross-modal knowledge fusion multi-agent cooperative processing method and system

Abstract

The invention belongs to the technical field of multi-agent collaboration, and provides an electric power cross-modal knowledge fusion multi-agent collaboration processing method and system, and the technical scheme is as follows: analyzing a task execution command instruction input by a user; key elements of the task are extracted, the task is decomposed into a plurality of sub-tasks according to the extracted key elements, and logic association and priorities among the sub-tasks are identified; according to the logical association and priority among the subtasks, allocating the subtasks to the corresponding agents; each agent executes the corresponding subtask according to a set execution path, the executed task result is shared to other agents to cooperatively complete the complex task, and finally a cooperative processing result is obtained through integration. Efficient execution of complex tasks such as document generation, review and management is realized, and smooth completion of the tasks is ensured.

Description

Technical Field

[0001] The present invention belongs to the field of multi-agent collaboration technology, and in particular, relates to a method and system for multi-agent collaborative processing of cross-modal knowledge fusion in electric power. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] The complex tasks performed in the power industry usually span multiple fields and steps, including data and knowledge retrieval, document generation, content review, storage management, etc. Traditional systems are weak in task decomposition and module collaboration, resulting in low efficiency and poor response. When users issue complex multi-task requirements across scenarios, the system cannot dynamically assign tasks to different modules for collaborative completion. This limitation makes it impossible for the system to flexibly coordinate and optimize resource allocation among multiple modules, thus affecting the integrity and efficiency of task execution. Summary of the invention

[0004] In order to solve at least one technical problem existing in the above-mentioned background technology, the present invention provides a method and system for collaborative processing of multi-agents with cross-modal knowledge fusion in electric power, which dynamically allocates complex multi-tasks across scenarios to different agent modules, and improves the integrity and efficiency of task execution through flexible coordination and optimized resource allocation among agents.

[0005] In order to achieve the above object, the present invention adopts the following technical solution: The first aspect of the present invention provides a method for cross-modal knowledge fusion multi-agent collaborative processing of electric power, comprising the following steps: Parse the task execution command instructions input by the user; Extract the key elements of the task, decompose the task into multiple subtasks based on the extracted key elements, and identify the logical connections and priorities between the subtasks; According to the logical association and priority between subtasks, each subtask is assigned to the corresponding agent; The logical relationship and priority between the subtasks are: Based on the acquired knowledge, cross-modal fusion is performed to build a knowledge graph, and the power multimodal data is retrieved according to the user task instructions and the constructed knowledge graph; the power multimodal data is received, and a power document is generated based on the power multimodal data; the generated power document is received, and the compliance review and quality assessment of the power document is performed to generate a review report; the data during the execution of each subtask is recorded on the blockchain for management and control; Each intelligent agent performs the corresponding subtask and follows the set execution path, sharing the results of the executed tasks with other intelligent agents, collaboratively completing complex tasks, and finally integrating to obtain collaborative processing results.

[0006] The second aspect of the present invention provides a power cross-modal knowledge fusion multi-agent collaborative processing system, comprising: The instruction parsing module is used to parse the task execution command instructions input by the user; The subtask identification module is used to extract the key elements of the task, decompose the task into multiple subtasks according to the extracted key elements, and identify the logical relationship and priority between the subtasks; The subtask allocation module is used to allocate each subtask to the corresponding agent according to the logical association and priority between the subtasks; The logical relationship and priority between the subtasks are: Based on the acquired knowledge, cross-modal fusion is performed to build a knowledge graph, and the power multimodal data is retrieved according to the user task instructions and the constructed knowledge graph; the power multimodal data is received, and a power document is generated based on the power multimodal data; the generated power document is received, and the compliance review and quality assessment of the power document is performed to generate a review report; the data during the execution of each subtask is recorded on the blockchain for management and control; The collaborative processing module is used for each intelligent agent to execute the corresponding subtask and execute it according to the set execution path, share the executed task results with other intelligent agents, collaboratively complete complex tasks, and finally integrate to obtain the collaborative processing results.

[0007] A third aspect of the present invention provides a computer-readable storage medium.

[0008] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps in the above-mentioned power cross-modal knowledge fusion multi-agent collaborative processing method.

[0009] A fourth aspect of the present invention provides a computer device.

[0010] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the above-mentioned method for cross-modal knowledge fusion and multi-agent collaborative processing of electric power are implemented.

[0011] A fifth aspect of the present invention provides a program product, which is a computer program product, including a computer program, which, when executed by a processor, implements the steps in the above-mentioned power cross-modal knowledge fusion multi-agent collaborative processing method.

[0012] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention innovatively proposes a method for collaborative processing of multiple intelligent agents based on cross-modal knowledge fusion in the power industry. For complex tasks in multiple fields in the power industry, based on the introduction of cross-modal knowledge fusion into the relationship between the complex tasks, including intelligent question and answer, document generation, review and management, this complex task is assigned to a dedicated intelligent agent. Each intelligent agent is responsible for performing a specific task and collaborates with other intelligent agents through shared status and real-time communication mechanisms. This solves the problem of being unable to dynamically allocate tasks to different modules for collaborative completion when complex multi-task requirements are encountered across scenarios. It flexibly coordinates and optimizes resource allocation among multiple modules, thereby improving the integrity and efficiency of task execution.

[0013] 2. The present invention innovatively proposes an intelligent question-and-answer retrieval technology that integrates incremental updates and multimodal fusion, and introduces an updated graph knowledge base maintenance method, so that the knowledge base can automatically integrate new multimodal data in the power industry, overcoming the limitations of traditional knowledge bases. While ensuring the consistency of multimodal data and the real-time nature of the knowledge base, it can provide the power industry with dynamic and accurate intelligent question-and-answer services, meet the industry's urgent needs for complex data processing and real-time knowledge acquisition, overcome the limitations of traditional knowledge bases, and achieve dynamic management of data and comprehensive improvement of multimodal question-and-answer.

[0014] 3. The present invention innovatively proposes a document generation method that integrates bionic long-term memory and hierarchical context merging. By simulating the memory mechanism of the human hippocampus, a long-term memory library is established. During the generation process, historical related knowledge is dynamically called to ensure that the generated content is highly consistent with the needs of the current task. At the same time, the method adopts a hierarchical context processing framework to hierarchically integrate the information of sentences, paragraphs and chapters to make the content coherent and logically consistent. It also integrates multimodal data (such as images, tables, etc.) with the text generation process to ensure that the generated documents meet the professional standards of the power industry, and can cope with the challenges of long-distance dependency and multimodal data integration in large language model text generation. It solves the problem of difficulty in generating complex documents such as construction plans and audit reports in the power industry, and significantly improves the coherence and information accuracy of long document generation.

[0015] 4. The present invention innovatively proposes an autonomous review method for power documents that combines causal reasoning and compliance review. Based on causal reasoning technology, it can accurately identify and construct causal chains with multiple causes and effects, so as to better understand the logical relationship in the document; at the same time, the causal chain in the document is compared with the laws and regulations and standards of the power industry to ensure that the document content complies with the latest industry specifications, solves the problem of omissions in manual review, reduces compliance risks, and not only improves the quality and consistency of the review, but also helps users identify potential risk points, provides automated feedback and improvement suggestions, and further improves the quality and security of the document.

[0016] 5. The present invention innovatively proposes a method for multimodal document management and control on the power industry big model chain, combines blockchain technology with multimodal big models, and realizes multimodal management and control of the entire life cycle of power industry documents. All document generation, modification and access operations will be recorded on the chain to form an unalterable operation log. The distributed ledger technology of blockchain ensures the transparency and data integrity of document management, so that each step of the operation can be traced and verified, and potential abnormalities or non-compliant behaviors can be detected. This solves the problem that the existing technology lacks traceability and anti-tampering mechanisms, making it difficult to detect problems in a timely manner and trace modification history, and further enhances the intelligence level of power industry document management, especially important policy documents or audit report management.

[0017] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0019] Figure 1 It is a flow chart of a method for collaborative processing of electric power cross-modal knowledge fusion and multi-agents provided by an embodiment of the present invention; Figure 2 It is a flow chart of an intelligent question-answering retrieval method combining incremental updating and multimodal fusion provided by an embodiment of the present invention; Figure 3 It is a flow chart of a method for autonomous review of electric power documents combining causal reasoning and compliance review provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0020] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0021] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0022] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0023] Embodiment 1 This embodiment provides a multi-agent collaborative processing method for cross-modal knowledge fusion of electric power, including the following steps: Step 1: Parse the task execution command instructions input by the user and extract the key elements of the task; In this embodiment, a large language model such as the GPT series is used to parse the commands entered by the user, understand the user's intention and extract the key elements of the task. For example, the user may enter the command: "Generate an audit report on the substation equipment inspection." The system converts the command into a standardized task description and extracts the key information in the task, such as "substation equipment inspection" and "audit report."

[0024] The specific analytical formula is: , in, is a natural language command entered by the user, It is the parsed and standardized task description.

[0025] Step 2: Decompose the task into multiple subtasks based on the extracted key elements, identify the logical associations and priorities between the subtasks, and assign each subtask to the corresponding agent based on the logical associations and priorities between the subtasks; The specific steps include: Step 201, decomposing the task into multiple subtasks according to the extracted key elements; In this embodiment, each task is decomposed into multiple subtasks, and the task decomposition formula is as follows: , in, It is a list of subtasks decomposed from the parsing task.

[0026] In this embodiment, the subtask list includes tasks such as intelligent question and answer, document generation and review, and multimodal data management.

[0027] Step 202: Identify the logical associations and priorities between the subtasks, and assign each subtask to a corresponding agent according to the logical associations and priorities between the subtasks; According to the priority, complexity and domain requirements of the task, the logical relationship and priority between the subtasks are identified. The task allocation formula is as follows: , in, is a specific subtask, is the agent responsible for this subtask; Assign intelligent question answering to intelligent question answering agents, document generation to document generation agents, document review to document review agents, and document management to document management agents; The specific logical relationships and priorities between the subtasks are as follows: the intelligent question-answering agent retrieves the power multimodal data according to the user's task instructions and the constructed knowledge graph; the document generation agent receives the power multimodal data and generates power documents based on the power multimodal data; the document review agent receives the generated power documents, conducts compliance review and quality assessment on the power documents, and generates a review report; the document management agent manages the data generated by each agent on the power big model chain.

[0028] Step 3: Each agent performs the corresponding task; The specific steps include: Step 301: The intelligent question-answering agent retrieves electric power multimodal data according to the user's task instructions and the constructed knowledge graph; The specific steps include: Step 3011: Acquire multimodal data of the power industry; In this embodiment, the multimodal data includes equipment operation data, regulatory changes, operating procedures, and on-site images.

[0029] Among them, equipment operation data usually contains real-time monitoring information, such as current, voltage, power and temperature, etc., through which the operating status and health of the equipment can be evaluated.

[0030] Regulatory change data covers the latest policies, regulations and safety standards issued by the government and industry, and is an important basis for guiding equipment operation and management.

[0031] The operating procedures include standardized operating procedures and emergency handling plans for various equipment to ensure that operations in daily and emergency situations comply with regulations.

[0032] On-site image data is captured by drones or collected by on-site monitoring, which can provide visual information of the equipment and surrounding environment to assist fault diagnosis and on-site inspection.

[0033] Step 3012: construct a knowledge graph based on cross-modal fusion of the acquired knowledge; For each type of data, it is converted into a structured representation of knowledge graph nodes and edges. Equipment operation data, regulatory data, operating procedure data, and image data are mapped to graph structures respectively. Nodes in and edge , where nodes represent entities such as equipment, procedures, and environment, and edges represent the relationships between these entities.

[0034] The specific steps include: Step 30121, convert different multimodal data into data nodes; Create different multimodal data nodes and obtain the node set by finding their union; Construct a node set of the knowledge graph to represent different types of power industry knowledge entities. Define the graph structure ,in Represents a collection of nodes. Represents the edge set between nodes. Create different node types for different types of multimodal data (equipment operation data, regulations, operating procedures, images, etc.): Device Node V dev : Indicates power equipment and its status information.

[0035] Regulation Node V reg : Represents policy and regulatory information for the power industry.

[0036] Operation Procedure Node V proc : Indicates the operating procedures and procedural steps of the equipment.

[0037] Image Node V img : Represents device-related image information (objects or scenes extracted through image segmentation).

[0038] Therefore, the node set can be represented as: , Step 30122: assign a feature vector to each node, expressed as: , in, , , , The feature vectors respectively represent the equipment, regulations, procedures and image nodes; these feature vectors can be obtained through multimodal feature extraction methods, such as using visual semantic segmentation to process image data, text vectorization to process regulation and procedure text data, etc.

[0039] Step 30123: Define the relationship between nodes, i.e. the edge in the knowledge graph .

[0040] Edge Set Used to describe the relationship between different node types. For example: The relationship between devices: such as connection relationship, dependency relationship, etc., is used to represent the physical or logical association between devices.

[0041] The relationship between equipment and regulations: For example, the compliance relationship indicates the regulatory provisions that the equipment must comply with during operation.

[0042] The relationship between equipment and operating procedures: such as operation dependency, which indicates the operating steps that a certain equipment needs to follow at different operation stages.

[0043] Relationship between equipment and images: image information of specific equipment in different environments, used for fault diagnosis and equipment status monitoring.

[0044] Assume Edge The weight is , indicating the strength or importance of the relationship, then the edge is represented as: , in, represents an edge between two nodes, Indicates the weight corresponding to the edge.

[0045] Step 30124: Optimize and remove duplicates on the edges between the data nodes to obtain optimized edges between the nodes; To ensure that the knowledge base structure is free of redundancy, the graph structure needs to be optimized, especially the deduplication of nodes and edges. First, for duplicate nodes of the same entity, the following conditions are used to determine whether they are redundant: if , the node and Merge into one node to reduce duplicate information.

[0046] in, Represents the similarity between two nodes, which can be calculated by the cosine similarity of the feature vectors. is the similarity threshold.

[0047] For edge redundancy detection, edge weight update rules are defined to prevent the occurrence of duplicate edges.

[0048] If there is already an edge between two identical nodes , then the weight of the newly added edge With existing edge weights To perform the merge: , This ensures that the graph structure has a reasonable weight distribution when representing multimodal data associations.

[0049] Step 30125: Generate a complete initial graph structure based on node construction, feature assignment, edge definition and structure optimization; through this initial construction step, the knowledge base has a complete representation of power industry information and can maintain data consistency during incremental updates.

[0050] Step 3013: Determine incremental change information based on the newly added multimodal data; Step 30131, extracting semantic feature representation of each modality's newly added data; For each type of newly added data (such as newly added equipment status information, regulatory texts, operating procedures, and on-site images), its semantic feature representation is first generated through feature extraction technology; let the newly added multimodal data set be Dnew ={ d 1, d 2, ..., dn}, where d i Indicates i For each data item d i , through the feature extraction function f Get its eigenvector ,Right now: , in, f Represents a feature extraction function, which uses different feature extraction methods for different modal data, such as using a text embedding model to process text data and using visual semantic segmentation to process image data.

[0051] Step 30132: Calculate semantic similarity between the semantic features of the newly added data and the semantic feature vectors of the corresponding nodes in the knowledge base of the knowledge graph constructed in step 3012 to determine whether the corresponding nodes need to be marked as change points; The features of the newly added data are represented The feature vector of the corresponding node in the knowledge base Perform semantic similarity calculation to detect changes between new data and existing data. The similarity calculation function for: , in, The value range of is [0,1]. The closer the value is to 1, the higher the similarity between the new data and the existing knowledge base data.

[0052] By setting a threshold θ, we can determine whether the new data is significantly different from the existing data.

[0053] when <θ, it is considered d and vj There is a big difference, so d Marked as the change point that needs to be updated. The value of θ can be adjusted according to the specific application scenario to ensure the accuracy and recall rate of change point detection.

[0054] Step 30133: Determine the change type of the change point, and for each change point d i , generate corresponding incremental update information; Once a change point is detected, it is further classified into types such as equipment status change, procedure update, or field environment change. Suppose the classification function of the change point is , then each change point is marked as a specific update type:

[0055] This classification helps in targeted updating of relevant nodes and relationships in the graph knowledge base.

[0056] For each change point d i , generate the corresponding incremental update information. Let the update information be , then the generation process of update information can be expressed as: , in, vj For the nodes that need to be updated, is the feature vector of the change point, and the "association edge" represents the edge between the change point and other related nodes in the knowledge base. For example, if a change in the device state is detected, d i The edges associated with the device nodes are updated to the latest status to reflect the current operating status.

[0057] Step 30134: Generated update information Add it to the knowledge base, update the feature vector of the corresponding node, and readjust the associated edges of the node.

[0058] Through local optimization, we ensure that the location and semantic relationship of the newly added information in the knowledge base are reasonable. For example, for a device node whose state has changed, the feature vector of the updated node is: , Among them, α∈[0,1] is the smoothing coefficient, which is used to control the degree of fusion of new information and existing information to prevent the excessive influence of new information on the node state.

[0059] Through automated knowledge base updates and semantic incremental maintenance methods, the system reduces the labor cost of knowledge base maintenance. In traditional methods, the power knowledge base often requires professionals to regularly update and review data, while the automatic update and semantic maintenance methods adopted by the patent significantly reduce labor consumption and improve system efficiency, allowing the system to be deployed on a large scale at a lower cost and maintain its efficient operation.

[0060] Step 3014: Based on the incremental change information, edge weight adjustment neighborhood optimization and consistency check, the system generates an optimized knowledge graph; The steps include: Step 30141: Determine the semantic similarity between the newly added node and the existing node to determine whether a connection needs to be established; For the newly added node information identified in change detection, it is not directly added independently, but integrated based on the association with the existing nodes.

[0061] For new nodes v new and existing nodes v existing , through the similarity function Calculate their correlation strength, if it exceeds the set threshold , it is considered that there is a strong correlation between the two, and an edge needs to be established between them. e : , Step 30142: After the new node is connected to the existing node, in order to ensure the accuracy of the association relationship, the weight of the edge is adjusted to reflect the actual association strength between the nodes.

[0062] The initial weight of the edge is set to winit, and it is adjusted according to the similarity of the nodes to ensure the dynamic nature of the relationship in the knowledge base. The final weight of the edge is w e The calculation formula is: , in, is the adjustment coefficient, which is used to balance the influence of the initial weight and similarity. This step ensures that the strength of the relationship between the new information and the nodes in the existing knowledge base reasonably reflects the actual situation and avoids overly weakening or strengthening certain associations.

[0063] Step 30143: After the integration of the newly added nodes and edges is completed, the affected node neighborhoods in the knowledge graph are optimized to remove redundant or weakly associated nodes.

[0064] Specifically, for each node v Neighborhood set of N ( v ) to update and remove nodes whose correlation is lower than the threshold to ensure the simplicity of the graph structure. v With neighboring nodes u The correlation degree is g( v , u ), the optimized representation of the neighborhood set is: , Among them, γ is the minimum correlation threshold of neighborhood optimization. This optimization step reduces unnecessary connections and improves the efficiency of graph representation.

[0065] Step 30144: After adding new nodes and edges and adjusting the neighborhood structure, a consistency check is performed to ensure that the updated knowledge base has no structural conflicts.

[0066] The checks include node uniqueness, edge rationality, and graph connectivity.

[0067] For redundant nodes or edges in the graph, the system marks and removes them after detecting them during the inspection process. If two nodes are highly similar in semantics (i.e., the similarity is close to 1), they are merged into one node to reduce information redundancy. The consistency check function C(Gnew) is used to verify whether the graph structure meets the consistency requirements: ; Step 30145: After incremental update, edge weight adjustment, neighborhood optimization and consistency check, the system generates an optimized knowledge graph Gnew=(Vnew,Enew), ensuring that it reflects the latest industry knowledge and maintains a simple and reasonable structure. This optimized graph provides a dynamic and accurate knowledge basis for subsequent queries and intelligent question and answer, while avoiding unnecessary redundant information, ensuring improved system performance and efficient retrieval.

[0068] Through automated knowledge base updates and semantic incremental maintenance methods, the system reduces the labor costs of knowledge base maintenance. In traditional methods, the power knowledge base often requires professionals to regularly update and review data, while the automatic update and semantic maintenance methods adopted by the patent significantly reduce manpower consumption and improve system efficiency, allowing the system to be deployed on a large scale at a lower cost and maintain its efficient operation. These advantages combined enable the patented intelligent question-answering retrieval system to provide dynamic, accurate and comprehensive services in the power industry to meet the ever-changing needs of the industry.

[0069] Step 3015: Fusing the extracted multimodal features; The steps include: Step 30151: For data of different modalities, use a suitable feature extraction model to generate its feature vector representation: In this embodiment, text data: The GPT model is used to process the text data and map the text content into feature vectors. x text , the formula is: , where T is the text data.

[0070] Image data: Use the Visual Transformer (ViT) model to process image data and extract deep feature vectors of the image , the formula is: , where I represents the input image.

[0071] Audio data: Use the Transformer for Audio (TFA) model to process audio data and extract audio feature vectors , the formula is: , where A is the audio data.

[0072] Video data: For video data, extract key frames and use ViT or other suitable video models for feature extraction to represent the video as a feature vector : ,in, A collection of keyframes representing video data.

[0073] Step 30152: construct positive samples and negative samples using the generated multimodal feature vectors, and calculate the similarities of the positive samples and the negative samples respectively; For each data instance, extract the feature vector of its text, image, audio, and video modalities, such as instance i The text features xtexti, image features ximagei, audio features xaudioi, and video features xvideoi of the same instance are the positive sample pairs, such as (xtexti, ximagei) and (xaudioi, xvideoi); while the negative sample pairs are composed of modal features of different instances, such as (xtexti, ximagej) and (xaudioi, xvideok), where i≠j≠k.

[0074] In order to achieve the alignment of multimodal features, the similarity is calculated for positive sample pairs and negative sample pairs respectively.

[0075] The similarity function s can choose cosine similarity or Euclidean distance. And negative sample pairs , the similarity is calculated as and ,in Aims to maximize the similarity of positive sample pairs, while It is used to minimize the similarity of negative sample pairs.

[0076] To this end, the contrast loss function Lcontrastive is defined, whose goal is to maximize the similarity of positive samples and minimize the similarity of negative samples by optimization. The specific loss function can be expressed as: , Among them, the first term is the loss of positive sample pairs, which is obtained by As close to 1 as possible to ensure that the positive sample pairs are highly aligned in the feature space. The second term is the loss of negative sample pairs, which is minimized And set the threshold δ to control the similarity of negative sample pairs to prevent the features between negative samples from being too close.

[0077] Step 30153: Introduce spatiotemporal information into similarity calculation to ensure that different modal features of the same instance remain consistent in time and space.

[0078] Assume that the modal features of the instance contain temporal information t and spatial information l , then the calculation of similarity is expanded to: , Among them, α and β are the attenuation coefficients of time and space, which are used to control the influence of time and space distance on similarity. This formula ensures the consistency of multimodal features of the same instance in time and space, making the alignment of multimodal features more reasonable and tight.

[0079] During the training process, the system continuously optimizes the similarity of positive sample pairs and reduces the similarity of negative sample pairs by minimizing the contrast loss function Lcontrastive, gradually realizing the alignment of multimodal features in the same feature space. In this way, the system generates a unified feature space so that data of different modalities can be highly consistent in the feature space, thereby providing more accurate multimodal retrieval results when facing fuzzy queries. Ultimately, through the aligned multimodal feature representation, the system can achieve a comprehensive understanding of multimodal data in intelligent question and answer and data retrieval, ensuring the accuracy and consistency of query responses.

[0080] Step 3016: Generate query and response of the intelligent question and answer system.

[0081] The specific steps include: Step 30161: Represent the user task instruction, i.e., query Q, as a query feature vector x Q , in order to match the multimodal features in the knowledge base in the feature space.

[0082] Assuming the query is in text form, the system converts it into a feature vector through an embedding model (such as the GPT model): , This feature vector x Q Located in the same alignment space as the multimodal features of the knowledge base, so as to calculate similarity with nodes in the graph knowledge base.

[0083] Step 30162: semantically match the query feature vector with the graph knowledge base to retrieve the nodes most relevant to the query; The system performs semantic matching in the graph knowledge base and calculates the query feature vector xQ and the characteristics x of each node in the graph V similarity to find the most relevant node or node combination.

[0084] Similarity s( x Q , x V ) can be calculated using cosine similarity: , The system retrieves the node V that is most relevant to the query from the knowledge base based on the similarity value. relevant , in order to generate targeted responses to queries.

[0085] Step 30163: Dynamically analyze the content and associated information of the selected node and generate corresponding response data; To provide more accurate suggestions, the system dynamically analyzes the content and associated information of the selected nodes. For example, if the query involves the status of the device, the system extracts the device operation data node and the related abnormal information node from the knowledge base, evaluates the current status and generates processing suggestions in combination with the corresponding historical data and abnormal situations.

[0086] Assuming the device status is S(t) and the anomaly detection result is E(t), the processing suggestion R can be generated by the following logic: , in, f It is a rule or model function that automatically generates corresponding response suggestions based on the current status of the device and abnormal conditions.

[0087] Finally, the system integrates multimodal information to generate a detailed response for the user. If the user's query requires a multimodal explanation (such as a text description, an image example, or an audio prompt), the system obtains data from the relevant modalities from the graph knowledge base and integrates it into a unified answer. Assuming that the final response contains a text answer Tresp and an image example Iresp, the generated multimodal response is: , The response can comprehensively cover different aspects of the user's query and provide the user with specific processing suggestions and reference information.

[0088] The intelligent question-and-answer interface and automatic response generation function greatly improve the user experience. Users can enter queries through a unified interface, and the system quickly performs semantic matching based on aligned multimodal features and dynamically updated knowledge bases, generating detailed multimodal response results such as device status analysis and processing suggestions. This question-and-answer mode allows users to query without being limited to a single modality and directly obtain multimodal integrated information, greatly enhancing the interactive experience of the system and the richness of the response.

[0089] Step 3017: Automatically adjust the structure and feature alignment strategy of the graph knowledge base based on new user queries and responses; The specific steps include: Step 30171: Analyze the user query patterns to identify common query types and key information needs.

[0090] For example, the system can analyze the frequency and content of queries to identify specific topics that users focus on (such as equipment operating status, abnormal diagnosis, or regulatory updates). Based on this pattern analysis, the system can dynamically generate query templates to identify users' high-frequency needs and strengthen the priority of relevant knowledge nodes.

[0091] After each response is generated, the system evaluates the effectiveness of the response to ensure that it meets user needs and provides accurate information. The effectiveness evaluation can calculate the matching degree between the user query Q and the response R by the following formula: , in, is the similarity between query and response features. When the match is low, the system marks the response as needing improvement and triggers the graph structure optimization process.

[0092] Step 30172: Based on the focus of the user's query and the effectiveness feedback of the response, the system adjusts the structure of the knowledge base by increasing the connection of important nodes or reducing redundant connections to ensure that the nodes that the user pays most attention to receive a higher access priority.

[0093] Assume that a node v If the access frequency of a node increases, the system can dynamically increase the associated edges of the node. e , and adjust its weight w e , so that it is displayed first in the search. The weight update formula is as follows: , Among them, γ is the adjustment coefficient, f 访问频率 Represents the access frequency function of the node. This step optimizes the graph structure to ensure that important information is more prominent in user queries, thereby improving the retrieval effect.

[0094] Step 30173: The system will dynamically adjust the alignment strategy of polymorphic features to adapt to changes in user needs.

[0095] If a modality data shows high importance in multiple queries (for example, image data is more critical in equipment fault diagnosis), the system will increase the feature weight of the modality. Suppose the alignment weight is adjusted to α m, for important modes m The feature weights are updated as: , Among them, λ is the adjustment parameter, f _ 模态重要性 is the importance function of the modality in a specific query. This dynamic adjustment of the alignment strategy ensures more accurate extraction of multimodal information in specific tasks.

[0096] Step 302: The document generation agent receives the power multimodal data and generates a power document according to the power multimodal data; The specific steps include: Step 3021: extract knowledge in the form of multimodal data from historical document data, build a long-term memory bank by simulating the memory mechanism of the human hippocampus with the help of bionics, and hierarchically store and dynamically update the knowledge in the long-term memory bank in three levels: sentence, paragraph, and chapter; The bionics concept here is to simulate the powerful human contextual long-term memory system, which relies on the interaction between the neocortex and the hippocampus. It consists of the following three parts: the neocortex, the parahippocampal region (PHR) and the hippocampus. In view of the above three parts, this embodiment designs a method for constructing a long-term memory bank. The function of the neocortex is simulated by the large language model LLM. LLM is responsible for extracting perceptual signals, similar to the neocortex converting perceptual stimuli into high-level feature representations. It is the information processing center of the entire system and is responsible for generating specific knowledge representations. The personalized PageRank algorithm (PPR) is more in line with the function of the parahippocampal area. PPR uses the structure of the knowledge graph in the retrieval process and is based on the query node (Query The PPR algorithm finds the subgraph most relevant to the context by searching for nodes in the knowledge graph, which links the features extracted by the neocortex with the associations in the knowledge graph. By guiding the retrieval process, it ultimately helps information flow to the "hippocampus", similar to the role of the parahippocampal region as a bridge connecting the neocortex and the hippocampus. The PPR algorithm achieves precise matching and filtering of the context. The constructed knowledge graph (KG) is analogous to the hippocampus. As the core "memory index" of the system, the knowledge graph records the nodes and relationships in the document and realizes pattern separation and pattern completion through its structure. During retrieval, the PPR algorithm activates query-related paths in the knowledge graph, which represent the extraction and reconstruction of memory. This design simulates the key role of the hippocampus in storing and retrieving associative memories.

[0097] The specific steps include: Step 30211: Analyze all documents and historical generation records, use the power large language model (LLM) to identify core concepts, keywords and key data points as noun phrases or events, and store them in text, image, and table formats.

[0098] The noun phrase or event here can be a single word, a term, a sentence, or a paragraph, depending on several factors: 1) Core concepts and subject terms: These are usually expressed as noun phrases, keywords or terms, which can be single words (such as "electricity", "substation"), phrases (such as "power equipment failure", "operating status") or more complex noun phrases (such as "equipment maintenance record"); these terms help to capture key information in the document and dynamically call it during the generation process.

[0099] 2) Key data points: This includes numbers, statistics, time information or other structured data (such as values ​​in a chart or rows and columns in a table); this data can be stored as a single data point (such as "voltage value", "operating time") or a more complex structure in a table.

[0100] 3) Sentences or paragraphs: For complex documents, especially those with contextual content, sentences or paragraphs may become storage units; for example, when generating an audit report, you may need to store a complete sentence or paragraph to preserve the integrity of the context (such as "the equipment has exceeded the specified number of working hours"). These sentences or paragraphs contain important background information or semantic context, which helps to integrate knowledge when generating documents later.

[0101] Step 30212: By simulating the memory mechanism of the human hippocampus, a long-term memory bank is constructed to store and dynamically update knowledge in a hierarchical structure, ensuring efficient retrieval and consistency of information in document generation.

[0102] In order to ensure the accessibility of information, the extracted knowledge is constructed into a multi-level knowledge graph. Each knowledge node represents a noun phrase or event. When establishing the knowledge graph, the memory characteristics of the human hippocampus are borrowed. By adding multiple synonymous relationships and hyponymous relationships, the knowledge graph has stronger relevance and scalability, and a long-term memory library in the form of a knowledge graph is obtained.

[0103] In the construction of the long-term memory library, referring to the memory indexing and retrieval process in HippoRAG, knowledge can be represented as a graph structure. The knowledge nodes are associated through relationships to obtain the knowledge graph G=(V,E) of the long-term memory library, where V represents the knowledge node set, E is the edge set connecting the knowledge nodes, and each knowledge node v∈V represents a noun phrase or event. Represents the semantic relationship between these knowledge.

[0104] The edges here include: 1) Multiple Synonymy A synonym relationship indicates that two nodes have semantically similar or equivalent meanings. For example, in the power industry, "substation" and "power transformer station" can be connected by a synonym relationship because they refer to the same type of facility.

[0105] 2) Hierarchical Relations The hierarchical relationship represents the hierarchical structure between nodes, that is, one node is the superior or subordinate of another node; the superior node (parent node) represents a more abstract concept, while the subordinate node (child node) represents a more specific instance or detail. For example, power equipment is the superior concept of substation, and distribution equipment is a type of power equipment, so it is a subordinate relationship.

[0106] 3) Other common relationship types Causal Relationships: Used to indicate the cause-effect relationship between events or behaviors, for example, "equipment failure" may be caused by "overload".

[0107] Temporal Relationships: Indicates the temporal order between nodes. For example, "Inspection Report" is generated after "Equipment Maintenance".

[0108] Spatial Relationships: Used to represent the spatial location relationship between nodes, such as "power lines are located near substations."

[0109] Functional Relationships: Indicates the functional dependency between nodes, for example, “substation” is responsible for “power dispatching”.

[0110] Determining the relationship between nodes usually depends on the following aspects: Semantic analysis: Perform semantic analysis on nodes through a large language model (such as LLM) to identify the potential relationships between them. For example, use a large language model to extract relevant descriptions from a document and automatically identify the causal relationship between "fault" and "repair."

[0111] Domain knowledge: Based on the expertise of the power industry, knowledge graphs are used to define standard relationships within the industry. For example, the relationship between power equipment and equipment maintenance is usually functional or causal.

[0112] Contextual relationship: By analyzing the context of nodes, it is determined whether there is a temporal, spatial or hierarchical relationship between them. For example, when describing a substation, it may be automatically associated with its geographical location and the equipment it contains.

[0113] The graph structure of the long-term memory library in this embodiment simulates the hierarchical storage method of different knowledge in the human brain, so that information can be directly located and retrieved to specific content according to context requirements in subsequent calls; this process not only improves the richness of the knowledge base, but also ensures the system's ability to respond quickly to complex power document generation needs.

[0114] Furthermore, after the information is extracted and constructed as a knowledge graph, the extracted knowledge is further refined and classified and stored at different levels such as sentences, paragraphs, and chapters. Each level corresponds to a specific content unit in document generation, making it easy to call it by level.

[0115] For example, sentence-level knowledge nodes are stored at the micro level and used to generate detailed descriptions; chapter-level nodes contain macro-background information and key policy content, which are used to guide the structure of the overall document.

[0116] In addition, a dynamic update mechanism is introduced to monitor the correlation between new tasks and historical generation records, and integrate the latest generated knowledge fragments into the memory bank. Through this continuous update, it can be ensured that the information in the memory bank always remains relevant to the current generation task. This mechanism effectively addresses the problem of knowledge expiration in long document generation, so that the most timely key information can be called in different generation scenarios, achieving a high degree of unity between professionalism and accuracy.

[0117] Step 3022: When generating a new document, dynamically call information highly relevant to the current task from the long-term memory through semantic analysis and personalized retrieval algorithms to ensure the coherence of the generated content and consistency with historical knowledge; After receiving the generated task, the task content is first semantically analyzed through natural language processing technology to identify the subject words, core requirements and important concepts. Based on the identified key information, a preliminary match is made in the memory library, and similarity calculations are performed with the nodes in the library to identify the memory fragments most relevant to the current task; through this associated node identification mechanism, not only can the basic knowledge required for the task be extracted, but also context support can be provided for subsequent content generation, achieving accurate information retrieval and effective integration.

[0118] The memory fragments here are the specific content or long content of the nodes in the long-term memory library, and the nodes are high-level summaries of the memory fragments and are general labels; each node represents the knowledge abstraction of a specific field, or a "conceptual label", for example, the node can be "substation equipment installation standards", "equipment failure records", etc.; these nodes point to actual, detailed memory fragments, which are high-level abstractions of specific content and help the system better organize and retrieve information. Memory fragments contain detailed, specific text content or multimodal data (such as images, tables, documents, etc.), which are longer content (such as the entire equipment maintenance record, or a detailed description of a certain equipment failure), and they provide a more specific knowledge base for the system.

[0119] The initial matching here uses a semantic matching algorithm, with the help of the node relationship and semantic similarity in the graph, to accurately locate the most relevant knowledge points. Taking "generating an audit report on substation equipment maintenance" as an example, the specific process is as follows: 1) Receive tasks and perform semantic analysis Suppose you receive a task to generate an audit report on substation equipment maintenance. The task content is as follows: “Please generate an audit report on the maintenance of substation equipment to check whether the equipment meets the installation standards, whether there are any faults, and whether the maintenance records are complete.” Through the large language model, the task content is analyzed to extract keywords, core requirements and important concepts: Keywords: Substation, Equipment Maintenance, Audit Report Core requirements: Check whether the equipment meets the installation standards, check whether the equipment has faults, and whether the maintenance records are complete 2) Perform a preliminary match in the memory bank The keywords and core requirements identified in the task will be used as input for preliminary node matching in the memory. The memory here is a graph containing many conceptual nodes (such as "equipment installation standards", "equipment maintenance records", "equipment failures", etc.), each of which represents an abstract knowledge unit in the knowledge graph.

[0120] The role of node relationships: Each node not only contains content (memory fragments) related to it, but also establishes semantic connections with other nodes through edges (relationships); for example, there may be an "inclusion" relationship between the "equipment installation standard" node and the "substation" node, indicating that the equipment installation standard applies to all equipment in the substation.

[0121] These node relationships are used in the matching process to determine which knowledge points should be prioritized. For example, the "equipment failure" node may find maintenance records related to "failure" through the "solution" relationship with the "equipment maintenance" node.

[0122] 3) Similarity calculation and semantic matching algorithm Once the nodes are determined, the semantic similarity between the task description and the nodes in the knowledge graph is calculated. The specific process is as follows: Vectorization: Each node (such as "equipment maintenance record", "equipment installation standard") and keywords in the task are converted into high-dimensional vectors through a large language model.

[0123] These vectors represent the semantic information of nodes and facilitate similarity calculation.

[0124] Calculate semantic similarity: The task content vector and the node vector are matched using cosine similarity (or other similarity algorithms) to calculate the semantic similarity between each task keyword and the node.

[0125] The calculation formula of cosine similarity is as follows: , in, and is the task description and the node vector, ⋅ represents the dot product, and is the norm of the vector, and the larger the value, the more similar it is.

[0126] Matching tasks and nodes: Based on the calculated similarity, the node most relevant to the task description and its associated memory fragments will be selected. For example, if the task mentions "check whether the equipment meets the installation standards", it will be matched to the "equipment installation standards" node, and the memory fragments related to the node will be selected. 4) Optimize matching with node relationships In addition to similarity calculations, matching results are optimized based on node relationships, for example: Hierarchical relationship: If the task requires checking equipment failure, the system will find a more specific failure type (such as "equipment overload failure") through the "Equipment Failure" node and further refine the results based on the "Failure Repair" node.

[0127] Causal relationship: If the task description mentions that "equipment failure" may require "equipment maintenance", the system will provide detailed information related to the fault repair through the causal relationship of the "fault repair" node and the fault repair record.

[0128] With the help of these relationships, the system can accurately locate the memory fragments most relevant to the task and provide the necessary contextual information for subsequent report generation.

[0129] Furthermore, after the initial matching of relevant nodes, a personalized PageRank algorithm is used to perform retrieval on the knowledge graph to dynamically call up key information in the memory bank. The algorithm adjusts the weights of different nodes to prioritize the retrieval of knowledge that is highly relevant to the current task and embeds these contents directly into the generation process. Through the dynamic calling mechanism, the knowledge that best fits the context can be called at each generation stage to avoid repeated calculations, thereby improving generation efficiency and consistency. For documents that require long-distance dependencies, such as complex construction plans or multi-chapter audit reports, historical information fragments related to the task, that is, the knowledge in the long-term memory bank, can be continuously called up to maintain consistency and logical coherence during the generation process, while ensuring the professionalism and accuracy of the generated content.

[0130] The following is an explanation of the PageRank algorithm: Assume that a task q is given and the personalized PageRank (PPR) algorithm is used to query the knowledge nodes most relevant to the task, which can be expressed as: , Among them, P(v i )、P(v j ) represents node v i 、v j The importance in the current query, N(v i ) is the node v i The set of neighbor nodes in the knowledge graph, is the initial distribution, and usually a higher initial value is set for query-related nodes. α is an adjustment parameter that controls the probability of random jumps. Through this algorithm, relevant nodes in the memory library can be dynamically called to ensure that the retrieved knowledge can accurately meet the needs of the current document generation task.

[0131] In the PageRank algorithm, the initial distribution refers to the "importance" weight initially assigned to each node in the graph, which is usually equal or set according to certain specific rules; it is usually closely related to the prior knowledge or task relevance of the nodes in the network; for each node v i , the initial distribution is usually It defines the "importance" or "score" of the node at the beginning of the algorithm, that is, each node v i At the beginning, each node has an initial distribution value, that is, the "initial importance" of the node in the network. These initial values ​​can be set according to the characteristics of the node, the requirements of the task, or the design of the algorithm.

[0132] In practical applications, the basis for setting the initial distribution is mainly based on the following considerations: (1) The initial values ​​of all nodes are equal: In the simplest case, the PageRank algorithm assumes that the initial importance values ​​of all nodes are equal, so that all nodes have the same influence at the beginning of the algorithm, and then dynamically adjusts the "importance" of each node through algorithm iteration. This method is suitable when there is no obvious priority or preference in the network. The formula is: , in, is the total number of nodes. In this case, the initial value is an average and has little to do with the initial matched nodes. It is more based on the assumption of "no prior knowledge".

[0133] (2) Set initial values ​​based on task relevance: If there is prior information about the task (such as the node type or domain required by the task), the initial value can be set based on the importance and relevance of the node.

[0134] Example: Assume that the task requires the generation of an audit report related to equipment installation standards. If the "Equipment Installation Standards" node (v i ) is mentioned in the task description or is particularly important, we can set a higher initial value for this node.

[0135] Node-task relevance: For example, if the task description mentions "equipment installation standards", then nodes related to "equipment installation standards" will get a higher initial value based on the results of the semantic matching algorithm. In this way, the initially matched nodes (such as the "equipment installation standards" node) will start with a higher initial distribution , to reflect its importance in the task.

[0136] (3) Set initial values ​​based on graph structure: Sometimes, the initial values ​​of nodes can also be set based on structural information in the graph. For example, nodes with higher in-degree (i.e., more edges connected to it) may be more important, so their initial values ​​may be higher.

[0137] For node v i , if it has more incoming edges (that is, more other nodes point to it), then its initial value It will be set higher to adjust the initial weight of the node based on the in-degree or out-degree.

[0138] There is a direct relationship between the initially matched nodes and the initial distribution. In the task description, nodes related to the task requirements will be assigned higher initial values ​​to ensure that these nodes can have a greater impact on the task results in the initial stage; for example, if the task is to generate an equipment maintenance report for a substation, the system will set a higher initial distribution for the "equipment maintenance record" node based on its relevance to the task, thereby maintaining its high importance in the entire algorithm.

[0139] Step 3023: Divide the document generation process into sentence, paragraph and chapter levels, and ensure the logical coherence and professionalism of the generated content at each level by integrating contextual background knowledge and key information in the long-term memory bank layer by layer.

[0140] The contextual background knowledge and key information here are retrieved from the long-term memory bank, the difference is: Contextual background knowledge refers to all information related to the current document generation task, usually including the task background, industry knowledge, context requirements, historical information of task description, etc. This information helps to ensure that the document generation is logically coherent and professional; contextual background knowledge mainly comes from the long-term memory bank, which stores historical information related to the task, previously generated content, and long-term stored knowledge points, including: (1) Generation records of previous tasks: including reports or document fragments similar to the current task.

[0141] (2) Industry norms and standards: such as relevant laws and regulations of the power industry, equipment operation standards, etc., which can provide the necessary background support for document generation.

[0142] (3) Context of historical tasks: For example, if the document generation task is about an audit report on equipment maintenance, the content of previous equipment maintenance audit reports can be used as background knowledge reference.

[0143] (4) This information helps the system maintain contextual consistency when generating documents and ensures the logical coherence between the newly generated content and the previously generated content.

[0144] Key information is the specific information mentioned in the task requirements. This information determines the core content of the generated document and provides the system with knowledge points that need to be focused on in the execution of specific tasks. Nodes are retrieved from the long-term memory library: In the long-term memory library, nodes represent specific pieces of knowledge, which are key points in document generation. For example, if the task requires the generation of an equipment maintenance report, the "equipment maintenance record" and "equipment failure log" nodes stored in the long-term memory library will be retrieved; if the task requires the inspection of equipment installation standards, the "equipment installation standards" node and related specific content in the long-term memory library will be extracted; these nodes contain detailed information related to the task (such as equipment specifications, historical records, standard requirements, etc.), and the system uses the information in these nodes to generate specific report paragraphs.

[0145] Contextual background knowledge mainly helps the system to ensure the coherence and consistency of content when generating documents. For example, when generating equipment maintenance reports, the system needs to understand the industry standards and the requirements of similar tasks in history. Therefore, the system retrieves relevant nodes and historical task information from the long-term memory to construct the background part of the document to ensure that the content meets industry standards; key information directly affects the specific content of the report. For example, if the task requires "check whether the equipment meets the installation standards", the system will retrieve the nodes related to "equipment installation standards" from the long-term memory, extract the specific standard requirements, and generate task-related paragraphs accordingly; therefore, it is necessary to integrate contextual background knowledge and key information in the long-term memory.

[0146] Specifically, at the sentence level, firstly, semantically clear sentences are generated by calling relevant knowledge (i.e. key information) in the long-term memory bank and combining it with the background information of the current task (i.e. contextual background knowledge). In this process, the theme and structure of the sentence are analyzed to ensure that each sentence is semantically independent and clear while being consistent with the context. In order to lay the foundation for paragraph generation, the semantic relationship between sentences is automatically detected while generating sentences, and the master-slave relationship, causal relationship and other associations are marked. These marking information are crucial in the subsequent paragraph integration; in this way, not only the accuracy of each sentence is ensured, but also the basis for the smooth connection of the paragraph level is provided.

[0147] According to the hierarchical processing method of HOMER with hierarchical context fusion, the sentence generation in the lth level can be expressed as: , in, represents the hidden state generated by the previous layer, is the knowledge retrieved from the long-term memory bank or the context, and f is the generating function.

[0148] Furthermore, at the paragraph level, the sentences generated in the previous step are integrated according to contextual markers (i.e., marked semantic relationships) to give the paragraph a coherent logical structure and thematic consistency. At the same time, contextual background knowledge in the long-term memory bank is referenced according to the theme of the paragraph. For example, relevant construction standards and regulations are called up in paragraphs involving power construction to make the content more comprehensive and in line with industry specifications.

[0149] The paragraph theme here is the core idea of ​​the paragraph content. It is the key basis for selecting relevant background knowledge and organizing sentences when generating paragraphs. The paragraph theme is mainly obtained in the following ways: (1) Extract from the task description: Directly extract the topic: The task objective or description usually contains the keywords or focus of the paragraph topic. For example: Task description: "Generate reports on electrical construction safety." The theme extracted from the large model: "Electric Power Construction Safety".

[0150] (2) Inference from contextual information: During paragraph generation, the large language model will automatically extract the topic based on the sentence content generated in the previous paragraph. For example: The previous article mentioned "equipment installation standards", and the topic of the current paragraph may continue to be "safety of equipment installation".

[0151] (3) Combine high-frequency nodes in the long-term memory bank: During the generation process, semantic analysis is used to find high-frequency knowledge nodes in the long-term memory bank that are relevant to the task. For example: If the paragraph content involves "construction standards", the knowledge nodes related to "construction specifications" are extracted, and the big model further refines the paragraph theme as "application scenarios of construction specifications".

[0152] (4) Extraction of main and subordinate clauses and core sentences: Within the paragraph, mark which sentences are the core content (main sentences), and then use the topic of the core sentence as the core of the paragraph.

[0153] Furthermore, at the chapter level, the paragraphs are further integrated and analyzed as a whole to ensure that the chapter content is logically and thematically coherent; at this time, the chapter background knowledge is called (the chapter theme is extracted through the task description, and the background knowledge related to the theme is directly matched as the chapter background knowledge), an introduction is generated for the beginning of the chapter, and a summary paragraph is generated at the end to provide overall support; through this hierarchical integration, the system can finely manage contextual information at each level, so that the generated documents maintain high-quality logical consistency and professionalism from sentences to chapters.

[0154] In order to merge contexts at the level, HOMER uses a recursive split and merge strategy. In the hierarchical merging process, Hierarchical Context Embedding Merging is introduced. The specific steps are as follows: 1. Divide long text into several chunks ; 2. Perform preliminary processing on each block and generate embedding through the self-attention mechanism H i .

[0155] The preliminary processing here includes dividing the long text into several blocks, segmenting and embedding each block, and using block embedding to fuse the adjacent text block embeddings. Through the self-attention mechanism or weighted averaging, the model merges the semantic information of different blocks, so that the generated content can reflect the common semantics of multiple blocks.

[0156] 3. In the high-level merging process, adjacent context blocks are gradually merged into a larger block. The merged embedding can be expressed by the following formula: , Here, g is the merging function, which may include weighted averaging, concatenation or other processing methods.

[0157] In the HOMER model, high-level usually refers to the incorporation of larger context units in document generation, including but not limited to: Merge sentences to form paragraphs: Merge multiple independently generated sentences into a logically coherent paragraph. The merging here usually needs to consider the semantic relationship between sentences (such as cause and effect, progression, etc.).

[0158] Merge paragraphs to form chapters: Merge multiple paragraphs to form a larger chapter unit. At this time, you need to analyze the relevance of the paragraph topics and generate transition content between the paragraphs.

[0159] Merge chapters into a complete document: At the document level, the chapter contents are integrated into a coherent whole to form the final generated document. When merging, it is necessary to ensure the logical consistency between chapters and generate a global introduction and summary.

[0160] Therefore, the high level is relative to the lower level sentence or word processing, and mainly refers to the merging of context units at the paragraph, chapter and document levels.

[0161] 4. Further context integration is performed on the merged blocks to ensure the consistency of global context information.

[0162] When performing high-level integration, large language models (such as GPT) not only refer to block embeddings, but also combine the specific text content and background knowledge of each block to ensure that the generated text is globally logically consistent, semantically fluent, and meets the task objectives, including the following steps: Weighted embeddings and contextual adjustments: Chunk embeddings are weighted based on the importance of each chunk, the relevance of the topic, and the connection between the context, so that the merged chunks are more in line with the global context; for example, some chunks may be more important than others in a certain chapter, and the model will adjust based on this.

[0163] Background knowledge injection: During the integration process, background knowledge is injected through two channels: Global background knowledge: The large model introduces standard knowledge (such as laws and regulations, industry specifications, etc.) in the long-term memory bank, making text generation more in line with real-world standards and requirements.

[0164] Local background knowledge: For a specific block, the system can also introduce knowledge in a specific field to supplement and improve the information based on the content of the block.

[0165] Generate logical and thematically consistent text: Finally, the integrated text will be further adjusted and optimized by the model to ensure the thematic consistency, logical fluency and professionalism of the text. The system will adjust the output according to the theme, target task and context of the paragraph or chapter to ensure that the text smoothly and accurately expresses the information.

[0166] Specifically, at each generation level, by integrating multimodal data forms such as text, images, and tables, we ensure that the content of the generated documents is not only rich and diverse but also complies with power industry standards, further improving the structural integrity of the documents and the effectiveness of information communication.

[0167] Furthermore, during the generation process, the needs of paragraph or chapter content are analyzed to determine whether it is necessary to embed multimodal data such as images and tables to enhance the comprehensiveness of information expression; for example, when generating equipment operation reports, relevant tables, drawings or images are dynamically called and embedded to more intuitively display the operating status and technical details of the equipment. By associating with multimodal information in the long-term memory library, the most appropriate multimodal data fragments are identified (these multimodal data fragments are vectorized pictures or tables, and the closest ones are found by matching the similarity with the text vectors, and then the original pictures or tables are inserted), and naturally combined with the text content, so that readers can understand the content from multiple perspectives, rather than relying solely on text descriptions, thereby effectively improving the readability and professionalism of the document.

[0168] In this scenario, the large language model will analyze the input text content, understand the complexity of the text, its structural requirements, and the needs of data presentation, and then determine whether multimodal data such as images and tables are needed to enhance the text expression based on these features. This process is described in detail below.

[0169] (1) Analyze the content requirements of paragraphs or chapters The large language model analyzes the following aspects to determine whether an image or table needs to be embedded: 1) Complexity of content Data-intensive content: When a paragraph or chapter involves a large number of numbers, statistics, parameter comparisons, experimental results, etc., a table is usually the most appropriate way to present this information. The model will determine whether it is necessary to convert it into a table based on the amount and complexity of the data; for example: if the text describes different performance indicators of multiple devices (such as power, efficiency, age, etc.), the model may think that a table is needed to clearly compare these data.

[0170] Visualization requirements: When the paragraph description involves spatial relationships, flow charts, or system architecture, images (such as flow charts, structure diagrams, schematic diagrams, etc.) are necessary, because this information is difficult to describe clearly in words in the text, and images can provide a more intuitive understanding; example: If the paragraph involves the workflow of the power system, the model may determine whether it is necessary to generate and embed a system flowchart based on the context requirement of "power system flowchart".

[0171] 2) Limitations of language expression Expression of abstract concepts: For some abstract concepts or complex processes, simple text descriptions may be difficult for readers to understand. At this time, charts can provide intuitive help; Example: When describing the installation process of power equipment, there may be complex steps and details, and the text expression may not be clear enough. At this time, a flowchart can be used to clearly display each step of the operation.

[0172] 3) Structured information requirements Structured data presentation: If the data contained in a paragraph needs to be presented in a list, comparison, or time series format, the model will determine whether it needs to be presented in a table. Example: When describing the regular maintenance cycle of power equipment, the model will recognize that this type of information is suitable for presentation in a table, which includes the equipment name, inspection frequency, maintenance content, etc.

[0173] (2) Specific steps to determine whether a chart is needed to assist in display 1) Extraction and analysis of content topics Analyze task objectives: The model extracts the main task objectives and themes of a paragraph or chapter and identifies whether the task requires a chart or image to assist in expression. For example, if the content of a paragraph is about the comparison of device performance, the model will determine that the content can be presented in a table.

[0174] Paragraph analysis: The model analyzes sentences and keywords in a paragraph to identify whether it involves information that needs to be presented in a structured manner, such as data, processes, comparisons, time series, etc. For example, for a paragraph containing keywords such as "comparison", "changing trends", and "system processes", the model may consider that a chart is helpful in presenting the content.

[0175] 2) Data and information density in a sentence or paragraph Identify data volume: The large language model analyzes the density of data mentioned in the paragraph, such as "equipment efficiency", "failure rate" and other quantitative data. When there is a lot of data, the model will suggest using a table to improve the readability of the information.

[0176] Image requirement analysis: For paragraphs describing spatial relationships, system layouts, or processes, the model will determine whether images are needed to aid understanding. For example, if a paragraph describes a complex equipment installation process or the architecture of a power system, the model will recommend inserting an image to improve clarity.

[0177] 3) Language model generation suggestions Based on the above analysis, the model generates a decision whether a chart needs to be inserted and generates or suggests the type of chart to be inserted as needed.

[0178] (3) Embedding multimodal data When the model determines that a chart or table needs to be inserted, it calls a multimodal image or table library. The library stores various standard charts, tables, and diagrams. After vectorization, the charts and tables can be retrieved, matched, and called in an efficient way. The matching is performed through the following steps: 1) Generate vectorized representation of content requirements Requirements analysis: The large model will analyze the content of the current paragraph and identify chart requirements, such as: A performance comparison of the equipment needs to be shown.

[0179] The step-by-step process of equipment installation needs to be demonstrated.

[0180] The relationship between failure rate and maintenance period needs to be shown.

[0181] These requirements are converted into vector representations that capture the semantics and structure of the requirement.

[0182] 2) Retrieval and matching of chart library Similarity calculation: The model matches the generated demand vector with all the charts or tables in the chart library. This step determines the matching degree by calculating the cosine similarity or other distance metrics between the demand vector and the vector of each chart / table in the library; Example: If the current paragraph needs to show the device performance comparison, the model will retrieve the tables related to "device performance comparison" from the table library, calculate the similarity between these tables and the input requirements, and find the best matching table.

[0183] 3) Choose an appropriate chart or table Based on the matching results, the model will select the most appropriate chart or table. The selection criteria include: content matching, format matching and display effect; 4) Creation and insertion of charts or tables Chart generation: For some specific data, the model may generate new charts, such as generating a line chart or a bar chart based on the data in a paragraph.

[0184] Insertion of charts or tables: The generated or selected charts / tables will be embedded in the text and displayed with the corresponding content. The description in the text will be consistent with the content of the chart / table to ensure that readers can understand the data or process in the text based on the chart.

[0185] Furthermore, after the multimodal data is embedded, the images, tables and other elements are carefully typeset according to the standards of the power industry documents to ensure that the data layout meets the requirements of the specifications. At the same time, a brief description text will be automatically generated for each multimodal element (i.e., multimodal data) to help readers quickly understand the core information of the chart or image; these descriptions not only include an overview of the chart content, but also indicate its specific use and significance in the current document. Through the typesetting and annotation of multimodal data, it is ensured that the document has both visual hierarchy and complies with industry standards, so that the professionalism and readability of the content are further improved, thereby meeting the high-standard information presentation requirements of the power industry in documents such as construction reports and equipment operation records.

[0186] The detailed layout here is as follows: (1) Document structure analysis, structural tagging of text and multimodal elements Perform structural analysis on the document and mark out elements such as titles, paragraphs, images, tables, etc., so that it can be laid out according to the type and hierarchical relationship of the content.

[0187] (2) Application of templates and styles Template usage: Automatic typesetting can be based on templates, which define the position, spacing, alignment, etc. of each element of the document. These templates can be designed according to the standards of power industry documents to ensure that all generated documents meet the specifications. For example, in power industry documents, tables and images usually have fixed alignment, and the width and height of the table are also specified. The automatic typesetting system will adjust these parameters according to the template.

[0188] (3) Automatic adjustment of page margins and line spacing Automatically adjust page margins and line spacing: The automatic typesetting system will automatically adjust the document's page margins, line spacing, paragraph spacing, etc. according to the standards of power industry documents, making the entire document look neater and more compliant.

[0189] In the process of iteratively generating document content layer by layer, by gradually progressing at the sentence, paragraph and chapter levels and dynamically calling the long-term memory library and contextual background knowledge, we ensure that the document content is logically coherent, accurate and in compliance with industry standards.

[0190] Furthermore, in the document generation process, starting from the sentence level and gradually building upwards, the logical consistency of content generation is ensured by dynamically calling relevant knowledge in the long-term memory bank at each level; first, the sentence level focuses on accurately presenting basic concepts and specific details to ensure the high relevance of sentences and contextual information; then, entering the paragraph level, relevant sentences are integrated, and background knowledge related to the paragraph topic is introduced to enhance the logical consistency and information integrity of the paragraph; finally, at the chapter level, the overall structure of the chapter content is analyzed, each paragraph is integrated and high-level background knowledge is introduced to ensure that the chapter content is coherent and organized; by dynamically calling relevant knowledge layer by layer, high-quality information transmission and logical construction are guaranteed in the entire document generation process.

[0191] The overall structure of the analysis chapter content here includes: (1) Chapter topic extraction and analysis In order to determine the core theme of the chapter and ensure that each paragraph and sentence serves the theme, the large language model will analyze the beginning, middle and end of the chapter to extract the subject words and keywords. Usually, the chapter title, paragraph subheadings and the first paragraph of the chapter will provide clear clues. The model can also determine the theme of the chapter through text classification or keyword extraction; for example: if the chapter title is "Installation and Maintenance of Power Equipment", the system will extract relevant keywords from the content of the chapter, such as "Equipment Installation", "Technical Requirements", "Maintenance Frequency", etc., to confirm that each part of the content meets this theme.

[0192] (2) Logical relationships between paragraphs and sentences In order to ensure that there is a clear logical connection between paragraphs within a chapter and avoid information gaps or duplication, the model will analyze the transitions and connections between paragraphs to ensure that each paragraph has a clear position and role in the chapter structure. The system uses natural language processing technology or large models to identify the relationships between paragraphs (such as progressive relationships, contrast relationships, causal relationships, etc.) and optimize the connections between paragraphs based on these relationships; for example: if paragraph 1 introduces "equipment installation standards" and paragraph 2 discusses "equipment commissioning and inspection", the model will ensure that the relationship between the two is clear and may insert transition sentences, such as "After the equipment is installed, the equipment needs to be debugged and inspected to ensure its normal operation." (3) Hierarchical structure of chapters To ensure that the chapter content is organized in a logical order and has a clear structure, at the chapter level, the model integrates paragraphs hierarchically; common chapter structures include: Introduction: This summarizes the main content of the chapter, usually including the background of the problem, the objectives and the general flow.

[0193] Main content: includes specific technical details, data analysis, step-by-step instructions, etc., divided into logical and chronological sections.

[0194] Conclusion: A summary of the chapter content, possibly with suggestions for further research or practice.

[0195] For example, in the chapter on installation and maintenance of power equipment, the model will integrate the equipment installation standards, commissioning steps, maintenance frequency, etc. into the main part, and may add equipment performance evaluation or long-term maintenance recommendations at the end.

[0196] (4) Completeness and coherence of information within the chapter In order to ensure that the content of each paragraph is self-consistent and each paragraph has its reasonable position in the overall chapter, and to avoid content duplication or deviation from the topic, the large language model dynamically analyzes and adjusts the chapter structure to ensure the integrity of the paragraph content. The detailed description within the paragraph should be highly relevant to the topic and logically consistent with the previous and next paragraphs. For example: If a paragraph describes "the installation standard of power equipment", the next paragraph may describe "how to check the compliance of equipment installation". There should be a smooth transition between the two to ensure that information is not missed and the topic is consistent.

[0197] The retrieval methods for high-level background knowledge and low-level background knowledge are similar. The difference is that the low-level background knowledge is matched with low-level topics and keywords, while the high-level background knowledge is matched with high-level subjects and keywords. After matching the high-level and low-level memory fragments, the large model will perform text processing, and GPT has strong summarization capabilities.

[0198] At the chapter level, the specific steps to analyze the structure of the chapter content and introduce high-level background knowledge are as follows: Analyze chapter structure: By analyzing the paragraph level, determine the core content of the chapter and the relationship between the paragraphs (such as progression, cause and effect, contrast, etc.). At the same time, the model will extract information such as chapter titles and paragraph subheadings to help identify the theme of the chapter.

[0199] Calling background knowledge: Based on paragraph content: Based on the topic of each paragraph, the model extracts relevant background knowledge from the long-term memory bank. For example, when describing the maintenance of power equipment, the model can call "equipment maintenance standards" as background knowledge to ensure that the content is complete and in line with industry standards.

[0200] Global background knowledge: When generating a chapter as a whole, the system will extract global background knowledge from the library based on the theme of the chapter, such as the latest regulations and technical standards in the power industry, to ensure that the chapter content is not only in-depth but also meets the latest industry requirements.

[0201] Integrate generated content: Combine the generated content of each paragraph with high-level background knowledge to ensure that each part of the content is logical and complete. Each part of the chapter will be connected to the previous and next paragraphs and information to ensure that the entire chapter has good coherence.

[0202] Furthermore, after the chapter generation is completed, the entire document content is automatically verified according to the standards of the power industry with the help of Wenxinyiyan or GPT, including logical consistency, information integrity and professionalism. The verification process will check whether each chapter meets the document requirements of the power industry, such as the standards for construction reports and equipment operation records, by comparing industry specifications and document content structure. In addition, optimization adjustments will be automatically made to modify parts with inconsistent structures or lengthy contents to ensure that the document meets high-standard hierarchical requirements when it is finally output; verified and optimized documents can meet the power industry's needs for complex documents, achieve logically rigorous, comprehensive content and highly professional output, and make it highly readable and reference value in practical applications.

[0203] Step 303: The document review agent receives the generated power document, performs compliance review and quality assessment on the power document, and generates a review report; The specific steps include: Step 3031: semantically parse the generated power document data to obtain a document semantic representation; In this embodiment, the power document data includes equipment failure reports, construction progress reports, audit files, etc. The documents usually use natural language to describe multiple events and their associations.

[0204] In this embodiment, a large language model (LLM), such as the GPT series model, is used for deep semantic analysis. LLM can capture the complex context and long-distance dependencies in the text, deeply understand the semantics of the event, and generate high-quality semantic vector representations.

[0205] The model will first process the document content into sentences, that is, divide the document into sentence units, so as to ensure that each sentence can be semantically parsed independently. Because the events and information in power documents are often scattered in multiple sentences, sentence processing can ensure that the semantic information of each sentence can be fully extracted.

[0206] The context-related capture model is used to deeply extract the core semantics and implicit causal information of each sentence, which can be expressed as follows: , in, Indicates that the input text is analyzed through a large language model The generated hidden semantic vector not only contains the basic semantic information of the sentence, but also combines the association between events in the context to form a representation with semantic and causal information. This semantic representation is crucial for event recognition and causal reasoning unique to power documents, because events in power industry documents often have complex backgrounds and technical terms, and LLM can effectively identify and parse this information.

[0207] Step 3032: Based on the semantic representation vector, extract events from each sentence identified in the document to obtain semantic representations of the events and context information of the events; Encode each extracted event to generate a semantic representation of the event ; The goal of event extraction is to find those events or behaviors from the document that are critical to causal reasoning. For example, in a power equipment failure report, common key events may include "equipment failure", "equipment maintenance", "fault repair", etc.; in a construction progress report, it may involve events such as "construction delay", "project progress", and "resource scheduling". Each event is not just a simple word or phrase, but also involves the semantic context of the event and its potential relationship with other events. LLM encodes the semantic meaning of events by generating semantic vectors. The system encodes these events to prepare for the subsequent causal reasoning steps.

[0208] During event recognition, the system uses contextual information to enhance the representation of events. .

[0209] For example, an event "equipment failure" identified in a sentence may mention related events such as "power outage" or "maintenance" in subsequent sentences. When constructing event vector representations, the system combines this contextual information to enhance the expressiveness of each event node, so that the representation of each event contains not only the independent information of the event, but also its potential association with other events. This process enables the semantic vector representation to describe the causal relationship between events at a deeper level.

[0210] In order to ensure the accuracy and completeness of the semantic representation of events, each event is encoded. This process is carried out by combining the contextual representation of the event with its own semantic representation to generate a comprehensive event representation vector. This can be described by the following formula: , in, represents the semantic representation of the event, while is the semantic representation of its context. The system combines these two pieces of information to generate a comprehensive event representation , which contains both the information of the event itself and the relationship and context of the event with other events in the document.

[0211] In this way, the system is able to extract structured event nodes and their associated semantic information from power documents. These event nodes and semantic representations provide a key foundation for causal reasoning, enabling the system to build and verify causal chains in subsequent steps.

[0212] Step 3033: constructing an initial event causal chain based on event semantic representation; The introduction of the hypergraph model enables the system to capture more complex multi-cause and multi-effect relationships in documents, especially the complex associations commonly found in power industry documents. For example, an equipment failure event may lead to multiple results at the same time, such as power grid outages and project delays. At the same time, multiple events (such as equipment aging and insufficient maintenance) may jointly lead to a result (such as equipment failure). The hypergraph model can simulate complex causal structures well.

[0213] The specific steps include: Step 30331: Add each event node in the document to the causal hypergraph model to identify the causal relationship of the events in the document; Each event node represents a specific event, while a hyperedge represents the relationship between multiple nodes. Unlike traditional graph models, the hypergraph model allows one edge to connect multiple nodes, so that it can represent a situation where multiple events jointly lead to a result. The system uses the event semantic representation generated in step 1 to combine the event with the hypergraph structure, establish corresponding node relationships for each event, and enable the model to better capture the complex causal chain structure in the document.

[0214] Step 30332: Use a causal reasoning algorithm (such as CausalNLP or DoWhy) to perform causal reasoning on these event nodes to construct event causal relationships; The goal of causal reasoning is to reveal the causal dependencies between events by analyzing the relationships between nodes and constructing causal chains. For example, the system can deduce the causal relationship between "grid outage" and "project delay" from the "equipment failure" event. Specifically, the causal reasoning algorithm analyzes the timing and association of each event and generates causal relationships using a causal reasoning model. This reasoning process uses the generation capability of a large language model (LLM) to express the reasoning results as a natural language causal chain description, making the logic of the causal chain clear and understandable.

[0215] Step 30333: perform deep modeling on the causal relationship, identify the core events in the causal chain and the relationship between the events, and construct the initial event causal chain; In order to further model the causal chain in depth, a hypergraph convolutional neural network (HGCN) is used to perform convolution calculations on the causal network. The introduction of HGCN can effectively handle the multi-node association problem in the hypergraph, and aggregate and learn the features of event nodes and their hyperedges through convolution operations.

[0216] Specifically, the aggregation and propagation of node features are achieved through the following formula: , in, Represents the hypergraph convolutional network in l The node representation obtained by the layer is is the incidence matrix of the hypergraph, and are the degree matrices of nodes and hyperedges respectively, is the weight matrix, To learn the parameters, is the activation function.

[0217] The hypergraph convolutional network can capture the complex associations between event nodes in a document. Through multi-layer convolution calculations, HGCN can learn the deep features of event nodes in the hypergraph structure, and transfer and aggregate these features between hyperedges, thereby learning the global causal structure in the document in the hypergraph.

[0218] After multiple layers of hypergraph convolution, the system can capture the complex causal chains and deep relationships between nodes in the document, and then identify the core events in the causal chain and their associations. For example, the system can not only identify the direct causal relationship of "power grid outage" caused by "equipment failure", but also identify intermediate events in the causal chain, such as "improper maintenance" may be one of the causes of equipment failure. Through multiple layers of feature aggregation operations, the hypergraph convolutional network can effectively mine the key nodes and paths in complex causal relationships, ensuring that the system can fully understand the causal logic in the document.

[0219] Step 3034: verify the constructed causal chain, and use the verified causal chain as the final causal chain; In this process, the system first needs to extract events and their associated paths from the hypergraph model. The hypergraph model has identified potential causal relationships through multi-node association and convolution operations, and the goal of this step is to further refine and verify the causal chain so that the final output causal chain is not only logically clear, but also accurately reflects the actual causal relationship in the document.

[0220] During the specific processing, the system will start from each central event (such as equipment failure) and gradually expand outward to find other nodes associated with the event. Through the hyperedges in the hypergraph, the system can identify causal paths where multiple events may work together. For example, equipment failure may not only cause a power outage, but also cause a project delay. In this case, the causal chain will contain multiple intermediate links, describing the complete path of the event from the cause to the final result. The generated causal chain not only contains direct causal relationships, but may also reveal hidden multi-level causal relationships.

[0221] Furthermore, the system uses causal reasoning algorithms to verify the generated causal chain.

[0222] During the verification process, causal reasoning algorithms (such as CausalNLP or DoWhy) mainly check whether the logic in each causal chain is reasonable and exclude irrelevant or weakly causally related paths. By analyzing the timing and semantic information of each event node and its associated nodes, the system can evaluate the rationality of the causal chain. If the system finds that the association paths between certain events have weak causality, or that some links in the association paths are logically contradictory or inconsistent, the system will automatically exclude these paths to ensure that the causal chain finally identified is logically clear and reasonable. When verifying the causal chain, the system relies on the core formula of causal reasoning for calculation. The system uses the following formula to calculate the causal strength between each event in the causal chain: , in, X, Y, Z Represents different events or variables,X An event or variable that represents an external intervention, usually one that controls or influences. Y Represents a result event, that is, the target event of interest, Z Represents an intermediate variable or covariate, that is, a variable that may affect the causal chain. X and Y Other factors in the relationship between Indicates external intervention Events, events The probability of occurrence, Denotes a "human intervention" or "influence" operation used to eliminate confounding factors in order to assess the effect of pure cause The causal effect of Indicates an event Z The probability of taking the value z, Express Z The sum of the conditional probabilities of all possible values ​​of .

[0223] For example, when X Indicates a device failure. Y Represents project delay. Through this formula, the system can estimate the impact of intervening in a certain event on other events and evaluate the credibility of its causal chain. The system can determine whether there is a clear causal relationship and the strength of the relationship by calculating the causal effect of "equipment failure" on "project delay".

[0224] During the verification process, the system will also carefully analyze the intermediate links in the chain. For example, if an intermediate event in the causal chain (such as "insufficient equipment maintenance") is the common cause of multiple final results (such as "project delay" and "grid outage"), the system will further analyze these multiple causes and multiple results to ensure that each link in the causal chain has a reasonable explanation.

[0225] In addition, the system will combine the structural information in the hypergraph model to optimize and simplify the redundant paths in the causal chain. For example, if there are different association paths between multiple events, the system will select the most logical and realistic path and prune the redundant paths. This process is completed through further reasoning and calculation of the causal reasoning algorithm to ensure that the final causal chain is both complete and concise.

[0226] Step 3035: Conduct compliance review on the causal chain; The core of this step is to compare the generated causal chain with the latest laws, standards and operating procedures of the power industry through the compliance rule engine to ensure that the events and their associations in each causal chain comply with industry standards. Compliance review is particularly important for the power industry, because operations involving safety, construction, maintenance, etc. must strictly comply with relevant laws and regulations to avoid potential safety hazards and legal risks.

[0227] The specific steps include: The system calls the compliance rule engine, which contains the latest laws, standards, and operating procedures of the power industry. These laws and regulations may cover all aspects from equipment operation to construction management. For example, the safety standards of the power industry, the operating specifications of power equipment, the requirements of construction management, and other important laws and regulations involving project compliance. By storing these laws and regulations in a structured manner, the rule engine can automatically compare them with the causal chain to determine whether the events in the chain meet the relevant legal requirements.

[0228] During the compliance check, the system reviews each event and its relationship in the causal chain one by one. To ensure that each step in the chain complies with industry regulations, the system performs specific rule checks for each event.

[0229] For example, in the causal chain caused by power equipment failure, the system will check whether the relevant equipment maintenance standards have been followed, whether regular inspections have been carried out as required, and whether safe operating specifications have been met during the event handling process. The system uses a formula to calculate whether each event complies with industry regulations, which is expressed as: , in, Indicates an event Compliance score, Indicates an event Regulatory standards The degree of conformity between Through this formula, the system can quantify the degree of match between each event and relevant regulatory standards, thereby determining whether it is compliant. When the compliance score is high, it means that the event node meets most regulatory requirements; otherwise, it indicates that the event may have a risk of violation.

[0230] For each pair of event associations in the causal chain, the system also performs compliance checks for logical consistency. The system verifies whether these relationships meet the logical requirements of regulations through the rule engine to ensure that the event associations in the causal chain are reasonable and compliant. For example, if a decision in the causal chain fails to fully consider potential risks (such as failure to take remedial measures in accordance with regulatory requirements in the event of a project delay), the system will mark these non-compliant parts through the rule engine and automatically generate prompts to indicate the specific violation clauses.

[0231] Another important function of the compliance rule engine is the ability to provide intelligent correction suggestions based on regulatory requirements in different scenarios. When the system detects that certain events or relationships in the causal chain do not meet regulatory standards, the rule engine automatically generates improvement suggestions based on the best practices of industry regulations. For example, the system may recommend increasing the frequency of inspections during equipment maintenance or adding contingency measures to project plans to reduce risks and ensure compliance.

[0232] Specifically, if the compliance score of an event node If it is below a certain threshold, the system will automatically generate prompts and give rectification suggestions to help users optimize the decision-making process.

[0233] Through the above steps, the system can generate a comprehensive compliance review report for the causal chain. The system not only marks the non-compliant parts, but also provides improvement suggestions to help users correct violations in a timely manner. This compliance review process effectively improves the reliability and security of the causal chain, making the final causal chain more accurate both logically and legally. By combining the natural language description generated by LLM, the system can output detailed compliance check results, allowing users to clearly understand each violation and its rectification measures, providing strong support for the standardized management of documents in the power industry.

[0234] Step 3036: Generate a review report based on the compliance review results; After completing the identification of the causal chain and compliance review, the large language model LLM integrates all the information and generates a detailed review report in a structured manner with the assistance of RAG knowledge base enhancement, etc., to help users fully understand the causal logic and compliance status in the document.

[0235] The specific steps include: Step 30361: extract the causal chain after compliance check; Each causal chain has been verified to ensure its logical consistency and compliance with the regulatory requirements of the power industry. Therefore, the system will build an overall causal structure diagram of the document based on these chains. This causal structure shows the complete path from the initial event (such as equipment failure) to the final result (such as project delay or power grid outage), and marks each intermediate event and its relationship.

[0236] Step 30362: List the compliance assessment results of each causal chain in detail in the report.

[0237] This part not only includes the logic verification of each chain, but also scores and explains the compliance of each event node. Specifically, the system will use the previous compliance check formula , for each event A compliance score is given and it is noted whether the event fully complies with the relevant power industry regulations. If certain events or paths do not comply with regulatory requirements, the system will clearly mark these non-compliant parts in the report and explain the specific clauses that have been violated.

[0238] Step 30363: For the identified non-compliant parts, the system will provide detailed feedback and improvement suggestions.

[0239] For example, if a risk point in a causal chain is not fully considered, the system will automatically generate suggestions through the compliance rule engine to remind users to add more preventive measures when dealing with such issues. The system proposes specific modification plans based on industry best practices and regulatory standards to help users better adjust the content of the document. This part of the report usually includes specific operational suggestions, such as "increase the frequency of regular equipment inspections" or "add emergency measures to the project plan", etc., to ensure that users can effectively deal with potential risks.

[0240] During the report generation process, the system will not only ensure the integrity of the report content, but also support multiple output formats according to user needs. The review report can be exported in multiple formats such as PDF and Word, which is convenient for users to review and save in different environments. In addition, the format and content structure of the report can be customized according to different application scenarios to ensure that the report not only meets the standard requirements of the power industry, but also has practicality and readability.

[0241] Against the backdrop of the accelerated digital transformation of the power industry, as the volume and complexity of documents increase, the system can efficiently process and manage a large number of documents to meet the needs of future intelligent management. Through the identification of causal chains and compliance checks, the system can also reveal potential risks in documents, helping managers better understand the content of documents and the basis for decision-making, thereby providing intelligent support for the management and operation of power systems and reducing risks in operations. These benefits make this patent of great application value, and can help the power industry achieve more efficient, smarter, and more compliant operational management.

[0242] Step 304: Record the data during the execution of each subtask on the blockchain for management and control; The specific steps include: Step 3041, document life cycle record and data mapping; When a document is generated, a unique identifier is first assigned to each document through blockchain technology to ensure the uniqueness and tamper-proofness of the document. Specifically, the system uses a cryptographic hash function (such as SHA-256) to generate a content hash value for the document. , the hash value serves as a unique identifier for the document; The document content hash value is calculated using the following formula: , in, Represents the original content of the document, this hash value It will serve as the unique identifier of the document in the blockchain, ensuring the uniqueness of the document and providing security for subsequent operations.

[0243] Whenever a document is created, modified, or updated, the system will record every operation of the document through the blockchain's smart contract. Each operation will generate a new transaction record, which contains the document identifier, timestamp, operation type (such as creation, modification, deletion), and related information of the operation. For example, the document operation record Toperation contains the following information: , in, is the hash value of the current document, The timestamp of the operation, is the identity of the user performing the operation, The type of operation (e.g. create, modify, delete), The hash of the previous version of the document.

[0244] These operation records will be permanently stored in the blockchain's distributed ledger, and each operation can be traced, ensuring that every step in the document life cycle can be traced and preventing any unauthorized tampering.

[0245] Furthermore, to ensure the integrity and immutability of the document version history, the encryption feature of blockchain technology provides tamper-proof protection for documents. All records of document operations are encrypted and stored in the blockchain to ensure that the data cannot be tampered with by unauthorized third parties. Each block contains the current document operation record and the hash value of the previous block, forming an immutable blockchain.

[0246] The specific encryption process is as follows: , in, is the hash value of the current block, It is a detailed record of the current document operation. is the hash value of the previous block; In this way, each operation record is linked to the hash value of the previous block through a hash function, ensuring that any attempt to modify the historical record will change the hash values ​​of all subsequent blocks, thereby being detected by the system to prevent tampering.

[0247] Furthermore, the system regularly checks the consistency of the document's content and operation records to ensure that the document's operation history on the blockchain is consistent with the actual content. If the document content changes, the system will recalculate the document's hash value and compare it with the hash value stored in the blockchain. If inconsistencies are found, the system will immediately trigger an alarm and mark it as a potential tampering event.

[0248] The document consistency detection formula is as follows: , in, The hash value of the current document, is the hash value of the document stored in the blockchain. If , indicating that the content of the document is inconsistent with the record stored on the blockchain, and the system will mark the document as a potentially tampered document.

[0249] Users can trace the complete operation history of a document by querying the blockchain, including detailed information on each generation, modification, approval, etc. The system provides a document traceability interface that allows users to query the operation history of a document by its hash value.

[0250] The query formula is as follows: , Through this query, users can obtain all historical operation records of the document, including operation time, operator, operation type and other information. This ensures that the document management process is fully transparent and traceable.

[0251] By combining blockchain technology with encrypted hash algorithms, the present invention realizes the complete traceability and tamper-proof functions of documents. Each operation of document generation, modification, approval, etc. will be recorded on the blockchain to form an unalterable operation history chain. Blockchain technology ensures the transparency and security of document operations through its distributed ledger and encryption characteristics, and prevents any unauthorized tampering. The system ensures the consistency of document content and operation records through a consistency detection mechanism, and can promptly detect and respond to potential tampering events. This solution provides a highly secure, transparent and intelligent document management system for the power industry, greatly enhancing the credibility and compliance of document management.

[0252] Step 3042: By combining the smart contract and multimodal model of blockchain, dynamic permission allocation and fine-grained access control of document operations are achieved.

[0253] The system manages user access rights through smart contracts to ensure that each user can only access and operate authorized documents. During each document operation, the system verifies user permissions through smart contracts to determine whether they meet the operation scope of the user role. If the user permissions meet the requirements, the operation is executed, otherwise the operation is rejected. The system uses blockchain technology to ensure the transparency and immutability of permission data, ensuring that each operation complies with the authorization rules.

[0254] First, the system dynamically assigns different permissions to each user based on the attributes of the document, the user's role, and the requirements of the task. Each user's role and permissions are defined and managed through smart contracts, and the system can automatically adjust the user's permissions based on the role. For example, if user U attempts to access document D and perform operation O, the system will first check whether the user has the permission to perform the operation.

[0255] This process is expressed by the following formula: , in, Represents a certain approval step or operation type (such as view, modify, delete). User is the user of the current operation. A result of 1 indicates that the user has the permission to perform the operation, and a result of 0 indicates that the user does not have the permission to perform the operation. On the basis of permission management, the system further adopts a multimodal model to refine access control. Especially when processing documents containing multiple modal data (such as text, images, audio, etc.), the system can dynamically assign access rights to each modal data based on the sensitivity of the document and the specific needs of the user. For example, the system may allow users to view the text part of the document, but not allow access to the image or audio part. The access control of each modality can be expressed by the following formula: , in, Represents the access rights of user U to modality M (such as image, text, or audio). Through this fine-grained control, the system can assign different access rights to each modality to ensure that users can access documents reasonably according to their permission scope.

[0256] Step 3043: Design and define the document approval process, using smart contract technology to automate the management of approval steps, role allocation, and time nodes.

[0257] In the smart contract, each step of the document approval process is defined as a "state", which are different stages of the document from submission to final completion, such as "pending review", "review passed" or "review rejected". At the same time, the document approval process is defined as a directed graph, in which each node represents an approval step, and the edge represents the transfer between steps. In order to ensure the smooth progress of the document approval process, the association between the state and the directed graph can be achieved through state transition and conditional judgment to control the transfer of documents from one stage to the next.

[0258] Specifically, the association between states and directed graphs and how to control the steps of the approval process can be completed through the following steps: Step 30431, status definition of each step Each approval step can be defined as a state. In the smart contract, each state is represented by a unique identifier (ID), and the change of state is determined by the approval result. Assume that the document approval process has three steps: S1: Submit for review (“pending review”), S2: Pass the preliminary review (“pass the preliminary review”), S3: Final approval (“final approval”) Each step is a status that represents a different stage of the document in the approval process.

[0259] Step 30432: Definition of directed graph A directed graph represents the relationship and transfer order of different steps in the approval process. Each node in the graph represents a step, and each edge represents the state transfer from one step to another. For example, a document first goes from "Submit for Review" to "Preliminary Review Passed" and then to "Final Approval".

[0260] The document approval process can be expressed as: S1 (submit for review) --> S2 (preliminary review passed) --> S3 (final approval); Step 30433: State transition control Smart contracts control the transition of documents from one state to another by defining the rules for state transitions. These transition rules can be determined by approval conditions to determine whether the document state transfer is allowed. For example: Status transition conditions: A document can only enter the final approval stage after passing the preliminary review, and only after the final approval can it be determined whether the document is complete. Each status transition has corresponding conditions, for example, "preliminary review passed" can enter "final approval".

[0261] In smart contracts, these conditions are implemented through if statements and state judgments: / / Smart contract pseudocode example enum ApprovalStatus { Submitting, InReview, Approved, Rejected} ApprovalStatus public currentStatus; The smart contract will check the conditions between each step to confirm whether the approval is completed. If not, the system will remain in the current step; if the approval is completed, it will proceed to the next step.

[0262] The recursive relationship of the smart contract execution process can be expressed by the following formula: , in, is the status of the current step, is the approval status of the current step (Approved or Rejected), It is the next step after the approval is completed.

[0263] Furthermore, in each approval step, the system automatically assigns approval roles and permissions through smart contracts. Each role (such as "initial reviewer", "auditor" or "final approver") is automatically assigned according to the complexity and requirements of the document, and only users corresponding to the role can perform related operations.

[0264] Smart contracts ensure that each user can only perform operations related to their functions by defining the permissions of roles. Each approval node is assigned a role, and role verification and permission control are implemented through contract code. For example, in the "Submit for Review" step, only users assigned to the "Submitter" role can perform the submission operation. Role assignment and permission verification are performed using the following formula: , in, Represents a certain approval step (such as submission for review, preliminary review, final approval, etc.). User is the user of the current operation. A result of 1 indicates that the user has the approval authority for this step, and a result of 0 indicates that the user does not have the authority to perform this step. Each approval step is managed by a smart contract to ensure that each step is completed within the specified time. If a step is not completed within the specified time, the system will automatically trigger a reminder or reassign the task. The time management formula is as follows: , in, is the completion time of the approval step (i.e. the difference between the current time and the due time), is the current system time, is the time cutoff point for this approval step. If If the preset time threshold is exceeded, the system will trigger the preset reminder mechanism to inform the relevant approval personnel to complete the approval in a timely manner.

[0265] Furthermore, after each document approval operation is completed, the system will record the approval result (such as approval or rejection), approval time, operator and other information through smart contracts, and write these records into the blockchain. The distributed ledger technology of the blockchain ensures that these records cannot be tampered with, increasing the transparency and traceability of the approval process. The record of each approval operation can be expressed as: , in, is the hash value of the current document, is the current approval step (e.g. "Submit for Review", "Preliminary Review", "Final Approval"), is the timestamp of the operation, The person who approves the operation. The approval results (such as "approved" or "rejected") will be written into the blockchain to form a historical record chain of document approval, and the details of each approval operation can be traced and verified.

[0266] Furthermore, the distributed ledger technology of blockchain ensures the transparency and immutability of all approval operations. Each approval operation is encrypted and recorded in the blockchain to prevent unauthorized modification while ensuring the integrity and credibility of the approval record. Each approval operation in the blockchain contains the following information: , in, It is a detailed record of the current approval operation. is the hash value of the previous block, It is the hash value of the current block, ensuring that each block is linked to the previous block, thus preventing data tampering.

[0267] By combining smart contracts and blockchain technology, the present invention realizes the automated management and transparency of the document approval process. Smart contracts automatically control the approval steps, role allocation and time nodes to ensure that the approval process is completed in a timely manner according to the predetermined process. Blockchain technology ensures the immutability and transparency of each approval operation, and each approval record can be traced, which enhances the efficiency, compliance and security of document approval in the power industry. Through the combination of smart contracts and blockchain, the document approval process has not only become more efficient and automated, but also ensured the security and credibility of the approval process, providing an innovative and efficient document management and approval system for the power industry.

[0268] Step 4: Each agent executes the corresponding subtask and follows the set execution path, sharing the results of the executed tasks with other agents to collaboratively complete complex tasks and finally integrate them to obtain collaborative processing results.

[0269] In this embodiment, the subtasks are decomposed layer by layer into meta-task sequences, the meta-task sequences are decomposed into atomic tasks, and the execution path is determined by combining the meta-task sequences and the atomic tasks; In order to efficiently handle complex tasks, the system adopts a multi-level planning task refinement and decomposition method, decomposing subtasks layer by layer into meta-task sequences and atomic tasks, and refining tasks through different levels. Tasks at each level depend on the completion of tasks at the upper level, and gradually break down complex tasks into smaller and more specific execution units.

[0270] Among them, the relationship between subtasks, metatasks and atomic tasks is: Subtask: Subtasks are usually larger task units obtained during the decomposition of complex tasks, representing a specific business goal or functional block. Subtasks may contain multiple submodules and usually involve higher-level goals or processes. Subtasks are usually the first level of task decomposition, containing relatively independent work units.

[0271] Meta-task sequence: Meta-task is a task level after the sub-task is further split, which divides the task into more structural, functional or logical parts. Meta-task sequence represents a functional module in the task, which is usually composed of multiple atomic tasks. Each meta-task can be regarded as a larger module of a business operation, which is further subdivided into a series of specific operation steps.

[0272] Atomic tasks: Atomic tasks are the most detailed units of task decomposition, representing the most basic operations or smallest functional modules in a task. Atomic tasks are usually operations that can be executed directly, such as a database query, a data processing step, a small-scale calculation, etc. Each atomic task has a clear execution goal, which is usually an operation that the system can execute independently.

[0273] In multi-level task planning, the process of decomposing tasks from top to bottom is detailed layer by layer. Specifically: The first layer (meta-task layer): decompose complex tasks into multiple meta-task sequences. Each meta-task sequence represents a large functional module, which usually contains a larger range of work objectives. For example, in the process of document generation, the meta-tasks may be "generating the first draft of the document", "reviewing the content of the document", etc. There may be dependencies and sequences between meta-tasks.

[0274] The second layer (atomic task layer): In each meta-task, it is further subdivided into multiple atomic tasks. Each atomic task is a unit that performs a specific small task. For example, under the meta-task of "generating a draft document", there may be atomic tasks such as "extracting device data", "generating document structure", "formatting text content", etc. Each atomic task is executed independently and completes part of the overall document generation.

[0275] In actual execution, the hierarchical relationship of tasks is organized through multi-level task decomposition to ensure that each task can be gradually advanced and the dependencies between each task can be clearly managed. Meta-task sequences and atomic tasks together constitute the specific execution path of the task, thereby ensuring that each step in the entire task chain can be executed efficiently and accurately. Hierarchical planning is expressed by the following formula: , in, Represents the decomposition of tasks at different levels. The levels can be divided into meta-tasks (high-level) and atomic tasks (low-level).

[0276] Optimize the allocation based on the hierarchical relationship of tasks, and adjust the execution order based on the dependencies of tasks. By performing hierarchical decomposition of tasks and task dependency analysis, the system can ensure that the allocation and execution of complex tasks meet actual needs, avoid resource conflicts and improve overall efficiency.

[0277] Task collaboration and information sharing: The multi-agent system works collaboratively by sharing status and data in real time, ensuring that there will be no conflicts in task processing between multiple agents and that the efficiency of task execution can be optimized collaboratively. The system uses a collaborative mechanism to enable agents to quickly exchange information and synchronize task progress. For example, when the document generation agent completes the preliminary document, it passes the document to the document review agent through the shared status, which then performs a compliance check. This process is expressed by the following sharing formula: , in, There are two agents, and the system shares the state To synchronize task progress and ensure collaborative execution of tasks.

[0278] Step 5: Result integration and feedback. The subtask results completed by all agents will be summarized by an integration module, and the final feedback will be provided.

[0279] For example, the preliminary documents of the document generation agent, the compliance report of the document review agent, and the archival records of the document management agent will be integrated into a complete result, which is integrated through the following formula: , in, is the result of the document generation agent, is the result of the document review agent, is the result of the document management agent, It is the complete document or report that is finally generated.

[0280] By integrating the results of different agents, the system can provide a unified and comprehensive document or task processing result to meet user needs.

[0281] Embodiment 2 This embodiment provides a power cross-modal knowledge fusion multi-agent collaborative processing system, including: The instruction parsing module is used to parse the task execution command instructions input by the user; The subtask identification module is used to extract the key elements of the task, decompose the task into multiple subtasks according to the extracted key elements, and identify the logical relationship and priority between the subtasks; The subtask allocation module is used to allocate each subtask to the corresponding agent according to the logical association and priority between the subtasks; The logical relationship and priority between the subtasks are: Based on the acquired knowledge, cross-modal fusion is performed to build a knowledge graph, and the power multimodal data is retrieved according to the user task instructions and the constructed knowledge graph; the power multimodal data is received, and a power document is generated based on the power multimodal data; the generated power document is received, and the compliance review and quality assessment of the power document is performed to generate a review report; the data during the execution of each subtask is recorded on the blockchain for management and control; The collaborative processing module is used for each intelligent agent to execute the corresponding subtask and execute it according to the set execution path, share the executed task results with other intelligent agents, collaboratively complete complex tasks, and finally integrate to obtain the collaborative processing results.

[0282] Embodiment 3 This embodiment provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the above-mentioned method for cross-modal knowledge fusion and multi-agent collaborative processing of electric power.

[0283] Embodiment 4 This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the method for multi-agent collaborative processing of cross-modal knowledge fusion of electric power are implemented as described above.

[0284] Embodiment 5 This embodiment provides a program product, which is a computer program product, including a computer program, characterized in that when the computer program is executed by a processor, the steps in the multi-agent collaborative processing method for cross-modal knowledge fusion of electric power are implemented as described above.

[0285] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0286] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0287] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0288] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0289] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0290] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A multi-agent collaborative processing method for cross-modal knowledge fusion in power, characterized by: The steps include: Parse the task execution command instructions input by the user; Extract the key elements of the task, decompose the task into multiple subtasks based on the extracted key elements, and identify the logical connections and priorities between the subtasks; According to the logical association and priority between subtasks, each subtask is assigned to the corresponding agent; The logical relationship and priority between the subtasks are: Based on the acquired knowledge, cross-modal fusion is performed to build a knowledge graph, and power multimodal data is retrieved according to user task instructions and the constructed knowledge graph; receiving electric power multimodal data, and generating electric power documents according to the electric power multimodal data; Receive generated power documents, conduct compliance review and quality assessment on the power documents, and generate review reports; Record the data during the execution of each subtask on the blockchain for management and control; Each intelligent agent performs the corresponding subtask and follows the set execution path, sharing the results of the executed tasks with other intelligent agents, collaboratively completing complex tasks, and finally integrating to obtain collaborative processing results.

2. The electric power cross-modal knowledge fusion multi-agent collaborative processing method according to claim 1, characterized in that: Based on the acquired knowledge, cross-modal fusion is performed to build a knowledge graph, and multi-modal power data is retrieved according to user task instructions and the constructed knowledge graph, including: Obtain multimodal data for the power industry; Convert the multimodal data of the power industry into a structured representation of knowledge graph nodes and edges; After optimizing and removing duplicate edges between data nodes, the optimized edges between nodes are obtained, and the initial knowledge graph is constructed; Determine the incremental change information based on the newly added multimodal data, and generate the optimized knowledge graph after adjusting the neighborhood optimization and consistency check according to the incremental change information and edge weights; Extract multimodal features, align the multimodal features in the same feature space, and obtain aligned multimodal features; According to the aligned multimodal features, the user task instructions are represented as query features, the query features are matched with the optimized knowledge graph, the nodes most relevant to the query are retrieved, the content and related information of the selected nodes are dynamically analyzed, and the corresponding multimodal response data are generated.

3. The electric power cross-modal knowledge fusion multi-agent collaborative processing method according to claim 2, characterized in that: Determine incremental change information based on the newly added multimodal data, including: Extract semantic feature representation of each modality’s newly added data; The semantic features of the newly added data are calculated with the semantic feature vectors of the corresponding nodes in the knowledge base. If the similarity result is less than the set threshold, the newly added data is marked as a change point that needs to be updated. Determine the change type of the change point, and generate corresponding incremental update information for each change point; The generated incremental update information is added to the knowledge base, the feature vector of the corresponding node is updated, and the associated edges of the node are readjusted.

4. The electric power cross-modal knowledge fusion multi-agent collaborative processing method according to claim 2, characterized in that: According to the incremental change information, edge weights are adjusted for neighborhood optimization and consistency checks, including: Determine the semantic similarity between the newly added node and the existing node. If the similarity exceeds the set threshold, establish an edge between the newly added node and the existing node. When a new node is connected to an existing node, the weight of the edge is adjusted according to the semantic similarity between the new node and the existing node to obtain the edge with adjusted weight; After the integration of new nodes and edges is completed, the affected node neighborhoods in the knowledge graph are optimized to remove redundant or weakly associated nodes; After adjusting the neighborhood structure, check whether the nodes or edges in the knowledge graph are redundant. If so, mark and remove the redundant nodes or edges.

5. The electric power cross-modal knowledge fusion multi-agent collaborative processing method according to claim 2, characterized in that: Align the multimodal features in the same feature space to obtain aligned multimodal features, including: For data of different modes, extract the feature vector of the corresponding mode; The generated multimodal feature vectors are used to construct positive samples and negative samples, and the similarities of the positive samples and negative samples are calculated respectively. When calculating the similarity, the event information and spatial information of the multimodal features are introduced; By maximizing the similarity of positive samples and minimizing the similarity of negative samples, multimodal features are aligned in the same feature space.

6. The electric power cross-modal knowledge fusion multi-agent collaborative processing method according to claim 1, characterized in that: The receiving of the electric power multimodal data and generating the electric power document according to the electric power multimodal data includes: Extract knowledge in the form of multimodal data from historical document data, build a long-term memory bank by simulating the memory mechanism of the human hippocampus with the help of bionics, and store and dynamically update the knowledge in the long-term memory bank in three levels: sentences, paragraphs, and chapters; The task of generating new documents is input into the electric power semantic big model, and the long-term memory is used as the external knowledge source of the electric power semantic big model; through semantic analysis and personalized retrieval algorithm, the knowledge of various levels related to the current task is dynamically searched from the long-term memory; according to the hierarchical order of sentences, paragraphs and chapters, the contextual background knowledge and the knowledge dynamically found from the long-term memory are iteratively integrated layer by layer to finally generate the complete document content.

7. The electric power cross-modal knowledge fusion multi-agent collaborative processing method according to claim 6, characterized in that: The process of building long-term memory includes: Construct a knowledge graph using knowledge as nodes and semantic relationships between knowledge as edges; Drawing on the memory characteristics of the human hippocampus, multiple synonymous and hyponymous relationships are added to the constructed knowledge graph; The nodes are further classified and stored at different levels such as sentences, paragraphs and chapters to obtain an experimental memory library.

8. The electric power cross-modal knowledge fusion multi-agent collaborative processing method according to claim 6, characterized in that: The semantic analysis and personalized retrieval algorithm are used to dynamically search for knowledge of various levels related to the current task from the long-term memory library, specifically: Use natural language processing technology to perform semantic analysis on the content of the task and identify key information, including keywords, core requirements, and important concepts; The similarity calculation between key information and nodes in the library is used to preliminarily identify knowledge nodes related to the current task; Using the personalized PageRank algorithm, based on the initially identified knowledge nodes, other knowledge nodes are retrieved on the knowledge graph of the long-term memory library, and finally knowledge at all levels related to the current task is obtained.

9. The electric power cross-modal knowledge fusion multi-agent collaborative processing method according to claim 1, characterized in that: The receiving of generated power documents, conducting compliance review and quality assessment on the power documents, and generating a review report include: Perform semantic analysis on power document data to obtain the semantic representation of the document; Based on the semantic representation of the document, events are extracted for each sentence identified in the document, and each event is encoded to obtain the semantic representation of the event; Construct the initial event causal chain based on the semantic representation of events; Use causal reasoning algorithms to verify the constructed initial event causal chain, compare each event and the correlation between events in the verified causal chain with the compliance rules of the power industry, calculate the compliance score, and generate correction suggestions for events or correlations between events with compliance scores lower than the set score value; Based on the verified causal chain that complies with the compliance rules of the power industry, the overall causal structure diagram of the power document is constructed, and a review report is generated.

10. The electric power cross-modal knowledge fusion multi-agent collaborative processing method according to claim 9, characterized in that: The event-based semantic representation constructs an initial event causal chain, including: Combine the semantic representation of events with the causal hypergraph model to establish corresponding node relationships for each event; each event node represents a specific event, and the hyperedge represents the relationship between multiple event nodes; Use causal reasoning algorithms to perform causal reasoning on event nodes and construct causal relationships between events; The causal hypergraph model is used to perform multi-layer convolution calculations on the causal relationship network between events. The deep features of event nodes are learned in the hypergraph, and the features are transferred and aggregated in the hyperedges to obtain the complex causal chain in the document and the deep relationship between nodes, thereby identifying the core events and the relationship between core events. The initial event causal chain is constructed based on the identified core events and the relationships between core events.

11. The electric power cross-modal knowledge fusion multi-agent collaborative processing method according to claim 9, characterized in that: The verification of the constructed initial event causal chain by using the causal inference algorithm includes: Check whether the logic in each causal chain is reasonable, and exclude irrelevant or weakly causally related paths based on the calculated causal strength between each event; Analyze the timing and semantic information of each event node and its associated nodes, and delete the paths with logical inconsistencies or contradictions.

12. The electric power cross-modal knowledge fusion multi-agent collaborative processing method according to claim 1, characterized in that: The data during the execution of each subtask is recorded on the blockchain for management and control, including: When a document is generated, a document hash value is generated for each generated power document, and the document hash value is checked for consistency with the document hash value stored in the blockchain. If they are inconsistent, the document is marked as a potentially tampered document; During each document operation, the user's permissions are verified through smart contracts to determine whether the user's operation complies with the assigned permissions. If yes, the operation is executed; if not, the operation is rejected. Deploy a smart contract for document approval, define each step of the document approval process as a state of the smart contract, and define the document approval process as a directed graph, where each node represents an approval step and the edge represents the transfer between steps. The approval process is controlled according to the smart contract to determine the conditions and approval roles of each step.

13. The electric power cross-modal knowledge fusion multi-agent collaborative processing method according to claim 12, characterized in that: Whenever a document is generated, modified, or updated, each operation on the document is recorded through the blockchain’s smart contract. Each operation generates a new transaction record, which includes the document’s identifier, timestamp, operation type, and related information about the operation.

14. The electric power cross-modal knowledge fusion multi-agent collaborative processing method according to claim 12, characterized in that: During each document operation, different permissions are dynamically assigned to each user based on the attributes of the document, the user's role, and the requirements of the task. The role and permissions of each user are defined and managed through smart contracts. When assigning different permissions to each user, access rights to each modal data are dynamically allocated based on the sensitivity of the document and the modality of the user's required data.

15. The electric power cross-modal knowledge fusion multi-agent collaborative processing method according to claim 12, characterized in that: In each approval step, the time node is managed by the smart contract. If a step is not completed within the specified time, a reminder will be automatically triggered or the task will be reassigned. After the approval operation is completed, the approval result, approval time and operator records will be written into the blockchain through the smart contract.

16. The electric power cross-modal knowledge fusion multi-agent collaborative processing method according to claim 1, characterized in that: The process of determining the execution of the set execution path includes: decomposing the subtasks into meta-task sequences layer by layer, decomposing the meta-task sequences into atomic tasks, and combining the meta-task sequences and the atomic tasks to determine the execution path.

17. The electric power cross-modal knowledge fusion multi-agent collaborative processing system is characterized by: include: The instruction parsing module is used to parse the task execution command instructions input by the user; The subtask identification module is used to extract the key elements of the task, decompose the task into multiple subtasks according to the extracted key elements, and identify the logical relationship and priority between the subtasks; The subtask allocation module is used to allocate each subtask to the corresponding agent according to the logical association and priority between the subtasks; The logical relationship and priority between the subtasks are: Based on the acquired knowledge, cross-modal fusion is performed to build a knowledge graph, and power multimodal data is retrieved according to user task instructions and the constructed knowledge graph; receiving electric power multimodal data, and generating electric power documents according to the electric power multimodal data; Receive generated power documents, conduct compliance review and quality assessment on the power documents, and generate review reports; Record the data during the execution of each subtask on the blockchain for management and control; The collaborative processing module is used for each intelligent agent to execute the corresponding subtask and execute it according to the set execution path, share the executed task results with other intelligent agents, collaboratively complete complex tasks, and finally integrate to obtain the collaborative processing results.

18. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in the power cross-modal knowledge fusion multi-agent collaborative processing method as described in any one of claims 1-16 are implemented.

19. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the power cross-modal knowledge fusion multi-agent collaborative processing method as described in any one of claims 1-16 are implemented.

20. A program product, the program product being a computer program product, comprising a computer program, characterized in that: When the computer program is executed by a processor, the steps in the power cross-modal knowledge fusion multi-agent collaborative processing method as described in any one of claims 1-16 are implemented.

Images