Multi-agent layered collaborative optimization method based on knowledge graph and graph neural network

Abstract

The invention discloses a multi-agent layered collaborative optimization method based on a knowledge graph and a graph neural network, and relates to the technical field of machine learning. According to the method, power-on self-test is carried out after software integrity verification, then multi-modal data is preprocessed to convert heterogeneous original data into a unified structured event object, and whether the corresponding candidate entities are divided into the entity candidate index table or not is judged based on the obtained confidence coefficient of the candidate entities; the method comprises the following steps: constructing a causal knowledge graph according to a verification result of a causal relationship of a to-be-verified causal edge, then constructing a space-time heterogeneous graph according to an embedding result of the causal knowledge graph, carrying out hierarchical collaborative optimization processing according to an output node representation matrix, and finally carrying out data twinborn simulation evaluation based on an optimal strategy parameter set. And the optimal strategy parameter set is screened according to the simulation evaluation result, so that the effectiveness of multi-agent hierarchical collaborative optimization is improved, and the problem of low effectiveness of multi-agent hierarchical collaborative optimization in the prior art is solved.

Description

Technical Field

[0001] This invention relates to the field of machine learning technology, and in particular to a multi-agent hierarchical collaborative optimization method based on knowledge graphs and graph neural networks. Background Technology

[0002] Modern large-scale complex systems (such as smart grids, automated production lines, and smart city transportation networks) are evolving towards large-scale, multi-agent, and highly dynamic systems. These systems typically consist of a large number of heterogeneous, autonomously decision-making agents that need to work collaboratively in dynamically changing environments to achieve globally optimal system performance. To achieve this goal, a collaborative optimization framework based on Multi-Agent Reinforcement Learning (MARL) and Graph Neural Networks (GNNs) is employed.

[0003] Existing technologies extract entities and relationships from text and form data to construct knowledge graphs. Subsequently, models such as TransE, ComplEx, or GraphSAGE are used to generate vector embeddings of entities and relationships as subsequent semantic features. A spatiotemporal multi-layer heterogeneous graph containing multiple types of nodes and edges is constructed. Attention mechanisms or heterogeneous graph neural networks are used to learn the representations of nodes and subgraphs. Combined with a time series module, the observed state of the agent is finally output. The leader agent guides the behavior of lower-level agents by adjusting exogenous weights, rules, or indirect rewards. Through batch simulation, the multi-dimensional indicators of each scheme are comprehensively evaluated, and the optimal or executable scheme is selected.

[0004] For example, Chinese Patent No. CN119443312B discloses a hierarchical multi-agent game adversarial and collaborative decision-making method based on federated learning, which includes: establishing a federated learning framework containing multiple agents, each located in a different region, and training a local model using data from its respective region; deploying a hierarchical reinforcement learning model within each agent within the federated learning framework; using the hierarchical reinforcement learning model to enable multiple agents to conduct adversarial training in a game environment; monitoring the agents based on the game adversarial training, and introducing a collaborative decision-making algorithm based on the monitoring data to coordinate the behavior of multiple agents; and uploading the local model parameters to a central server for global optimization when a preset number of training rounds are reached or specific conditions are met.

[0005] For example, Chinese Patent No. CN117350410B discloses a micro-network swarm collaborative operation optimization method based on multi-agent federated reinforcement learning, which includes: 1) constructing a micro-network swarm interaction optimization operation model and a Markov decision model that consider economic benefits; 2) constructing a micro-network swarm continuous bidirectional auction model that considers transaction priority; and 3) constructing a multi-agent federated reinforcement learning method based on attention mechanism to optimize and manage the micro-network swarm.

[0006] The above-mentioned technology has at least the following technical problems: In existing technologies, when processing complex observation data, existing knowledge graph embedding and graph neural network technologies have difficulty effectively distinguishing between causal relationships and false associations or confusing features. The node and edge representations they learn often encode the correlations in the data rather than causality, causing the backend decision model to reason based on false or biased associations, which seriously reduces the credibility of the decision.

[0007] In leader-follower (Stackelberg) or hierarchical MARL architectures, the leader's intent is often conveyed through simple weights / thresholds or indirect rewards. Followers struggle to structurally conditionalize the leader's macro-intention into their policies, leading to cross-level collaboration failures and poor convergence performance. Furthermore, relying solely on multi-agent reinforcement learning for policy optimization is highly susceptible to getting trapped in local optima due to credit allocation and non-stationarity. While genetic / evolutionary algorithms possess global search capabilities, they suffer from low sample efficiency and difficulty in efficiently integrating with gradient information.

[0008] Meanwhile, in long-term, non-steady-state operating environments, existing technologies employing static hyperparameters or single-path online update mechanisms struggle to effectively cope with sudden environmental changes. When external conditions drastically change, the system lacks rapid and robust adaptive capabilities, leading to significant performance fluctuations during training and deployment phases. Furthermore, existing graph structure update and decay mechanisms are overly idealistic, assuming slow, linear changes in causal relationships. They fail to capture complex dynamics in the real world, such as sudden changes, nonlinear variations, and even causal reversals, resulting in low effectiveness of multi-agent hierarchical collaborative optimization. Summary of the Invention

[0009] To address the low effectiveness of existing multi-agent hierarchical collaborative optimization techniques, this invention provides a multi-agent hierarchical collaborative optimization method based on knowledge graphs and graph neural networks. The technical solution is as follows: S1. Software integrity is verified based on the hash digest of the acquired main program image file. If the software integrity verification fails, an integrity exception warning is triggered; otherwise, a power-on self-test is performed. S2. After the power-on self-test, multimodal data from heterogeneous sources is preprocessed to convert the heterogeneous raw data into unified structured event objects, and candidate entities and their corresponding confidence scores are extracted. S3. Based on the confidence scores, it is determined whether to assign the corresponding candidate entity to the entity candidate index table to update the existing entity candidate index table, thereby forming a time-series knowledge graph, from which a structured knowledge graph including the entity candidate index table is generated. S4. Extract causal edges to be verified from the event object set and verify causal relationships. Construct a causal knowledge graph based on the verification results. The structured event object set is a collection obtained by integrating the structured event objects generated for each multimodal data. S5. Construct a spatiotemporal heterogeneous graph based on the embedding results of the causal knowledge graph. Perform hierarchical collaborative optimization processing to achieve global optimization based on the output node representation matrix to obtain the optimal policy parameter set. S6. Perform data twin simulation evaluation based on the optimal policy parameter set. Then, filter the optimal policy parameter set based on the simulation evaluation results to determine the optimal policy and distribute the optimal policy to the preset terminal.

[0010] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: 1. By verifying software integrity and performing power-on self-tests, errors in subsequent operation due to software tampering or abnormal basic environment are prevented. Then, by unifying the format and structure of multimodal data, processing difficulties caused by heterogeneity are eliminated, improving data processing efficiency and accuracy. By extracting candidate entities and calculating confidence scores, foundational data is provided for subsequent knowledge graph construction, and confidence score information helps to filter out more reliable entities. Next, based on the confidence score of candidate entities, it is determined whether to classify them into the entity candidate index table, updating the existing entity candidate index table. Temporally sequenced knowledge graphs can more accurately reflect the dynamic evolution of knowledge in the real world, recording changes in entity relationships over time. Causal knowledge graphs clarify the causal relationships between events, aiding in causal reasoning and decision-making, and improving the understanding and response to complex situations. Simultaneously, a spatiotemporal heterogeneous graph is constructed based on the embedding results of the causal knowledge graph. This graph integrates spatial and temporal information into its structure, more accurately describing the dynamic characteristics of multi-agent hierarchical collaborative optimization. Through graph neural networks… Algorithms such as networks process spatiotemporally heterogeneous graphs, outputting node representation matrices. Based on these matrices, hierarchical collaborative optimization is performed, enabling the optimization process to make decisions based on richer information. This hierarchical collaborative optimization can solve optimization problems in large-scale complex systems, improving the overall performance and efficiency of multi-agent hierarchical collaborative optimization. Finally, through data twin simulation evaluation, a comprehensive performance assessment of the optimal policy parameter set can be conducted without actually running the system. By selecting the optimal policy, better results can be achieved in real-world systems, improving the operational efficiency and performance of multi-agent hierarchical collaborative optimization, thereby enhancing its effectiveness. Existing technologies may only focus on one aspect of knowledge graph construction or optimization algorithms, while this application organically combines multiple technical fields such as knowledge graphs, graph neural networks, multi-agent hierarchical collaborative optimization, and data twin simulation evaluation to form a complete solution that can more comprehensively address multi-agent hierarchical collaborative optimization problems.

[0011] 2. By acquiring entity information and its confidence levels from different sources, the identification channels for entities are enriched, improving the comprehensiveness and accuracy of entity identification. Confidence levels from different sources can complement and verify each other, providing a more reliable data foundation for subsequent aggregated confidence level processing. Compared to existing technologies that rely solely on confidence levels from a single source for entity identification and causal relationship judgment, by integrating confidence level information from multiple sources, aggregated confidence levels can more accurately reflect the authenticity and reliability of entities in the knowledge graph. By setting a writing threshold, entities are filtered to ensure that only entities with high confidence levels enter the entity candidate index table, improving the quality of the index table. Simultaneously, for entities with abnormal confidence levels, pre-defined personnel are promptly notified for processing, facilitating the discovery and resolution of potential problems. Then, candidate entities in the entity candidate index table are normalized and unified indexed. Normalization eliminates differences in entity features, enabling more accurate similarity comparisons between different entities. Unified indexing provides a unique identifier for each entity, improving the efficiency and maintainability of multi-agent hierarchical collaborative optimization. By performing similarity processing on candidate entities in the entity candidate index table and entities in the existing entity index table one by one, it is possible to determine... The system determines whether candidate entities are the same as existing entities, avoiding duplicate entry and confusion. Then, it performs entity matching and judgment based on entity similarity thresholds, achieving automatic entity merging and updating. Potential causal relationships are extracted from the structured event object set, providing a data source for causal analysis. A causal verification model is established to quantitatively analyze the causal relationship between candidate cause events and result events. Compared to the simple causal analysis methods used in existing technologies, this method combines causal regression models and autoregressive baseline models, along with residual squared processing and multi-source confidence weights, to more accurately analyze causal relationships and avoid misjudgments of false causal relationships. If a causal relationship exists, causal edges are generated and causal strength is calculated, providing important information for the construction of a causal knowledge graph and enabling causal relationships to be represented quantitatively. Finally, causal edges are screened by comparing the set relationship writing threshold and multi-source confidence scores, ensuring that only high-confidence causal edges enter the causal knowledge graph, improving the accuracy and reliability of the knowledge graph. Low-confidence causal edges are marked as candidate edges for further review and processing, thus improving the quality of causal knowledge graph construction.

[0012] 3. By reconstructing the virtual environment model, the operating environment and characteristics of the actual system can be accurately simulated, providing a reliable foundation platform for policy simulation. This makes the simulation results closer to reality and improves the accuracy of policy evaluation. Compared to existing technologies that only focus on a single type of data or a simple simulation environment, by loading the optimal policy parameter set, the virtual environment model can run according to the optimal policy. This allows for intuitive observation and analysis of the optimal policy's performance in the simulated environment, providing a basis for subsequent performance evaluation. By collecting various performance indicators, the performance of the optimal policy in similar real-world environments can be comprehensively and quantitatively evaluated, providing data support for policy optimization and selection. Then, by injecting disturbance noise and obtaining robustness indicators, the uncertainty in the actual system is simulated, evaluating the stability and reliability of the policy in the face of disturbance. Robustness indicators can help select policies that are more adaptable in practical applications, improving the ability of multi-agent hierarchical collaborative optimization to operate in complex environments. Existing technologies may have overlooked certain aspects in policy evaluation. The consistency verification of causal logic ensures that the decision-making process of the strategy conforms to the expected causal logic, avoiding unreasonable decision-making behavior. When the causal consistency index fails to meet the requirements, the influence of nodes in the graph neural network can be re-optimized by adjusting the cross-layer attention weights, improving the causal reasoning logic of the strategy and thus enhancing its rationality and effectiveness. Finally, multiple evaluation indicators of different agent strategies are summarized and normalized, eliminating the dimensional differences between different indicators and making the data comparable. Through non-dominated ranking and crowding distance calculation mechanisms, the set of strategies that achieve a relatively optimal balance across multiple objectives can be effectively identified, providing a high-quality candidate set for further screening of optimal strategies, improving screening efficiency and accuracy. Multi-objective screening in the Pareto front policy set can comprehensively consider the priority and importance of multiple objectives to find the optimal strategy that meets the actual needs. The screening terminates and outputs the results when the convergence condition is met, ensuring the efficiency and rationality of the screening process. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 A flowchart of a multi-agent hierarchical collaborative optimization method based on knowledge graphs and graph neural networks provided in this application embodiment; Figure 2 A flowchart of power-on self-test and preprocessing provided for embodiments of this application; Figure 3A flowchart for causal relationship extraction and verification provided in the embodiments of this application; Figure 4 A flowchart for data twin simulation evaluation provided in the embodiments of this application. Detailed Implementation

[0015] This application provides a multi-agent hierarchical collaborative optimization method based on knowledge graphs and graph neural networks, which solves the problem of low effectiveness in existing multi-agent hierarchical collaborative optimization. After software integrity verification, a power-on self-test is performed. Then, multimodal data is preprocessed. Based on the confidence level of the obtained candidate entities, it is determined whether to classify the corresponding candidate entities into the entity candidate index table. A causal knowledge graph is constructed based on the verification results of the causal relationships of the causal edges to be verified. Next, a spatiotemporal heterogeneous graph is constructed based on the embedding results of the causal knowledge graph. Hierarchical collaborative optimization processing is performed based on the output node representation matrix. Finally, data twin simulation evaluation is performed based on the optimal policy parameter set. The optimal policy parameter set is then filtered based on the simulation evaluation results, thus improving the effectiveness of multi-agent hierarchical collaborative optimization.

[0016] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0017] Example 1: This embodiment of the invention provides a multi-agent hierarchical collaborative optimization method based on knowledge graphs and graph neural networks. For example... Figure 1 The flowchart shown is for a multi-agent hierarchical collaborative optimization method based on knowledge graphs and graph neural networks. The processing flow of this method may include the following steps: S1 automatically downloads the main program image file from a trusted, pre-defined remote address (such as a file server or container image repository) when the system (such as an embedded device or server) starts or updates. It then performs software integrity verification based on the hash digest of the acquired main program image file. If the software integrity verification fails, an integrity exception warning is triggered; otherwise, a power-on self-test is performed. This ensures the reliability, accuracy, and security of the basic environment, including software, hardware, and clock synchronization. It is a complete copy of the main program at a specific moment, containing information such as the program's code, data, and runtime environment. It is typically used for program deployment, backup, and recovery. It is a fixed-length string calculated by applying a hash algorithm to data (such as files). It is unique, and the hash digests generated by different data are almost impossible to be the same. It is often used for data integrity verification. By comparing the hash digest of the original data with the hash digest of the recalculated data, it is determined whether the data has been tampered with. This prevents subsequent multi-agent hierarchical collaborative optimization from failing due to software tampering or abnormal basic environment, and provides a basic guarantee for the stable operation of the entire multi-agent hierarchical collaborative optimization.

[0018] It should be added that the SHA-256 hash function is called to calculate the digest value: ,in, This represents a secure hash algorithm with an output length of 256 bits, and it reads the stored signature digest. ,like If the software integrity verification passes, the gateway will send an "integrity exception warning" to the remote management terminal and record the status in the log. If the verification passes, then set ;in, This is the system firmware file; This is the hash digest of the currently running image; This is an official signature summary; This is the integrity check flag.

[0019] S2, after power-on self-test, preprocesses multimodal data from heterogeneous sources (structured forms, unstructured text, sensor time series, images / videos, audio, etc.) to convert the heterogeneous raw data into unified structured event objects. Simultaneously, it extracts candidate entities (objects that may represent actual entities initially extracted from the raw data during data processing) and their corresponding confidence scores (used to measure the credibility of candidate entities as real entities). This unifies the format and structure of multimodal data, eliminating processing difficulties caused by heterogeneity and improving the efficiency and accuracy of data processing. By extracting candidate entities and calculating confidence scores, it provides foundational data for subsequent knowledge graph construction, and the confidence score information helps to filter out more reliable entities.

[0020] S3, based on confidence level, determines whether to classify the corresponding candidate entity into the entity candidate index table (used to store candidate entities and their related information, such as a table structure for confidence level, facilitating the management and querying of candidate entities), in order to update the existing entity candidate index table, thereby forming a temporal knowledge graph. It extracts causal edges to be verified from the structured event object set including the entity candidate index table and verifies causal relationships. Based on the verification results, it constructs a causal knowledge graph (the knowledge graph particularly emphasizes the causal relationships between entities; by verifying and determining the causal connections between events, a knowledge graph capable of representing causal reasoning is constructed). The structured event object set is a collection obtained by integrating the structured event objects generated for each multimodal data. The temporal knowledge graph can more accurately reflect the dynamic evolution of knowledge in the real world, providing richer historical information for multi-agent hierarchical collaborative optimization. The causal knowledge graph clarifies the causal connections between events, which helps in causal reasoning and decision-making, improving the understanding and coping ability of multi-agent hierarchical collaborative optimization to handle complex situations.

[0021] S4. Based on the embedding results of the causal knowledge graph, a spatiotemporal heterogeneous graph is constructed (where nodes and edges contain not only spatial information but also temporal information, capable of simultaneously describing the distribution and changes of objects in space and time), and a node representation matrix is ​​output (a matrix obtained by processing the nodes in the spatiotemporal heterogeneous graph through algorithms such as graph neural networks, where each row or column represents the feature representation of a node). Based on the output node representation matrix, hierarchical collaborative optimization processing is performed to achieve global optimization, resulting in the optimal policy parameter set. The spatiotemporal heterogeneous graph can simultaneously consider spatial and temporal factors; the node representation matrix provides an effective feature representation for hierarchical collaborative optimization, enabling the optimization process to make decisions based on richer information; hierarchical collaborative optimization processing can solve large-scale and complex multi-agent hierarchical collaborative optimization problems, improving the overall performance and efficiency of multi-agent hierarchical collaborative optimization.

[0022] S5 performs data twin simulation evaluation based on the optimal policy parameter set to assess the performance of the optimal policy parameter set in a virtual environment. Based on the simulation evaluation results, the optimal policy parameter set is selected and distributed to preset terminals. Data twin simulation evaluation enables comprehensive performance evaluation of the optimal policy parameter set without actually running the system, reducing experimental costs and risks. The selected optimal policy performs better in real-world systems, improving the operational efficiency and performance of multi-agent hierarchical collaborative optimization.

[0023] like Figure 2 The flowchart of the power-on self-test and preprocessing shown further illustrates that the power-on self-test includes clock synchronization detection, circuit continuity detection, and sensor accuracy detection for global time synchronization, as well as analog-to-digital converter noise detection for evaluating the performance of the acquisition link and signal synchronization detection for evaluating the acquisition synchronization accuracy.

[0024] Specifically, clock synchronization detection includes: The gateway connects to a preset number of standard time servers. (That is, a server that can provide accurate and standard time, usually based on a high-precision atomic clock or other time reference, to provide time synchronization services for other devices in the network) sends a synchronization request and obtains the request sending time, server receiving time, server return time and local receiving response time, thereby obtaining the network round-trip delay and clock deviation.

[0025] The network round-trip time is obtained by processing the network round-trip time using the obtained request sending time, server receiving time, server return time, and local response receiving time; that is... In the formula, k is the number of the standard time server. The time the request was sent; This refers to the server's receiving time. The server return time; For local response time, The network round-trip delay for the k-th server is the total time required for a data packet to travel from the local system to the k-th time server and back.

[0026] The time of request sent, server received, server returned, and local response received are processed for clock skew; that is, clock skew. In the formula, This represents the clock deviation relative to the k-th server, i.e., the time difference between the local clock and the clock of the k-th server.

[0027] If the network round-trip delay does not exceed the network filtering delay limit and the clock deviation is not greater than the synchronization accuracy threshold, then select the server with the smallest clock deviation from the corresponding standard time servers and record it as the time synchronization server; otherwise, prompt the preset personnel time synchronization abnormality.

[0028] The network filtering delay limit is the maximum allowable network round-trip delay; the synchronization accuracy threshold is the maximum allowable clock deviation; the network filtering delay limit and the synchronization accuracy threshold are set by preset personnel based on experience, for example, by setting them to the average value over a historical time period.

[0029] It is important to understand that by synchronizing with a standard time server and selecting the server with the smallest clock deviation as the time synchronization server, it is possible to ensure that the device's time is highly consistent with the standard time, providing a time reference for the normal operation of the device and the accurate recording of data. Strict screening of network round-trip latency and clock deviation can avoid system failures and data errors caused by time asynchrony, and improve the reliability and stability of multi-agent hierarchical collaborative optimization.

[0030] Specifically, circuit continuity detection involves: detecting the continuity status of the input circuit; processing the measured voltage drop, test current, and test lead resistance into a continuity resistance value; if the obtained continuity resistance is not greater than the upper limit of continuity resistance, the circuit is considered to be conducting normally and no further processing is required; otherwise, the current continuity resistance is recorded, and a faulty circuit is detected by the pre-set personnel. The continuity resistance is calculated as follows: ;in, To measure lead resistance (usually taken as...) ), For on-resistance, To test the current, This is the voltage drop.

[0031] The upper limit of the on-resistance is the maximum allowable on-resistance, which is set by the user based on experience; for example, it can be set to 0.05Ω.

[0032] It is important to understand that by detecting the on-resistance of a circuit, problems such as open circuits, short circuits, or poor contacts can be detected in a timely manner, ensuring that the circuit can conduct normally. When the circuit conduction is abnormal, the on-resistance is recorded and the designated personnel are notified, which helps to quickly locate the fault point and improve the efficiency of fault diagnosis.

[0033] Specifically, sensor accuracy detection includes: The sensor measures the signal twice to obtain the primary signal and the secondary signal.

[0034] The actual transformation ratio is obtained by comparing the secondary signal and the primary signal; the comparison processing in this application is a ratio calculation.

[0035] The actual transformer ratio and the nominal transformer ratio are compared by absolute deviation to obtain the transformer ratio error; the transformer ratio error is: In the formula, the nominal ratio is The actual ratio is The ratio error is .

[0036] If the ratio error is not greater than the upper limit of the ratio error (set by the preset personnel based on experience, for example, it can be set to 1%), it means that the sensor accuracy is qualified and no processing is required; otherwise, the preset personnel will be prompted that the sensor accuracy is abnormal.

[0037] It is important to understand that by comparing the actual transformer ratio with the nominal transformer ratio and calculating the transformer ratio error, it is possible to promptly identify sensor accuracy issues, ensure the accuracy and reliability of sensor measurement data, and provide an accurate foundation for subsequent data processing and analysis.

[0038] Specifically, the analog-to-digital converter (ADC) noise detection works as follows: if the standard deviation of the ADC's sampled values ​​within a preset time period is not greater than the upper limit of the standard deviation, and the measured total signal-to-noise ratio (SNR) is not less than the theoretical SNR, then the sampling link is considered normal and no processing is required. Otherwise, noise information is recorded, and the preset personnel are alerted to the presence of noise interference. The upper limit of the standard deviation and the theoretical SNR are set by the preset personnel based on experience. By detecting the standard deviation of the ADC's sampled values ​​and the measured total SNR, noise interference problems in the sampling link can be detected in a timely manner, ensuring the quality of the acquired data and avoiding data distortion and errors caused by noise.

[0039] Specifically, signal synchronization detection involves obtaining the average synchronization delay and standard deviation of the acquired signal. If the average synchronization delay is not greater than the upper limit of the average synchronization delay and the standard deviation of the synchronization delay is not greater than the upper limit of the standard deviation of the synchronization delay, the acquisition synchronization accuracy is considered acceptable and no further processing is required. Otherwise, a message indicating an abnormal signal synchronization is displayed to the preset personnel. The synchronization delay is calculated by subtracting the actual start time of data acquisition from the trigger command time. The upper limit of the average synchronization delay is the maximum allowable synchronization delay, and the upper limit of the standard deviation of the synchronization delay is the maximum allowable standard deviation of the synchronization delay. These upper limits are set by the preset personnel based on experience.

[0040] By detecting the average synchronization delay and standard deviation of the acquired signals, the synchronization accuracy between different acquisition channels can be ensured, guaranteeing the consistency of the acquired data in time and space, and providing an accurate basis for subsequent data fusion and analysis.

[0041] Further preprocessing includes time grid alignment, privacy desensitization, temporal missing data imputation, outlier handling, and parameter encoding.

[0042] Specifically, time grid alignment is as follows: The source timestamps and clock offsets from each multimodal data acquisition are coupled to obtain the corrected timestamps. Then, grid alignment is performed by combining these timestamps with the time grid intervals to obtain the grid-aligned timestamps. In the formula, The time grid interval is the unit length of the time axis defined and governed by the system. For grid-aligned timestamps, The corrected timestamp, i.e., the original timestamp Apply global clock skew After correction, an accurate and standard timestamp is obtained.

[0043] The absolute difference between the corrected timestamp and the grid-aligned timestamp is obtained and recorded as the alignment error. If the alignment error is greater than the alignment error threshold (configured threshold, usually half of the time grid interval), the multimodal data is marked as an unaligned parameter and discarded. Otherwise, no processing is performed. This ensures that the data of different modalities can accurately correspond in the time dimension, avoids data analysis errors caused by time asynchrony, and improves the accuracy and reliability of subsequent data processing and analysis. Discarding unaligned parameters can prevent erroneous data from interfering with the overall analysis results.

[0044] Privacy desensitization refers to performing irreversible desensitization operations on parameters including personally identifiable information (i.e., using a keyed hash message authentication code for each sensitive adoption number), and performing differential privacy protection operations on parameters that provide statistical access (i.e., adding Laplace noise to the data). At the same time, the privacy desensitization operations are recorded for auditing purposes; thereby effectively protecting personal privacy and preventing the leakage of sensitive personal information.

[0045] Temporal missing value imputation is specifically performed as follows: missing points are detected and imputed on the time grid, and the imputation method is selected according to the missing value length. Specifically, if a single point is missing, linear interpolation is used; if the missing length is not greater than the missing length threshold (set by the preset personnel based on experience), linear state-space Kalman filtering prediction and update recovery are used; if neither of the above conditions is met, the corresponding missing value is marked as a missing segment and no further processing steps are performed.

[0046] Imputing missing values ​​in time-series data makes the data sequence more complete and improves data usability. Different imputation methods can be flexibly selected according to the missing information, ensuring imputation accuracy while minimizing the impact on data authenticity. Marking missing segments can avoid analytical biases caused by incorrect imputation.

[0047] Outlier handling involves using the median absolute deviation method to identify outliers in multimodal data over a preset time period, and calculating a robust z-score (i.e., ...) based on the median and median absolute deviation. In the formula, i is the window sequence number. Let z be the robust z-score of the i-th window sequence. Let m be the median (where m is the median and MAD is the median absolute deviation) and m be the window sequence. If the absolute value of the robust z-score is greater than the outlier threshold (set by a predefined user, such as 3), it is considered an outlier. Outliers are removed, and time-series missing data is imputed. This effectively identifies and removes outliers from the data, preventing them from misleading the data analysis results and improving the accuracy and reliability of the data. Subsequent missing data imputation ensures the integrity of the data sequence, facilitating subsequent data processing and analysis.

[0048] Parametric encoding refers to performing feature encoding on different modalities and projecting them into node feature vectors. Specifically, this involves applying a pre-trained language encoder to the text to obtain text vectors and then performing L2 normalization (i.e., the specific formula for L2 normalization in this application is: ...). In the formula, For text vectors, (The L2 norm of the text vector); an image vector is obtained by applying a visual encoder to an image or video frame, and the image vector is standardized (i.e., the difference between the image vector and the average value of the image vector is compared with the standard deviation of the image vector) and then L2 normalized; time-domain features and frequency-domain features are obtained based on sensor sequence features. The time-domain features are the average value and variance of the sensor sequence features; the frequency-domain features are extracted by fast Fourier transform to obtain the power spectrum energy of the sensor sequence features, and the time-domain features (average value and variance of the sensor sequence features) and the frequency-domain features (power spectrum energy) are concatenated to obtain the sensor sequence vector.

[0049] Converting data from different modalities into a unified vector representation facilitates subsequent data fusion and analysis. Feature encoding enables the extraction of key features from the data, reducing data dimensionality, improving data processing efficiency, and preserving important information, thus providing high-quality feature input for subsequent model training and data analysis.

[0050] Furthermore, based on the confidence level, a decision is made on whether to assign the corresponding candidate entity to the entity candidate index table. The specific process is as follows: By analyzing text vectors using a named entity recognizer, text entities and their corresponding confidence scores are output. Simultaneously, image vectors from an object detector are analyzed to output visual entities and their corresponding confidence scores. Structured fields (i.e., sensor sequence vectors) are mapped to obtain structured field entities, and confidence scores are set accordingly. Entity information is obtained from different sources (the text itself and potentially related image text information), increasing the comprehensiveness and diversity of entity recognition. The recognition results from different tools can complement each other, improving the recall rate of entity recognition.

[0051] All confidence levels pointing to the same entity Perform aggregate confidence processing to obtain the aggregate confidence score, i.e. In the formula, To aggregate the confidence levels, d is the confidence level number, d=1,2,...,D, where D represents the total number of confidence levels. Let d be the confidence level. By aggregating the results from multiple sources, the credibility of the entity can be assessed more accurately, avoiding the bias that may exist in the results from a single source, and improving the accuracy and stability of entity confidence assessment.

[0052] If the aggregate confidence score is not less than the write threshold (set by a preset user, for example, 0.8), the corresponding candidate entity will be added to the entity candidate index table. Otherwise, the preset user will be notified that the corresponding entity's confidence score is abnormal, or a manual review will be conducted to determine whether to add the corresponding candidate entity to the entity candidate index table. By setting the write threshold, the quality of entities entering the entity candidate index table can be controlled, ensuring that the entities in the index table have high credibility and improving the accuracy and effectiveness of subsequent operations based on the index table.

[0053] like Figure 3 The flowchart shown further illustrates the process of extracting and verifying causal relationships. It details the extraction of causal edges to be verified from a structured event object set, including an entity candidate index table, and the verification of causal relationships. The specific process is as follows: Candidate entities in the entity candidate index table are normalized and uniformly indexed. Normalization eliminates differences in entity names and attributes, while uniform indexing provides a unique identifier for each entity, facilitating rapid querying and association during subsequent data processing and model building, thereby improving data processing efficiency.

[0054] The similarity scores of candidate entities in the candidate entity index table and entities in the existing entity index table are calculated one by one. This similarity processing involves splitting the entity string into sets of words or characters and then calculating the similarity score using Jaccard. Through similarity processing, the similarity relationships between candidate entities and existing entities can be accurately identified, providing a basis for subsequent entity matching.

[0055] If the entity similarity is not less than the entity similarity threshold (set by a predefined group, for example, 0.85), the corresponding entities are determined to be the same entity, and the index timestamp is updated to generate an updated version, which serves as the base set of knowledge graph nodes. Otherwise, a new index entity is created. Updating the index timestamp records the latest state of the entity, facilitating subsequent data management and maintenance; creating a new index entity ensures that newly discovered entities are correctly recorded and processed.

[0056] The causal edges to be verified are extracted from a set of structured event objects. A causal verification model is then established based on the observations corresponding to these edges within a preset time period. This model includes a causal regression model and an autoregressive baseline model. Establishing these two models allows for prediction of the outcome event from the perspectives of including and excluding candidate causal events, respectively. By comparing the prediction results of the two models, the existence of causal relationships can be verified more accurately, improving the scientific rigor and reliability of the causal relationship verification.

[0057] The specific constraint expression for the causal regression model is: ; In the formula, t is the time number, and g is the autoregressive order (indicating the order of prediction). At that time, it is necessary to consider (Historical values ​​from the past number of time steps), g=1,2,...,G, where G is the total number of autoregressive orders, and f is the cross-regression order (indicating the order in prediction). The number of candidate causal events (x) and the number of past time steps are represented by f = 1, 2, ..., F, where F is the total number of cross-regression orders. The current target value is the observed value of the outcome event of the causal edge to be verified at time t. These are historical causal candidate values, representing observations of candidate causal events at a past time tf that is earlier than the current time t. The historical target value is the observed value of the outcome event y at a past time tg, which is earlier than the current time t. The autoregressive coefficient represents the g-th historical value of the outcome event y itself. For the current value The degree or weight of influence The cross-regression coefficient represents the f-th historical value of the candidate causal event x. For the target current value The degree or weight of influence The residual term represents the random error or noise that the model cannot explain at time t. It is usually assumed to be white noise with a mean of 0 and constant variance.

[0058] The specific constraint expression for the autoregressive baseline model is: ; In the formula, The current baseline target value, This is the historical target value. The baseline autoregressive coefficients are... This represents the baseline residual term.

[0059] The results of the causal validation model are processed by residual squares (i.e., the F-statistic between the sum of squared residuals of the baseline model and the sum of squared residuals of the causal regression model is calculated) to obtain the causal residual processing results; that is... In the formula, The sum of squared residuals of the baseline model. Let N be the sum of squared residuals of the causal regression model, N be the sample size, and N−G−F be the degrees of freedom. By comparing the sum of squared residuals of the causal regression model and the autoregressive baseline model, we can clearly see the improvement in the model's predictive performance after introducing historical values ​​of candidate causal events, thus more effectively determining whether a causal relationship exists. If the causal residual processing results meet the preset conditions, i.e. ,in, The critical value of the F-statistic at a significance level α (e.g., set to 0.05) is determined by a pre-defined group. This generates causal edges, and the causal strength (i.e., the critical value of the F-statistic at a significance level α) is obtained based on the causal residual processing results. If the result is positive, it indicates that there is no causal relationship between the edges to be verified. Calculating causal strength can quantify the strength of causal relationships, providing more valuable information for subsequent causal knowledge graph construction and application.

[0060] By using a pre-trained causal relationship extraction model, causal pairs are identified from trigger words (such as "cause", "due to", "therefore") in sentences, and text confidence is obtained. The pre-trained causal relationship extraction model can quickly and accurately identify causal pairs from text, improving the efficiency of causal relationship extraction.

[0061] Multi-source confidence weights are introduced to weight and couple textual semantic confidence and causal strength to obtain a multi-source confidence score. The sum of the weights for each multi-source confidence score is 1, and these weights are set by pre-defined personnel. For example, typical values ​​are: a weight of 0.4 for textual semantic confidence and a weight of 0.6 for causal strength. Multi-source confidence weights consider the different importance of textual semantic information and causal verification results. Through weighted coupling, information from different sources is integrated, providing a more comprehensive and accurate assessment of the reliability of causal relationships, thus improving the accuracy and reasonableness of causal relationship judgments.

[0062] If the multi-source confidence score is less than the relation writing threshold (set by a preset person, for example, 0.75), the causal edge is marked as a candidate edge; otherwise, the causal edge is written into the causal knowledge graph. Judging based on the multi-source confidence score and the relation writing threshold can effectively filter out causal edges with higher reliability for writing into the causal knowledge graph, ensuring the quality and accuracy of the causal knowledge graph.

[0063] It should be added that the graph structure update and decay mechanism, specifically, follows this process: the causal knowledge graph is a dynamic time-series graph. ,in, Let be the set of entity nodes at time t. Given a set of causal edges, the edge weights are subject to time decay and update in each cycle: Define an edge weight decay function for any slave node. Pointing to node Causal edge The specific formula for calculating its weight at time t is as follows: ,in, Causal edge The initial causal strength; Causal edge The last time it was observed, Causal edge The weights; The time decay factor (default is 0). / s). If causal edge If the weight of an edge is less than the minimum weight threshold set by a pre-defined person, the edge is marked as "deactivated" and will no longer participate in the embedding training. The deactivated record is then archived.

[0064] For edges with repeated observations, a sliding update strategy is used to smoothly update their initial causal strength. Through smoothing coefficient (Typically 0.8) Weighted fusion of historical intensity and new observed intensity avoids drastic intensity fluctuations caused by noise or outliers in a single observation, thus achieving stable updates of causal intensity.

[0065] Specifically, the causal knowledge graph embedding computation follows this process: An improved TransE-Causal model is used to embed triples... The mapping is done as a low-dimensional embedding vector to capture causal directionality and intensity.

[0066] Define the entity and relation embedding matrix Define the causal energy function: ,in, For relational transformation functions (linear layer or hyperbolic mapping); Causal confidence level of the edge; This is the confidence weighting coefficient.

[0067] Loss function: In the formula, The total loss embedded in the causal knowledge graph is the overall objective function that needs to be minimized. It consists of two parts: marginal ranking loss and regularization term. For a positive sample triple, a specific, real-world triple, where h is the head entity, r is the relation, and For tail entities; E(h,r, The energy of a positive sample triple (h, r, ...) is a measure of the energy of the triple (h, r, ...). The more reasonable the score, the more likely the triple is to be true. is the regularization coefficient, a hyperparameter used to control the strength of the regularization term and prevent the model from overfitting; For L2 regularization, apply to all entity vectors and relation vector Sum of the squares of the L2 norm; where For negative sample triples After training, the embedding vector set is obtained.

[0068] Furthermore, based on the output node representation matrix, a hierarchical collaborative optimization process is performed to achieve global optimization. The specific process is as follows: The process receives the node representation matrix and iterates the leader state vector based on the leader parameters and node representation matrix from the previous iteration. In graph neural networks, the node representation matrix is ​​a mathematical representation of the features of each node in the graph. Through specific algorithms (such as convolution operations), the original information of the nodes (such as attributes and connections) is transformed into low-dimensional vectors, with each vector representing a node. These vectors are arranged into a matrix according to certain rules, which can capture the features of nodes in the graph structure and the relationships between nodes, providing basic data for subsequent analysis and optimization. In hierarchical collaborative optimization, the leader parameters are a set of parameters related to the leader state and decision rules, used to update or calculate the leader state vector, such as iteration step size, normalization coefficient, weight matrix, etc. The leader state vector reflects the leader's state information in the current iteration process. By combining the leader parameters and node representation matrix from the previous iteration for iterative updates, the leader state vector contains the leader's cognition and decision basis for the current state of the system, which is used to guide the generation of subsequent leader strategies.

[0069] A gradient evolution hybrid optimization mechanism is adopted. The current gradient direction is updated based on a multi-objective utility function that includes system efficiency indicators, control costs, and uncertainty risks. A genetic evolution algorithm is then used to perform mutation operations on the gradient direction with a mutation probability (set by a predefined set, for example, 0.05). That is, the multi-objective utility function is: ,in, For system efficiency indicators; To control costs; This is due to uncertainty and risk. The gradient direction indicates the direction in which the function value grows the fastest. Theoretically, updating the policy vector along the gradient direction can maximize the utility function. The Guided Evolution Strategies Controller (Guided ES controller) adjusts the evolutionary search range according to the gradient direction to improve convergence.

[0070] Gradient optimization uses the gradient information of the objective function to determine the search direction, quickly finding the function's extreme points; while evolutionary algorithms simulate selection, crossover, and mutation in biological evolution, performing a global search in the solution space. The gradient-evolutionary hybrid optimization mechanism combines the two, using gradient information to guide the evolutionary algorithm's search direction, improving optimization efficiency and effectiveness.

[0071] The generated optimal leader policy vector is output to the follower module. The observable states of each follower agent in the follower module are extracted from the output of the hierarchical graph neural network. Each follower agent, relative to the leader, executes corresponding actions based on the leader's output policy vector and its own observable state. When processing graph-structured data, the hierarchical graph neural network generates outputs containing information about each node. From these outputs, the observable state information of each follower agent is extracted; this state information forms the basis for the follower agent's decision-making.

[0072] Define a policy function and a value function for evaluating long-term rewards for each follower agent. The policy function defines the rules by which the follower agent chooses actions given its own observable state. For example, a deep neural network (such as a convolutional neural network) can be used to construct the policy function, taking the state as input and outputting the probability distribution of actions. The value function is used to evaluate the long-term rewards that the follower agent can obtain when executing a certain policy. By calculating the value function, the merits of different policies can be evaluated, thereby guiding the agent to choose the optimal policy. The calculation of the value function usually adopts methods in reinforcement learning, such as Monte Carlo methods and temporal difference learning.

[0073] A reinforcement learning framework employing centralized training and distributed execution is used to collaboratively train the policies of various agents. During centralized training, all follower agents share information and collaboratively learn the optimal policy through centralized computation methods (such as centralized computation of value functions and policy gradients). For example, a central controller can be used to collect the state information and reward signals of all agents for unified policy updates. During distributed execution, each agent independently executes corresponding actions based on its perceived local state information. This reinforcement learning framework combines the advantages of centralized training in utilizing global information with the flexibility of distributed execution, thereby improving the overall performance of multi-agent hierarchical collaborative optimization.

[0074] After a certain number of training sessions, once the policy converges, each follower agent independently executes actions according to the corresponding policy, and the system's periodic summary benefit (reflecting the comprehensive effect obtained from executing actions within the current period, including goal achievement and resource utilization efficiency, etc., and is fed back to the leader module so that the leader can adjust the policy based on the feedback information) is fed back to the leader module so that the leader can adjust the policy based on the feedback information.

[0075] A genetic algorithm is used to select offspring from the policy set formed by the policies of each agent (i.e., the initial population of the genetic algorithm), and the population is updated by correcting the gradient direction to output the optimal policy parameter set. Meanwhile, if the global fitness gain (the degree of improvement in fitness of the new generation relative to the previous generation during the iteration of the genetic algorithm) is less than the fitness gain threshold for three consecutive generations, it indicates that the algorithm may be trapped in a local optimum or the convergence speed is slowing down. Adjust the mutation probability and mutation intensity according to the preset ratio; otherwise, do not process it. When adjusting the mutation intensity, the preset ratio can be set to 1.2, and when adjusting the mutation probability, the preset ratio can be set to 0.8.

[0076] like Figure 4 The flowchart shown further illustrates that the data twin simulation evaluation is based on the optimal policy parameter set, and the specific process is as follows: By reading multimodal data from historical time periods through a digital twin engine, information about physical entities at different times and in different aspects is collected to reconstruct a virtual environment model. The reconstructed virtual environment model can highly realistically reflect the characteristics and behaviors of the actual physical environment, providing a reliable simulation platform for subsequent strategy evaluation. By using multimodal data, the complexity of the physical environment can be captured more comprehensively and accurately, improving the accuracy and credibility of the virtual model.

[0077] The optimal strategy parameter set is loaded and injected into the virtual environment model; the optimal strategy parameter set is applied to the virtual environment, which can simulate the operation of the strategy in the actual system under the simulation environment, thereby evaluating the effectiveness and feasibility of the strategy in advance.

[0078] The optimal strategy is run in the simulation environment of the virtual environment model, and performance indicators (such as cumulative simulation gains and average latency) are collected. By collecting performance indicators, we can intuitively understand the performance of the optimal strategy in the virtual environment, which provides a basis for further optimization of the strategy.

[0079] Injecting disturbance noise into the simulation environment of the pseudo-environment model and obtaining robustness indices to evaluate policy stability, i.e. In the formula, For robustness indicators, To accumulate profits for simulation, The simulation cumulative gain after noise injection; injecting disturbance noise can test the adaptability and stability of the strategy in the face of various disturbances in the real environment. By obtaining robustness indicators, it is possible to determine whether the strategy can maintain good performance in complex and variable environments, thereby improving its reliability in real-world environments.

[0080] Consistency verification is performed on the simulation logs and causal knowledge graph in the digital twin engine to verify the causal logic consistency of policy decisions. By verifying the consistency of causal logic, the interpretability and credibility of the policy can be improved, making the decision-making process more transparent and reliable. If inconsistencies in causal logic are found, the policy can be adjusted in a timely manner to ensure the effectiveness and correctness of the policy.

[0081] If the causal consistency index is less than the causal consistency index limit (set by preset personnel, the inflow can be set to 0.9), then the cross-layer attention weight is adjusted to re-evaluate the node's influence; otherwise, it means that the strategy conforms to the causal reasoning logic and no processing is performed. By adjusting the cross-layer attention weight, the internal information transmission mechanism of the model can be optimized, the influence of the node can be re-evaluated, and the causal logic consistency of the strategy may be improved.

[0082] The specific constraint expression for the causal consistency index is: ; In the formula, The causal consistency index is a scalar value between 0 and 1, representing the degree of consistency between the agent's policy behavior and the overall causal graph. The causal edge set is the set of all causal relationships that need to be verified, extracted from the causal knowledge graph. A specific causal edge is a directed edge in a knowledge graph, representing a path from the cause node. To the result node causal relationship This is an indicator function, a discriminant function. It returns 1 when the condition within the brackets [·] is true, and 0 when the condition is false. In this formula, it is used for counting; 1 point is awarded for each simulated edge that matches the graph definition. This refers to the causal effect symbols observed in the simulation, specifically the causal nodes analyzed in the digital twin simulation log. State changes and result nodes The correlation symbol between state changes is calculated logically as follows: The increase was mainly accompanied by If the value increases, the sign is positive; if... The increase was mainly accompanied by If the value decreases, the sign is negative. The causal relationship symbol defined in a knowledge graph is an edge in a causal knowledge graph. The symbol for the recorded causal strength.

[0083] Furthermore, the optimal strategy parameter set is screened based on the simulation evaluation results. The specific process is as follows: This study summarizes evaluation results including the cumulative benefits of each agent's strategy (i.e., the total benefit obtained by the agent after a period of time or a series of operations during the execution of a specific strategy), robustness indicators, causal consistency indicators, and resource consumption rate (i.e., the ratio of the amount of resources consumed per unit time to the total amount of available resources during the execution of the strategy). A multi-objective indicator matrix is ​​constructed, and the multimodal data is normalized (converted to the range of 0-1 through mathematical transformation) to obtain a standardized multi-objective dataset. Constructing the multi-objective indicator matrix comprehensively and systematically demonstrates the performance of different agent strategies on multiple key indicators, providing a foundation for subsequent comprehensive analysis. Normalization eliminates the influence of differences in units and numerical ranges between different indicators, making each indicator comparable in subsequent analysis and comparison.

[0084] Using non-dominated ranking and crowding distance calculation mechanisms, the policies of each agent are hierarchically divided to obtain the Pareto front policy set. Non-dominated ranking refers to the ranking of agents in multi-objective optimization problems based on their dominance relationships. If an agent is no worse than another agent in all objectives and is better than another agent in at least one objective, then the former is said to dominate the latter. Non-dominated ranking divides all agents into different levels according to this dominance relationship. Individuals in the first level are not dominated by any other agents, individuals in the second level are dominated only by individuals in the first level, and so on. Crowding distance calculation is used to measure the density of solutions around a given solution in the objective space in multi-objective optimization problems. The larger the crowding distance, the sparser the other solutions around the given solution, indicating that this solution has greater value in maintaining the diversity of the solution set. In multi-objective optimization problems, the Pareto front refers to the set of all solutions that are not dominated by other solutions, and the Pareto front policy set is the set of policies that satisfy the Pareto optimality condition. Non-dominated ranking can quickly stratify strategies according to their superiority or inferiority in multiple objectives, while the crowding distance calculation mechanism ensures the diversity of the Pareto front strategy set and avoids all optimal solutions from being concentrated in a certain local region, thus providing a richer and more representative set of candidate strategies for subsequent multi-objective screening.

[0085] The obtained Pareto front policy set undergoes multi-objective screening. Screening terminates when any preset convergence condition is met, and the final optimal policy and corresponding policy parameters are output. The preset convergence conditions are: Pareto front stability (measured by the distance between each solution in the current Pareto solution set and its nearest neighbor in the previous generation, then averaging the distances across all solutions) is less than a preset threshold (e.g., 0.001); front diversity index (assessed by the uniformity and coverage of the Pareto front distribution, measured by the variance or spacing statistics of the front solutions in the target space) is less than a preset diversity threshold; and the maximum number of iterations reaches a preset number of iterations (e.g., 50). Through multi-objective screening, the optimal policy that best meets the actual needs can be found from the Pareto front policy set, achieving a reasonable balance between multiple conflicting objectives. The preset convergence conditions prevent excessive algorithm iteration, save computational resources, and ensure that a sufficiently good solution is found within a reasonable timeframe, improving the efficiency and practicality of the entire multi-agent hierarchical collaborative optimization process.

[0086] Example 2: In most real-world systems, the in-degree and out-degree distributions of nodes follow a power-law distribution (a few nodes have a large number of connections). If the distribution is too uniform (e.g., close to a Poisson distribution), it indicates that the causal knowledge graph may be full of random, meaningless connections; or the confidence scores of the two channels, causal strength and textual semantic confidence, often give diametrically opposed signals, but the fused multi-source confidence score is still very high.

[0087] The Peter-Clark algorithm (a constraint-based causal discovery algorithm that infers causal relationships between variables through a series of conditional independence tests, such as the chi-square test, and gradually constructs a causal graph structure; its basic idea is to start from a completely undirected graph and continuously delete edges that do not have conditional independence relationships, eventually obtaining a directed acyclic graph) and the Granger test algorithm (a statistical method for testing causal relationships between time series data. Its core idea is that if the past value of variable X helps predict the future value of variable Y, but the past value of variable Y does not help predict the future value of variable X, then X is considered a Granger cause of Y, i.e., there is a causal relationship from X to Y) are used to analyze the observations corresponding to the causal edges to be verified within a preset time period, and to determine whether a causal relationship exists between the observations of the result event and the observations of the cause event.

[0088] If the PC algorithm determines that there is no causal relationship between the observed values ​​of the causal event and the observed values ​​of the result event, and the Granger test shows that there is a causal relationship, then the causal relationship is marked as a pending causal relationship, and the causal strength of the causal edge is reduced according to a preset ratio.

[0089] If the PC algorithm determines that there is no causal relationship between the observed values ​​of the causal event and the observed values ​​of the result event, and the Granger test shows that there is no causal relationship, then it means that there is no causal relationship between the observed values ​​of the causal event and the observed values ​​of the result event.

[0090] If the PC algorithm determines that there is a causal relationship between the observed values ​​of the causal event and the observed values ​​of the result event, and the Granger test shows that there is no causal relationship, then the causal relationship is marked as a pending causal relationship, and the causal strength of the causal edge is reduced according to a preset ratio.

[0091] If the PC algorithm determines that there is a causal relationship between the observed values ​​of the causal event and the observed values ​​of the result event, and the Granger test shows that there is a causal relationship, then a causal relationship is directly assigned to the observed values ​​of the causal event and the result event.

[0092] It is important to understand that the PC algorithm and the Granger test algorithm judge causal relationships from different perspectives, and using them together can improve the accuracy of causal relationship judgment. The PC algorithm can uncover the conditional independence relationship between variables and construct a causal graph structure; the Granger test algorithm uses the characteristics of time series to determine the causal direction between variables. The two complement each other and provide a more comprehensive basis for judging causal relationships.

[0093] Marking potential causal relationships and reducing their causal strength can reduce their impact on the knowledge graph without immediately deleting potentially erroneous causal relationships. This provides a buffer time for further evaluation and avoids the loss of important causal relationships due to erroneous deletion.

[0094] After decaying the edge weights in the causal knowledge graph within a preset decay period, an online change point detection algorithm (such as Bayesian Online Change Point Detection, ADWIN) is run in real time on each causal edge. The core idea of ​​ADWIN is to maintain a dynamic window W, in which the data is considered stationary. When a new data point x_t arrives, it is added to the window. Then, the algorithm attempts to find a split point within the window to divide the window into two sub-windows W_0 and W_1. If the difference between the means of the two sub-windows is large enough (exceeding a confidence boundary related to the window size), a change point is considered to have occurred, and the window W is shrunk to W_1 (keeping the new data and discarding the old data).

[0095] If a change point exists, a local causal relationship reassessment is triggered; otherwise, no action is taken. Local causal relationship reassessment involves re-invoking the causal verification model and re-evaluating causal relationships. If a causal edge does not have a causal relationship, it is deleted; otherwise, a new causal edge is generated and assigned causal strength. The online change point detection algorithm can monitor changes in the observation sequence corresponding to causal edges in real time, promptly detecting potential changes in causal relationships. Local causal relationship reassessment, based on the change point detection results, re-evaluates and updates local causal relationships, ensuring the accuracy and timeliness of the causal knowledge graph.

[0096] Example 3: If the average value of the causal consistency index within the preset time period is not less than the causal consistency index limit, it indicates that the macro-causal logic framework of the causal knowledge graph is reliable. At this time, executing Example 1 may cause the correct connections in the causal knowledge graph to be cut off, thereby reducing the effectiveness of multi-agent collaborative optimization.

[0097] If the causal consistency index is not less than the causal consistency index limit, no processing is performed; otherwise, the causal edge corresponding to the condition in the indicator function being false is obtained, and the corresponding causal edge is recorded as a causal inconsistency edge, and the knowledge graph is adjusted and optimized. By setting the causal consistency index limit, knowledge graphs with basically reasonable causal relationships and knowledge graphs with causal inconsistencies can be quickly screened out, which helps to optimize knowledge graphs with problems and improve optimization efficiency. Clearly identifying the edges with causal inconsistencies in the knowledge graph provides specific targets for subsequent knowledge graph adjustment and optimization. By focusing on these inconsistent edges, the causal relationships of the knowledge graph can be improved more effectively.

[0098] The knowledge graph is adjusted and optimized as follows: In the next round of training, the meta-parameters are adjusted to corresponding corrected meta-parameters. Corrected meta-parameters represent the results of mapping causal consistency indicators and meta-parameters into their respective meta-parameter mapping sets. The meta-parameter mapping sets are collections obtained from a pre-set database representing the mapping relationships between causal consistency indicators, current meta-parameters, and corresponding corrected meta-parameters. These sets are trained using meta-parameter training data, which includes causal consistency indicators based on historical time periods, current meta-parameters, and meta-parameters set by professionals based on empirical rules. By mapping causal consistency indicators and meta-parameters to corrected meta-parameters and using these corrected meta-parameters in the next round of training, the model can pay more attention to the rationality of causal relationships during training, thereby improving the quality of knowledge graph construction.

[0099] Simultaneously, when acquiring cross-layer attention weights at the level where causal inconsistencies exist, a causal penalty term is introduced and coupled with the attention score (in the attention mechanism, a score used to represent the correlation between input elements) to correct the attention score. The causal penalty term represents the result obtained by mapping the causal consistency index into the causal penalty term mapping set. The causal penalty term mapping set is a collection of mapping relationships between causal consistency indices and corresponding causal penalty terms obtained from a pre-set database. The causal penalty term mapping set is trained using causal penalty term training data, which includes causal consistency indices based on historical time periods and causal penalty terms set by professional technicians based on empirical rules. Introducing a causal penalty term in the calculation of cross-layer attention weights and coupling it with the attention score can correct the attention score, making the model pay more attention to reasonable causal relationships and reduce attention to causal inconsistencies when interacting with information across layers. This can further improve the expression of causal relationships in the knowledge graph, enhance the performance of multi-agent hierarchical collaborative optimization, and improve the accuracy of the knowledge graph.

[0100] In summary, this application's embodiments prevent subsequent operational errors due to software tampering or abnormal basic environments by verifying software integrity and performing power-on self-tests. Then, by unifying the format and structure of multimodal data, it eliminates processing difficulties caused by heterogeneity, improving data processing efficiency and accuracy. By extracting candidate entities and calculating confidence scores, it provides foundational data for subsequent knowledge graph construction, and the confidence score information helps filter out more reliable entities. Next, based on the confidence score of candidate entities, it determines whether to classify them into the entity candidate index table and updates the existing entity candidate index table. The temporal sequence knowledge graph can more accurately reflect the dynamic evolution of knowledge in the real world, recording changes in entity relationships over time. The causal knowledge graph clarifies the causal relationships between events, aiding in causal reasoning and decision-making, and improving the understanding and response to complex situations. Simultaneously, based on the embedding results of the causal knowledge graph, a spatiotemporal heterogeneous graph is constructed. This graph integrates spatial and temporal information into its structure, more accurately describing the dynamic characteristics of multi-agent hierarchical collaborative optimization. This application processes spatiotemporal heterogeneous graphs using algorithms such as graph neural networks, outputting node representation matrices. Based on these matrices, hierarchical collaborative optimization is performed, enabling the optimization process to make decisions based on richer information. This hierarchical collaborative optimization can solve optimization problems in large-scale complex systems, improving the overall performance and efficiency of multi-agent hierarchical collaborative optimization. Finally, through data twin simulation evaluation, a comprehensive performance assessment of the optimal policy parameter set can be conducted without actually running the system. The selected optimal policy can achieve better results in real-world systems, improving the operational efficiency and performance of multi-agent hierarchical collaborative optimization, thereby enhancing its effectiveness. Existing technologies may only focus on one aspect of knowledge graph construction or optimization algorithms, while this application organically combines multiple technical fields such as knowledge graphs, graph neural networks, multi-agent hierarchical collaborative optimization, and data twin simulation evaluation to form a complete solution that can more comprehensively address multi-agent hierarchical collaborative optimization problems.

[0101] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0102] This invention is described with reference to flowchart illustrations and / or block diagrams of systems, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0103] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0104] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0105] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A multi-agent hierarchical collaborative optimization method based on knowledge graphs and graph neural networks, characterized in that, The method includes: S1. Perform software integrity verification based on the hash digest of the obtained main program image file. If the software integrity verification fails, trigger an integrity exception warning; otherwise, perform a power-on self-test. S2, after power-on self-test, preprocesses multimodal data from heterogeneous sources to convert heterogeneous raw data into unified structured event objects, and extracts candidate entities and corresponding confidence scores. S3, based on the confidence level, determine whether to classify the corresponding candidate entity into the entity candidate index table, so as to update the existing entity candidate index table and thus form a temporal knowledge graph. Extract the causal edges to be verified from the structured event object set including the entity candidate index table and verify the causal relationship. Construct a causal knowledge graph based on the verification results. The structured event object set is a set obtained by integrating the structured event objects generated for each multimodal data. S4. Construct a spatiotemporal heterogeneous graph based on the embedding results of the causal knowledge graph, and perform hierarchical collaborative optimization processing based on the output node representation matrix to achieve global optimization, thereby obtaining the optimal policy parameter set. S5 performs data twin simulation evaluation based on the optimal strategy parameter set, thereby filtering the optimal strategy parameter set according to the simulation evaluation results to determine the optimal strategy, and distributing the optimal strategy to preset terminals.

2. The multi-agent hierarchical collaborative optimization method based on knowledge graphs and graph neural networks according to claim 1, characterized in that: The power-on self-test includes clock synchronization detection, circuit continuity detection, and sensor accuracy detection for global time synchronization, as well as analog-to-digital converter noise detection for evaluating the performance of the acquisition link and signal synchronization detection for evaluating the acquisition synchronization accuracy. The clock synchronization detection specifically includes: The gateway sends synchronization requests to a preset number of standard time servers and obtains the request sending time, server receiving time, server return time, and local receiving response time, thereby obtaining the network round-trip delay and clock deviation. If the network round-trip delay does not exceed the network filtering delay limit and the clock deviation is not greater than the synchronization accuracy threshold, then select the server with the smallest clock deviation from the corresponding standard time servers and record it as the time synchronization server; otherwise, prompt the preset personnel time synchronization abnormality. The circuit continuity detection specifically involves: detecting the continuity status of the input circuit, processing the measured voltage drop, test current, and test lead resistance; if the obtained continuity resistance is not greater than the upper limit of continuity resistance, it indicates that the circuit is conducting normally and no processing is required; otherwise, the current continuity resistance is recorded and a preset personnel is notified that the circuit is not conducting normally. The upper limit of the on-resistance is the maximum allowable on-resistance; The sensor accuracy detection specifically involves: The sensor measures the signal twice to obtain a primary signal and a secondary signal. The actual transformation ratio is obtained by comparing and processing the secondary signal and the primary signal; The actual ratio and the nominal ratio are compared by absolute deviation to obtain the ratio error; If the ratio error is not greater than the upper limit of the ratio error, it means that the sensor accuracy is qualified and no processing is required; otherwise, the preset personnel will be prompted that the sensor accuracy is abnormal. The analog-to-digital converter noise detection is specifically as follows: if the standard deviation of the sampled values ​​of the analog-to-digital converter within a preset time period is not greater than the upper limit of the standard deviation of the sampled values, and the measured total signal-to-noise ratio is not less than the theoretical signal-to-noise ratio, then the sampling link is normal and no processing is performed; otherwise, the noise information is recorded and a preset person is alerted that there is noise interference. The signal synchronization detection specifically involves: acquiring the average synchronization delay and the standard deviation of the synchronization delay of the acquired signal; if the average synchronization delay is not greater than the upper limit of the average synchronization delay and the standard deviation of the synchronization delay is not greater than the upper limit of the standard deviation of the synchronization delay, it indicates that the acquisition synchronization accuracy is qualified and no processing is required; otherwise, a message is sent to the preset personnel indicating that the acquisition signal synchronization is abnormal.

3. The multi-agent hierarchical collaborative optimization method based on knowledge graphs and graph neural networks according to claim 1, characterized in that: The preprocessing includes temporal grid alignment, privacy desensitization, temporal missing data imputation, outlier handling, and parameter encoding. The time grid alignment specifically refers to: The source timestamp and clock deviation of each multimodal data acquisition are coupled to obtain the corrected timestamp, and then grid alignment is performed by combining the time grid interval to obtain the grid aligned timestamp. Obtain the absolute difference between the corrected timestamp and the grid alignment timestamp, and record it as the alignment error. If the alignment error is greater than the alignment error threshold, mark the multimodal data as an unaligned parameter and discard it; otherwise, do not process it. The privacy desensitization refers to performing irreversible desensitization on parameters including personally identifiable information, and performing differential privacy protection on parameters that provide statistical access, while recording the privacy desensitization operations; The temporal missing interpolation is specifically as follows: missing points are detected and interpolated on the time grid, and the missing value interpolation method is selected according to the missing length. Specifically, if a single point is missing, linear interpolation is used; if the missing length is not greater than the missing length threshold, linear state space interpolation is used; if none of the above conditions are met, the corresponding missing value is marked as a missing segment and no further processing is performed. The outlier processing refers to using the median absolute deviation method to identify outliers in multimodal data over a preset time period, removing outliers and performing temporal missing data imputation. The parameter encoding refers to performing feature encoding on different modalities and projecting it into node feature vectors. Specifically, it involves applying a pre-trained language encoder to text to obtain text vectors, applying a visual encoder to images or video frames to obtain image vectors, obtaining time-domain and frequency-domain features based on sensor sequence features, and concatenating the time-domain and frequency-domain features to obtain sensor sequence vectors.

4. The multi-agent hierarchical collaborative optimization method based on knowledge graphs and graph neural networks according to claim 1, characterized in that: The specific process for determining whether to classify the corresponding candidate entity into the entity candidate index table based on confidence level is as follows: The named entity recognizer outputs text entities and their corresponding confidence scores, while the object detector outputs visual entities and their corresponding confidence scores. The structured fields are mapped to obtain structured field entities, and the confidence level is set. Aggregate the confidence scores of all entities pointing to the same entity to obtain the aggregate confidence score. If the aggregate confidence level is not less than the write threshold, the corresponding candidate entity will be assigned to the entity candidate index table; otherwise, the system will prompt that the confidence level of the corresponding entity is abnormal.

5. The multi-agent hierarchical collaborative optimization method based on knowledge graphs and graph neural networks according to claim 1, characterized in that: The specific process for extracting causal edges to be verified from the structured event object set, including the entity candidate index table, and verifying causal relationships is as follows: Extract the causal edges to be verified from the set of structured event objects, and establish a causal verification model based on the observations corresponding to the causal edges to be verified within a preset time period. The causal verification model includes a causal regression model and an autoregressive baseline model. The results of the causal verification model are processed by squared residuals to obtain the causal residual processing results. If the causal residual processing result meets the preset conditions, a causal edge is generated, and the causal strength is obtained based on the causal residual processing result; otherwise, it indicates that there is no causal relationship between the causal edges to be verified. By using a pre-trained causal relationship extraction model, causal pairs are identified and text confidence scores are obtained. By introducing multi-source confidence weights, the text semantic confidence and causal strength are weighted and coupled to obtain multi-source confidence. If the multi-source confidence is less than the relation writing threshold, then the causal edge is marked as a pending edge; otherwise, the causal edge is written into the causal knowledge graph.

6. The multi-agent hierarchical collaborative optimization method based on knowledge graphs and graph neural networks according to claim 1, characterized in that: The hierarchical collaborative optimization process, which uses the output node representation matrix to achieve global optimization, is performed as follows: Receive the node representation matrix and iterate the leader state vector based on the leader parameters and node representation matrix from the previous iteration; A gradient evolution hybrid optimization mechanism is adopted, which updates the current gradient direction based on a multi-objective utility function that includes system efficiency index, control cost and uncertainty risk, and performs mutation operation on the gradient direction with mutation probability through a genetic evolution algorithm. The generated optimal leader strategy vector is output to the follower module; Extract the self-observable state of each follower agent in the follower module from the output of the hierarchical graph neural network; Define a policy function and a value function for evaluating long-term returns for each follower agent; A reinforcement learning framework with centralized training and distributed execution is used to collaboratively train the policies of each agent. After the strategy converges, each follower agent independently executes actions according to the corresponding strategy, and the system periodically summarizes the gains and feeds them back to the leader module. A genetic algorithm is used to select offspring from the policy set formed by the policies of each agent, and the population is updated by correcting the gradient direction, and the optimal policy parameter set is output. Meanwhile, if the global fitness gain is less than the fitness gain threshold for three consecutive generations, the mutation probability and mutation intensity will be adjusted according to a preset ratio; otherwise, no action will be taken.

7. The multi-agent hierarchical collaborative optimization method based on knowledge graphs and graph neural networks according to claim 1, characterized in that: The specific process for data twin simulation evaluation based on the optimal strategy parameter set is as follows: The virtual environment model is reconstructed by reading multimodal data from historical time periods through a digital twin engine. Load the optimal strategy parameter set and inject it into the virtual environment model; Run the optimal strategy and collect performance metrics in the simulation environment of the virtual environment model; Injecting disturbance noise into the simulation environment of the pseudo-environment model and obtaining robustness indicators to evaluate the stability of the strategy; Consistency verification is performed on the simulation logs and causal knowledge graph in the digital twin engine to verify the consistency of the causal logic of policy decisions. If the causal consistency index is less than the causal consistency index limit, the cross-layer attention weight is adjusted to reassess the node influence; otherwise, it means that the strategy conforms to the causal reasoning logic and no action is taken.

8. The multi-agent hierarchical collaborative optimization method based on knowledge graphs and graph neural networks according to claim 1, characterized in that: The process of selecting the optimal strategy parameter set based on the simulation evaluation results is as follows: The evaluation results, including the cumulative benefits of each agent's strategy, robustness index, causal consistency index, and resource consumption rate, are summarized to construct a multi-objective index matrix. The multi-modal data are then normalized to obtain a standardized multi-objective dataset. Using non-dominated sorting and crowding distance calculation mechanisms, the policies of each agent are hierarchically divided to obtain the Pareto front policy set. The obtained Pareto front policy set is subjected to multi-objective screening. The screening is terminated when the preset convergence condition is met, and the final optimal policy and corresponding policy parameters are output.

9. The multi-agent hierarchical collaborative optimization method based on knowledge graphs and graph neural networks according to claim 5, characterized in that: Also includes: Using the PC algorithm and Granger test algorithm, the observed values ​​corresponding to the causal edges to be verified within a preset time period are analyzed, and it is determined whether there is a causal relationship between the observed values ​​of the result event and the observed values ​​of the cause event. If the PC algorithm determines that there is no causal relationship between the observed values ​​of the causal event and the observed values ​​of the result event, and the Granger test shows that there is a causal relationship, then the causal relationship is marked as a candidate causal relationship, and the causal strength of the causal edge is reduced according to a preset ratio. If the PC algorithm determines that there is no causal relationship between the observed values ​​of the causal event and the observed values ​​of the result event, and the Granger test shows that there is no causal relationship, then it means that there is no causal relationship between the observed values ​​of the causal event and the observed values ​​of the result event. If the PC algorithm determines that there is a causal relationship between the observed values ​​of the causal event and the observed values ​​of the result event, and the Granger test shows that there is no causal relationship, then the causal relationship is marked as a candidate causal relationship, and the causal strength of the causal edge is reduced according to a preset ratio. If the PC algorithm determines that there is a causal relationship between the observed values ​​of the causal event and the observed values ​​of the result event, and the Granger test shows that there is a causal relationship, then it means that there is a causal relationship between the observed values ​​of the causal event and the observed values ​​of the result event, and the causal strength is directly assigned. After attenuating the edge weights in the causal knowledge graph within a preset attenuation period, an online change point detection algorithm is run in real time for each causal edge. If a change point exists, a local causal relationship reassessment is triggered; otherwise, no action is taken. The local causal relationship re-evaluation means re-invoking the causal verification model and making a causal relationship judgment. If there is no causal relationship on the causal edge, the causal edge is deleted; otherwise, the causal edge is regenerated and assigned causal strength.

10. The multi-agent hierarchical collaborative optimization method based on knowledge graphs and graph neural networks according to claim 7, characterized in that: Also includes: If the causal consistency index is not less than the causal consistency index limit, no processing is performed; otherwise, the causal edge corresponding to the condition in the indicator function being false is obtained, the corresponding causal edge is recorded as a causal inconsistency edge, and the knowledge graph is adjusted and optimized. The knowledge graph adjustment and optimization specifically involves adjusting the meta-parameters to the corresponding corrected meta-parameters during the next round of training. The corrected meta-parameters represent the results obtained by mapping the causal consistency index and meta-parameters into the corresponding meta-parameter mapping sets. Meanwhile, when obtaining cross-layer attention weights at the level where the causal inconsistency edge is located, a causal penalty term is introduced and coupled with the attention score to correct the attention score. The causal penalty term represents the result obtained by mapping the causal consistency index into the causal penalty term mapping set.

Images