Multi-modal knowledge graph construction method and system for energy industry Internet

Abstract

The invention discloses a multi-modal knowledge graph construction method and system for an energy industry internet, and belongs to the technical field of energy industry internet security. The method comprises the steps of collecting multi-source modal data of the energy industry internet, determining a dynamic weight of each modal type of data in the multi-source modal data, and performing data cleaning on the multi-modal data; for the multi-modal data after data cleaning, feature enhancement processing is carried out according to the dynamic weight, a data set is generated, and the generated data set is divided; and based on the divided data set, establishing semantic association of energy industry internet structuring to establish a knowledge graph, and performing graph optimization and graph semantic enhancement based on service constraint attention on the knowledge graph to obtain a multi-modal knowledge graph. According to the method, the problems of fixed multi-modal data weight and poor adaptive energy time sequence characteristics are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of energy industry internet security, and more particularly, to a multi-modal knowledge graph construction method and system for energy industry internet. BACKGROUND

[0002] Although the multi-modal knowledge graph construction technology for energy industry internet has been initially explored, there are still many core technology deficiencies, which seriously limit its deep application in complex energy industrial control scenes such as converter stations and substations, and it is difficult to meet the actual needs of abnormal traffic accurate detection and full-link tracing.

[0003] Firstly, the scene adaptability is seriously insufficient: the existing technology focuses on general industrial scenes and is not designed for the special properties of energy industry internet. It neither fully adapts to the exclusive industrial control protocol traffic characteristics of the SCADA system of the converter station data acquisition and monitoring control system and the station control layer of the substation (such as the timing interaction logic of IEC104 protocol and the function code mapping relationship of power special Modbus protocol), nor combines with the energy core business process (such as the power regulation instruction link of the converter station and the inspection sequence of the substation) to construct the correlation logic, resulting in the disconnection of the multi-modal knowledge graph and the energy business logic, and the inability to accurately reflect the actual association relationship of "device-protocol-business".

[0004] Secondly, the depth of multi-modal data fusion is lacking: the existing scheme either only fuses part of the modal data (such as single device account and traffic log), or does not establish a cross-modal semantic association mechanism, which cannot realize the deep fusion of industrial control protocol traffic data (dynamic timing type), device static attribute data (structured), and business process rule data (text logic type), resulting in the difficulty of the graph to provide multi-dimensional collaborative context analysis capability, and the inability to support multi-perspective tracing of abnormal traffic.

[0005] Thirdly, the full-link context memory capability is missing: the existing technology generally ignores the timing continuity of industrial control instruction interaction, does not construct the full-link timing storage mechanism of "device sends instruction→protocol transmission→business process triggers→result feedback", and cannot restore the complete trigger path of abnormal traffic. When facing abnormality caused by multi-device collaboration, it is difficult to locate the root device and key interaction node, and the tracing capability is greatly limited.

[0006] In addition, the energy business rules are not integrated: the existing graph construction does not implant energy industry exclusive business constraints (such as the threshold limit of ±5% of the power regulation amplitude of the converter station and the trigger condition of the fault isolation instruction of the substation), but only relies on general data association logic, resulting in business logic errors in graph instance association (such as marking instructions that do not meet the power regulation specification as normal association), which cannot support abnormal compliance judgment at the business level, further reducing the accuracy of abnormal detection.

[0007] In summary, the core problem of the prior art is that a multi-modal knowledge graph that adapts to the energy industrial internet scene, integrates multi-modal semantic association, has full-link memory, and incorporates business rules cannot be constructed, which leads to the inability to provide reliable knowledge support for the accurate detection and full-link tracing of energy industrial control abnormal traffic. SUMMARY

[0008] To solve the above problems, the present application provides a multi-modal knowledge graph construction method for the energy industrial internet, comprising:

[0009] Collecting multi-source modal data of the energy industrial internet, and determining the dynamic weight of each modal type data in the multi-source modal data, and performing data cleaning on the multi-modal data;

[0010] For the multi-modal data after data cleaning, performing feature enhancement processing according to the dynamic weight, generating a data set, and dividing the generated data set;

[0011] Based on the divided data set, establishing structured semantic association of the energy industrial internet to establish a knowledge graph, and performing graph optimization and graph semantic reinforcement based on business constraint attention on the knowledge graph to obtain a multi-modal knowledge graph.

[0012] Optionally, collecting multi-source modal data of the energy industrial internet, and determining the dynamic weight of each modal type data in the multi-source modal data, comprises:

[0013] Collecting industrial control protocol traffic data, device static data and business process data of the energy industrial internet, determining the scene importance of the industrial control protocol traffic data, device static data and business process data, and using a Gaussian membership function to assign dynamic weights to the industrial control protocol traffic data, device static data and business process data according to the scene importance.

[0014] Optionally, based on the divided data set, the structured semantic association of the energy industrial internet is established by a semantic mapping algorithm to establish a knowledge graph, comprising:

[0015] Through the semantic mapping algorithm, the multi-modal data in the divided data set is converted into ontology instances of the energy industrial internet, and the structured semantic association of devices, protocols and businesses is established to establish the semantic skeleton of the knowledge graph. Through a semantic extraction model, the knowledge features of entities, relationships and events are extracted from the multi-modal data, and the knowledge features are mapped to the semantic framework to establish a knowledge graph.

[0016] Optionally, the knowledge graph is optimized, comprising:

[0017] The parameters of the dynamic knowledge graph are dynamically adjusted by adapting the Diffusion model to the sparsity of the energy data to supplement missing instances of the knowledge graph or correct incorrect associations of the knowledge graph.

[0018] The parameters of the dynamic knowledge graph include:

[0019] An adaptive diffusion rate is calculated and dynamically adjusted.

[0020] Gaussian noise is added to the forward diffusion process, and a denoising process is learned for the reverse generation process to recover instance features.

[0021] Optionally, the graph semantic reinforcement based on business constraint attention includes:

[0022] The attention mechanism of the knowledge graph is adjusted by incorporating the energy business rules to enhance the semantic association strength of devices, protocols and businesses.

[0023] The attention mechanism includes business constraint attention mask construction and business constraint attention calculation.

[0024] Optionally, the method further includes:

[0025] After graph optimization and graph semantic reinforcement based on business constraint attention, the consistency of the knowledge graph instances and the energy business standards is detected based on the strong time sequence characteristics of the energy flow and the time sequence weighted longest common subsequence LCS algorithm, and whether the knowledge graph conforms to the energy industry operation specification is determined, and the instance association that does not conform to the business logic is removed.

[0026] Optionally, the method further includes:

[0027] For the multi-modal knowledge graph for energy industrial internet constructed, a multi-objective optimization function is defined to adjust the model parameters so that the comprehensive performance of the multi-modal knowledge graph is optimal, and the multi-modal knowledge graph with optimal comprehensive performance is deployed in the energy industrial internet.

[0028] The defined multi-objective optimization function includes a loss function that combines Diffusion generation loss, attention association loss, consistency loss and modal difference penalty term.

[0029] In another aspect, the application also provides a multi-modal knowledge graph construction system for energy industrial internet, which includes:

[0030] The data acquisition unit is configured to acquire multi-source modal data of the energy industrial internet, determine dynamic weights of each type of modal data in the multi-source modal data, and perform data cleaning on the multi-modal data.

[0031] a data processing unit, configured to perform feature enhancement processing on the multi-modal data after data cleaning according to the dynamic weight, generate a data set, and divide the generated data set;

[0032] a graph construction unit, configured to establish structured semantic association of the energy industrial internet based on the divided data set, establish a knowledge graph, and perform graph optimization and graph semantic reinforcement based on business constraint attention on the knowledge graph, to obtain a multi-modal knowledge graph.

[0033] In another aspect, the present application also provides a computing device, comprising: one or more processors;

[0034] a processor, configured to execute one or more programs;

[0035] When the one or more programs are executed by the one or more processors, the method as described above is implemented.

[0036] In another aspect, the present application also provides a computer readable storage medium, having a computer program stored thereon, which, when executed, implements the method as described above.

[0037] Compared with the prior art, the present application has the following beneficial effects:

[0038] The present application provides a multi-modal knowledge graph construction method for the energy industrial internet, comprising: collecting multi-source modal data of the energy industrial internet, determining dynamic weights of each modal type data in the multi-source modal data, and performing data cleaning on the multi-modal data; performing feature enhancement processing on the multi-modal data after data cleaning according to the dynamic weight, generating a data set, and dividing the generated data set; based on the divided data set, establishing structured semantic association of the energy industrial internet, establishing a knowledge graph, and performing graph optimization and graph semantic reinforcement based on business constraint attention on the knowledge graph, to obtain a multi-modal knowledge graph. The present application solves the problem of fixed multi-modal data weight and poor adaptation to energy time sequence characteristics. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 The flowchart of the method of the present application is shown in the figure;

[0040] Figure 2 The structural diagram of the system of the present application is shown in the figure. DETAILED DESCRIPTION

[0041] Reference will now be made to the drawings to describe the exemplary embodiments of the present application in greater detail. The present application can be variously embodied and is not limited to the embodiments described herein, which are provided for the purpose of full and complete disclosure of the present application and to fully convey the scope of the present application to those skilled in the art. The terms used in the exemplary embodiments represented in the drawings are not limited to the terms used in the description. In the drawings, like reference numerals refer to like elements throughout.

[0042] Unless otherwise defined, the terms (including technical terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art. In addition, it is to be understood that the terms defined by commonly used dictionaries are to be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

[0043] Embodiment 1:

[0044] The present application proposes a multi-modal knowledge graph construction method S100 for an energy industry internet, as shown in Figure 1 The method comprises the following steps:

[0045] S101, collecting multi-source modal data of the energy industry internet, and determining dynamic weights of each modal type data in the multi-source modal data, and performing data cleaning on the multi-modal data;

[0046] S102, for the multi-modal data after data cleaning, performing feature enhancement processing according to the dynamic weights, generating a data set, and dividing the generated data set;

[0047] S103, based on the divided data set, establishing a structured semantic association of the energy industry internet, to establish a knowledge graph, and performing graph optimization and graph semantic reinforcement based on business constraint attention on the knowledge graph, to obtain a multi-modal knowledge graph.

[0048] The steps S101-S103 are described in detail as follows:

[0049] Among them, collecting multi-source modal data of the energy industry internet, and determining dynamic weights of each modal type data in the multi-source modal data, comprises:

[0050] Collecting industrial control protocol flow data, device static data and business process data of the energy industry internet, determining the scene importance of the industrial control protocol flow data, device static data and business process data, and using a Gaussian membership function to assign dynamic weights to the industrial control protocol flow data, device static data and business process data according to the scene importance.

[0051] This step is mainly to obtain the energy industry internet "device-protocol-service" full-dimensional original data, and to initialize the multi-modal data dynamic weight based on the energy scene timing characteristics (such as the difference between peak and valley load), to provide a scene-adapted weight basis for subsequent data fusion, covering the data feature requirements of converter station and substation core scenes.

[0052] Specifically includes:

[0053] 1. Industrial protocol traffic data collection: Deploy traffic probes (such as version 6.0.0 Zeek traffic probe) on the mirror port of the core switch of the SCADA system in the converter station, and on the communication link between the station control layer and the interval layer of the substation, capture the request / response messages and data payload of the industrial protocol (such as IEC104, Modbus, DNP3, S7Comm protocol) at the preset sampling frequency (such as 1000Hz), and store them in the preset format (such as PCAP / PCAPNG format); The collection period covers the peak, flat and valley segments of the power system (such as peak segment 9:00-11:00, 19:00-21:00, valley segment 0:00-8:00), and records the fluctuation coefficient of the traffic data in each period (such as peak segment traffic fluctuation coefficient about 0.35, valley segment about 0.12, which can be adjusted according to actual scene measurement).

[0054] 2. Static data collection of equipment: Obtain the account data (such as equipment model, factory number, supported protocol version) and hardware parameters (such as communication port rate, CPU load threshold) of the converter station equipment (such as converter valve) and substation equipment (such as transformer) through the standard interface of industrial control network (such as OPCUA interface), and store them in the preset database (such as MySQL database, version can be selected as 8.0 and above); Define the freshness of equipment data (such as equipment data freshness = 1-(current time-data update time) / equipment maintenance period, the freshness should not be less than 0.8, and the equipment maintenance period can be set according to the power equipment maintenance regulations).

[0055] 3. Business process data collection: Organize business experts with energy industry experience (such as power system dispatching engineers and substation operation and maintenance engineers with more than 5 years of experience), sort out the core business processes of energy (such as converter station power regulation and substation inspection process), and define business rules (such as converter station power regulation trigger threshold and substation inspection interval), and build a business rule library; Store the process documents in the preset format (such as XML format), and define the activation rate of business rules (such as the number of activated rules / total number of rules, the peak segment activation rate is about 0.75, and the valley segment is about 0.4, which will change with the scene).

[0056] 4. Dynamic weight initialization: Based on the scene importance of each modal data, use Gaussian membership function to calculate the initial weight, realize the nonlinear dynamic distribution of weight (adapt to the timing characteristics of energy scene), the formula is as follows:

[0057]

[0058] wherein, m represents the modal type (1 for equipment modal, 2 for flow modal, 3 for business modal), ξ m (t) is the characteristic value of modal m at time t (such as equipment modal taking data freshness, flow modal taking fluctuation coefficient, business modal taking rule activation rate), ξ m,max is the characteristic maximum value of modal m (such as equipment modal 1, flow modal 0.4, business modal 0.8, which can be determined by scene measurement statistics), σ m is the characteristic standard deviation of modal m (such as equipment modal 0.15, flow modal 0.08, business modal 0.12, which is obtained based on multiple sets of energy scene data statistics).

[0059] wherein, the multi-modal data is subjected to data cleaning, and for the multi-modal data after data cleaning, feature enhancement processing is performed according to the dynamic weight, mainly removing redundant and incorrect information in the original data through multi-dimensional cleaning, combining with the energy scene feature to design a feature enhancement method, improving the data quality and discrimination, and providing high availability data support for subsequent modeling.

[0060] Specifically, it includes:

[0061] 1. Multi-dimensional data cleaning: using a protocol analysis tool (such as Scapy library, version can be selected as 2.5.0 and above) to parse flow data files, extract industrial protocol core information (such as protocol header, field structure, data payload), and remove low-layer protocol data (such as TCP / IP header information); using a data processing tool (such as Pandas library, version can be selected as 2.0.3 and above) to remove duplicate device data (such as based on device ID + serial number composite primary key), and correct errors (such as based on device model regular expression matching); removing redundant steps for business data (such as repeated device status confirmation steps), and unifying process step naming specification.

[0062] 2. Feature enhancement: design adaptive enhancement strategies for different modal data: for flow data, use time-frequency analysis method (such as wavelet transform, can select db4 wavelet basis or other wavelet basis suitable for energy data, decomposition layer number can be set to 3 layers) to extract time-frequency features; for equipment data, use encoding and normalization method (such as one-hot encoding + embedding layer processing classification features, Min-Max normalization processing numerical features, embedding layer dimension, normalization range can be preset); for business data, use semantic encoding method (such as Word2Vec, window size can be set to 5, vector dimension can be set to 128, specific according to business text length adjustment) to convert process steps into semantic vectors.

[0063] 3. Dataset division: based on stratified sampling method, the enhanced dataset is divided into training set, validation set and test set, and the division ratio is dynamically adjusted according to the data volume (for example, when the data volume is less than 100,000, the training set is 60%, the validation set is 20%, and the test set is 20%; when the data volume is greater than or equal to 100,000, the training set is 70%, the validation set is 10%, and the test set is 20%), to ensure that the distribution of each modal data in the subset is consistent with the original dataset, and meet the needs of model training, validation and testing.

[0064] Among them, based on the divided dataset, the structured semantic association of the energy industrial internet is established through the semantic mapping algorithm, to establish the knowledge graph, including:

[0065] Through the semantic mapping algorithm, the multi-modal data in the divided dataset is converted into the ontology instance of the energy industrial internet, and the structured semantic association of the equipment, protocol and business is established, the semantic skeleton of the knowledge graph is established, the knowledge features of entities, relationships and events are extracted from the multi-modal data through the semantic extraction model, and the knowledge features are mapped to the semantic framework to establish the knowledge graph.

[0066] This step is mainly to build an ontology framework integrated with energy business constraints, convert multi-modal data into ontology instances through a semantic mapping algorithm, and establish a structured semantic association of "equipment-protocol-business" to provide a semantic skeleton consistent with the energy scene for the knowledge graph.

[0067] Specifically, it includes:

[0068] 1. Energy-specific ontology modeling: use ontology construction tools (such as Protégé tool, version 5.5.0 and above) to build ontology (support OWL2DL language), and use inference engine (such as HermiT inference engine, version 1.4.3.456 and above) to verify ontology consistency. Define core classes and properties:

[0069] Device class: sub-classes of converter station equipment and substation equipment (such as converter valve, smoothing reactor, transformer, circuit breaker), data attributes include rated power, communication port rate, etc. (attribute units can be pre-set, such as rated power unit MW, communication port rate unit Mbps);

[0070] Protocol class: sub-classes of common industrial control protocols (such as IEC104 protocol, Modbus protocol), data attributes include transmission period, function code range, etc. (such as transmission period unit ms);

[0071] Business process class: sub-classes of energy core business process (such as converter station power regulation process, substation inspection process), data attributes include trigger threshold, process time, etc. (such as trigger threshold unit MW, process time unit s);

[0072] Object property: define "support protocol" (domain device class, value domain protocol class), "associated process" (domain protocol class, value domain business process class), describe the association relationship between classes. Implant energy-specific SWRL rules (such as "converter valve with rated power exceeding preset value needs to support specified industrial control protocol"), ensure that the ontology conforms to the energy business logic.

[0073] 2. Instance semantic mapping: adopt semantic extraction model (such as BERT-BiLSTM-CRF model, pre-training model can be selected bert-base-chinese, hidden layer dimension, CRF iteration number can be preset) to extract entities, relationships and events from multi-modal data:

[0074] Entity extraction: extract device ID, rated parameter and other entities from device data, and map them to device class instances (such as converter valve instance, URI can be defined according to preset rules);

[0075] Relationship extraction: extract the interaction relationship between devices and protocols (such as "a device sends a protocol instruction") from traffic data, and map it to object property instances;

[0076] Event extraction: extract business process trigger events (such as "a device starts power regulation process at a certain time") from business data, and map them to business process class instances.

[0077] Among them, the knowledge graph is optimized, including:

[0078] By adapting the Diffusion model of energy data sparsity, the parameters of the dynamic knowledge graph are dynamically adjusted to supplement the missing instances of the knowledge graph or correct the wrong associations of the knowledge graph;

[0079] Among them, the parameters of the dynamic knowledge graph include:

[0080] Calculate the adaptive diffusion rate and dynamically adjust the adaptive diffusion rate;

[0081] For the forward expansion process, add Gaussian noise, and for the reverse generation process, learn the denoising process to recover the instance features.

[0082] This step mainly adjusts the model parameters by adapting the Diffusion model of energy data sparsity to supplement missing instances, correct wrong associations, improve the integrity and accuracy of the graph instances, and solve the problem of missing device / process data in the energy scenario.

[0083] Specifically, it includes:

[0084] 1. Adaptive diffusion rate calculation: For the sparsity of different modal instances (such as 1-actual instance number / theoretical instance number, theoretical instance number=device number x protocol number x business number), the diffusion rate is dynamically adjusted to ensure that the modal instances with high sparsity are more fully optimized, and the formula is as follows:

[0085]

[0086] Where m is the modal type (1=device, 2=flow, 3=business), β m,base is the basic diffusion rate of modal m (such as device modal 0.004, flow modal 0.002, business modal 0.003, which can be adjusted according to instance characteristics), S m is the instance sparsity of modal m, S m,thres is the sparsity threshold (such as 0.3), σ S is the sparsity standard deviation (such as 0.1), and the tanh function realizes the nonlinear increase of the diffusion rate when the sparsity exceeds the threshold.

[0087] 2. Forward diffusion process: Gaussian noise is gradually added to the instance feature vector, and the noise distribution is dynamically adjusted according to the importance of the instance (such as device rated power, flow fluctuation coefficient, business rule activation rate), to ensure that important instances have lower noise injection intensity and retain core features.

[0088] 3. Reverse generation process: a neural network with modal cross-attention (such as a Transformer encoder, the number of layers can be set to 4 and the number of attention heads can be set to 8) is used to learn the denoising process, restore instance features, and consider the correlation between multiple modalities (such as the association between device parameters and protocol types, the association between business processes and flow timing), and the formula is as follows:

[0089]

[0090] Where μ θ (x t ,m) is the mean vector of modal m instance features (calculated by existing technology such as a Transformer encoder, with multiple modalities concatenated as input), Σ θ (x t ,m) is a low-rank covariance matrix (such as Σ θ = LL T + λI, L is a low-rank matrix, λ is a regularization term, I is an identity matrix, and the rank and regularization term value can be preset), and θ is a network parameter (trained using an optimizer such as AdamW, with learning rate, weight decay, and iteration number preset).

[0091] Where graph semantic reinforcement based on business constraint attention includes:

[0092] By introducing an attention mechanism with energy business rules into the knowledge graph, the correlation weight between instances is adjusted to enhance the semantic correlation strength of devices, protocols, and business.

[0093] The attention mechanism includes business constraint attention mask construction and business constraint attention calculation.

[0094] This step mainly adjusts the correlation weight between instances by introducing an attention mechanism with energy business rules to enhance the semantic correlation strength of "device-protocol-business", improves the fitting degree of the graph to the energy business logic, and solves the problem of disconnection between the semantic correlation of the existing graph and the business.

[0095] Specifically, it includes:

[0096] 1. Business constraint attention mask construction: based on the energy business rule library, a nonlinear penalty (reducing the attention weight) is applied to the instance pair that violates the business rules, and the instance pair that meets the rules maintains the normal weight. The attention mask matrix is constructed to ensure that the semantic correlation conforms to the business logic.

[0097] 2. Business constraint attention calculation: the mask matrix is integrated into the self-attention mechanism to calculate the correlation weight between instances, and a time sequence weight (such as higher weight for recent instances, adapting to the strong time sequence characteristics of energy flow) is introduced. The formula is as follows:

[0098]

[0099] Where Q is the query matrix, K is the key matrix (the dimension can be set to n x d k , n is the number of instances, d k is the feature dimension, such as d_k=64), M is the attention mask matrix, is the Hadamard product, b attn is the attention bias (which can be initialized and updated during training), V j is the value matrix, τ i,j is the interaction time difference between instances i and j, τ max is the maximum interaction time difference (such as 3600s, adapting to the energy business cycle), and the exp term is the time sequence weight.

[0100] After graph optimization and business constraint attention-based graph semantic reinforcement of the knowledge graph, based on the strong time sequence characteristics of energy flow and the time sequence weighted longest common subsequence LCS algorithm, the consistency of the knowledge graph instances with the energy business standard is detected, and whether the knowledge graph conforms to the energy industry operation specification is determined, and the instance correlation that does not conform to the business logic is removed.

[0101] This step mainly combines the strong temporal characteristics of energy flow to design the Temporally Weighted Longest Common Subsequence (LCS) algorithm to detect the consistency between the graph instances and energy business standards, ensure that the graph complies with energy industry operating specifications, and eliminate instance associations that do not conform to business logic.

[0102] Specifically, it includes:

[0103] 1. Time-series weight calculation: Based on the temporal characteristics of the instance sequence, a time-series weight is assigned to each instance (more recent instances have higher weights, and the decay coefficient can be preset), as shown in the following formula:

[0104] w t =exp(-γ·(t) now -t))

[0105] Among them, t now t is the current time, γ is the time the instance occurred, and γ is the time decay coefficient (e.g., 0.0003, which can be adjusted according to the characteristics of the business time).

[0106] 2. Time-weighted LCS calculation: Based on dynamic programming, time-series weights are incorporated into the traditional LCS to improve the adaptability of consistency detection to energy time-series characteristics. The recursive formula is as follows:

[0107]

[0108] Where A is the graph instance sequence, B is the business standard sequence, i and j are sequence indices, and t is the sequence index. A (i) represents the time when A[i] occurs, t B (j) represents the standard time of B[j], and 0.1 represents the weighting coefficient of non-matching items (to reduce the contribution of non-matching items to LCS, which can be adjusted according to consistency requirements).

[0109] 3. Consistency determination: Set a consistency score threshold (e.g., 0.82, determined based on energy business consistency detection experiments). When the detection score is lower than the threshold, return to the "Adaptive Diffusion Optimization" step to reprocess instance association.

[0110] Specifically, for the constructed multimodal knowledge graph for the energy industrial internet, the model parameters are adjusted by defining a multi-objective optimization function to optimize the overall performance of the multimodal knowledge graph, and the multimodal knowledge graph with the optimal overall performance is deployed in the energy industrial internet;

[0111] The defined multi-objective optimization function includes a loss function that integrates Diffusion generation loss, attention association loss, consistency loss, and modality difference penalty term.

[0112] This step mainly adjusts the model parameters through a multi-objective optimization function to ensure the optimal comprehensive performance of the model, and at the same time, deploys the optimized graph to the energy industrial internet environment to provide practical support for abnormal traffic detection and tracing.

[0113] Specifically includes:

[0114] 1. Multi-objective model optimization: define the total loss function, fuse Diffusion generation loss, attention correlation loss, consistency loss and modal difference penalty term (avoid overfitting of a certain modal), realize multi-objective collaborative optimization, and the formula is as follows:

[0115] L total = λ1L Diff + λ2L Attn + λ3L Cons + λ4·Var(L Diff ,L Attn ,L Cons )

[0116] Wherein, L Diff is the Diffusion generation loss (such as the sum of the squares of the L2 norms of the generated instances and the real instances), L Attn is the attention correlation loss (such as the negative logarithmic entropy of the attention weight), L Cons is the consistency loss (such as the value: 1-consistency score), Var(·) is the variance of the three losses, λ is the loss weight (such as λ1=0.4, λ2=0.3, λ3=0.2, λ4=0.1, determined based on multi-objective optimization experiments); the total loss is minimized using an optimizer (such as the Adam optimizer, the initial learning rate can be set to 1e-3, and the cosine annealing strategy is used for updating); and the convergence condition (such as the loss change being less than a threshold within a continuous preset number of iterations) is set.

[0117] 2. Graph deployment: select hardware devices that adapt to the industrial environment (such as industrial-grade servers, the configuration can be referred to: CPU such as Intel Xeon Gold series, memory such as 128GB and above, hard disk such as 2TB SSD and above), install an operating system (such as Ubuntu Server system, version can be selected 22.04 and above), deploy a graph database (such as Neo4j database, version can be selected 5.10 and above, can enable plug-in support for multi-modal data import) to store the graph, and deploy an interface framework (such as FastAPI framework, version can be selected 0.103.1 and above) to build RESTful API interface (requires the interface response time to meet the real-time requirements of the industry).

[0118] 3. Application docking: dock the graph with the energy industrial internet abnormal detection system to provide core functions:

[0119] Abnormal traffic tracing: input abnormal traffic identification, return the full-link sequence of "device instance->protocol instance->service instance" and the associated weight (based on dynamic weight algorithm calculation), ensure that the tracing result is accurate and consistent with the business logic;

[0120] Business compliance check: input business process identification, return the violated SWRL rule and severity, support business-level abnormal compliance judgment.

[0121] In summary, the present application aims at the core pain points of multi-modal knowledge graph construction in the energy industry internet (converter station, substation) scene, and proposes an innovative technical path of "dynamic adaptation-deep optimization-business integration". Compared with the existing technology, the innovation points are as follows:

[0122] 1. Design a dynamic modal weight algorithm based on Gaussian membership, combine the peak-valley load difference of the energy scene (such as traffic fluctuation coefficient, business rule activation rate) to realize the nonlinear dynamic distribution of multi-modal data weight, solve the problem that traditional fixed weight cannot adapt to the time sequence characteristics of energy data;

[0123] 2. Propose an adaptive Diffusion instance optimization method, dynamically adjust the diffusion rate based on instance sparsity, fuse multi-modal correlation using low-rank covariance matrix, effectively supplement missing instances, and significantly improve graph integrity;

[0124] 3. Construct a business constraint attention mechanism, generate an attention mask through SWRL rules, integrate time sequence weight calculation instance correlation strength, significantly improve the fit degree of graph semantic correlation and energy business logic, solve the problem of existing graph and business disconnection;

[0125] 4. Innovate the time sequence weighted LCS consistency detection algorithm, design instance time sequence weight combined with the strong time sequence characteristics of energy flow, improve the adaptability of consistency detection to energy scene, and ensure that the graph meets the industry operation specification. Through the above innovations, the finally constructed multi-modal knowledge graph can accurately support the detection and full-link tracing of abnormal traffic in the energy industry internet, filling the gap of existing technology in energy scene adaptability and semantic correlation depth.

[0126] The present application solves the problem of fixed multi-modal data weight and poor adaptation to energy time sequence characteristics;

[0127] The application realizes nonlinear dynamic allocation of multi-modal data of equipment, traffic and service by combining the peak-valley load difference of energy scene (such as flow fluctuation coefficient, service rule activation rate) based on the dynamic modal weight algorithm of Gaussian membership, automatically increases the weight of traffic data in the peak segment to focus on high concurrency interaction, and automatically increases the weight of equipment data in the valley segment to strengthen the association of static attributes, without manual adjustment of weight parameters. Compared with the defects that the traditional fixed weight cannot adapt to the time sequence fluctuation of energy data, the technology improves the quality of multi-modal data fusion, provides more actual scene-based basic data for subsequent graph construction, and can accurately match the data association needs of different scenes such as peak period of power regulation of converter station and valley period of inspection of transformer substation.

[0128] The application solves the problem of missing graph instances and insufficient integrity caused by sparse energy data;

[0129] The graph instance optimization method based on adaptive Diffusion of the application supplements the missing equipment-protocol association instances and service-traffic interaction instances in the converter station / transformer substation scene by dynamically adjusting the diffusion rate (the higher the sparsity, the nonlinear increase of the diffusion rate) and the low-rank covariance matrix (fusion of multi-modal correlation). Compared with the traditional fixed parameter generation model which is easy to generate invalid instances that deviate from the energy logic, the technology significantly improves the accuracy of missing instance supplementation, effectively solves the graph “broken link” problem caused by incomplete equipment account and incomplete service process record, and ensures that the graph covers the full-link instances of “equipment->protocol->service”, providing complete knowledge support for anomaly tracing.

[0130] The application solves the problem of disconnection between graph semantic association and energy business logic and low fitting degree;

[0131] The application strengthens semantic association by generating attention masks based on SWRL rules (applying nonlinear punishment to instances that violate business rules) and integrating time sequence weight (higher weight of recent time interaction instances) through the business constraint attention mechanism. Compared with the existing technology which only relies on data feature calculation for association and ignores energy business constraints (such as converter valve power regulation exceeding threshold and missing transformer station inspection steps), the technology improves the fitting degree of graph instance association and energy business logic, can accurately identify implicit association errors such as “mismatch between equipment supporting protocol and business demand” and “time sequence interaction violating process specification”, and ensures that the graph semantics conforms to the exclusive business logic such as power regulation of converter station and fault isolation of transformer substation.

[0132] The application solves the problem that consistency detection ignores energy time sequence characteristics and compliance judgment is not accurate;

[0133] The application innovates the time sequence weighted LCS consistency detection algorithm, assigns time sequence weights (higher weight for recent time instances) to instances, and integrates the weight factor in LCS calculation, to accurately match the time sequence characteristics (such as the sequence of command interaction and the time period of business process) of energy industrial control flow. Compared with the traditional LCS which only focuses on sequence matching and ignores the time dimension, the technology can effectively distinguish between "normal process with time sequence compliance" and "violation process with time sequence disorder" (such as performing switch operation before obtaining device status in a substation), improve the accuracy of consistency detection, reduce the deviation of atlas compliance caused by time sequence misjudgment, and ensure that the atlas meets the operation specifications of the energy industry.

[0134] The application solves the problems of single atlas model optimization and poor adaptability of engineering deployment.

[0135] The application realizes model collaborative optimization through a multi-objective total loss function (combining Diffusion generation loss, attention correlation loss, consistency loss and modal difference penalty term), avoids overfitting of a certain modal caused by single-objective optimization, and uses industrial-grade hardware deployment (such as Intel Xeon Gold series servers) and FastAPI interface framework to adapt to the real-time requirements of industrial environments such as converter stations and substations (interface response time meets industrial standards) and support incremental training to quickly adapt to new devices (such as new converter valves) and business upgrades (such as power regulation process updates). Compared with the limitations of traditional model optimization and deployment relying on high-performance cloud, the technology makes the model have better comprehensive performance, and can realize local real-time application without cloud support, avoids the adaptation lag caused by full retraining, and ensures that the atlas can play a long-term stable role in the energy industry environment.

[0136] Compared with the prior art, the application breaks through the core limitations of "poor multi-modal weight adaptation, incomplete atlas instances, business-related semantic correlation, time sequence consistency detection, and engineering deployment difficulty", and constructs a multi-modal knowledge graph that is more suitable for the energy industrial internet (converter station, substation) scene, which can be widely used in industrial control abnormal flow detection and full-link tracing in the power system and new energy field, and provides knowledge support for the safe and stable operation of energy critical infrastructure.

[0137] Embodiment 2:

[0138] In still another aspect, the application also provides a multi-modal knowledge graph construction system 200 for the energy industrial internet, as shown in Figure 2 , comprising:

[0139] The data acquisition unit 201 is configured to acquire multi-source modal data of the energy industrial internet, determine the dynamic weight of each modal type data in the multi-source modal data, and perform data cleaning on the multi-modal data.

[0140] The data processing unit 202 is configured to perform feature enhancement processing on the multi-modal data after data cleaning according to the dynamic weight, generate a data set, and divide the generated data set;

[0141] The graph construction unit 203 is configured to establish structured semantic association of the energy industry internet based on the divided data set, establish a knowledge graph, and perform graph optimization and graph semantic reinforcement based on business constraint attention on the knowledge graph, to obtain a multi-modal knowledge graph.

[0142] The present application solves the problems of fixed multi-modal data weight and poor adaptation to energy time sequence characteristics.

[0143] Embodiment 3

[0144] Based on the same inventive concept, the present application further provides a computer device, which comprises a processor and a memory. The memory is used to store a computer program, the computer program comprises program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is suitable for implementing one or more instructions, and is specifically suitable for loading and executing one or more instructions in the computer storage medium to implement a corresponding method process or a corresponding function, so as to implement the steps of the method in the above embodiments.

[0145] Embodiment 4

[0146] Based on the same inventive concept, the present application also provides a storage medium, specifically a computer readable storage medium (Memory), which is a memory device in a computer device, used for storing programs and data. It can be understood that the computer readable storage medium herein can include an internal storage medium in the computer device, and of course can also include an extended storage medium supported by the computer device. The computer readable storage medium provides a storage space, which stores an operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and the instructions can be one or more computer programs (including program codes). It should be noted that the computer readable storage medium herein can be a high-speed RAM memory, or a non-volatile memory such as at least one disk memory. One or more instructions stored in the computer readable storage medium can be loaded and executed by the processor to realize the steps of the method in the above embodiments.

[0147] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of the present application can be implemented in various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0148] The present application is described with reference to flowcharts and / or block diagrams of the method, device (system), and computer program product according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one or more flows and / or blocks Figure 1 The device that implements the functions specified in one or more flows and / or blocks.

[0149] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the Figure 1 function specified in the flow or flows and / or blocks Figure 1 of the block or blocks.

[0150] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the Figure 1 function specified in the flow or flows and / or blocks Figure 1 of the block or blocks.

[0151] While the preferred embodiments of the application have been described, additional variations and modifications can be made to the embodiments by those of skill in the art once they have the benefit of the present disclosure. Therefore, the appended claims are intended to encompass within their scope all possible variations and modifications of the preferred embodiments. 1

[0152] It is apparent that a person skilled in the art can make a variety of changes and modifications to the application without departing from the spirit and scope of the application. Thus, if these modifications and variations of the application fall within the scope of the claims and their equivalents, it is intended to include them in the application.

Claims

1. A method for constructing a multi-modal knowledge graph for an energy industrial internet, characterized in that, The method comprises the following steps: Collecting multi-source modal data of the energy industrial internet, determining dynamic weights of each modal type data in the multi-source modal data, and performing data cleaning on the multi-modal data; For the multi-modal data after data cleaning, performing feature enhancement processing according to the dynamic weights, generating a data set, and dividing the generated data set; Based on the divided data set, the structured semantic association of the energy industrial internet is established to establish a knowledge graph, and the knowledge graph is optimized and the semantic of the knowledge graph is strengthened based on the business constraint attention, and a multi-modal knowledge graph is obtained. 2.The method of claim 1, wherein, The method comprises the following steps: Collecting industrial control protocol flow data, device static data and business process data of the energy industrial internet, determining the scene importance of the industrial control protocol flow data, device static data and business process data, and assigning dynamic weights to the industrial control protocol flow data, device static data and business process data according to the scene importance by using Gaussian membership function. 3.The method of claim 1, wherein, Based on the divided data set, the structured semantic association of the energy industrial internet is established to establish a knowledge graph, comprising: Through the semantic mapping algorithm, the multi-modal data in the divided data set is converted into the ontology instance of the energy industrial internet, and the structured semantic association of the device, protocol and business is established, the semantic skeleton of the knowledge graph is established, the knowledge features of entities, relationships and events are extracted from the multi-modal data by using the semantic extraction model, and the knowledge features are mapped to the semantic framework to establish the knowledge graph. 4.The method of claim 1, wherein, The method comprises the following steps: By adapting the Diffusion model of energy data sparsity, the parameters of dynamic knowledge graph are dynamically adjusted to supplement the missing instances of the knowledge graph or correct the wrong association of the knowledge graph. The parameters of the dynamic knowledge graph comprise: The adaptive diffusion rate is calculated and dynamically adjusted; For the forward expansion process, Gaussian noise is added, and for the reverse generation process, a denoising process is learned to recover the instance features. 5.The method of claim 1, wherein, The method comprises the following steps: By introducing the attention mechanism of energy business rules into the knowledge graph, the association weight between instances is adjusted to enhance the semantic association strength of devices, protocols and businesses; The attention mechanism comprises business constraint attention mask construction and business constraint attention calculation. 6.The method of claim 1, wherein, The method further comprises the following steps: After the knowledge graph is optimized and the semantic of the knowledge graph is strengthened based on the business constraint attention, the consistency of the knowledge graph instance and the energy business standard is detected based on the strong time sequence characteristics of the energy flow and the time sequence weighted longest common subsequence LCS algorithm, whether the knowledge graph conforms to the energy industry operation specification is detected, and the instance association that does not conform to the business logic is removed. 7.The method of claim 1, wherein, The method further comprises the following steps: For the constructed multi-modal knowledge graph facing the energy industry internet, a multi-objective optimization function is defined to adjust the model parameters so that the comprehensive performance of the multi-modal knowledge graph is optimal, and the multi-modal knowledge graph with optimal comprehensive performance is deployed in the energy industry internet. The defined multi-objective optimization function includes a loss function that fuses a Diffusion generation loss, an attention correlation loss, a consistency loss, and a modal difference penalty term.

8. An energy industry internet-oriented multi-modal knowledge graph construction system, characterized in that, It includes: A data acquisition unit is configured to acquire multi-source modal data of the energy industry internet, determine dynamic weights of each type of modal data in the multi-source modal data, and perform data cleaning on the multi-modal data. A data processing unit is configured to perform feature enhancement processing on the multi-modal data after data cleaning according to the dynamic weights, generate a data set, and divide the generated data set. A graph construction unit is configured to establish structured semantic associations of the energy industry internet based on the divided data set, establish a knowledge graph, and perform graph optimization and graph semantic reinforcement based on business constraint attention on the knowledge graph to obtain a multi-modal knowledge graph.

9. A computer device, comprising: It includes: One or more processors; The processor is configured to execute one or more programs; When the one or more programs are executed by the one or more processors, the method of any one of claims 1-7 is implemented.

10. A computer-readable storage medium, characterized in that, The computer program is stored thereon, and when executed, the method of any one of claims 1-7 is implemented.

Images