Electric power Internet of Things industrial control equipment safety assessment method based on multi-modal large model

Abstract

The invention discloses an electric power Internet of Things industrial control equipment safety assessment method based on a multi-modal large model. The method mainly comprises the steps of data collection and preprocessing, multi-modal large model processing, safety risk assessment and result display and decision support. The method comprises the following steps: firstly, collecting multi-modal data of the power Internet of Things industrial control equipment, and preprocessing to ensure the consistency and accuracy of the data; secondly, performing deep fusion and feature extraction on the multi-source heterogeneous data by adopting a multi-modal large model technology and combining an orthogonal sequential fusion (OSF) strategy; then, establishing and training a security risk assessment model, quantifying the fault probability, the risk level and the potential influence, and providing more accurate security assessment in combination with background information provided by the knowledge graph; and finally, displaying an evaluation result through a visual interface. According to the invention, the automation level and accuracy of safety risk assessment of the power Internet of Things industrial control equipment are effectively improved, and stable operation and safety management of a power system are ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of power Internet of Things industrial control network security, and in particular to a security risk assessment method for power Internet of Things industrial control equipment based on a multi-modal large model. Background Art

[0002] Information network is the basis for realizing automatic monitoring and automatic control of power Internet of Things equipment. Once there is a problem with the information network, it will affect the control system or monitoring system of the power Internet of Things application, and the operating power equipment may be damaged. With the upgrading of industrial control facilities, there are great information security risks, especially in the power Internet of Things system serving external parties, such as shore power network. Security monitoring or evaluation of power Internet of Things systems or equipment is of great significance to timely strengthen the safety management and protection of the control system and avoid safety accidents.

[0003] At present, most of the existing equipment status assessment technologies based on deep learning methods are based on artificial intelligence technology, and intelligently classify and process the operating status information of industrial control equipment, such as monitoring data and monitoring images. These technologies are limited to the annotation of data sets of "fault images" or "characteristic status parameters" and the establishment of feature libraries, and cannot provide text descriptions of faults and risks, historical correlation data of faults, and other information such as fault causal analysis. Therefore, it is difficult to provide a comprehensive evaluation of the digitalization of industrial control equipment or networks.

[0004] In order to accurately assess the risk level of the power Internet of Things network and provide scientific management and maintenance solutions, it is necessary to establish an industrial control equipment risk assessment system to dynamically collect monitoring data and management data of power Internet of Things industrial control equipment, apply large models to process multimodal industrial control equipment data, and use artificial intelligence technology to achieve security risk assessment, so as to effectively improve the automation level of security risk assessment of the power Internet of Things. Summary of the invention

[0005] The purpose of this invention is to propose a security assessment method for industrial control equipment of the power Internet of Things based on a multimodal large model. This method processes the multi-source heterogeneous data generated by the industrial control equipment through the multimodal large model technology, and combines it with the knowledge graph technology to achieve a comprehensive security risk assessment of the industrial control equipment of the power Internet of Things.

[0006] To achieve the above purpose, the present invention proposes a method for security assessment of power Internet of Things industrial control equipment based on a multimodal large model, the steps of which are as follows:

[0007] S1. Data collection and preprocessing: responsible for collecting real-time monitoring data and management data from power IoT industrial control equipment and preprocessing; the preprocessing process includes data cleaning, labeling, formatting, encoding and segmentation; at the same time, using knowledge graphs to integrate and associate data from different sources to improve the consistency and accuracy of data processing;

[0008] S2, multi-modal large model processing, responsible for processing and analyzing multi-modal data from power IoT industrial control equipment; adopts orthogonal sequential fusion (OSF) technology, by gradually fusing data of different modes, improves the complementarity and diversity of feature representation, thereby improving the accuracy and reliability of risk assessment; in addition, through training, evaluation and updating models, in-depth analysis and understanding of the security status of industrial control equipment are achieved; at the same time, combined with the entity and relationship information in the knowledge graph, the feature representation of the model is optimized to improve its ability to understand the security status of industrial control equipment;

[0009] S3. Security risk assessment: using artificial intelligence technology and knowledge graph technology to model the extracted features and conduct risk assessment on the safety of industrial control equipment; the assessment results include quantitative indicators such as failure probability, risk level, and potential impact;

[0010] S4. Result display and decision support: The evaluation results are presented to users intuitively, such as displaying risk levels and failure trends through a visual interface. At the same time, decision support is provided, such as automatically generating maintenance recommendations and early warning notifications, and combining knowledge graphs to provide entity relationship diagrams and attribute information to enhance the interpretability and operability of the data.

[0011] Preferably, in step S1, the data collection collects data of different modes from the power Internet of Things industrial control equipment, including but not limited to text data, image data, audio data and video data; wherein the text data may include equipment logs, alarm information, and operation records; the image data may include monitoring images, thermal imaging images, and fault images of the equipment; the audio data may include equipment operation sounds and alarm sounds; and the video data may include real-time monitoring videos and fault videos.

[0012] Preferably, in step S1, the data preprocessing performs preliminary cleaning, formatting and encoding on the multi-source heterogeneous data collected from the industrial control equipment to ensure the quality and consistency of the data. The specific steps are as follows:

[0013] S11. Data cleaning: remove noise and errors in the data, including removing duplicate data, correcting erroneous records, and processing missing values;

[0014] S12. Data annotation: Annotate the collected data, including entity recognition of text, object detection and classification of images, and speech recognition of audio;

[0015] S13, data formatting: converting the same modality data of different formats and structures into a unified format;

[0016] S14. Data coding: Perform specific coding processing on data of different modalities to ensure the compatibility and interpretability of the data in subsequent processing;

[0017] S15. Data segmentation: Split the data into training set, validation set, and test set to facilitate model training and evaluation, ensuring that the model can learn from the data and effectively evaluate its performance;

[0018] S16. Knowledge graph integration: Use knowledge graph technology to integrate and associate data from different sources to further improve the consistency and accuracy of data processing.

[0019] Preferably, in step S2, the multimodal large model processing steps are as follows:

[0020] S21, multimodal data fusion;

[0021] S22, model training and optimization;

[0022] S23, Model evaluation and validation;

[0023] S24, extract the final fusion features.

[0024] Preferably, in step S21, the multimodal data fusion adopts the OSF data fusion strategy to form a unified feature representation of data sets of different modalities, fully consider the heterogeneity and correlation of data of different modalities, ensure the complementarity and integrity of the data, and avoid information redundancy. The specific steps are as follows:

[0025] S211, initialize the single modality model, train independent models for each modality, and evaluate their performance on specific tasks;

[0026] S212, modal sorting: sorting the modes according to the performance of the single-modal model, starting from the mode with the weakest performance and gradually transitioning to the mode with the strongest performance;

[0027] S213, sequential fusion, first, select the two lowest-ranked modalities for fusion, and use the orthogonal loss function to train the fusion model; specifically, assuming there are N modalities and N-1 fusion layers L i =[embd 1,i ,embd 2,i ], the orthogonal loss function expression is:

[0028]

[0029] Among them, L OSF is the orthogonal loss function of OSF, L is the loss function of a given task, preds and outputs are the predicted values ​​and true values ​​of the model, i represents the fusion layer number, and λ 1,i and λ 2,i They are used to adjust the strength of L1 and L2 regularization, embd1,i and embd 2,i Represents the fusion layer L i Two different embedding vectors, embd 1,i ·embd 2,i Represents the dot product operation;

[0030] Then, the result of each fusion step is fused with the next modality, and the model is trained using the orthogonal loss function. This process is repeated until all modalities are fused in sequence.

[0031] Preferably, in step S22, a back propagation algorithm is used to optimize the loss function, and during the training process, different modalities are weighted to maximize the diversity and complementarity of the fused embedding.

[0032] Preferably, in step S23, by performing a comprehensive evaluation of the key evaluation indicators on an independent test set, this evaluation process is specifically aimed at the performance of modal fusion, fully considering the diversity and complexity of data under different safety states, and ensuring the practicality of the evaluation results for the modal fusion effect; at the same time, the verification data and rules in the knowledge graph are used to perform additional verification on the fused model to accurately evaluate the enhancing effect of modal fusion on model performance. After completing the above steps, the final fusion features are extracted for subsequent analysis and application.

[0033] Preferably, in step S3, the security risk assessment includes establishing a security assessment model, model training and verification, risk quantification, risk threshold setting, and continuous learning and updating, and the specific steps are as follows:

[0034] S31. Establish a security assessment model: Based on the extracted features, select a suitable artificial intelligence algorithm to establish a security assessment model; the choice of model depends on the characteristics of the data and the requirements of the assessment task; combine the entity and relationship information in the knowledge graph to optimize the feature representation of the model and improve its ability to understand the security status of industrial control equipment;

[0035] S32. Model training and validation: Use labeled historical data to train the safety assessment model. The labeled data includes normal operating conditions and known abnormal or fault conditions. During the training process, the model will learn how to predict the safety risks of the equipment based on the input features and the additional information provided by the knowledge graph. After training, the model is validated using an independent test set to evaluate its accuracy and reliability. If necessary, optimize by adjusting the model parameters to ensure the best performance of the model.

[0036] S33, Risk Quantification: After model training is completed, the model can perform risk assessment on new data instances and generate quantitative indicators, including failure probability, risk level and potential impact; failure probability refers to the possibility of equipment failure, risk level is a comprehensive score of equipment safety risks, and potential impact assesses the possible consequences of failure;

[0037] S34, Risk threshold setting: Set risk thresholds to facilitate decision-making. When the assessment result exceeds the threshold, the system triggers an alarm and notifies relevant personnel to take action. Threshold setting needs to comprehensively consider the equipment operating environment, safety standards and business requirements, while considering the security rules and constraints extracted from the knowledge graph;

[0038] S35. Continuous learning and updating: Continuous learning and updating can be achieved through online learning or regular retraining of the model. At the same time, the entity and relationship information in the knowledge graph is updated to maintain the effectiveness of the evaluation model.

[0039] Preferably, in step S4, the result display and decision support mainly include:

[0040] S41. Visual interface: The assessment results are presented to users in the form of charts, dashboards, and heat maps. The risk level of different areas or equipment is intuitively displayed using color-coded risk maps. Combined with knowledge graphs, relationship diagrams and attribute information of entities are provided to enhance the interpretability and operability of data.

[0041] S42, Risk Report: The system automatically generates a detailed assessment report, including risk level, failure probability, and potential impact; the report can be provided in PDF, Excel or HTML format; the report content is combined with the entity information in the knowledge graph to provide more comprehensive background and analysis;

[0042] S43, Real-time monitoring and alarm: The system provides real-time monitoring function to continuously track the operating status and security risks of industrial control equipment; when it detects that the potential security risk exceeds the set threshold, the system automatically triggers an alarm and notifies relevant personnel through email, SMS or application push;

[0043] S44, Maintenance suggestions: Based on the evaluation results, the system automatically generates maintenance suggestions to guide users to take corresponding preventive measures or repair operations; the suggestions may include replacing faulty parts, adjusting equipment configuration, and strengthening safety protection. In addition, the system combines maintenance records and failure modes in the knowledge graph to provide more accurate maintenance suggestions.

[0044] S45. Early warning notification: The system issues early warning notifications based on risk assessment results to help users take timely actions to avoid or mitigate potential losses; early warning notifications are customized according to different risk levels and urgency levels;

[0045] S46. Decision support tools: The system provides decision support tools, such as trend analysis, historical comparison, and simulation forecasting, to help users better understand the reasons behind risks and develop effective risk management strategies;

[0046] S47. User interaction: includes query, filtering, and sorting related interactive functions, enabling users to view and analyze data according to their needs; using knowledge graphs, providing entity navigation and search functions, allowing users to access and understand data more conveniently.

[0047] Therefore, the present invention proposes a security assessment method for power Internet of Things industrial control equipment based on a multimodal large model, and its beneficial effects are as follows:

[0048] (1) The present invention adopts advanced multimodal large model technology to deeply process and analyze the multi-source heterogeneous data generated by the power Internet of Things industrial control equipment, thereby achieving a comprehensive and accurate security risk assessment, thereby significantly improving the safety of equipment operation.

[0049] (2) The present invention adopts the orthogonal sequential fusion (OSF) method combined with expert knowledge sorting for multimodal data fusion. By gradually fusing different modal data from power Internet of Things industrial control equipment, it allows for selective weighting of modalities, thereby avoiding unbalanced representation and limited fusion control between modalities. This method promotes orthogonal representation, extracts complementary information, and effectively avoids redundant information between modalities, ultimately generating fusion features. This process can improve the efficiency and effectiveness of data fusion.

[0050] (3) The present invention realizes the automation of security risk assessment and improves the efficiency and real-time performance of security management of industrial control equipment in the power Internet of Things. The system can monitor the operating status of industrial control equipment in real time, identify potential security risks in a timely manner, and notify relevant personnel through an automatic early warning mechanism to ensure that timely measures are taken.

[0051] (4) The present invention provides users with intuitive risk information and maintenance suggestions by providing a visual interface and detailed assessment reports, helping users to formulate effective security management strategies and preventive measures; at the same time, the model in the invention has the ability of online learning and regular updating, can adapt to changes in the operating environment and security threats of industrial control equipment, and maintain the effectiveness and accuracy of the assessment model.

[0052] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is an overall flow chart of the security assessment method of power Internet of Things industrial control equipment based on a multi-modal large model of the present invention;

[0054] Figure 2 It is a structural diagram of the power Internet of Things industrial control equipment security assessment method based on a multi-modal large model of the present invention;

[0055] Figure 3 It is a structural diagram of the data preprocessing step of the present invention;

[0056] Figure 4 This is a flowchart of the multimodal large model processing of the present invention. DETAILED DESCRIPTION

[0057] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments.

[0058] Unless otherwise defined, technical or scientific terms used in the present invention shall have the common meanings understood by one having ordinary skills in the field to which the present invention belongs.

[0059] like Figure 1-2 As shown, the present invention provides a method for security assessment of power Internet of Things industrial control equipment based on a multi-modal large model, and the specific steps are as follows:

[0060] S1. Data collection and preprocessing: responsible for collecting real-time monitoring data and management data from power IoT industrial control equipment and preprocessing; the preprocessing process includes data cleaning, labeling, formatting, encoding and segmentation; at the same time, using knowledge graphs to integrate and associate data from different sources to improve the consistency and accuracy of data processing;

[0061] Collect data of different modes from power IoT industrial control equipment, including but not limited to:

[0062] Text data, including equipment logs, alarm information, and operation records; equipment logs include equipment operation logs, fault logs, and maintenance logs; alarm information includes alarm records and descriptions generated by the system; and operation records include records and descriptions of user operations on the equipment.

[0063] Image data, including monitoring images, thermal imaging images and fault images of the equipment; among them, the equipment monitoring image is an image of the equipment operating status monitored in real time by a camera; the thermal imaging image is an image of the equipment temperature distribution obtained using a thermal imaging device; the fault image is an on-site image recording the equipment failure.

[0064] Audio data, including equipment operation sound and alarm sound; equipment operation sound is the sound characteristics recorded when the equipment is operating normally and abnormally; alarm sound is the alarm audio record generated by the system.

[0065] Video data includes real-time monitoring video and fault video; among them, real-time monitoring video is the video data that records the operation status of the equipment in real time through the video monitoring system; fault video is the on-site video that records when the equipment fails.

[0066] like Figure 3 As shown in the figure, the multi-source heterogeneous data generated by the collected industrial control equipment is preliminarily cleaned, formatted and encoded to ensure the quality and consistency of the data and provide suitable data input for the subsequent multi-modal large model processing. The specific steps are as follows:

[0067] (1) Data cleaning: remove noise and errors from the data, including removing duplicate data, correcting erroneous records, and handling missing values. The main goal of data cleaning is to ensure the integrity and reliability of the data. For text data, the cleaning step includes removing irrelevant information (such as HTML tags, special characters, etc.) and correcting spelling errors; for image data, the cleaning step includes removing blurry or noisy images; for audio and video data, the cleaning process includes removing silent segments and erroneous records. The application of rule-based cleaning algorithms can automatically identify and correct common errors, while cluster cleaning is used to identify and remove abnormal data points.

[0068] (2) Data labeling: Labeling the collected data, including entity recognition of text, object detection and classification of images, and speech recognition of audio. Labeling is an important step in data processing, especially for the training of artificial intelligence algorithms. Text data is automatically labeled with entities such as equipment name, fault type, etc. through named entity recognition (NER) technology; image data is automatically labeled with equipment parts and fault areas through object detection algorithms; audio data is labeled with equipment operating status and abnormal sounds through speech recognition technology. Rule-based labeling is suitable for standardized tasks, while deep learning labeling tools are used for complex and unstructured data. During the labeling process, human intervention can be combined to improve the accuracy and consistency of labeling.

[0069] (3) Data formatting: Convert homomodal data of different formats and structures into a unified format. The purpose of data formatting is to eliminate the structural differences between the same type of data and ensure the compatibility of different data sets in subsequent processing. For example, all text data is converted to UTF-8 encoding format to ensure that the language model can parse it correctly; all image data is standardized to the same resolution and color space to meet the input requirements of the image processing model. In this way, data formatting improves the training efficiency and performance of the model.

[0070] (4) Data encoding: Specific encoding processing is performed on data of different modalities to ensure the compatibility and interpretability of the data in subsequent processing. For example, word embedding encoding (such as Word2Vec or BERT) is performed on text data to capture the semantic relationship and contextual information of words; image data is encoded using a pre-trained CNN model (such as ResNet, VGG); and audio data is encoded using Mel spectrum encoding to capture the frequency characteristics of the audio signal. The selection of each encoding method is based on its performance in a specific application scenario to ensure the effectiveness of data representation and the optimization of model performance.

[0071] (5) Data segmentation: Use data segmentation algorithms, such as random segmentation and proportional segmentation, to segment the data; divide the data into training set, validation set, and test set to facilitate model training and evaluation, ensuring that the model can learn from the data and effectively evaluate its performance.

[0072] (6) Knowledge graph integration: Using knowledge graph technology, data from different sources are integrated and associated to further improve the consistency and accuracy of data processing. As a structured semantic network, the knowledge graph represents entities related to power Internet of Things industrial control equipment (such as equipment, components, events) and their relationships (such as causal relationships, temporal relationships) as nodes and edges. By constructing a knowledge graph, information from different data sources is associated, for example, the equipment failure description in the text log is associated with the corresponding image, audio and video data to form a complete event representation. This integration not only improves the consistency of data processing, but also enhances the semantic understanding and reasoning capabilities of the data, providing richer contextual information for subsequent multimodal large model processing.

[0073] S2, multimodal large model processing, is responsible for processing and analyzing multimodal data from industrial control equipment of the power Internet of Things; the orthogonal sequential fusion (OSF) technology is used to gradually fuse data of different modes to improve the complementarity and diversity of feature representation, thereby improving the accuracy and reliability of risk assessment; in addition, through training, evaluation and updating the model, an in-depth analysis and understanding of the security status of industrial control equipment is achieved; at the same time, the entity and relationship information in the knowledge graph is combined to optimize the feature representation of the model and improve its ability to understand the security status of industrial control equipment; the specific processing steps are as follows:

[0074] (1) Multimodal data fusion;

[0075] (2) Model training and optimization;

[0076] (3) Model evaluation and validation;

[0077] (4) Extract the final fusion features.

[0078] like Figure 4As shown in the figure, multimodal data fusion adopts the OSF data fusion strategy to form a unified feature representation of data sets of different modalities, fully considering the heterogeneity and correlation of data of different modalities, ensuring the complementarity and integrity of the data, and avoiding information redundancy. The specific steps are as follows:

[0079] A1. Initialize a single modality model, train independent models for each modality, and evaluate their performance on specific tasks.

[0080] A2. Modal sorting: sorting the modalities according to the performance of the single-modal model, starting from the modality with the weakest performance and gradually transitioning to the modality with the strongest performance; in addition, the modal sorting step of the present invention allows the integration of expert knowledge, and the fusion order is determined in advance by presetting an order that is beneficial to the model.

[0081] A3. Sequential fusion. First, select the two modalities with the lowest ranking for fusion and use the orthogonal loss function to train the fusion model. Specifically, assuming there are N modalities and N-1 fusion layers L i =[embd 1,i ,embd 2,i ], the orthogonal loss function expression is:

[0082]

[0083] Among them, L OSF is the orthogonal loss function of OSF, L is the loss function of a given task, preds and outputs are the predicted values ​​and true values ​​of the model, i represents the fusion layer number, and λ 1,i and λ 2,i They are used to adjust the strength of L1 and L2 regularization, embd 1,i and embd 2,i Represents the fusion layer L i Two different embedding vectors, embd 1,i ·embd 2,i Represents the dot product operation;

[0084] The L1 regularization term encourages orthogonality between the embedding vectors by making the absolute value of their dot products close to zero, while the L2 regularization term achieves the same goal by making the square of the dot product close to zero. Together, these two regularization terms ensure orthogonality between the embedding vectors, helping the model extract unique and complementary information from each modality.

[0085] The result of each fusion step is fused with the next modality, and the model is trained continuously using the orthogonal loss function. This process is repeated until all modalities are fused in sequence.

[0086] Model training and optimization,The back-propagation algorithm is used to optimize the loss function. During the training process, different modalities are weighted to maximize the diversity and complementarity of the fused embedding.

[0087] Model evaluation and verification, through comprehensive evaluation of key evaluation indicators on an independent test set, this evaluation process is specifically aimed at the performance of modal fusion, fully considering the diversity and complexity of data under different safety states, and ensuring the practicality of the evaluation results for the modal fusion effect; at the same time, the verification data and rules in the knowledge graph are used to perform additional verification on the fused model to accurately evaluate the enhancing effect of modal fusion on model performance.

[0088] After completing the above steps, the final fusion features are extracted for subsequent analysis and application.

[0089] S3. Security risk assessment: Use artificial intelligence technology combined with knowledge graph technology to model the extracted features and conduct risk assessment on the safety of industrial control equipment; the assessment results include quantitative indicators such as failure probability, risk level, and potential impact.

[0090] Security risk assessment includes establishing a security assessment model, model training and verification, risk quantification, risk threshold setting, and continuous learning and updating. The specific steps are as follows:

[0091] (1) Establish a security assessment model: Based on the extracted features, select appropriate artificial intelligence algorithms (such as deep neural networks, random forests, etc.) to establish a security risk assessment model; the choice of model depends on the characteristics of the data and the requirements of the assessment task; combine the entity and relationship information in the knowledge graph to optimize the feature representation of the model and improve its ability to understand the security status of industrial control equipment.

[0092] In the process of model construction, the entity and relationship information in the knowledge graph can be further combined, which not only optimizes the feature representation of the model, but also significantly improves the model's ability to understand the safety status of industrial control equipment. For example, by integrating the equipment's operating history, fault records, and related environmental factors (such as temperature and pressure) into the model, the model can more accurately capture potential risk patterns and causal relationships.

[0093] In addition, ensemble learning methods can be used to combine the advantages of multiple algorithms to further improve the robustness and prediction accuracy of the model. Ensemble learning can build a strong model by combining multiple weak models (such as multiple decision trees in a random forest), thereby reducing the bias and variance problems that may be caused by a single model and improving the overall prediction ability of the safety status of industrial control equipment.

[0094] (2) Model training and verification: The safety assessment model is trained using labeled historical data; the labeled data includes normal operating conditions and known abnormal or fault conditions. During the training process, the model gradually learns how to predict the safety risks of the equipment based on the input features and the additional information provided by the knowledge graph through repeated iterations. Specifically, the model uses the input features (such as equipment temperature, vibration frequency, etc.) and the associated information in the knowledge graph (such as the comparison of historical failure modes and current operating conditions) for joint analysis to identify potential risk factors and predict possible future failures.

[0095] After training, the model is validated using an independent test set to evaluate its accuracy and reliability. The test set usually contains data samples that are different from the training set to ensure the generalization ability of the model. Validation indicators include accuracy, recall, precision, and F1 score, which can fully reflect the performance of the model in practical applications.

[0096] If necessary, you can optimize the model by adjusting the model parameters to ensure the best performance of the model. For example, adjusting the learning rate of the neural network, regularization parameters, or increasing the number of training iterations of the model can help improve the predictive ability and robustness of the model. In addition, you can use the cross-validation method to further evaluate the stability of the model to ensure that it performs well on different data sets.

[0097] (3) Risk quantification: After model training is completed, the model can perform risk assessment on new data instances; the model outputs quantitative indicators, including failure probability, risk level and potential impact; failure probability refers to the possibility of equipment failure, which is predicted and output by the model; risk level is a comprehensive score of equipment safety risks, and potential impact assesses the possible consequences of failure, such as equipment downtime and production loss.

[0098] (4) Risk threshold setting: Set risk thresholds to facilitate decision-making; when the assessment result exceeds the threshold, the system triggers an alarm and notifies relevant personnel to take action; threshold setting needs to comprehensively consider the equipment operating environment, safety standards and business needs, while considering the security rules and constraints extracted from the knowledge graph. For example, for certain key equipment, a lower threshold can be set to ensure that risks can be dealt with as soon as they begin to appear; for secondary equipment, a higher threshold can be set to reduce unnecessary intervention and costs; set a reasonable threshold range based on the equipment's operating environment; refer to industry safety standards and specifications to set risk thresholds; set the sensitivity of the risk level based on the specific needs of the enterprise.

[0099] (5) Continuous learning and updating: The operating environment and security threats of industrial control equipment change over time, and the assessment model needs to be updated regularly to maintain its effectiveness. Continuous learning and optimization can be achieved through online learning or regular retraining of the model. This method allows the model to continuously adapt to new environmental changes and equipment status, ensuring that it can maintain efficient prediction capabilities in a rapidly changing environment. At the same time, the entity and relationship information in the knowledge graph is regularly updated to enhance the model's adaptability to the latest data. Ensure that the model can maintain efficient and accurate risk assessment capabilities when facing new threats.

[0100] S4. Result display and decision support: The evaluation results are presented to users intuitively, such as displaying risk levels and failure trends through a visual interface. At the same time, decision support is provided, such as automatically generating maintenance recommendations and early warning notifications, and combining knowledge graphs to provide entity relationship diagrams and attribute information to enhance the interpretability and operability of the data.

[0101] Results presentation and decision support mainly include:

[0102] (1) Visual interface: The assessment results are presented to users in the form of charts, dashboards, heat maps, etc. These visualization tools help users quickly understand and identify risk levels and failure trends; use dashboards to display key risk indicators, such as failure probability, risk level, etc., so that users can see at a glance; use color-coded risk maps to intuitively display the risk levels of different areas or equipment; combine with knowledge graphs to provide entity relationship diagrams and attribute information to enhance the interpretability and operability of data.

[0103] (2) Risk Report: The system automatically generates a detailed assessment report, including key information such as risk level, failure probability, and potential impact. The report is provided in PDF, Excel, or HTML format, making it easy for users to save and share. The report content is combined with entity information in the knowledge graph to provide more comprehensive background and analysis, helping users better understand risk sources and their associated factors.

[0104] (3) Real-time monitoring and alarm: The system provides real-time monitoring functions to continuously track the operating status and security risks of industrial control equipment to ensure timely detection of potential problems. When it is detected that the potential security risk exceeds the set threshold, the system automatically triggers an alarm and notifies relevant personnel via email, SMS or application push to ensure timely response.

[0105] (4) Maintenance recommendations: Based on the evaluation results, the system automatically generates maintenance recommendations to guide users to take appropriate preventive measures or repair operations; recommendations may include replacing faulty parts, adjusting equipment configuration, strengthening safety protection, etc., and combined with maintenance records and failure modes in the knowledge graph, more accurate maintenance recommendations are provided to ensure the effectiveness and pertinence of maintenance measures.

[0106] (5) Early warning notification: The system issues early warning notifications based on risk assessment results to help users take timely actions to avoid or mitigate potential losses. Early warning notifications are customized according to different risk levels and urgency levels to ensure the accuracy and timeliness of the notifications.

[0107] (6) Decision support tools: Provide trend analysis tools to help users identify long-term risk trends and changes; provide historical data comparison functions to help users understand the differences between current risks and historical data; provide simulation prediction tools to help users predict possible future risk situations and formulate effective risk management strategies.

[0108] (7) User interaction: Users can search for specific equipment or risk information according to their needs; provide data filtering function, users can filter data according to different conditions and view specific data sets; provide sorting function, users can sort data according to indicators such as risk level and failure probability; use knowledge graph to provide entity navigation and search functions, so that users can more conveniently access and understand data; by combining the result display and decision support modules of the knowledge graph, the present invention not only provides a powerful security risk assessment function, but also ensures the ease of use and practicality of the assessment results, helping users to effectively manage and control the security risks of power Internet of Things industrial control equipment.

[0109] Therefore, the present invention is a security assessment method for electric power Internet of Things industrial control equipment based on a multimodal large model. By adopting advanced multimodal large model technology, multi-source heterogeneous data generated by electric power Internet of Things industrial control equipment is deeply processed and analyzed, thereby achieving a comprehensive and accurate security risk assessment, thereby significantly improving the safety of equipment operation; at the same time, knowledge graph technology is used to integrate and associate data from different sources, optimize the feature representation of the model, significantly improve the consistency and accuracy of data processing, and enhance the model's understanding and prediction capabilities of the equipment's security status; the automation of security risk assessment is realized, and the efficiency and real-time performance of security management of electric power Internet of Things industrial control equipment are improved.

[0110] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. Security assessment method of power Internet of Things industrial control equipment based on multi-modal large model, It is characterized in that Here are the steps: S1. Data collection and preprocessing: collect real-time monitoring data and management data from power IoT industrial control equipment, and perform data preprocessing; S2, multi-modal large model processing, responsible for processing and analyzing multi-modal data from power Internet of Things industrial control equipment; S3. Security risk assessment: using artificial intelligence technology and knowledge graph technology to model the extracted features and conduct risk assessment on the security of industrial control equipment; S4. Result display and decision support: the evaluation results are intuitively displayed to users and decision support is provided.

2. According to the method for security assessment of power Internet of Things industrial control equipment based on multi-modal large model as described in claim 1, It is characterized in that In step S1, the data collection collects data of different modes from the power Internet of Things industrial control equipment, including but not limited to text data, image data, audio data and video data.

3. According to the method for security assessment of power Internet of Things industrial control equipment based on multi-modal large model as described in claim 1, It is characterized in that In step S1, the data preprocessing performs preliminary cleaning, formatting and encoding on the multi-source heterogeneous data generated by the collected industrial control equipment to ensure the quality and consistency of the data. The specific steps are as follows: S11. Data cleaning: remove noise and errors in the data, including removing duplicate data, correcting erroneous records, and processing missing values; S12. Data annotation: Annotate the collected data, including entity recognition of text, object detection and classification of images, and speech recognition of audio; S13, data formatting: converting the same modality data of different formats and structures into a unified format; S14. Data coding: Perform specific coding processing on data of different modalities to ensure the compatibility and interpretability of the data in subsequent processing; S15. Data segmentation: Split the data into training set, validation set, and test set to facilitate model training and evaluation, ensuring that the model can learn from the data and effectively evaluate its performance; S16. Knowledge graph integration: Use knowledge graph technology to integrate and associate data from different sources to further improve the consistency and accuracy of data processing.

4. According to the method for security assessment of power Internet of Things industrial control equipment based on multi-modal large model as described in claim 1, It is characterized in that In step S2, the multimodal large model processing steps are as follows: S21, multimodal data fusion; S22, model training and optimization; S23, model evaluation and validation; S24, extract the final fusion features.

5. According to the method for security assessment of power Internet of Things industrial control equipment based on multi-modal large model as described in claim 4, It is characterized in that In step S21, the multimodal data fusion adopts the OSF data fusion strategy to form a unified feature representation of data sets of different modalities. The steps are as follows: S211, initialize the single modality model, train independent models for each modality, and evaluate their performance on specific tasks; S212, modal sorting: sorting the modes according to the performance of the single-modal model, starting from the mode with the weakest performance and gradually transitioning to the mode with the strongest performance; S213, sequential fusion, first, select the two lowest-ranked modalities for fusion, and use the orthogonal loss function to train the fusion model. Specifically, assuming there are N modalities and N-1 fusion layers L i =[embd 1i ,embd 2i ], the orthogonal loss function expression is: Among them, L OSF is the orthogonal loss function of OSF, L is the loss function of a given task, preds and outputs are the predicted values ​​and true values ​​of the model, i represents the fusion layer number, and λ 1,i and λ 2,i They are used to adjust the strength of L1 and L2 regularization, embd 1,i and embd 2,i Represents the fusion layer L i Two different embedding vectors, embd 1,i ·embd 2,i Represents the dot product operation; Then, the result of each fusion step is fused with the next modality, and the model is trained using the orthogonal loss function. This process is repeated until all modalities are fused in sequence.

6. According to the method for security assessment of power Internet of Things industrial control equipment based on multi-modal large model as described in claim 4, It is characterized in that In step S22, the back propagation algorithm is used to optimize the loss function. During the training process, different modalities are weighted to maximize the diversity and complementarity of the fused embedding.

7. According to claim 4, a method for safety assessment of power Internet of Things industrial control equipment based on a multi-modal large model, It is characterized in that In step S23, by comprehensively evaluating the key evaluation indicators on an independent test set, this evaluation process is specifically aimed at the performance of modal fusion, fully considering the diversity and complexity of data under different safety states, and ensuring the practicality of the evaluation results for the modal fusion effect; At the same time, the verification data and rules in the knowledge graph are used to perform additional verification on the fused model to accurately evaluate the enhancing effect of modal fusion on model performance. After completing the above steps, the final fusion features are extracted for subsequent analysis and application.

8. According to the method for security assessment of power Internet of Things industrial control equipment based on multi-modal large model as claimed in claim 1, It is characterized in that In step S3, the security risk assessment includes establishing a security assessment model, model training, risk quantification, risk threshold setting, and continuous learning and updating. The specific steps are as follows: S31. Establish a security assessment model: Based on the extracted features, select a suitable artificial intelligence algorithm to establish a security assessment model; S32, Model training and verification: Use labeled historical data to train the safety assessment model. The labeled data includes normal operating status and known abnormal or fault status. After the training is completed, use an independent test set to verify the model to evaluate its accuracy and reliability; S33, Risk quantification: After model training is completed, the model can perform risk assessment on new data instances and generate quantitative indicators, including failure probability, risk level and potential impact; S34. Risk threshold setting: Setting risk thresholds to facilitate decision making; S35. Continuous learning and updating: Continuous learning and updating can be achieved through online learning or regular retraining of the model. At the same time, the entity and relationship information in the knowledge graph is updated to maintain the effectiveness of the evaluation model.

9. According to the method for security assessment of power Internet of Things industrial control equipment based on multi-modal large model as claimed in claim 1, It is characterized in that In step S4, the result display and decision support mainly include: S41. Visual interface: The assessment results are presented to users in the form of charts, dashboards, and heat maps. The risk level of different areas or equipment is intuitively displayed using color-coded risk maps. Combined with knowledge graphs, relationship diagrams and attribute information of entities are provided to enhance the interpretability and operability of data. S42, Risk Report: The system automatically generates a detailed assessment report, including risk level, failure probability, and potential impact; the report can be provided in PDF, Excel or HTML format; the report content is combined with the entity information in the knowledge graph to provide more comprehensive background and analysis; S43, Real-time monitoring and alarm: The system provides real-time monitoring function to continuously track the operating status and safety risks of industrial control equipment; S44, Maintenance suggestions: Based on the evaluation results, the system automatically generates maintenance suggestions to guide users to take corresponding preventive measures or repair operations; S45, Early Warning Notification: The system issues early warning notifications based on risk assessment results to help users take timely actions to avoid or mitigate potential losses; S46. Decision support tools: The system provides decision support tools, such as trend analysis, historical comparison, and simulation forecasting, to help users better understand the reasons behind risks and develop effective risk management strategies; S47. User interaction: includes query, filtering, and sorting related interactive functions, enabling users to view and analyze data according to their needs; using knowledge graphs, providing entity navigation and search functions, allowing users to access and understand data more conveniently.

Images