Intelligent important target system monitoring and analyzing system based on large model

Abstract

The invention discloses an important target system intelligent monitoring analysis system based on a large model, and the system comprises a data collection module, a data preprocessing module, an intelligent analysis module based on the large model, a result output module, and a system management and configuration module. And efficient, accurate and intelligent monitoring and analysis of key infrastructures, important military targets and high-value industrial facilities are realized. Particularly, the invention provides a large model monitoring analysis method based on domain knowledge enhancement and a large model optimization monitoring analysis scheme based on multi-modal data collaboration, and the monitoring effect and the adaptive capacity of the system are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent monitoring and analysis technology, and specifically to an intelligent monitoring and analysis system for an important target system based on a large model, which can be widely used in monitoring and analysis scenarios of key infrastructure (such as power facilities, communication base stations, etc.), important military targets (such as military bases, weapons and equipment depots, etc.) and high-value industrial facilities (such as large chemical plants, nuclear power plants, etc.). Background Art

[0002] With the rapid development of science and technology, accurate monitoring and in-depth analysis of various important targets have become key factors in ensuring social security, economic stability and national defense security. These targets cover a wide range of fields, from infrastructure such as power grids and transportation systems to military facilities related to national security. However, although traditional monitoring and analysis methods have improved monitoring efficiency and accuracy to a certain extent, they have gradually revealed their inherent limitations in the face of increasingly complex real-world challenges, especially when it comes to application scenarios that require high intelligence and adaptability.

[0003] Although traditional knowledge graph technology can effectively organize and represent a large amount of structured information, it relies heavily on predefined entities and relationship patterns. This static knowledge structure design makes it difficult for knowledge graphs to adapt to changing monitoring needs, especially when facing new equipment or complex fault modes, knowledge updating and expansion become extremely difficult. For example, in the power system, with the access of new energy and the development of smart grids, new equipment and technologies emerge in an endless stream, and the existing knowledge graph may not be able to reflect these changes in a timely manner, thus affecting the effectiveness and reliability of the monitoring system. The reasoning mechanism of knowledge graphs is mostly based on fixed logical rules, which is inadequate when dealing with complex and changing actual environments. Modern monitoring systems need to have strong context-awareness and flexible reasoning mechanisms to make accurate judgments and predictions in different situations. However, traditional knowledge graph technology has obvious shortcomings in this regard. Its reasoning process often lacks sufficient flexibility and adaptability, and cannot make timely adjustments according to changes in the environment. In modern monitoring systems, the data types from different sources are diverse, including but not limited to images, videos, sounds, texts, etc. There are complex associations between these data. How to effectively integrate these multimodal data and extract valuable information from them is a major challenge currently faced. Traditional knowledge graph technology often fails to achieve the desired effect when dealing with such problems due to the lack of an effective cross-modal data fusion mechanism, which limits the further improvement of monitoring accuracy and efficiency. With the dynamic changes in the monitoring environment, the monitoring system needs to have a certain degree of adaptive ability to automatically adjust its parameters and behaviors according to changes in external conditions. However, traditional knowledge graph technology is particularly insufficient in this regard. Its structure and reasoning rules are relatively fixed, and once determined, they are difficult to change, which leads to the system's poor response speed and adaptability when dealing with new situations, and cannot meet the rapidly changing real needs.

[0004] In summary, traditional monitoring and analysis methods can no longer meet the requirements of modern society for efficient, accurate, and intelligent monitoring and analysis of important target systems. Therefore, it is urgent to develop a new monitoring and analysis system. Summary of the invention

[0005] The present invention provides an important target system intelligent monitoring and analysis system based on a large model, comprising:

[0006] Data acquisition module, used to collect various types of data related to important targets from multiple sensors;

[0007] Data preprocessing module, used to perform cleaning, noise reduction, and normalization preprocessing operations on the collected data;

[0008] The intelligent analysis module based on the large model is used to perform intelligent analysis on the pre-processed data, identify the target status, and detect abnormal situations;

[0009] The result output module is used to output the results obtained by the intelligent analysis module to relevant users or systems in an intuitive form;

[0010] The system management and configuration module is used to manage and configure the entire system; the intelligent analysis module based on the big model includes: knowledge graph construction and embedding new model, which is used to embed the knowledge graph into the semantic space of the big model to achieve deep integration of knowledge and model;

[0011] A new algorithm based on the adaptive fine-tuning strategy is used to dynamically adjust the parameter update step of the large model according to the characteristics and monitoring task requirements, thereby improving the adaptability and generalization ability of the model.

[0012] Furthermore, the intelligent analysis module based on the large model also includes:

[0013] The new multimodal data collaboration algorithm is used to extract and fuse features of multimodal data such as images, videos, and audio collected by base-related sensors to form a unified multimodal feature representation.

[0014] Furthermore, the new multimodal data collaboration algorithm further includes:

[0015] A new modal fusion algorithm based on the cross-modal attention mechanism is used to calculate the attention weights of each modal feature vector and achieve effective fusion of multimodal data.

[0016] Furthermore, the intelligent analysis module based on the large model also includes:

[0017] A new demand-based dynamic data screening algorithm is used to dynamically select the most relevant modal data for analysis according to the specific tasks of base monitoring, thereby improving the accuracy and efficiency of monitoring.

[0018] Furthermore, the data preprocessing module includes:

[0019] The image data preprocessing submodule is used to perform preprocessing operations such as cleaning, noise reduction, and normalization on the image data;

[0020] The video data preprocessing submodule is used to perform preprocessing operations such as splitting and frame feature extraction on video data;

[0021] The audio data preprocessing submodule is used to perform preprocessing operations such as noise reduction and feature extraction on audio data.

[0022] Furthermore, the result output module includes:

[0023] The chart output submodule is used to output the monitoring results in the form of charts;

[0024] The report output submodule is used to output the monitoring results in the form of a report;

[0025] The alarm information output submodule is used to output alarm information when an abnormal situation is detected.

[0026] Furthermore, the system management and configuration module includes:

[0027] The parameter setting submodule is used to set the working parameters of each module of the system;

[0028] User rights management submodule, used to manage user access rights;

[0029] The system update submodule is used to update the system software.

[0030] Furthermore, the system also has the ability of self-learning and self-optimization, and can continuously learn new data patterns during use, automatically adjust analysis algorithms, and improve monitoring accuracy and response speed.

[0031] Beneficial technical effects of the invention:

[0032] The system realizes efficient, accurate and intelligent monitoring and analysis of key infrastructure, important military targets and high-value industrial facilities through data acquisition module, data preprocessing module, intelligent analysis module based on large model, result output module and system management and configuration module. In particular, the present invention proposes a large model monitoring and analysis method based on domain knowledge enhancement and a large model optimization monitoring and analysis scheme based on multimodal data collaboration, which significantly improves the monitoring effect and the adaptive ability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 : is the algorithm principle diagram of the present invention. DETAILED DESCRIPTION

[0034] Example 1

[0035] Large model monitoring and analysis method based on domain knowledge enhancement (taking power facility monitoring as an example):

[0036] The system of the present invention mainly includes the following modules: Data acquisition module: responsible for collecting various types of data related to important targets from a variety of sensors (such as cameras, radars, sensor networks, etc.), including images, videos, audio, location information, etc. Data preprocessing module: performs preprocessing operations such as cleaning, noise reduction, and normalization on the collected data so that subsequent modules can better process the data. Intelligent analysis module based on large models: core module, using pre-trained large models, combined with specific algorithms and parameters designed for important target systems, to perform intelligent analysis on preprocessed data, identify target states, detect abnormal situations, etc. Result output module: outputs the results obtained by the intelligent analysis module to relevant users or systems in an intuitive form (such as charts, reports, alarm information, etc.). System management and configuration module: responsible for managing and configuring the entire system, including parameter settings, user authority management, system updates, etc.

[0037] Construct a knowledge graph in the field of power facilities, including various entity information (such as models, parameters, operating status, etc. of substations, transmission lines, transformers, etc.) and association relationships. Propose an embedding model based on Multi-Scale Graph Convolutional Network (MSGCN) to embed the domain knowledge graph into the semantic space of the large model.

[0038] For each node in the knowledge graph i , in each layer l of MSGCN, calculate its embedding vector The update formula is as follows:

[0039]

[0040] Among them, N(i) represents the set of neighbor nodes of node i, is the learnable weight between node i and neighbor node j in layer l, W (l) is the weight matrix of the lth layer, σ is the rectified linear unit activation function, It means concatenating the embedding vectors of node i and neighbor node j in layer l. Through iterative calculation of multiple layers of MSGCN, the embedding vector of node i in the semantic space is finally obtained. The deep integration of knowledge graph in the field of power facilities and big model semantic space has been achieved.

[0041] Assume that the parameter of the large model is θ. During the training process, according to the characteristics of power facilities (such as the complexity of different types of substations, the length and environment of transmission lines, etc.) and the monitoring task requirements (such as detecting the fault type of power facilities, abnormal power fluctuations, etc.), an adaptive fine-tuning strategy based on dynamic Bayesian network (DBN) is adopted to dynamically adjust the parameter update step size.

[0042] Firstly, a DBN model reflecting the relationship between the characteristics of power facilities and monitoring tasks is constructed, whose nodes include the power facility characteristic variable set X and the monitoring task difficulty variable set Y.

[0043] Define the conditional probability distribution P(Y|X) to describe the probability distribution of the difficulty of the monitoring task under given power facility characteristics, and the conditional probability distribution P(X|Y) to describe the probability distribution of power facility characteristics under given monitoring task difficulty.

[0044] According to Bayes' theorem, the posterior probability distribution P(X, Y) is calculated as follows:

[0045] P(X, Y) = P(Y|X)·P(X)

[0046] Based on the calculated posterior probability distribution P(X, Y), an index I that measures the comprehensive impact of power facility characteristics and monitoring tasks is defined as follows:

[0047]

[0048] Among them, f(x, y) is a function that measures the combined impact of power facility characteristics x and monitoring task difficulty y.

[0049] Calculate the adaptive parameter update step size α t as follows:

[0050] α t =α0·exp(-β·I t )

[0051] Among them, α0 is the initial parameter update step size, and β is an adjustable hyperparameter used to control the sensitivity of step size adjustment. Through this DBN-based adaptive fine-tuning strategy, the adaptability and generalization ability of the model for the specific important target of power facilities are improved.

[0052] The parameter F is defined to measure the degree of integration between the knowledge graph in the field of power facilities and the semantic space of the large model. The specific calculation method is:

[0053]

[0054] in, represents the embedding vector of node i in the actual fusion state, Represents the embedding vector of node i in the ideal fusion state. Optimizing this parameter can ensure the best fusion effect and enable the large model to accurately use the domain knowledge of power facilities for analysis.

[0055] The parameter R is introduced to evaluate the rationality of the parameter update step in the adaptive fine-tuning strategy. The calculation method is as follows:

[0056]

[0057] Where T is the total number of training steps, θ t represents the parameters of the large model at step t, α t It represents the parameter update step size at step t. By monitoring this parameter in real time and adjusting it, the stability and effectiveness of model training can be guaranteed.

[0058] Example 2

[0059] Large model optimization monitoring and analysis scheme based on multimodal data collaboration (taking military base monitoring as an example)

[0060] The system of the present invention mainly includes the following modules: Data acquisition module: responsible for collecting various types of data related to important targets from a variety of sensors (such as cameras, radars, sensor networks, etc.), including images, videos, audio, location information, etc. Data preprocessing module: performs preprocessing operations such as cleaning, noise reduction, and normalization on the collected data so that subsequent modules can better process the data. Intelligent analysis module based on large models: core module, using pre-trained large models, combined with specific algorithms and parameters designed for important target systems, to perform intelligent analysis on preprocessed data, identify target states, detect abnormal situations, etc. Result output module: outputs the results obtained by the intelligent analysis module to relevant users or systems in an intuitive form (such as charts, reports, alarm information, etc.). System management and configuration module: responsible for managing and configuring the entire system, including parameter settings, user authority management, system updates, etc.

[0061] Feature extraction is performed on multimodal data such as image I, video V, audio A, etc. collected from relevant sensors of the military base.

[0062] For image data I, an improved convolutional neural network (CNN) is used to extract features and obtain the image feature vector f I Assume that the convolution kernel size of CNN is k, the step size is s, the padding method is p, and special filters for military target recognition (such as filters for recognizing camouflage patterns, specific weapon equipment shapes, etc.) are added to the convolution layer. After passing through a certain layer l of CNN, the feature vector f I The calculation of is as follows:

[0063] f I =CNN l (I; k, s, p)

[0064] For video data V, split the video into a frame sequence {V1, V2, ..., V n}, for each frame, a CNN similar to the image is used to extract features and obtain a frame feature vector sequence , and then the video feature vector f is obtained through average pooling operation V :

[0065]

[0066] For audio data A, the Mel Frequency Cepstral Coefficient (MFCC) method based on the audio characteristics of military scenes is used to extract features and obtain the audio feature vector f A For example, the features commonly found in military bases, such as gunfire sounds and vehicle driving sounds, are optimized and extracted.

[0067] A new modal fusion algorithm based on the cross-modal attention mechanism is proposed to fuse these features to form a unified multimodal feature representation f M For each modal eigenvector f I 、f V 、f A , calculate the fused feature vector f M ,as follows:

[0068]

[0069] Among them, α m is the attention weight of each modality, calculated as follows:

[0070]

[0071] Among them, sim(f m , f M ) is a learnable function used to measure the correlation between each modal feature vector and the fused feature vector. Here, a function based on the matching degree of military target features is used, that is, comparing the matching degree of the military target features reflected by each modal feature vector (such as the appearance of weapons and equipment in the image, the posture of personnel in the video, and the specific sound in the audio) with the overall features of the military target expected to be reflected by the fused feature vector. In this way, the effective fusion of multimodal data of military bases is achieved so that the large model can better process and analyze.

[0072] Assume that the current analysis demand of the large model is represented by vector D, and the importance of each modal data is represented by vector W (where each element of W corresponds to the importance weight of the modal data such as image, video, and audio). The analysis demand and importance weight here are determined according to the specific tasks of military base monitoring (such as monitoring personnel activities, vehicle entry and exit, and weapon equipment deployment around the military base).

[0073] A new dynamic data screening algorithm based on the Probability Graph Model (PGM) is proposed to calculate the correlation score S between each modal data and the analysis requirements. m as follows:

[0074] S m =P(D|f m )

[0075] Among them, P(D|f m ) is given a modal eigenvector f m Next, analyze the probability of demand D. By calculating the correlation score of each modal data, the most relevant modal data can be dynamically selected for analysis to improve the accuracy and efficiency of monitoring.

[0076] Example 3

[0077] Comparative test data table display

[0078] The following uses power facility monitoring and military base monitoring as examples to demonstrate comparative test data in different aspects between the system of the present invention and traditional monitoring and analysis methods (represented by knowledge graph algorithms):

[0079]

[0080]

[0081] It can be clearly seen from the above comparative test data that the intelligent monitoring and analysis system of important target systems based on large models of the present invention has significant advantages over traditional knowledge graphs and other algorithms in multiple key indicators, and can more effectively meet the monitoring and analysis needs of various important target systems. In view of the limitations of traditional knowledge graphs and other algorithms in knowledge representation and fusion, the present invention proposes a new embedding model based on multi-scale graph convolutional network (MSGCN) for embedding the domain knowledge graph of important target systems (such as power facility domain knowledge graph) into the semantic space of large models. Through the multi-layer iterative calculation of MSGCN, the deep fusion of domain knowledge and large models is realized, so that the large model can better understand and utilize domain knowledge, and overcome the problems of difficult knowledge update and limited representation of traditional knowledge graphs. When facing new targets or situations, traditional methods do not update knowledge in time, resulting in limited analysis accuracy. However, this innovation allows large models to quickly adapt to new situations and improve the accuracy and comprehensiveness of monitoring and analysis of various important targets by dynamically integrating domain knowledge. In the process of large model training, in order to adapt to the characteristics of different important targets and the requirements of monitoring tasks, the present invention adopts an adaptive fine-tuning strategy based on dynamic Bayesian network (DBN) to dynamically adjust the parameter update step size. By constructing a DBN model that reflects the relationship between target characteristics and monitoring tasks, the posterior probability distribution is calculated and the comprehensive impact index is defined accordingly, and then the adaptive parameter update step size is obtained. Compared with the traditional method of adjusting parameters with fixed step size or simple rules, this strategy can accurately adjust the step size according to the actual situation of the target and task, improve the adaptability and generalization ability of the model to specific important targets, and avoid problems such as training instability or overfitting caused by unreasonable step size. In response to the problem of multi-source data fusion, the present invention proposes a set of innovative algorithms for multimodal data collaboration. For multimodal data such as images, videos, and audio collected from different sensors, improved feature extraction methods are used respectively (such as special filters for military target recognition for image data extraction), and then the features of each modality are fused through a new modal fusion algorithm based on a cross-modal attention mechanism to form a unified multimodal feature representation, so that the large model can better process and analyze. Traditional algorithms have difficulties in fusing multi-source data and cannot give full play to the advantages of each data. This innovation can capture the information of important targets more comprehensively and improve the accuracy and reliability of monitoring and analysis through effective multi-modal data collaborative processing, especially in complex environments. It can better identify target states and detect abnormal situations. In order to improve the operating efficiency of the system and reduce unnecessary data processing, the present invention designs a dynamic data screening algorithm based on a probabilistic graphical model (PGM). According to the current analysis requirements of the large model and the importance of each modal data, the PGM is used to calculate the correlation score between each modal data and the analysis requirements, and the screening score is obtained by combining the importance weights, and then the most relevant data is dynamically screened for processing. Traditional data processing methods often treat all collected data equally, resulting in waste of resources and low efficiency.This innovation can significantly improve the operating efficiency of the system while ensuring the monitoring and analysis effect by screening data in a targeted manner, so that the system can respond and output analysis results more quickly. The present invention has established a comprehensive parameter definition and evaluation system for each innovative algorithm. For example, specific parameters are defined for the degree of domain knowledge fusion, the rationality of parameter update step size, the fusion effect of multimodal data features, the accuracy of data screening, and the rationality of importance weights, and the system performance is optimized by real-time monitoring and adjustment of these parameters. Traditional algorithms lack a fine parameter evaluation and optimization mechanism for key links in their own operation process. Through this set of parameter systems, this innovation can accurately control the operating status of each part of the system, promptly discover and solve possible problems, ensure that the system always maintains good performance and stability, and further improve the accuracy and reliability of monitoring and analysis.

Claims

1. An intelligent monitoring and analysis system for important target systems based on a large model, characterized in that: include: Data acquisition module, used to collect various types of data related to important targets from multiple sensors; Data preprocessing module, used to perform cleaning, noise reduction, and normalization preprocessing operations on the collected data; The intelligent analysis module based on the large model is used to perform intelligent analysis on the pre-processed data, identify the target status, and detect abnormal situations; The result output module is used to output the results obtained by the intelligent analysis module to relevant users or systems in an intuitive form; The system management and configuration module is used to manage and configure the entire system; the intelligent analysis module based on the big model includes: knowledge graph construction and embedding new model, which is used to embed the knowledge graph into the semantic space of the big model to achieve deep integration of knowledge and model; A new algorithm based on adaptive fine-tuning strategy is used to dynamically adjust the parameter update step of the large model according to the characteristics and monitoring task requirements, thereby improving the adaptability and generalization ability of the model. The parameter R is introduced to evaluate the rationality of the parameter update step in the adaptive fine-tuning strategy. The calculation method is as follows: Where, T is the total number of training steps, θ t represents the parameters of the large model at step t, α t It represents the parameter update step size at step t. By monitoring this parameter in real time and adjusting it, the stability and effectiveness of model training can be guaranteed.

2. The system according to claim 1, characterized in that The intelligent analysis module based on the large model also includes: The new multimodal data collaboration algorithm is used to extract and fuse features of multimodal data such as images, videos, and audio collected by base-related sensors to form a unified multimodal feature representation.

3. The system according to claim 4, characterized in that The new multimodal data collaboration algorithm further includes: A new modal fusion algorithm based on the cross-modal attention mechanism is used to calculate the attention weights of each modal feature vector and achieve effective fusion of multimodal data.

4. The system according to claim 1, characterized in that The intelligent analysis module based on the large model also includes: New algorithm for dynamic data screening based on demand, A new dynamic data screening algorithm based on the Probability Graph Model (PGM) is proposed to calculate the correlation score S between each modal data and the analysis requirements. m as follows: S m =P(D|f m ) Among them, P(D|f m ) is given a modal eigenvector f m Next, analyze the probability of demand D. By calculating the correlation score of each modal data, the most relevant modal data can be dynamically selected for analysis to improve the accuracy and efficiency of monitoring.

5. The system according to claim 1, characterized in that The data preprocessing module comprises: The image data preprocessing submodule is used to perform preprocessing operations such as cleaning, noise reduction, and normalization on the image data; The video data preprocessing submodule is used to perform preprocessing operations such as splitting and frame feature extraction on video data; The audio data preprocessing submodule is used to perform preprocessing operations such as noise reduction and feature extraction on audio data.

6. The system according to claim 1, characterized in that The result output module includes: The chart output submodule is used to output the monitoring results in the form of charts; The report output submodule is used to output the monitoring results in the form of a report; The alarm information output submodule is used to output alarm information when an abnormal situation is detected.

7. The system according to claim 1, characterized in that The system management and configuration module includes: The parameter setting submodule is used to set the working parameters of each module of the system; User rights management submodule, used to manage user access rights; The system update submodule is used to update the system software.

8. The system according to claim 1, characterized in that The system also has the ability of self-learning and self-optimization. It can continuously learn new data patterns during use, automatically adjust analysis algorithms, and improve monitoring accuracy and response speed.

Images