Substation intelligent inspection system based on multi-modal large and small model fusion

Abstract

The present application provides a substation intelligent inspection system based on multi-modal large and small model fusion, which relates to the field of substation inspection technology, including: a data reading module; a data processing module; a first risk coefficient obtaining module, which is used to perform status assessment on multiple regional data sets through a small early warning model, and output multiple first risk coefficients; an early warning area setting module, which is used to call the early warning large model to perform status assessment on multiple risk data sets, output multiple second risk coefficients, and set the area where the second risk coefficient is greater than the second risk threshold as the early warning area; an operation and maintenance management module. This application can solve the technical problems in the existing substation inspection method that the speed, accuracy and stability of regional risk warnings are poor due to the inability to effectively fuse and analyze multi-modal monitoring data, which affects the quality and efficiency of substation inspections. It can improve the accuracy and reliability of regional monitoring and early warning, and ensure the safe and stable operation of the power system.

Description

Technical Field

[0001] The present application relates to the technical field of substation inspection, and in particular to a substation intelligent inspection system based on multi-modal large and small model fusion. Background Art

[0002] With the continuous development of power systems and the improvement of automation levels, substations, as the core components of power systems, are crucial to the safe operation of power grids and the quality of power supply. Traditional manual inspection methods have problems such as low efficiency, high cost, and high risk. Therefore, intelligent inspection technology based on multimodal monitoring data has gradually become a research hotspot.

[0003] At present, there are technical problems in the existing substation inspection methods, such as the inability to effectively integrate and analyze multimodal monitoring data, resulting in poor speed, accuracy and stability of regional risk warning, which affects the quality and efficiency of substation inspection. Summary of the invention

[0004] The purpose of this application is to provide a substation intelligent inspection system based on multi-modal large and small model fusion, so as to solve the technical problems in the existing substation inspection methods, such as the inability to effectively fuse and analyze multi-modal monitoring data, resulting in poor speed, accuracy and stability of regional risk warning, and affecting the quality and efficiency of substation inspection.

[0005] In view of the above problems, the present application provides a substation intelligent inspection system based on multi-modal large and small model fusion, including: a monitoring data set reading module, which is used to read the monitoring data set of the target substation at a fixed point based on a multi-source monitoring network, wherein the monitoring data set includes an image set, a thermal imaging set and a voiceprint set; a data processing module, which is used to pre-process and regionally locate and fuse the image set, thermal imaging set and voiceprint set to obtain multiple regional data sets; a first risk coefficient obtaining module, which is used to perform state assessment on the multiple regional data sets through a small early warning model, optimize the state assessment results according to real-time interference data, and output multiple first risk coefficients, wherein the small early warning model is obtained after distillation and dimensionality reduction of the large early warning model, and the large early warning model is constructed based on the principle of integrated learning; an early warning area setting module, which is used to identify the regional data set whose first risk coefficient is greater than the first risk threshold as a risk data set, call the large early warning model to perform state assessment on multiple risk data sets, output multiple second risk coefficients, and set the area where the second risk coefficient is greater than the second risk threshold as a warning area, wherein the second risk threshold is greater than the first risk threshold; an operation and maintenance management module, which is used to perform operation and maintenance management on the target substation based on the warning area.

[0006] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0007] 1. Real-time monitoring of the target substation is carried out through a multi-source monitoring network, and the monitoring data set is read at a fixed point, including an image set, a thermal imaging set, and a voiceprint set; then the monitoring data set is preprocessed and regionally positioned and fused to obtain multiple regional data sets; on the other hand, based on the principle of integrated learning, a large warning model is constructed according to the risk prediction operator, wherein the risk prediction operator includes at least BP neural network, support vector machine, and random forest, and the large warning model is trained to convergence through the sample data set; then the large warning model is subjected to knowledge distillation and dimensionality reduction to obtain a small warning model that meets the expected response speed; then the small warning model is used to evaluate the status of multiple regional data sets, and the status evaluation results are evaluated according to the real-time interference data during the status evaluation process. Optimize and output multiple first risk coefficients; further identify the regional data set where the first risk coefficient is greater than the first risk threshold as a risk data set, call the early warning model to perform status assessment on the multiple risk data sets, and output multiple second risk coefficients; then set the area where the second risk coefficient is greater than the second risk threshold as the early warning area, wherein the second risk threshold is greater than the first risk threshold; finally, perform operation and maintenance management on the target substation based on the early warning area; the efficiency and accuracy of the fusion analysis of multimodal monitoring data can be improved, thereby improving the speed and accuracy of regional risk prediction, and at the same time can significantly enhance the system's adaptability to various environments, improve the accuracy and reliability of regional monitoring and early warning, and ensure the safe and stable operation of the power system.

[0008] 2. By constructing a large warning model and a small warning model, since the small warning model is obtained by distilling and reducing the dimensionality of the large warning model, it has a simple structure and a fast warning speed. First, real-time monitoring is performed through the small warning model to timely detect potential fault signals; then, after the potential fault signal is discovered, the large warning model is called for precise analysis, thereby improving the speed and accuracy of regional risk warnings.

[0009] 3. By collecting the real-time interference factors of multimodal data during the early warning analysis process, and evaluating the data credibility based on the real-time interference factors, and then compensating the model early warning results based on the evaluation results, it is possible to effectively reduce the early warning errors caused by external interference, reduce the probability of false alarms and missed alarms, and thus further improve the accuracy and reliability of substation monitoring and early warning.

[0010] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented according to the contents of the specification, and in order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are specifically cited below. It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easy to understand through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the present application or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0012] Figure 1 This application provides a structural schematic diagram of a substation intelligent inspection system based on multi-modal large and small model fusion.

[0013] Figure 2 This application provides a flow chart of constructing a small early warning model in a substation intelligent inspection system based on multi-modal large and small model fusion.

[0014] Description of reference numerals:

[0015] Monitoring data set reading module 11, data processing module 12, first risk coefficient obtaining module 13, early warning area setting module 14, operation and maintenance management module 15. DETAILED DESCRIPTION

[0016] This application provides a substation intelligent inspection system based on multi-modal large and small model fusion, which solves the technical problem that the existing substation inspection method cannot effectively integrate and analyze multi-modal monitoring data, resulting in poor speed, accuracy and stability of regional risk warning, which affects the quality and efficiency of substation inspection. It can improve the efficiency and accuracy of multi-modal monitoring data fusion analysis, thereby improving the speed and accuracy of regional risk prediction, and at the same time significantly enhance the system's adaptability to various environments, improve the accuracy and reliability of regional monitoring and early warning, and ensure the safe and stable operation of the power system.

[0017] Below, the technical solutions in the present application will be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments of the present application. It should be understood that the present application is not limited to the example embodiments described herein. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application. It should also be noted that, for the convenience of description, only the parts related to the present application are shown in the accompanying drawings, rather than all of them.

[0018] For examples, please see the attached Figure 1 The present application provides a substation intelligent inspection system based on multi-modal size model fusion, including:

[0019] The monitoring data set reading module 11 is used to read the monitoring data set of the target substation at a fixed point based on the multi-source monitoring network, wherein the monitoring data set includes an image set, a thermal imaging set and a voiceprint set.

[0020] Specifically, first, a multi-source monitoring network is built, including optical / infrared cameras, drones, voiceprint recognition equipment, and inspection robots, to achieve all-round monitoring of substations. The multi-source monitoring network can cover every corner of the substation and provide real-time and accurate data collection. Then, through the multi-source monitoring network, the monitoring data of the target substation is read at a fixed point, including image sets, thermal imaging sets, and voiceprint sets. These data contain the status information of the substation equipment and provide a basis for subsequent data analysis and processing. By obtaining the monitoring data set, multimodal data from different sensors and equipment can be integrated to provide comprehensive data support for the operation and maintenance and risk management of the substation.

[0021] The data processing module 12 is used to perform preprocessing and regional positioning fusion on the image set, thermal imaging set and voiceprint set to obtain multiple regional data sets.

[0022] Specifically, first, the image set, thermal imaging set and voiceprint set are preprocessed based on the data preprocessing strategy. The data preprocessing strategy can be set by technicians in this field according to actual conditions, wherein the image preprocessing strategy includes preprocessing steps such as denoising, cropping, scaling, and color standardization to improve the quality and availability of image data; the thermal imaging preprocessing strategy includes preprocessing steps such as temperature correction and thermal map generation to ensure the accuracy of thermal imaging data, and the preprocessed thermal imaging data can better reflect the thermal state of the equipment and help to discover potential thermal anomalies; the voiceprint preprocessing strategy includes preprocessing steps such as noise elimination and feature extraction to improve the quality and availability of voiceprint data, and the preprocessed voiceprint data can better reflect the sound characteristics of the equipment during operation, and help to discover potential faults.

[0023] Then, through target detection, segmentation and other methods, the image dataset, thermal imaging dataset and voiceprint dataset are regionally located. The located data can more accurately reflect the equipment status of a specific area, which helps to improve the accuracy of data analysis. The specific area can be set according to actual conditions, such as a single device or cable area. The preprocessed image data, thermal imaging data and voiceprint data are further clustered and fused according to specific areas to obtain multiple regional datasets. Data fusion can make full use of the complementarity between different modal data, thereby improving the accuracy and stability of subsequent data analysis and prediction.

[0024] The first risk coefficient obtaining module 13 is used to perform status assessment on the multiple regional data sets through a small early warning model, optimize the status assessment results according to real-time interference data, and output multiple first risk coefficients, wherein the small early warning model is obtained through distillation and dimensionality reduction of the large early warning model, and the large early warning model is constructed based on the principle of integrated learning.

[0025] Specifically, first, based on the principle of integrated learning, a large early warning model is constructed according to the risk prediction operator, wherein the risk prediction operator includes at least BP neural network, support vector machine and random forest. Technical personnel in this field can also add additional prediction operators according to actual needs. These operators have their own advantages and can capture different aspects of the data, thereby improving the generalization ability of the model. Then, based on the sample data set, each risk prediction operator is trained independently to ensure that each model has sufficient data and resources for training, wherein each model has the opportunity to learn different features of the data, thereby improving the diversity of the model and obtaining multiple trained early warning models; further based on the principle of integrated learning, multiple early warning models are combined to construct a large early warning model, wherein integrated learning can combine the advantages of multiple risk prediction operators by combining the prediction results of multiple models, thereby improving the accuracy and robustness of the prediction.

[0026] Knowledge distillation is a technique that transfers the knowledge of a large, complex model to a smaller model. In an early warning system, the large model is usually a complex model trained with a large amount of data and resources, while the small model is a relatively simple model designed to replicate the performance of the large model as much as possible while maintaining a smaller model size. Then, a smaller model is designed based on the risk prediction operator, which has a simpler structure and fewer parameters than the large model; then, using knowledge distillation technology, the small model is trained based on the large warning model and sample data set to obtain a small warning model that meets the expected convergence constraints. By constructing a small warning model based on the principle of knowledge distillation, a rapid response to the early warning system can be achieved.

[0027] The multiple regional data sets are further input into the early warning small model for status assessment, and in the process of status assessment, the status assessment results are optimized according to the real-time interference data, wherein the real-time interference data refers to the interference factor during data collection, which affects the accuracy of the data, that is, the data credibility is assessed according to the real-time interference factor, and then the status assessment results are optimized and compensated according to the assessment results, and multiple first risk coefficients are output, wherein the first risk coefficient corresponds to the substation area one by one, and is used to characterize the risk level of the substation area, wherein the larger the first risk coefficient is, the higher the risk level of the characterization area is.

[0028] By optimizing the status assessment results according to real-time interference data during the early warning analysis process, the early warning errors caused by external interference can be effectively reduced, and the probability of false alarms and missed alarms can be reduced, thereby further improving the accuracy and reliability of substation monitoring and early warning.

[0029] The warning area setting module 14 is used to identify the regional data set where the first risk coefficient is greater than the first risk threshold as a risk data set, call the warning model to perform status assessment on multiple risk data sets, output multiple second risk coefficients, and set the area where the second risk coefficient is greater than the second risk threshold as a warning area, wherein the second risk threshold is greater than the first risk threshold.

[0030] Specifically, first, configure the first risk threshold and the second risk threshold, wherein the first risk threshold and the second risk threshold can be set according to the actual scenario and needs, and the second risk threshold is greater than the first risk threshold; then, judge multiple first risk coefficients based on the first risk threshold, and identify the regional data set where the first risk coefficient is greater than the first risk threshold as a risk data set, wherein the risk data set represents the data set of the high-risk area; further call the early warning model to evaluate the status of multiple risk data sets, and optimize the status evaluation results according to the real-time interference data, and output multiple second risk coefficients, the second risk coefficient represents the risk level of the corresponding risk data set, and the second risk coefficient can provide a detailed evaluation of each risk data set. Then set the area where the second risk coefficient is greater than the second risk threshold as the early warning area.

[0031] By constructing a large warning model and a small warning model, since the small warning model is obtained by distilling and reducing the dimensionality of the large warning model, it has a simple structure and a fast warning speed. First, real-time monitoring is performed through the small warning model to timely detect potential fault signals; then, after the potential fault signal is discovered, the large warning model is called for precise analysis, thereby improving the speed and accuracy of regional risk warnings.

[0032] The operation and maintenance management module 15 is used to perform operation and maintenance management on the target substation based on the early warning area.

[0033] Specifically, the target substation is finally operated and managed based on the early warning area, where the early warning area is an area with potential faults or risks. The early warning level is set according to the second risk coefficient of the early warning area, where the greater the risk coefficient, the higher the early warning level; further, an operation and maintenance plan is formulated according to the early warning level to carry out operation and maintenance management of the target substation, including inspection, maintenance and repair, etc., to ensure timely detection and handling of potential faults and improve operation and maintenance efficiency and quality.

[0034] By providing a substation intelligent inspection system based on multimodal large and small model fusion, it is possible to solve the technical problems in existing substation inspection methods that the speed, accuracy and stability of regional risk warnings are poor due to the inability to effectively fuse and analyze multimodal monitoring data, which affects the quality and efficiency of substation inspections. The target substation is monitored in real time through a multi-source monitoring network, and the monitoring data set is read at a fixed point, including an image set, a thermal imaging set and a voiceprint set; then the monitoring data set is preprocessed and regionally positioned and fused to obtain multiple regional data sets; on the other hand, based on the principle of integrated learning, a large warning model is constructed according to the risk prediction operator, wherein the risk prediction operator includes at least BP neural network, support vector machine and random forest, and the large warning model is trained to convergence through a sample data set; then the large warning model is subjected to knowledge distillation and dimensionality reduction to obtain a small warning model that meets the expected response speed; then the small warning model is used to evaluate the status of multiple regional data sets, and the status evaluation results are evaluated according to the real-time interference data during the status evaluation process. The optimization is performed to output multiple first risk coefficients; the regional data sets with the first risk coefficient greater than the first risk threshold are further identified as risk data sets, and the early warning model is called to perform status assessment on the multiple risk data sets to output multiple second risk coefficients; then the area with the second risk coefficient greater than the second risk threshold is set as the early warning area, wherein the second risk threshold is greater than the first risk threshold; finally, the target substation is operated and maintained based on the early warning area; the efficiency and accuracy of the fusion analysis of multimodal monitoring data can be improved, thereby improving the speed and accuracy of regional risk prediction, and at the same time, the system's adaptability to various environments can be significantly enhanced, the accuracy and reliability of regional monitoring and early warning can be improved, and the safe and stable operation of the power system can be ensured.

[0035] Further, to construct a large early warning model, this application includes:

[0036] Taking the attribute information of the target substation as a constraint, homologous information retrieval is performed based on power big data to obtain a sample image set, a sample thermal imaging set and a sample voiceprint set; the sample image set is feature calibrated according to predetermined image detection features to generate a sample image feature set, and a first training set is constructed based on the sample image set and the sample image feature set; the sample thermal imaging set is feature calibrated according to predetermined thermal imaging detection features to generate a sample thermal imaging feature set, and a second training set is constructed based on the sample thermal imaging set and the sample thermal imaging feature set; the sample voiceprint set is feature calibrated according to predetermined voiceprint detection features to generate a sample voiceprint feature set, and a third training set is constructed based on the sample voiceprint set and the sample voiceprint feature set; the first training set, the second training set and the third training set are used to perform supervised training on the convolutional neural network respectively, to obtain a feature extraction layer constructed by an image feature extraction unit, a thermal imaging feature extraction unit and a voiceprint feature extraction unit, and the early warning large model is constructed based on the feature extraction layer.

[0037] Specifically, first, the attribute information of the target substation is obtained, including data such as the scale of the power station and the type of equipment; then, with the attribute information of the target substation as a constraint, homologous information retrieval is performed based on the power big data, that is, sample data related to the target substation is retrieved to obtain sample image sets, sample thermal imaging sets and sample voiceprint sets.

[0038] Then, the sample image set is calibrated according to the predetermined image detection features, wherein appropriate image detection features are selected according to the characteristics and failure modes of the substation equipment, such as abnormal color, surface wear, surface corrosion and other features, to generate a sample image feature set, and then a first training set is constructed based on the sample image set and the sample image feature set; the sample thermal imaging set is calibrated according to the predetermined thermal imaging detection features, wherein the thermal imaging features include temperature anomalies, temperature gradient features, etc., to generate a sample thermal imaging feature set, and a second training set is constructed based on the sample thermal imaging set and the sample thermal imaging feature set; the sample voiceprint set is calibrated according to the predetermined voiceprint detection features, wherein the predetermined voiceprint features include abnormal frequency, abnormal amplitude, etc., to generate a sample voiceprint feature set, and a third training set is constructed based on the sample voiceprint set and the sample voiceprint feature set.

[0039] A convolutional neural network is a deep learning model that is mainly used for computer vision tasks such as image recognition and feature extraction. It has a strong feature learning ability and can automatically extract features from input data. Then, the first training set is used to perform supervised training on the convolutional neural network to obtain an image feature extraction unit, which is used to extract key features from image data. The second training set is used to perform supervised training on the convolutional neural network to obtain a thermal imaging feature extraction unit, which is used to extract key features from thermal imaging data. The third training set is used to perform supervised training on the convolutional neural network to obtain a voiceprint feature extraction unit, which is used to extract key features from voiceprint data. Finally, a feature extraction layer is built based on the image feature extraction unit, the thermal imaging feature extraction unit and the voiceprint feature extraction unit, and a large early warning model is built based on the feature extraction layer.

[0040] Further, the early warning model is constructed based on the feature extraction layer. The present application includes:

[0041] Configure a risk prediction operator, wherein the risk prediction operator includes at least a BP neural network, a support vector machine and a random forest; retrieve and obtain a sample data set, wherein the sample data set includes a sample image warning data set, a sample thermal imaging warning data set and a sample voiceprint warning data set; use the sample image warning data set, the sample thermal imaging warning data set and the sample voiceprint warning data set to perform supervised training on the risk prediction operator respectively, and obtain an image warning unit, a thermal imaging warning unit and a voiceprint warning unit to build a warning analysis layer; construct the warning large model based on the feature extraction layer and the warning analysis layer.

[0042] Specifically, first, configure the risk prediction operator, wherein the risk prediction operator includes at least BP neural network, support vector machine and random forest. The technicians in this field can also add additional risk prediction operators according to actual needs; BP neural network is a feedforward neural network that can be iteratively optimized in machine learning, and it is trained by back propagation algorithm. The network consists of input layer, hidden layer and output layer, and minimizes the prediction error by adjusting the network weights, can handle nonlinear problems, and has strong generalization ability; support vector machine is an algorithm for classification and regression analysis, which maximizes the classification interval by finding an optimal hyperplane, thereby achieving the best fit to the data, suitable for high-dimensional data, can handle nonlinear problems, and has good generalization ability; random forest builds multiple decision trees, and then improves the accuracy and stability of prediction by voting or taking the average, can handle high-dimensional data, and has good generalization ability.

[0043] Then, taking the attribute information of the target substation as a constraint, a sample data set is obtained based on the power big data retrieval, where the sample data set includes a sample image warning data set, a sample thermal imaging warning data set and a sample voiceprint warning data set. The sample image warning data includes sample image features and sample risk coefficients, the sample thermal imaging warning data includes sample thermal imaging features and sample risk coefficients, and the sample voiceprint warning data includes sample voiceprint features and sample risk coefficients.

[0044] Then, the sample image warning data set, the sample thermal imaging warning data set and the sample voiceprint warning data set are used to perform supervised training on the risk prediction operators respectively, and the image warning unit, the thermal imaging warning unit and the voiceprint warning unit are obtained to build a warning analysis layer. For example, the sample image warning data set is divided into Q parts, where Q is the number of risk prediction operators, and further Q data sets are randomly selected with replacement Q times to obtain the first data set, and then the same method is used to iteratively select Q times to obtain Q data sets; the Q data sets are further used to perform supervised training on the BP neural network, the support vector machine and the random forest, for example: supervised training is performed through the back propagation algorithm and the gradient descent algorithm to obtain an image warning unit that meets the expected accuracy index.

[0045] Finally, based on the principle of integrated learning, a warning analysis layer is built according to the image warning unit, thermal imaging warning unit and voiceprint warning unit, and a large warning model is constructed based on the feature extraction layer and the warning analysis layer, in which the output layer of the feature extraction layer and the input layer of the warning analysis layer are connected.

[0046] Further, the early warning model is constructed based on the feature extraction layer and the early warning analysis layer. The present application includes:

[0047] The prediction accuracy of the image warning unit, thermal imaging warning unit and voiceprint warning unit is obtained, and based on the coefficient of variation method, the credible weight is configured according to the prediction accuracy; a weighted fusion layer is built according to the credible weight, and the warning large model is constructed based on the feature extraction layer, the warning analysis layer and the weighted fusion layer.

[0048] Specifically, first, the prediction accuracy of the image warning unit, thermal imaging warning unit and voiceprint warning unit is obtained, where the prediction accuracy is the output accuracy rate, which can be obtained through verification testing; then, based on the coefficient of variation method, the credible weight is configured according to the prediction accuracy, where the higher the prediction accuracy, the larger the corresponding credible weight. For example, if the prediction accuracy of the warning unit is high, then its coefficient of variation is small, so it obtains a higher weight in the weighted fusion layer. On the contrary, if the prediction accuracy of the warning unit is low, then its coefficient of variation is large, so it obtains a lower weight in the weighted fusion layer. Then, according to the credible weight, a weighted fusion layer is built, and the weighted fusion layer is used to perform weighted calculation on the output results of the image warning unit, thermal imaging warning unit and voiceprint warning unit, and output the weighted calculation results; finally, a large warning model is constructed based on the feature extraction layer, the warning analysis layer and the weighted fusion layer, where the output layer of the warning analysis layer is connected to the input layer of the weighted fusion layer.

[0049] Further, a small early warning model is constructed, as shown in the attached Figure 2 As shown, this application includes:

[0050] A simplified warning analysis layer of the warning small model is constructed based on the risk prediction operator, wherein the model dimension of the simplified warning analysis layer is much smaller than that of the warning analysis layer; a sample data set is obtained, the sample data set is input into the warning analysis layer of the warning large model, and a soft label data set is output, wherein the soft label data includes the probability distribution of each category; the sample data set and the soft label data set are fused to construct an enhanced training set; based on a predetermined loss function, the simplified warning analysis layer is supervised and trained using the enhanced training set to obtain a simplified warning analysis layer that meets expected convergence constraints, wherein the expected convergence constraints include expected response speed and expected recognition accuracy; the warning small model is constructed based on the feature extraction layer, the simplified warning analysis layer and the weighted fusion layer.

[0051] Specifically, first, a simplified warning analysis layer of the early warning small model is constructed based on the risk prediction operator. The model dimension of the simplified early warning analysis layer is much smaller than the early warning analysis layer, with fewer parameters and a shallower network structure. For example, a simplified model structure is designed, such as reducing the number of layers, the number of neurons, and the complexity of the network. Simplified model design can reduce computational complexity and improve the speed of model training and reasoning. Then retrieve and obtain a sample data set, and input the sample data set into the early warning analysis layer of the early warning large model, and output a soft label data set, where the soft label data includes the probability distribution of each category, that is, it contains the relative confidence of the large model for each category, which provides more information than hard labels.

[0052] Then, the sample data set and the soft label data set are fused to construct an enhanced training set, which contains the original training data and soft label data, and is used to guide the training of the simplified early warning analysis layer. Further, based on a predetermined loss function, the simplified early warning analysis layer is supervised and trained using the enhanced training set. During the training process, the simplified early warning analysis layer will learn how to adjust its parameters so that its predicted distribution is closer to the combination of the soft label and the original hard label provided by the teacher model; until a simplified early warning analysis layer that meets the expected convergence constraints is obtained, wherein the expected convergence constraints include the expected response speed and the expected recognition accuracy, which can be set according to actual conditions.

[0053] Further, a predetermined loss function is constructed, and the present application includes:

[0054] The expression of the predetermined loss function is:

[0055] ; ; ;in, is a hyperparameter used to adjust the weight of the soft label loss. is the cross entropy loss of the warning analysis layer, To simplify the cross entropy loss of the early warning analysis layer, I is the number of categories, T is the temperature parameter, is the true probability of the i-th category in the early warning analysis layer, is the probability distribution of the i-th category in the early warning analysis layer, To simplify the true probability of the i-th category in the early warning analysis layer, To simplify the probability distribution of the i-th category in the early warning analysis layer.

[0056] Specifically, in the predetermined loss function, is the overall loss function, is the cross entropy loss of the warning analysis layer, To simplify the cross entropy loss of the early warning analysis layer, is a hyperparameter used to adjust the weight of soft label loss; I is the number of categories, T is the temperature parameter, is the true probability of the i-th category in the early warning analysis layer, is the probability distribution of the i-th category in the early warning analysis layer, To simplify the true probability of the i-th category in the early warning analysis layer, To simplify the probability distribution of the i-th category in the early warning analysis layer.

[0057] Further, the status assessment results are optimized according to the real-time interference data. The present application includes:

[0058] Obtain an interference indicator set of monitoring data, perform fixed-point collection based on the interference indicator set, and obtain image interference data, thermal imaging interference data, and voiceprint interference data; perform interference intensity assessment based on the image interference data, thermal imaging interference data, and voiceprint interference data to determine the image interference intensity, thermal imaging interference intensity, and voiceprint interference intensity; and optimize the status assessment result based on the image interference intensity, thermal imaging interference intensity, and voiceprint interference intensity.

[0059] Specifically, first, obtain the interference indicator set of monitoring data, where the interference indicator set includes image interference indicator set, thermal imaging interference indicator set and voiceprint interference indicator set. For example, the image interference indicator set includes lighting conditions, equipment occlusion, etc., the thermal imaging interference indicator set includes temperature gradient, radiation interference, etc., and the voiceprint interference indicator set includes environmental noise, equipment noise, etc.; then, fixed-point collection is performed based on the interference indicator set, where the collection time node is consistent with the monitoring time node of the multi-source monitoring network, to obtain image interference data, thermal imaging interference data and voiceprint interference data.

[0060] Further, interference intensity assessment is performed based on the image interference data, thermal imaging interference data and voiceprint interference data. For example, assessment can be performed through statistical analysis, machine learning, building an expert system and other methods to determine the image interference intensity, thermal imaging interference intensity and voiceprint interference intensity; finally, the state assessment result is optimized based on the image interference intensity, thermal imaging interference intensity and voiceprint interference intensity.

[0061] Further, the state assessment result is optimized based on the image interference intensity, thermal imaging interference intensity and voiceprint interference intensity. The application includes:

[0062] The image interference intensity, thermal imaging interference intensity and voiceprint interference intensity are input into a predetermined matching library, and image correction weights, thermal imaging correction weights and voiceprint correction weights are output, wherein the interference intensity and correction weights are anti-correlated; the weighted fusion layer of the early warning small model is optimized using the image correction weights, thermal imaging correction weights and voiceprint correction weights, and status evaluation is performed on the multiple regional data sets based on the optimized weighted fusion layer.

[0063] Specifically, first, a predetermined matching library is constructed, which contains a series of mapping relationships between interference intensity and correction weights. The corresponding correction weights can be automatically calculated according to the input interference intensity, and can be constructed based on sample data; then the image interference intensity, thermal imaging interference intensity and voiceprint interference intensity are input into the predetermined matching library, and the image correction weights, thermal imaging correction weights and voiceprint correction weights are output, wherein the interference intensity and the correction weights are anti-correlated, that is, the higher the interference intensity, the lower the correction weights, and vice versa, the lower the interference intensity, the higher the correction weights. This anti-correlation ensures that more compensation is given when the interference level is high to improve the quality of the data. Finally, the weighted fusion layer of the early warning model is optimized using the image correction weights, thermal imaging correction weights and voiceprint correction weights, that is, the weight distribution of the weighted fusion layer is adjusted according to the correction weights to optimize the prediction performance of the model, and the state of the multiple regional data sets is evaluated through the optimized weighted fusion layer.

[0064] By collecting real-time interference factors of multimodal data during the early warning analysis process, evaluating the data credibility based on the real-time interference factors, and then compensating the model early warning results based on the evaluation results, the early warning errors caused by external interference can be effectively reduced, and the probability of false alarms and missed alarms can be reduced, thereby further improving the accuracy and reliability of substation monitoring and early warning.

[0065] In summary, the substation intelligent inspection system based on multi-modal large and small model fusion provided by this application has the following technical effects:

[0066] 1. Real-time monitoring of the target substation is carried out through a multi-source monitoring network, and the monitoring data set is read at a fixed point, including an image set, a thermal imaging set, and a voiceprint set; then the monitoring data set is preprocessed and regionally positioned and fused to obtain multiple regional data sets; on the other hand, based on the principle of integrated learning, a large warning model is constructed according to the risk prediction operator, wherein the risk prediction operator includes at least BP neural network, support vector machine, and random forest, and the large warning model is trained to convergence through the sample data set; then the large warning model is subjected to knowledge distillation and dimensionality reduction to obtain a small warning model that meets the expected response speed; then the small warning model is used to evaluate the status of multiple regional data sets, and the status evaluation results are evaluated according to the real-time interference data during the status evaluation process. Optimize and output multiple first risk coefficients; further identify the regional data set where the first risk coefficient is greater than the first risk threshold as a risk data set, call the early warning model to perform status assessment on the multiple risk data sets, and output multiple second risk coefficients; then set the area where the second risk coefficient is greater than the second risk threshold as the early warning area, wherein the second risk threshold is greater than the first risk threshold; finally, perform operation and maintenance management on the target substation based on the early warning area; the efficiency and accuracy of the fusion analysis of multimodal monitoring data can be improved, thereby improving the speed and accuracy of regional risk prediction, and at the same time can significantly enhance the system's adaptability to various environments, improve the accuracy and reliability of regional monitoring and early warning, and ensure the safe and stable operation of the power system.

[0067] 2. By constructing a large warning model and a small warning model, since the small warning model is obtained by distilling and reducing the dimensionality of the large warning model, it has a simple structure and a fast warning speed. First, real-time monitoring is performed through the small warning model to timely detect potential fault signals; then, after the potential fault signal is discovered, the large warning model is called for precise analysis, thereby improving the speed and accuracy of regional risk warnings.

[0068] 3. By collecting the real-time interference factors of multimodal data during the early warning analysis process, and evaluating the data credibility based on the real-time interference factors, and then compensating the model early warning results based on the evaluation results, it is possible to effectively reduce the early warning errors caused by external interference, reduce the probability of false alarms and missed alarms, and thus further improve the accuracy and reliability of substation monitoring and early warning.

[0069] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

[0070] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application belong to the scope of the present application and its equivalent technology, the present application is also intended to include these modifications and variations.

Claims

1. Intelligent inspection system for substations based on multi-modal large and small model fusion, It is characterized in that include: A monitoring data set reading module is used to read the monitoring data set of the target substation at a fixed point based on a multi-source monitoring network, wherein the monitoring data set includes an image set, a thermal imaging set and a voiceprint set; A data processing module, used for preprocessing and regional positioning fusion of the image set, thermal imaging set and voiceprint set to obtain multiple regional data sets; A first risk coefficient obtaining module is used to perform status evaluation on the multiple regional data sets through a small early warning model, optimize the status evaluation results according to real-time interference data, and output multiple first risk coefficients, wherein the small early warning model is obtained through distillation and dimensionality reduction of a large early warning model, and the large early warning model is constructed based on an integrated learning principle; An early warning area setting module is used to identify a regional data set where the first risk coefficient is greater than a first risk threshold as a risk data set, call the early warning model to perform status assessment on multiple risk data sets, output multiple second risk coefficients, and set a region where the second risk coefficient is greater than a second risk threshold as an early warning region, wherein the second risk threshold is greater than the first risk threshold; An operation and maintenance management module, used for performing operation and maintenance management on the target substation based on the early warning area; Among them, the construction of the early warning model includes: Taking the attribute information of the target substation as a constraint, homologous information retrieval is performed based on power big data to obtain sample image sets, sample thermal imaging sets, and sample voiceprint sets; Performing feature calibration on the sample image set according to predetermined image detection features to generate a sample image feature set, and constructing a first training set based on the sample image set and the sample image feature set; Performing feature calibration on the sample thermal imaging set according to predetermined thermal imaging detection features to generate a sample thermal imaging feature set, and constructing a second training set based on the sample thermal imaging set and the sample thermal imaging feature set; Performing feature calibration on the sample voiceprint set according to predetermined voiceprint detection features to generate a sample voiceprint feature set, and constructing a third training set based on the sample voiceprint set and the sample voiceprint feature set; Using the first training set, the second training set and the third training set to perform supervised training on the convolutional neural network respectively, to obtain an image feature extraction unit, a thermal imaging feature extraction unit and a voiceprint feature extraction unit to build a feature extraction layer, and to construct the early warning large model based on the feature extraction layer; Constructing the early warning model based on the feature extraction layer includes: Configure risk prediction operators, wherein the risk prediction operators at least include BP neural network, support vector machine and random forest; Retrieve and obtain a sample data set, wherein the sample data set includes a sample image warning data set, a sample thermal imaging warning data set, and a sample voiceprint warning data set; The risk prediction operator is supervised and trained using the sample image warning data set, the sample thermal imaging warning data set, and the sample voiceprint warning data set, respectively, to obtain an image warning unit, a thermal imaging warning unit, and a voiceprint warning unit to build a warning analysis layer; The early warning model is constructed based on the feature extraction layer and the early warning analysis layer.

2. According to the substation intelligent inspection system based on multi-modal size model fusion according to claim 1, It is characterized in that The early warning model is constructed based on the feature extraction layer and the early warning analysis layer, including: Obtaining the prediction accuracy of the image warning unit, the thermal imaging warning unit, and the voiceprint warning unit, and configuring a credible weight according to the prediction accuracy based on a coefficient of variation method; A weighted fusion layer is constructed according to the trusted weights, and the early warning model is constructed based on the feature extraction layer, the early warning analysis layer and the weighted fusion layer.

3. According to the substation intelligent inspection system based on multi-modal size model fusion according to claim 2, It is characterized in that Construct a small early warning model, including: Constructing a simplified early warning analysis layer of the early warning small model based on the risk prediction operator, wherein the model dimension of the simplified early warning analysis layer is much smaller than that of the early warning analysis layer; Acquire a sample data set, input the sample data set into the early warning analysis layer of the early warning large model, and output a soft label data set, wherein the soft label data includes a probability distribution of each category; Fusing the sample data set and the soft label data set to construct an enhanced training set; Based on a predetermined loss function, the simplified early warning analysis layer is supervised and trained using the enhanced training set to obtain a simplified early warning analysis layer that satisfies expected convergence constraints, wherein the expected convergence constraints include expected response speed and expected recognition accuracy; The small warning model is constructed based on the feature extraction layer, the simplified warning analysis layer and the weighted fusion layer.

4. According to the substation intelligent inspection system based on multi-modal size model fusion according to claim 3, It is characterized in that Construct a predetermined loss function, including: The expression of the predetermined loss function is: ； ； ； in, is a hyperparameter used to adjust the weight of the soft label loss. is the cross entropy loss of the warning analysis layer, To simplify the cross entropy loss of the early warning analysis layer, I is the number of categories, T is the temperature parameter, is the true probability of the i-th category in the early warning analysis layer, is the probability distribution of the i-th category in the early warning analysis layer, To simplify the true probability of the i-th category in the early warning analysis layer, To simplify the probability distribution of the i-th category in the early warning analysis layer.

5. According to the substation intelligent inspection system based on multi-modal size model fusion according to claim 3, It is characterized in that Optimize condition assessment results based on real-time interference data, including: Obtaining an interference indicator set of monitoring data, and performing fixed-point collection based on the interference indicator set to obtain image interference data, thermal imaging interference data, and voiceprint interference data; Perform interference intensity assessment according to the image interference data, thermal imaging interference data and voiceprint interference data to determine the image interference intensity, thermal imaging interference intensity and voiceprint interference intensity; The state assessment result is optimized based on the image interference intensity, the thermal imaging interference intensity and the voiceprint interference intensity.

6. According to claim 5, the substation intelligent inspection system based on multi-modal size model fusion, It is characterized in that Optimizing the state assessment result based on the image interference intensity, the thermal imaging interference intensity, and the voiceprint interference intensity includes: Inputting the image interference intensity, thermal imaging interference intensity and voiceprint interference intensity into a predetermined matching library, and outputting image correction weights, thermal imaging correction weights and voiceprint correction weights, wherein the interference intensity and the correction weights are inversely correlated; The image correction weight, the thermal imaging correction weight and the voiceprint correction weight are used to optimize the weighted fusion layer of the early warning small model, and the status of the multiple regional data sets is evaluated based on the optimized weighted fusion layer.

Images