Substation scene change detection method and system
By using the SAM encoding module and the space-time module of the pre-trained visual model, the change detection of the substation scene is solved, and the problems of inaccurate detection results and high labor costs in the prior art are achieved, and efficient and accurate equipment change detection is achieved.
Patent Information
- Application Number
- CN202510289936.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-12
- Publication Date
- 2025-06-20
AI Technical Summary
Due to the influence of various factors, the existing detection methods for substation equipment change have inaccurate results and consume a lot of labor and time.
The SAM encoding module of the pre-trained visual large model extracts the reference map and the map to be measured of the substation scene, obtains the multi-scale spatiotemporal difference map through the spatiotemporal domain module, and uses the SAM decoding module to generate a mask of the changing equipment.
The cost of manual annotation and data annotation is reduced, the accuracy and work efficiency of detection are improved, and the generalization and robustness of change detection are enhanced.
Smart Images
Figure CN120182225A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of equipment defect detection technology, and more specifically, to a method and system for detecting equipment changes in a substation scene based on a SAM encoding module of a visual large model. Background Art
[0002] With the rapid development of economy and power grid, the number of transmission and substation equipment has increased significantly, the demand for automated management and control has continued to increase, and detection applications in power scenarios are emerging.
[0003] At present, change detection technology is widely used in various fields, including disaster assessment, environmental monitoring, land management, and urban change analysis. In terms of detection and status assessment of power equipment, it is of great significance to discover special situations such as hardware failures and external environmental anomalies, and is crucial to the development of the power grid industry. However, due to the influence of various factors, such as changes in image quality, noise, registration errors, illumination changes, changes in complex landscapes, and spatial heterogeneity, these have an impact on the detection of power equipment changes in power systems and add huge challenges.
[0004] Most existing change detection methods are based on semantic segmentation technology, which has high requirements for data annotation, consumes a lot of manpower and time, and is inefficient and expensive. In addition, errors or omissions in manual annotation and deviations caused by individual subjective differences will also affect the accuracy of change target detection to a certain extent.
[0005] Therefore, there is an urgent need for a substation scene change detection method and system that can reduce manpower input, improve work efficiency and detection reliability through change detection technology in applications such as foreign object intrusion and equipment abnormal status monitoring. Summary of the invention
[0006] In view of the above problems, the purpose of the present invention is to provide a substation scene change detection method and system to solve the technical problems that the existing substations are affected by various factors, which affects the detection of changes in power equipment in the power system, increases the difficulty and interference of detection, leads to inaccurate detection results, and wastes manpower.
[0007] The present invention provides a method for detecting a substation scene change, which includes: The obtained reference image and the image to be tested about the substation scene are subjected to feature extraction through the SAM encoding module of the pre-trained visual large model to obtain the reference feature image and the feature image to be tested; The preset spatio-temporal domain module is used to perform spatial feature extraction on the reference feature map and the to-be-tested feature map respectively to obtain the reference spatial dimension feature and the to-be-tested spatial dimension feature, and a multi-scale spatio-temporal difference map is obtained through a preset time-domain fusion algorithm based on the reference spatial dimension feature and the to-be-tested spatial dimension feature; The SAM decoding module of the visual large model is used to perform decoding according to the multi-scale spatio-temporal difference map to obtain the mask map of the changed equipment in the substation scene; The change result of the substation scene is determined according to the mask map.
[0008] Optionally, the SAM encoding module of the pre-trained visual large model is used to perform feature extraction on the obtained reference map and to-be-tested map of the substation scene to obtain a reference feature map and a to-be-tested feature map, including: Obtain the reference map and the to-be-tested map of the substation scene; Preprocess the reference map and the to-be-tested map respectively to obtain a reference standard map and a to-be-tested standard map; The SAM encoding module of the visual large model is used to perform feature extraction on the reference standard map and the to-be-tested standard map by using the SAM model weights obtained during pre-training to obtain a reference feature map and a to-be-tested feature map.
[0009] Optionally, the preset spatio-temporal domain module is used to perform spatial feature extraction on the reference feature map and the to-be-tested feature map respectively to obtain the reference spatial dimension feature and the to-be-tested spatial dimension feature, and a multi-scale spatio-temporal difference map is obtained through a preset time-domain fusion algorithm based on the reference spatial dimension feature and the to-be-tested spatial dimension feature, including: The spatial domain module in the spatio-temporal domain module is used to perform convolution operations on the reference feature map and the to-be-tested feature map respectively to obtain the reference spatial dimension feature and the to-be-tested spatial dimension feature; The time domain module in the spatio-temporal domain module is used to obtain a first time domain difference value and a second time domain difference value based on the reference spatial dimension feature and the to-be-tested spatial dimension feature, add the first time domain difference value and the second time domain difference value to obtain a time domain dimension feature, and generate a scale spatio-temporal difference map according to the time domain dimension feature.
[0010] Optionally, the obtaining of the first time domain difference value and the second time domain difference value includes: Align the reference spatial dimension feature and the to-be-tested spatial dimension feature in time, use the difference value obtained by subtracting the to-be-tested spatial dimension feature from the reference spatial dimension feature as the first time domain difference value, and use the difference value obtained by subtracting the reference spatial dimension feature from the to-be-tested spatial dimension feature as the second time domain difference value.
[0011] Optionally, the SAM decoding module using the vision large model decodes according to the multi-scale spatio-temporal difference map to obtain the matte map of the changed devices in the substation scene, including: The SAM decoding module performs a first upsampling on the multi-scale spatio-temporal difference map to obtain a normalized spatio-temporal difference map; Performs SAM decoding on the normalized spatio-temporal difference map to obtain an initial matte map; Filters according to a preset elimination rule to obtain a valid matte map; Performs a second upsampling on the valid matte map to obtain a normalized matte map, and performs noise filtering and optimized connected component processing on the normalized matte map to obtain the matte map of the changed devices in the substation scene.
[0012] Optionally, the SAM encoding module of the pre-trained vision large model, the preset spatio-temporal domain module, and the SAM decoding module of the vision large model are integrated in a pre-trained substation scene change detection model; wherein, before pre-training the substation scene change detection model, it includes: Defines the names of different object objects in the substation based on a preset three-dimensional model of the power system, and maps a corresponding class label to each object object to construct a three-dimensional scene; Makes the unmanned aerial vehicle move along a preset line trajectory of the three-dimensional scene through a preset simulation software to obtain a reference sequence; Changes the three-dimensional scene to obtain a changed scene, and makes the unmanned aerial vehicle in the simulation software move on the changed scene according to the preset line trajectory of the three-dimensional scene to obtain a sequence to be measured; Performs generalization processing on the sequence to be measured and the reference sequence to obtain training data to be measured and reference training data; Determines scene change data based on the training data to be measured and the reference training data, and generates a change label for each pixel in the scene change data; wherein, the change label will be used for calculating the loss function when training the substation scene change detection model; Randomly selects image pairs with changes as positive samples and image pairs without changes as negative samples from all the training data to be measured and reference training data; Mixes the positive samples and the negative samples to form a sample data set; Divides the sample data set according to a preset specific ratio to form a training set and a validation set; wherein, the ratio of positive samples and negative samples in the training set and the validation set is a preset positive-negative ratio.
[0013] Optionally, the steps of training the substation scene change detection model include: Use the training set and the validation set to repeatedly train and validate a preset substation scene change detection network structure; during the training process, Use a preset combined loss function to repeatedly train and optimize the substation scene change detection network structure; and, regard the changed devices in the scene change data in the training set as the foreground, and regard the unchanged objects in the training set as the background, until the total loss of the predicted mask map of the changed devices generated by the substation scene change detection network structure through the segmentation task reaches a preset training threshold, then stop the training, and use the substation scene change detection network structure generated by the last training as the substation scene change detection model.
[0014] Optionally, the step of calculating the total loss includes: Obtain a weight loss according to the weight values of the foreground and the background, and calculate an edge loss according to the predicted segmentation region of the predicted mask map and the known changed region; Linearly weight the weight loss and the edge loss to obtain the total loss.
[0015] The present invention also provides a substation scene change detection system for implementing the substation scene change detection method as described above. The system includes a substation scene change detection model, and the substation scene change detection model includes: The SAM encoding module of the large vision model is used to extract features from the obtained reference map and the map to be measured of the substation scene to obtain a reference feature map and a map to be measured feature map; The spatio-temporal domain module is used to respectively extract spatial features from the reference feature map and the map to be measured feature map to respectively obtain a reference spatial dimension feature and a map to be measured spatial dimension feature, and obtain a multi-scale spatio-temporal difference map based on the reference spatial dimension feature and the map to be measured spatial dimension feature through a preset time domain fusion algorithm; The SAM decoding module of the large vision model is used to decode according to the multi-scale spatio-temporal difference map to obtain a mask map of the changed devices in the substation scene; The result extraction module is used to determine the change result of the substation scene according to the mask map.
[0016] Optionally, it further includes a pre-training module for pre-training the substation scene change detection model, and includes: Define the names of different object objects in the substation based on a preset three-dimensional model of the power system, and map a corresponding class label to each object object to construct a three-dimensional scene; Make the unmanned aerial vehicle move along a preset line trajectory of the three-dimensional scene through a preset simulation software to obtain a reference sequence; Modify the three-dimensional scene to obtain a modified scene, and cause the unmanned aerial vehicle in the simulation software to move on the modified scene according to the preset line trajectory of the three-dimensional scene to obtain a sequence to be measured; Perform generalization processing on the sequence to be measured and the reference sequence to obtain training data to be measured and reference training data; Determine scene change data based on the training data to be measured and the reference training data, and generate change labels for each pixel in the scene change data; wherein, the change labels will be used for loss function calculation when training the substation scene change detection model; Randomly select image pairs with changes as positive samples and image pairs without changes as negative samples from all the training data to be measured and reference training data to be used as the training set and the validation set; Repeatedly train and validate a preset substation scene change detection network structure using the training set and the validation set; during the training process, Repeatedly train and optimize the substation scene change detection network structure using a preset combined loss function; and regard the changed devices in the scene change data in the training set as the foreground and the unchanged objects in the training set as the background until the total loss of the predicted mask map of the changed devices generated by the substation scene change detection network structure through the segmentation task reaches a preset training threshold, then stop training, and use the substation scene change detection network structure generated in the last training as the substation scene change detection model.
[0017] It can be seen from the above technical scheme that the substation scene change detection method and system provided by the present invention perform feature extraction on the obtained reference image and the image to be tested about the substation scene through the SAM encoding module of the pre-trained visual large model to obtain the reference feature image and the image to be tested, and then perform spatial feature extraction on the reference feature image and the image to be tested through the preset time-space domain module to obtain the reference space dimension features and the space dimension features to be tested respectively, and obtain a multi-scale space-time difference map based on the reference space dimension features and the space dimension features to be tested through the preset time-domain fusion algorithm, and then decode the multi-scale space-time difference map based on the SAM decoding module to obtain the mask map of the changed equipment in the substation scene, and then obtain the detection result based on the mask map, so that it can be more clearly known what changes have occurred in the substation scene, and the method of generating reference image and image to be tested data for detection reduces the number of According to the annotation cost, by generating data and change labels, the cost of data annotation and the dependence on manpower and time costs of expert experience can be saved. The SAM encoding module based on the large visual model extracts difference features, which can effectively increase the generalization and robustness of change detection in substation scenarios. Small sample data is used to perform fine-tuning on the basis of the large electric vision pre-training model to achieve small sample or even zero sample electric scene vision tasks, increase generalization, and use the automatic generation of spatiotemporal domain prompt word module to use the attention mechanism to better extract difference features. In addition, by using the simulation software's drone to build a data set, a large amount of data can be generated to meet the fine-tuning needs of the large visual model, and at the same time, the dependence on data annotation can be eliminated. Therefore, through this change detection technology, in applications such as foreign object intrusion and equipment abnormal status monitoring, it can reduce manpower investment and improve work efficiency and detection reliability. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] By referring to the following description in conjunction with the accompanying drawings, and with a more comprehensive understanding of the present invention, other objects and results of the present invention will become more clear and easy to understand. In the accompanying drawings: Figure 1 is a flow chart of a method for detecting substation scene changes according to an embodiment of the present invention; Figure 2 4 is a logic block diagram of a substation scene change detection system according to an embodiment of the present invention. DETAILED DESCRIPTION
[0019] Changes in image quality, noise, registration errors, illumination changes, changes in complex landscapes, and spatial heterogeneity will all affect the detection of power equipment changes in power systems and increase the detection challenges. Most existing change detection methods are based on semantic segmentation technology, which has high requirements for data annotation, consumes a lot of manpower and time costs, and is inefficient and expensive to annotate. In addition, errors or omissions in manual annotation and deviations caused by individual subjective differences will also affect the accuracy of change target detection to a certain extent.
[0020] In view of the above problems, the present invention provides a method and system for detecting substation scene changes. The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.
[0021] In order to illustrate the substation scene change detection method and system provided by the present invention, Figure 1 , Figure 2 The embodiments of the present invention are exemplarily indicated.
[0022] The following description of exemplary embodiments is merely illustrative in nature and is not intended to limit the present invention and its application or use. Technologies and devices known to ordinary technicians in the relevant field may not be discussed in detail, but in appropriate cases, the technologies and devices should be considered as part of the specification.
[0023] like Figure 1 As shown, the present invention provides a method for detecting substation scene changes, comprising: S1: extracting features of the obtained reference image and the image to be tested about the substation scene through the SAM encoding module of the pre-trained visual large model to obtain a reference feature image and a feature image to be tested; S2: performing spatial feature extraction on the reference feature map and the feature map to be tested respectively through a preset spatiotemporal domain module to obtain reference spatial dimension features and test spatial dimension features respectively, and obtaining a multi-scale spatiotemporal difference map based on the reference spatial dimension features and test spatial dimension features through a preset time domain fusion algorithm; S3: using the SAM decoding module of the visual large model to decode according to the multi-scale spatiotemporal difference map to obtain a mask map of the changed equipment in the power transformation scene; S4: Determine a change result of the substation scene according to the mask image.
[0024] Step S1 is a process of extracting features from the obtained reference image and the image to be measured of the substation scene through the SAM encoding module of the pre-trained large vision model to obtain the reference feature map and the feature map to be measured; in this embodiment, extracting features from the obtained reference image and the image to be measured of the substation scene through the SAM encoding module of the pre-trained large vision model to obtain the reference feature map and the feature map to be measured includes: S11: Obtain the reference image and the image to be measured of the substation scene; S12: Preprocess the reference image and the image to be measured respectively to obtain the reference standard image and the to-be-measured standard image; S13: Enable the SAM encoding module of the large vision model to extract features from the reference standard image and the to-be-measured standard image using the SAM model weights obtained during pre-training to obtain the reference feature map and the to-be-measured feature map; In a more specific embodiment, simply put, it is to use the encoding network of the SAM (Segment Anything Model) encoding module of the large vision model to extract image features, input the reference image and the image to be measured into the feature extraction network respectively, and generate the corresponding reference feature map and the to-be-measured feature map. The goal of this step is to provide high-quality feature representations for subsequent difference detection; First, it is necessary to define the reference image (T0) and the image to be measured (T1) input to the network, preprocess them to unify the size and format, ensure the standardization and consistency of data input, and then input the preprocessed reference image and the image to be measured into the SAM encoding module of the large vision model respectively. Using the SAM model weights obtained during pre-training, by fixing the model weights and only fine-tuning the weight parameters of the latter layer, the overfitting problem caused by a small amount of data can be effectively avoided, and at the same time, the pre-trained large model can better adapt to the characteristics of the power scene. The reference image and the image to be measured are processed in the shared feature extraction network, and the corresponding reference feature map and the to-be-measured feature map are output respectively.
[0025] Step S2 is a process of respectively extracting spatial features from the reference feature map and the to-be-measured feature map through a preset spatio-temporal domain module to obtain the reference spatial dimension feature and the to-be-measured spatial dimension feature, and obtaining a multi-scale spatio-temporal difference map based on the reference spatial dimension feature and the to-be-measured spatial dimension feature through a preset time domain fusion algorithm, where, including: S21: Perform convolution operations on the reference feature map and the to-be-measured feature map respectively through the spatial domain module in the spatio-temporal domain module to obtain the reference spatial dimension feature and the to-be-measured spatial dimension feature; S22: Based on the reference spatial dimension feature and the to-be-tested spatial dimension feature, the time domain module in the spatio-temporal domain module obtains a first time domain difference and a second time domain difference, adds the first time domain difference and the second time domain difference to obtain a time domain dimension feature, and generates a scale spatio-temporal difference map according to the time domain dimension feature.
[0026] Among them, obtaining the first time domain difference and the second time domain difference includes: Align the reference spatial dimension feature and the to-be-tested spatial dimension feature in time, use the difference between the reference spatial dimension feature minus the to-be-tested spatial dimension feature as the first time domain difference, and use the difference between the to-be-tested spatial dimension feature minus the reference spatial dimension feature as the second time domain difference.
[0027] Specifically, step S2 is the process of the automatic generation of the spatio-temporal domain prompt word module to extract difference features. The spatio-temporal domain module (TSF) is used to extract and fuse the features of the changing device from the time domain and the spatial domain respectively, and generate a multi-scale difference feature map. The spatio-temporal domain module contains two core components: the time domain module (MTF, Multi-Temporal Fusion) and the spatial domain module, also known as the spatial domain module (MSF, Multi-Scale Fusion), which are used to capture the change information of the device in the time dimension and the spatial dimension. First, the reference feature map and the to-be-tested feature map need to be input into the spatial domain module respectively to extract and fuse multi-scale difference features from the spatial dimension. This module can capture the change situations at different spatial scales by performing convolution operations at different spatial scales, ensuring that the difference features can reflect both global changes and pay attention to local detail changes. Then, the spatial domain differences between the reference feature map and the to-be-tested feature map are fused in the time domain respectively, that is, based on [reference map MSF, to-be-tested map MSF], the first time domain difference is obtained, and based on [to-be-tested map MSF, reference map MSF], the second time domain difference is obtained. Then, the two are input into the time domain feature module (time domain module), and the attention mechanism (Additive Attention) is used to calculate the similarity and difference between the feature maps, that is, add the above two differences. During the process, the regions that change in the time dimension are focused on, that is, the attention mechanism adaptively adjusts the model's attention to different time features to ensure that the device state changes are effectively captured. Finally, the multi-scale spatio-temporal difference feature map output by the time domain module can implicitly reflect the change features of the device.
[0028] Step S3 is a process in which the SAM decoding module of the visual large model decodes according to the multi-scale spatio-temporal difference map to obtain the mask map of the changed equipment in the substation scene. Among them, the process of using the SAM decoding module of the visual large model to decode according to the multi-scale spatio-temporal difference map to obtain the mask map of the changed equipment in the substation scene includes: S31: The SAM decoding module performs a first upsampling on the multi-scale spatio-temporal difference map to obtain a normalized spatio-temporal difference map; S32: Perform SAM decoding on the normalized spatio-temporal difference map to obtain an initial mask map; S33: Filter according to a preset removal rule to obtain a valid mask map; S34: Perform a second upsampling on the valid mask map to obtain a normalized mask map, and perform noise filtering and optimized connected component processing on the normalized mask map to obtain the mask map of the changed equipment in the substation scene.
[0029] In this specific embodiment, the multi-scale spatio-temporal difference map generated in the above steps is input into the SAM decoding module for the first upsampling operation. This first upsampling operation is used to restore the spatial resolution of the feature map and restore it to a size close to the original image. Through the SAM decoding module, the mask map of the detection result is output, and the pixel value of the output mask map is a continuous value between [0,1]; and a threshold is set to filter out the detection results with low scores and remove the regions with high uncertainty. The setting of this threshold is adjusted according to the probability or confidence distribution output by the model. Only the regions above the threshold are retained as the final mask map to ensure that the output changed regions have high accuracy; then, the mask map is subjected to a second upsampling to enlarge it to the same size as the input image. Then, post-processing is performed to optimize the quality of the mask. The specific post-processing operations include: Noise filtering: Use morphological operations (such as dilation, erosion) or small connected component deletion to remove isolated small noise points; Optimizing connected components: Through connected component analysis, ensure the continuity of the changed regions, further eliminate the interference of irrelevant regions, and improve the integrity and accuracy of the mask.
[0030] In this specific embodiment, the SAM encoding module of the pre-trained visual large model, the preset spatio-temporal domain module, and the SAM decoding module of the visual large model are integrated in the pre-trained substation scene change detection model; among them, before pre-training the substation scene change detection model, it includes: S01: Define the names of different object objects in the substation based on a preset three-dimensional model of the power system, and map a corresponding class label to each object object to construct a three-dimensional scene; S02: Use a preset simulation software to move the drone along a preset line trajectory of the 3D scene to obtain a reference sequence; S03: Make changes to the 3D scene to obtain a changed scene, and make the drone in the simulation software move on the changed scene according to the preset line trajectory of the 3D scene to obtain a sequence to be measured; S04: Perform generalization processing on the sequence to be measured and the reference sequence to obtain training data to be measured and reference training data; S05: Determine scene change data based on the training data to be measured and the reference training data, and generate a change label for each pixel in the scene change data; wherein, the change label will be used for calculating the loss function during the training of the substation scene change detection model; S06: Randomly select image pairs with changes as positive samples and image pairs without changes as negative samples from all the training data to be measured and reference training data; S07: Mix the positive samples and the negative samples to form a sample data set; S08: Divide the sample data set according to a preset specific ratio to form a training set and a validation set; wherein, the ratio of positive samples and negative samples in the training set and the validation set is the preset positive-negative ratio.
[0031] More specifically, first construct a 3D scene. Based on the UE4 power system 3D model, define the names of different object objects and map corresponding class labels to each object for subsequent semantic analysis and recognition. Then, during the capture of the reference sequence, the simulation software Airsim uses the drone to move along the preset trajectory to obtain an RGBD sequence, and use this RGBD sequence as the reference sequence. This RGBD sequence can represent the color, depth, and semantic information of pixels (multi-modal sensor data). Then, perform 3D scene changes, including different types of changes, such as changing the states of the object objects obtained in S11, such as appearance, disappearance, replacement, etc.; and different environmental changes, such as weather, air dust, light, shadow, etc. After that, during the capture of the sequence to be measured, make the drone traverse the changed scene according to the same trajectory as before to obtain a new RGBD sequence, and use this RGBD sequence as the sequence to be measured.
[0032] After comparing the RGBD sequence to be measured with the reference sequence, it is used to determine scene changes and generate a change label (ground truth) for each pixel. This change label will be used for calculating the loss function during the model training process and serve as the basis for judging the accuracy of the model to ensure that the model can correctly identify the changes in the scene.
[0033] In addition, online data augmentation can be used to increase the generalization ability. Randomly select one of the two corresponding images in the above-mentioned sequence to be measured and the reference sequence, and perform affine transformations with random horizontal, vertical or rotation angles to change the position and amplitude to random values, which are used to simulate the jitter that may occur during acquisition. Random brightness and contrast are changed to simulate light and shadow changes, and random pixel value blocks of noise appear at random positions to simulate the possible noise interference during the shooting process, so as to obtain the generalized training data to be measured and the reference training data.
[0034] Finally, through the extraction of the corresponding matching frames of the training data to be measured and the reference training data, randomly select the image pairs with changes as positive samples, and randomly select the image pairs without changes as negative samples. After mixing them into a sample data set, divide the training set and the validation set. In this embodiment, the training set and the validation set are divided at a ratio of 8:2 according to different scenarios for model training, and in this specific embodiment, the positive-negative ratio is 3:1, that is, in this specific embodiment, the quantity ratio of positive samples to negative samples is 3:1.
[0035] In this specific embodiment, the steps of training the substation scene change detection model include: S51: Repeatedly train and validate the preset substation scene change detection network structure using the training set and the validation set; during the training process, S52: Repeatedly train and optimize the substation scene change detection network structure using a preset combined loss function; and regard the changed devices in the scene change data in the training set as the foreground, and regard the unchanged objects in the training set as the background until the total loss of the predicted mask map of the changed devices generated by the substation scene change detection network structure through the segmentation task reaches the preset training threshold, then stop training, and use the substation scene change detection network structure generated by the last training as the substation scene change detection model.
[0036] The steps of calculating the total loss include: Obtain the weight loss according to the weight values of the foreground and the background, and calculate the edge loss according to the predicted segmentation region of the predicted mask map and the known changed region; Linearly weight the weight loss and the edge loss to obtain the total loss.
[0037] Generally speaking, in more specific embodiments, a combined loss function is used for training and optimizing the model. Among them, the changing devices are regarded as the foreground, while the other unchanged objects in the scene are regarded as the background. The goal of the model is to generate a mask map of the changing devices through a segmentation task, accurately identifying the regions of these changing devices. The performance of the model is optimized by minimizing the total loss. Specifically, the Weights loss and the Dice loss (boundary loss) are linearly weighted and combined to calculate the total loss of the model. And the F1 score and the MIOU (Mean Intersection over Union) are used as evaluation metrics of the model to measure the segmentation effect of the model; The Weights loss calculation is mainly used to handle the problem of foreground and background imbalance in the data. In the images of the power scene, the foreground (such as the device change region) often occupies a small part, while the background may occupy most of the region, which is likely to cause the model to be biased towards the background prediction. Therefore, the Weights loss assigns different weights to different classes (foreground and background), making the model pay more attention to the prediction of the foreground region during the training process. The calculation formula of the Weights loss is: ; where wi is the weight value according to the foreground (changing device) and the background (unchanged object), pi is the predicted value of the model, and yi is the true label value, which can usually be the cross-entropy loss or other regression loss functions.
[0038] The Dice loss calculation is performed. The Dice loss is a boundary loss function, especially suitable for segmentation tasks, aiming to maximize the overlap between the predicted segmentation region and the true target region. By measuring the similarity between the predicted segmentation region and the true segmentation region, it can effectively handle the problem of class imbalance, especially having a good effect on the accurate segmentation of the boundary region. The calculation formula of the Dice loss is: ; where pi represents the predicted segmentation region of the model, and yi represents the true target region. The goal of this loss function is to maximize the intersection of the model prediction region and the true region.
[0039] Then the combined total loss calculation is carried out. To improve the performance of the model, the Weights loss and the Dice loss are linearly weighted and combined to obtain the total loss function. The parameters of the model are optimized by minimizing this total loss. The calculation formula of the total loss is: ; where α and β are weight coefficients, used to control the relative importance of the two losses. According to the experimental results, in this specific embodiment, α = 0.5 and β = 1.5 are adjusted to obtain the best segmentation effect.
[0040] Finally, model training and evaluation are carried out: Model training: By minimizing the total loss, use the gradient descent algorithm or its variants (such as Adam, RMSprop, etc.) to optimize the model parameters. During the training process, iterate continuously until the loss converges to the optimal value; Model evaluation: In the model training and validation phases, use F1score and MIOU as the main evaluation metrics to measure the segmentation performance of the model. F1score: Measures the balance between the precision and recall of the model in detecting the foreground (changing devices). MIOU: Measures the degree of overlap between the predicted foreground region (changing devices) and the real region. The higher the MIOU, the better the segmentation effect of the model.
[0041] Finally, the trained model can locate and output the changed area. That is, using the trained model, the reference image and the image to be measured are sent into the model to obtain the mask image of the final changed device. After erosion, dilation, and calculation of the minimum bounding rectangle, the finally obtained bounding box is used to represent the position of the changed device in the image.
[0042] Step S4 is the process of determining the change result of the substation scene according to the mask image, that is, performing erosion, dilation, and calculation processing of the minimum bounding rectangle on the mask image to obtain the position of the bounding box. The changed device is located in the image within this bounding box, and thus the change result is obtained.
[0043] As described above, the substation scene change detection method provided by the present invention has the following beneficial effects: 1. Reduce manual workload and improve efficiency and reliability. Through change detection technology, in applications such as foreign object intrusion and equipment abnormal state monitoring, manpower input can be reduced, and work efficiency and detection reliability can be improved; 2. Based on the three-dimensional substation model to simulate the real scene and generate the reference image and the image to be measured data for detection, reduce the data annotation cost. By generating data and change labels, the cost of data annotation and the dependence on the manpower and time cost of experts can be saved; 3. Based on the SAM coding module of the vision large model for feature extraction to improve the model robustness. Use small sample data to fine-tune on the basis of the pre-trained large model of power vision, realize small sample or even zero sample power scene vision tasks, and increase generalization.
[0044] 4. The automatic generation of spatio-temporal domain prompt word module uses the attention mechanism and can better extract differential features. Compared with the previous methods that require manual design and prior knowledge to fuse features, this module can better fit the differential features.
[0045] Such as Figure 2As shown, the present invention also provides a substation scene change detection system 100, which implements the substation scene change detection method as described above. The system includes a substation scene change detection model, and the substation scene change detection model includes: The SAM encoding module 110 of the visual large model is used to extract features from the obtained reference image and the image to be tested about the substation scene to obtain a reference feature image and a feature image to be tested; The spatiotemporal domain module 120 is used to extract spatial features from the reference feature map and the feature map to be tested respectively to obtain reference spatial dimension features and the spatial dimension features to be tested respectively, and obtain a multi-scale spatiotemporal difference map based on the reference spatial dimension features and the spatial dimension features to be tested by a preset time domain fusion algorithm; The SAM decoding module 130 of the visual large model is used to decode according to the multi-scale spatiotemporal difference map to obtain the mask map of the changed equipment in the power transformation scene; The result extraction module 140 is used to determine the change result of the substation scene according to the mask image.
[0046] It also includes a pre-training module 150, which is used to pre-train the substation scene change detection model, including: Based on the preset three-dimensional model of the power system, the names of different objects in the substation are defined, and corresponding class labels are mapped to each object to construct a three-dimensional scene; Using preset simulation software, the drone is moved along a preset route track of the three-dimensional scene to obtain a reference sequence; The three-dimensional scene is changed to obtain a changed scene, and the drone in the simulation software moves on the changed scene according to a preset line trajectory of the three-dimensional scene to obtain a sequence to be tested; Performing generalization processing on the sequence to be tested and the reference sequence to obtain training data to be tested and reference training data; Determine scene change data based on the training data to be tested and the benchmark training data, and generate a change label for each pixel in the scene change data; wherein the change label will be used to calculate the loss function when training the substation scene change detection model; Randomly extract image pairs with changes from all the training data to be tested and the benchmark training data as positive samples, and randomly extract image pairs without changes as negative samples to serve as training sets and validation sets; The training set and the verification set are used to repeatedly train and verify the preset substation scene change detection network structure; during the training process, The substation scene change detection network structure is repeatedly trained and optimized using a preset combined loss function; moreover, the changed devices in the scene change data in the training set are regarded as foregrounds, and the unchanged objects in the training set are regarded as backgrounds, until the total loss of the predicted mask map of the changed devices generated by the substation scene change detection network structure through the segmentation task reaches a preset training threshold, then the training is stopped, and the substation scene change detection network structure generated in the last training is used as the substation scene change detection model.
[0047] For a more specific implementation manner of this substation scene change detection system, refer to the embodiments of the above-mentioned substation scene change detection method, which will not be elaborated here.
[0048] As described above by way of example with reference to the drawings, the substation scene change detection method and system proposed according to the present invention are described. However, those skilled in the art should understand that various improvements can be made to the above-mentioned substation scene change detection method and system proposed by the present invention without departing from the content of the present invention. Therefore, the protection scope of the present invention should be determined by the content of the appended claims.
Claims
1. A method for detecting substation scene changes, characterized in that: include: The obtained reference image and the image to be tested about the substation scene are subjected to feature extraction through the SAM encoding module of the pre-trained visual large model to obtain the reference feature image and the feature image to be tested; Performing spatial feature extraction on the reference feature map and the feature map to be tested respectively through a preset spatiotemporal domain module to obtain reference spatial dimension features and test spatial dimension features respectively, and obtaining a multi-scale spatiotemporal difference map based on the reference spatial dimension features and test spatial dimension features through a preset time domain fusion algorithm; Adopting the SAM decoding module of the visual large model to decode according to the multi-scale spatiotemporal difference map to obtain a mask map of the changed equipment in the power transformation scene; The change result of the substation scene is determined according to the mask image.
2. The substation scene change detection method according to claim 1, characterized in that: The SAM encoding module of the pre-trained visual large model extracts features from the obtained reference image and the image to be tested about the substation scene to obtain the reference feature image and the feature image to be tested, including: Obtain the reference image and the image to be tested of the substation scene; Preprocessing the reference image and the image to be tested respectively to obtain a reference standard image and a standard image to be tested; The SAM encoding module of the visual large model uses the SAM model weights obtained during pre-training to perform feature extraction on the benchmark standard image and the standard image to be tested to obtain a benchmark feature image and a feature image to be tested.
3. The substation scene change detection method according to claim 1, characterized in that: The method of performing spatial feature extraction on the reference feature map and the feature map to be tested respectively through a preset spatiotemporal domain module to obtain reference spatial dimension features and test spatial dimension features respectively, and obtaining a multi-scale spatiotemporal difference map based on the reference spatial dimension features and the test spatial dimension features through a preset time domain fusion algorithm, including: Performing convolution operations on the reference feature map and the feature map to be tested respectively through the spatial domain module in the spatiotemporal domain module to obtain reference spatial dimension features and spatial dimension features to be tested; The time domain module in the space-time domain module obtains a first time domain difference and a second time domain difference based on the reference space dimension feature and the space dimension feature to be measured, adds the first time domain difference and the second time domain difference to obtain the time domain dimension feature, and generates a scale space-time difference map according to the time domain dimension feature.
4. The substation scene change detection method according to claim 3, characterized in that: The obtaining of the first time domain difference and the second time domain difference comprises: The reference spatial dimension feature and the spatial dimension feature to be measured are aligned in time, the difference between the reference spatial dimension feature and the spatial dimension feature to be measured is used as the first time domain difference, and the difference between the spatial dimension feature to be measured and the reference spatial dimension feature is used as the second time domain difference.
5. The method for detecting substation scene changes according to claim 4, characterized in that: The SAM decoding module using the visual large model performs decoding according to the multi-scale spatiotemporal difference map to obtain a mask map of the changed equipment in the power transformation scene, including: The SAM decoding module performs a first upsampling on the multi-scale spatiotemporal difference map to obtain a standard spatiotemporal difference map; Performing SAM decoding on the standard spatiotemporal difference map to obtain an initial mask map; Filter according to the preset elimination rules to obtain a valid mask image; The effective mask image is upsampled for a second time to obtain a standard mask image, and the standard mask image is subjected to noise filtering and connected domain optimization processing to obtain a mask image of a changed device in the power transformation scenario.
6. The method for detecting substation scene changes according to claim 4, characterized in that: The SAM encoding module of the pre-trained visual large model, the preset spatiotemporal domain module, and the SAM decoding module of the visual large model are integrated in the pre-trained substation scene change detection model; wherein, before pre-training the substation scene change detection model, it includes: Based on the preset three-dimensional model of the power system, the names of different objects in the substation are defined, and corresponding class labels are mapped to each object to construct a three-dimensional scene; Using preset simulation software, the drone is moved along a preset route track of the three-dimensional scene to obtain a reference sequence; The three-dimensional scene is changed to obtain a changed scene, and the drone in the simulation software moves on the changed scene according to a preset line trajectory of the three-dimensional scene to obtain a sequence to be tested; Performing generalization processing on the sequence to be tested and the reference sequence to obtain training data to be tested and reference training data; Determine scene change data based on the training data to be tested and the benchmark training data, and generate a change label for each pixel in the scene change data; wherein the change label will be used to calculate the loss function when training the substation scene change detection model; Randomly extract image pairs with changes from all the training data to be tested and the benchmark training data as positive samples, and randomly extract image pairs without changes as negative samples; Mixing the positive samples and the negative samples to form a sample data set; The sample data set is divided according to a preset specific ratio to form a training set and a verification set; wherein the ratio of positive samples to negative samples in the training set and the verification set is a preset positive-negative ratio.
7. The method for detecting substation scene changes according to claim 6, characterized in that: The step of training the substation scene change detection model comprises: The training set and the verification set are used to repeatedly train and verify the preset substation scene change detection network structure; during the training process, The substation scene change detection network structure is repeatedly trained and optimized using a preset combined loss function; and the changed devices in the scene change data in the training set are regarded as the foreground, and the unchanged objects in the training set are regarded as the background, until the total loss of the predicted mask map of the changed devices generated by the segmentation task of the substation scene change detection network structure reaches a preset training threshold, then the training is stopped, and the substation scene change detection network structure generated by the last training is used as the substation scene change detection model.
8. The method for detecting substation scene changes according to claim 7, characterized in that: The steps of calculating the total loss include: Obtaining a weight loss according to weight values of the foreground and the background, and calculating an edge loss according to a predicted segmentation area and a predicted change area of the predicted mask image; The weight loss and the edge loss are linearly weighted to obtain the total loss.
9. A substation scene change detection system, characterized in that: For implementing the substation scene change detection method according to any one of claims 1 to 8, the system includes a substation scene change detection model, and the substation scene change detection model includes: The SAM encoding module of the visual large model is used to extract features from the obtained reference image and the image to be tested about the substation scene to obtain the reference feature image and the feature image to be tested; A spatiotemporal domain module, configured to extract spatial features from the reference feature map and the feature map to be tested respectively to obtain reference spatial dimension features and reference spatial dimension features to be tested respectively, and to obtain a multi-scale spatiotemporal difference map based on the reference spatial dimension features and reference spatial dimension features to be tested by a preset time domain fusion algorithm; A SAM decoding module of the visual large model is used to decode according to the multi-scale spatiotemporal difference map to obtain a mask map of the changed equipment in the power transformation scene; A result extraction module is used to determine the change result of the substation scene according to the mask image.
10. The substation scene change detection system according to claim 9, characterized in that: It also includes a pre-training module for pre-training the substation scene change detection model, which includes: Based on the preset three-dimensional model of the power system, the names of different objects in the substation are defined, and corresponding class labels are mapped to each object to construct a three-dimensional scene; Using preset simulation software, the drone is moved along a preset route track of the three-dimensional scene to obtain a reference sequence; The three-dimensional scene is changed to obtain a changed scene, and the drone in the simulation software moves on the changed scene according to a preset line trajectory of the three-dimensional scene to obtain a sequence to be tested; Performing generalization processing on the sequence to be tested and the reference sequence to obtain training data to be tested and reference training data; Determine scene change data based on the training data to be tested and the benchmark training data, and generate a change label for each pixel in the scene change data; wherein the change label will be used to calculate the loss function when training the substation scene change detection model; Randomly extract image pairs with changes from all the training data to be tested and the benchmark training data as positive samples, and randomly extract image pairs without changes as negative samples to serve as training sets and validation sets; The training set and the verification set are used to repeatedly train and verify the preset substation scene change detection network structure; during the training process, The substation scene change detection network structure is repeatedly trained and optimized using a preset combined loss function; and the changed devices in the scene change data in the training set are regarded as the foreground, and the unchanged objects in the training set are regarded as the background, until the total loss of the predicted mask map of the changed devices generated by the segmentation task of the substation scene change detection network structure reaches a preset training threshold, then the training is stopped, and the substation scene change detection network structure generated by the last training is used as the substation scene change detection model.