Multi-scenario substation equipment defect change detection method based on optical flow feature fusion

Abstract

The present invention discloses a multi-scenario substation defect change detection method based on optical flow feature fusion, including a twin optical flow generation module and a defect change detection module. The twin optical flow generation module is used to extract general difference features between template images and detection images in multiple scenes, and output forward optical flow images and reverse optical flow images after visualization; the defect change detection module is used to extract and fuse features of forward optical flow images, reverse optical flow images, template images and detection images, and use rectangular frames to locate defect change areas at multiple scales. The present invention uses template images + detection images + bidirectional optical flow images as the input of the defect change detection network, integrates optical flow features and appearance features, can effectively extract general difference features between image pairs in multiple scenes, and improve the accuracy of substation defect detection; the present invention is suitable for defect change detection of power equipment in multi-scenario monitoring, and the model has strong adaptability and robustness.

Description

Technical Field

[0001] The present invention relates to intelligent monitoring defect change detection of industrial equipment, and in particular to a multi-scenario substation defect change detection method based on optical flow feature fusion. Background Art

[0002] There are many kinds of power transmission and transformation equipment distributed in the power scene, such as substation cabinets, electromagnetic switches, insulators, etc. In the daily operation process, due to the complex working environment, common defects such as cabinet door opening, insulator defect, and foreign object intrusion, such as personnel, vehicles, and kites, often occur in the equipment status change. In order to maintain the normal operation of the power transmission and transformation equipment, it is necessary to monitor the equipment status changes and the intrusion of foreign objects in the scene. At present, the main method is for personnel to view the video, which is time-consuming, labor-intensive and inefficient.

[0003] In recent years, computer vision technology has developed more maturely and is widely used in industrial production and monitoring. This year, some scholars have used target detection methods to explore specific fields such as personnel detection, vehicle detection, and defect detection. However, in general, the targets to be detected need to have common features, and the detection scenes are relatively single. In the defect detection of power equipment, there are few samples, many types of defects, and many scenes, which makes it difficult to extract general features. Therefore, the change information between the template image and the test image can be effectively used to extract general difference features. In addition, since the template image and the test image are not taken under the same conditions, weather changes, lighting changes, and camera motion will make it difficult to extract general difference features. Therefore, the relevant research methods of predecessors are difficult to directly apply to the defect change detection of power equipment in multiple scenes.

[0004] It is worth noting that optical flow features are widely used in video recognition tasks to extract motion features between video frames, which is similar to the defect change detection task. By performing optical flow estimation on the template image and the test image, the pixel-level offset between the two can be generated, and the optical flow map can be generated after further visualization, which has good positioning performance for the defect change area. At the same time, the field of optical flow estimation has a large amount of open source data, and general difference features can be extracted after pre-training as a supplement to the harsh conditions of multiple scenes and few samples. How to effectively combine optical flow estimation methods and target detection technology, apply them to defect change detection of substation equipment in multiple scenes, and obtain a detection model with high robustness and high accuracy is a huge challenge. Summary of the invention

[0005] Purpose of the invention: The purpose of the present invention is to provide a multi-scenario substation equipment defect change detection method based on optical flow feature fusion, which solves the problem of multi-scenario power equipment defect change detection.

[0006] Technical solution: The multi-scenario substation equipment defect change detection method based on optical flow feature fusion described in the present invention includes two parts: a twin optical flow generation module and a defect change detection module, specifically:

[0007] Twin optical flow generation module: extracts general difference features between template images and test images in multiple scenes, and outputs forward optical flow images and reverse optical flow images after visualization;

[0008] Defect change detection module: extracts and fuses features of forward optical flow images, reverse optical flow images, template images and detection images, and uses rectangular frames to locate defect change areas at multiple scales.

[0009] The twin optical flow generation module includes the following steps:

[0010] (1) Construct a pre-trained optical flow dataset T based on the FlyingChairs open source dataset; the pre-trained optical flow dataset T uses image pairs as samples, and the samples include template images Tn and detection images Td. The sample labels are suffix files of flo, corresponding to the change from template image Tn to detection image Td, and the horizontal and vertical offsets of each pixel in the template image Tn;

[0011] (2) Build the twin optical flow generation module. This step is implemented through the following sub-steps:

[0012] (2.1) Build a twin encoding network. The twin encoding network is a three-level encoding network with two identical structures and shared weights. Each level of the encoding network consists of multiple cascaded convolutional layers, ReLU activation functions, and Max-Pool. The template image Tn and the detection image Td are input into the twin encoding network respectively, and the first-level feature F1, the second-level feature F2, and the third-level feature F3 are generated after splicing at each level.

[0013] (2.2) Build a decoding and reconstruction network. The decoding and reconstruction network is a three-level decoding network. Each level of the decoding network consists of multiple cascaded deconvolution layers, ReLU activation functions and upsampling. The input of the third-level decoding network is the third-level feature F3, and the output is the third-level reconstruction feature R3. The input of the second-level decoding network is the concatenation of the second-level feature F2 and the third-level reconstruction feature R3, and the output is the second-level reconstruction feature R2. The input of the first-level decoding network is the concatenation of the first-level feature F1 and the second-level reconstruction feature R2, and the output is the first-level reconstruction feature R1, which corresponds to the final prediction result.

[0014] (3) Train the twin optical flow generation module. This step is implemented through the following sub-steps:

[0015] (3.1) Use the pre-trained optical flow dataset T to train the twin optical flow module and set hyperparameters such as the number of training rounds, learning rate, and batch size;

[0016] (3.2) During the training process, EPEloss is used as the loss function:

[0017]

[0018] Where M×N is the number of pixels of the template image Tn; It is the prediction result output by the twin optical flow generation module, corresponding to the change from the template image Tn to the detection image Td, the lateral offset and longitudinal offset of each pixel point, and x, y are the actual lateral offset and longitudinal offset of each pixel point provided by the suffix flo file;

[0019] (3.3) During the training process, the SGD optimizer is used to update the network weights of the twin optical flow generation module. After the model training reaches the set training round, the trained weights in the twin optical flow generation module are saved;

[0020] (4) Test the twin optical flow generation module. This step is implemented through the following sub-steps:

[0021] (4.1) Load the twin optical flow generation module and the trained weights, input the template image Tn and the detection image Td into the trained twin optical flow module for testing, predict the lateral and longitudinal offsets of each pixel, and generate a flo suffix file;

[0022] (4.2) Normalize the offset data in the generated flo suffix file to an interval of (0, 1), and visualize it according to the Munsell color system to generate an optical flow image.

[0023] The defect change detection module includes the following steps:

[0024] (1) Data preprocessing: This step is implemented through the following sub-steps:

[0025] (1.1) Based on the monitoring image data of the actual substation equipment area, a substation equipment defect detection dataset I is constructed. The image pairs are used as samples. The samples include the template image In and the detection image Id. The label file is an XML suffix file. The camera movement, weather and lighting changes are ignored. The VOC rectangular box format is used to annotate the defect change area between the template image In and the detection image Id. The category is anomaly.

[0026] (1.2) The template image In and the detection image Id are swapped in order, and the trained twin optical flow generation module is input twice for testing. After visualization, the forward optical flow image Mf and the reverse optical flow image Mb are generated to form the bidirectional optical flow dataset M;

[0027] (1.3) Data enhancement is performed on the template image In, the detection image Id, and the bidirectional optical flow dataset M using random mirroring, random cropping, random noise, and other methods. The size of each image is changed to 416×416×3, and the size after splicing is changed to 416×416×12;

[0028] (2) Build a defect change detection module. This step is implemented through the following sub-steps:

[0029] (2.1) Build Darknet-53 to extract features. Darknet-53 consists of multiple DBL modules and res-n modules. The DBL module consists of a convolutional layer, a BN layer, and a ReLU activation function. The res-n module consists of multiple DBL modules and res residual units. Modify the input channel of Darknet-53 to 12 channels, remove the last fully connected layer, perform 5 downsamplings in sequence, and extract the downsampled features D1, D2, D3, D4, and D5.

[0030] (2.2) Build a feature pyramid fusion feature. Use the extracted downsampled features D3, D4, and D5 to build a feature pyramid. D5 passes through the DBL module and is upsampled before being spliced ​​with D4 to generate fusion feature A1. D4 passes through the DBL module and is upsampled before being spliced ​​with D3 to generate fusion feature A2.

[0031] (2.3) Use fused features for multi-scale prediction. Input the fifth down-sampled feature D5 and fused features A1 and A2 into the DBL module and convolution layer, and output prediction values ​​of three scales with sizes of 13*13*3*5, 26*26*3*5, and 52*52*3*5 respectively. The three scales correspond to the number of division units S=13, 26, and 52, respectively, and the number of anchors at each scale K=3;

[0032] (3) Training the defect change detection module. This step is implemented through the following sub-steps:

[0033] (3.1) Use the template image In, the detection image Id and the training set data of the bidirectional optical flow dataset M to train the defect detection module, and set hyperparameters such as training rounds, learning rate, batch size, etc.

[0034] (3.2) During the training process, weighted loss is used As a loss function:

[0035]

[0036] Among them, x, y, w, h, C are the prediction results output by the defect change detection module, which correspond to the center coordinates, width, length, and confidence predicted based on a certain anchor of a certain unit, S×S is the number of units in which the image data is divided, and K is the number of anchors at each scale; if the j anchor of the i unit is responsible for the prediction, then If the jAnchor of unit i is not responsible for prediction, then Weighted Loss The first two items are the regression loss of the bounding box position, and the last two items are the cross entropy loss of the confidence level;

[0037] (3.3) During the training process, the Adam optimizer is used to update the network weights of the defect change detection module. After the model training reaches the set number of iterations, the trained weights in the defect change detection module are saved;

[0038] (4) Test the defect change detection module. This step is implemented through the following sub-steps:

[0039] (4.1) Load the defect change detection module and the trained weights, use the test set data of the substation defect detection dataset I and the bidirectional optical flow dataset M as the input of the defect change detection module, and generate test results;

[0040] (4.2) Post-process the test results and use NMS non-maximum suppression to post-process the prediction results of the three scales. Set the confidence threshold con_t=0.5 and the interaction ratio threshold iou_t=0.4 to filter out the prediction results with low confidence and large interaction ratio, and generate the final defect change detection results.

[0041] A multi-scenario substation defect change detection system based on optical flow feature fusion, including a twin optical flow generation module and defect change detection; the twin optical flow generation module is used to extract general difference features between template images and detection images in multiple scenes, and output forward optical flow images and reverse optical flow images after visualization; the defect change detection module is used to extract and fuse features of forward optical flow images, reverse optical flow images, template images and detection images, and use rectangular frames to locate defect change areas at multiple scales.

[0042] A computer storage medium stores a computer program, which, when executed by a processor, implements the above-mentioned multi-scenario substation equipment defect change detection method based on optical flow feature fusion.

[0043] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for detecting defect changes in multi-scenario substation equipment based on optical flow feature fusion is implemented.

[0044] Beneficial effects: Compared with the prior art, the present invention has the following advantages: 1. The present invention adopts template image + detection image + bidirectional optical flow image as the input of the defect change detection network, integrates optical flow features and appearance features, and can effectively extract general difference features between image pairs in multiple scenes, thereby improving the accuracy of defect detection in substation equipment; 2. The present invention is suitable for defect change detection of power equipment in multi-scene monitoring, and the model has strong adaptability and robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a schematic diagram of the overall structure of this method;

[0046] Figure 2 It is a schematic diagram of the structure of the twin optical flow generation module of this method;

[0047] Figure 3 It is a structural diagram of the defect change detection module of the present method;

[0048] Figure 4 It is a flowchart of the training process of the twin optical flow generation module of this method;

[0049] Figure 5 is a flow chart of the training process of the defect change detection module of the present method;

[0050] Figure 6 is a schematic diagram of the experimental results of this method, where Figure 6 a is the template image, Figure 6 b is the detection picture, Figure 6 c is the forward optical flow map, Figure 6 d is the reverse optical flow map, which uses a rectangular frame to frame the defect area predicted by the method in the detection image. DETAILED DESCRIPTION

[0051] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0052] The present invention takes the image data extracted from the monitoring video of multiple scenes of a substation as an example. The image data includes template images and detection images under multiple scenes. The substation equipment defect change detection task requires that the defect change detection accuracy mAP be maintained above 80% and the defect discrimination accuracy Acc be greater than 90%.

[0053] like Figure 1 As shown, the present invention is a multi-scenario substation defect change detection method based on optical flow feature fusion. The method constructs a twin optical flow generation module and a defect change detection module to achieve accurate detection of multi-scenario substation defect changes. The following is an explanation in conjunction with a specific embodiment:

[0054] like Figure 2and Figure 4 As shown, the steps of the twin optical flow generation module are as follows:

[0055] (1) Construct a pre-trained optical flow dataset T based on the FlyingChairs open source dataset. The pre-trained optical flow dataset T uses image pairs as samples. The samples include template images Tn and detection images Td. The sample labels are suffix files of flo, corresponding to the horizontal and vertical offsets of each pixel in the template image Tn from the template image Tn to the detection image Td.

[0056] (2) Build the twin optical flow generation module. This step is implemented through the following sub-steps:

[0057] (2.1) Build a twin encoding network. The twin encoding network is a three-level encoding network with two identical structures and shared weights. Each level of the encoding network consists of multiple cascaded convolutional layers, ReLu activation functions, and Max-Pool. The template image Tn and the detection image Td are input into the twin encoding network respectively, and the first-level feature F1, the second-level feature F2, and the third-level feature F3 are generated after splicing at each level.

[0058] (2.2) Build a decoding and reconstruction network. The decoding and reconstruction network is a three-level decoding network. Each level of the decoding network consists of multiple cascaded deconvolution layers, ReLu activation functions and upsampling. The input of the third-level decoding network is the third-level feature F3, and the output is the third-level reconstruction feature R3; the input of the second-level decoding network is the concatenation of the second-level feature F2 and the third-level reconstruction feature R3, and the output is the second-level reconstruction feature R2; the input of the first-level decoding network is the concatenation of the first-level feature F1 and the second-level reconstruction feature R2, and the output is the first-level reconstruction feature R1, which corresponds to the final prediction result.

[0059] (3) Train the twin optical flow generation module. This step is implemented through the following sub-steps:

[0060] (3.1) Use the pre-trained optical flow dataset T to train the twin optical flow module and set hyperparameters such as the number of training rounds, learning rate, and batch size.

[0061] (3.2) During the training process, EPEloss is used as the loss function:

[0062]

[0063] Where M×N is the number of pixels of the template image Tn. It is the prediction result output by the twin optical flow generation module, corresponding to the change from the template image Tn to the detection image Td, the lateral offset and longitudinal offset of each pixel. x,y are the actual lateral offset and longitudinal offset of each pixel provided by the suffix flo file.

[0064] (3.3) During the training process, the SGD optimizer is used to update the network weights of the twin optical flow generation module. After the model training reaches the set training rounds, the trained weights in the twin optical flow generation module are saved.

[0065] (4) Test the twin optical flow generation module. This step is implemented through the following sub-steps:

[0066] (4.1) Load the twin optical flow generation module and the trained weights, input the template image Tn and the detection image Td into the trained twin optical flow module for testing, predict the lateral and longitudinal offsets of each pixel, and generate a flo suffix file.

[0067] (4.2) Normalize the offset data in the generated flo suffix file to an interval of (0, 1), and visualize it according to the Munsell color system to generate an optical flow image.

[0068] like Figure 3 and Figure 5 As shown, the steps of the defect change detection module are as follows:

[0069] (1) Data preprocessing: This step is implemented through the following sub-steps:

[0070] (1.1) Based on the monitoring image data of the actual substation equipment area, a substation equipment defect detection dataset I is constructed. The image pairs are used as samples. The samples include template images In and detection images Id. The label files are xml suffix files. Camera movement, weather and lighting changes are ignored. The VOC rectangular box format is used to annotate the defect change area between the template image In and the detection image Id, and the category is anomaly.

[0071] (1.2) The template image In and the detection image Id are swapped in order and input into the trained twin optical flow generation module twice for testing. After visualization, the forward optical flow image Mf and the reverse optical flow image Mb are generated to form the bidirectional optical flow dataset M.

[0072] (1.3) The template image In, the detection image Id, and the bidirectional optical flow dataset M are enhanced by random mirroring, random cropping, random noise addition, etc. The size of each image is changed to 416×416×3, and the size after splicing is changed to 416×416×12.

[0073] (2) Figure 3 As shown, build a defect change detection module, which is implemented through the following sub-steps:

[0074] (2.1) Build Darknet-53 to extract features. Darknet-53 consists of multiple DBL modules and res-n modules. The DBL module consists of a convolutional layer, a BN layer, and a ReLU activation function. The res-n module consists of multiple DBL modules and res residual units. The input channel of Darknet-53 is modified to 12 channels, and the last fully connected layer is removed. Downsampling is performed five times in sequence to extract the downsampled features D1, D2, D3, D4, and D5.

[0075] (2.2) Build a feature pyramid to fuse features. Use the extracted downsampled features D3, D4, and D5 to build a feature pyramid. D5 passes through the DBL module and is upsampled before being concatenated with D4 to generate fused feature A1. D4 passes through the DBL module and is upsampled before being concatenated with D3 to generate fused feature A2.

[0076] (2.3) Use fused features for multi-scale prediction. Input the fifth down-sampled feature D5 and fused features A1 and A2 into the DBL module and convolution layer, and output prediction values ​​of three scales with sizes of 13*13*3*5, 26*26*3*5, and 52*52*3*5 respectively. The three scales correspond to the number of division units S=13, 26, and 52 respectively, and the number of anchors at each scale K=3.

[0077] (3) Figure 5 As shown, the defect change detection module is trained. This step is implemented by the following sub-steps:

[0078] (3.1) Use the template image In, the detection image Id and the training set data of the bidirectional optical flow dataset M to train the defect detection module, and set hyperparameters such as training rounds, learning rate, and batch size.

[0079] (3.2) During the training process, weighted loss is used As a loss function:

[0080]

[0081]

[0082] Among them, x, y, w, h, C are the prediction results output by the defect change detection module, which correspond to the center coordinates, width, length, and confidence predicted based on a certain anchor of a certain unit, S×S is the number of units in which the image data is divided, and K is the number of anchors at each scale. If the j anchor of the i unit is responsible for the prediction, then If the jAnchor of unit i is not responsible for prediction, then Weighted Loss The first two items are the regression loss of the bounding box position, and the last two items are the cross entropy loss of the confidence.

[0083] (3.3) During the training process, the Adam optimizer is used to update the network weights of the defect change detection module. After the model training reaches the set number of iterations, the trained weights in the defect change detection module are saved.

[0084] (4) Test the defect change detection module. This step is implemented through the following sub-steps:

[0085] (4.1) Load the defect change detection module and the trained weights, use the test set data of the substation defect detection dataset I and the bidirectional optical flow dataset M as the input of the defect change detection module, and generate test results.

[0086] (4.2) Post-process the test results and use NMS non-maximum suppression to post-process the prediction results of the three scales. Set the confidence threshold con_t=0.5 and the interaction ratio threshold iou_t=0.4 to filter out the prediction results with low confidence and large interaction ratio, and generate the final defect change detection results.

[0087] The dataset of defect change detection of substation equipment contains 1893 samples, of which about 480 samples contain defects, accounting for about 25.4%. 80% (1514) samples are randomly selected to train the detection model, and the remaining 20% ​​(379) samples are used as the test set. Table 1 lists the detection results of the multi-scenario defect change detection method based on optical flow feature fusion and the related ablation model on the test set. As shown in Table 1, different types of images are used as the input of the defect detection module in the ablation model. There are three types: template image + detection image, bidirectional optical flow image, template image + detection image + bidirectional optical flow image. The detection results are measured and evaluated using the defect detection accuracy mAP and the defect discrimination accuracy Acc.

[0088] Table 1 Comparative performance of multi-scenario substation equipment defect change detection methods based on optical flow feature fusion

[0089]

[0090] According to the detection results in Table 1, using template images + detection images + bidirectional optical flow images as the input of the defect detection network and fusing optical flow features and appearance features can improve the detection results of defect changes in power equipment in multiple scenarios, verifying the effectiveness of the twin optical flow generation module.

Claims

1. A multi-scenario substation equipment defect change detection method based on optical flow feature fusion, characterized in that: It includes two parts: twin optical flow generation module and defect change detection module, specifically: Twin optical flow generation module: extracts general difference features between template images and test images in multiple scenes, and outputs forward optical flow images and reverse optical flow images after visualization; The twin optical flow generation module includes the following steps: (1) Construct a pre-trained optical flow dataset T based on the FlyingChairs open source dataset; the pre-trained optical flow dataset T uses image pairs as samples, and the samples include template images Tn and detection images Td. The sample labels are suffix files of flo, corresponding to the change from template image Tn to detection image Td, and the horizontal and vertical offsets of each pixel in the template image Tn; (2) Build the twin optical flow generation module. This step is implemented through the following sub-steps: (2.1) Build a twin encoding network. The twin encoding network is a three-level encoding network with two identical structures and shared weights. Each level of the encoding network consists of multiple cascaded convolutional layers, ReLU activation functions, and Max-Pool. The template image Tn and the detection image Td are input into the twin encoding network respectively, and the first-level feature F1, the second-level feature F2, and the third-level feature F3 are generated after splicing at each level. (2.2) Build a decoding and reconstruction network. The decoding and reconstruction network is a three-level decoding network. Each level of the decoding network consists of multiple cascaded deconvolution layers, ReLU activation functions and upsampling. The input of the third-level decoding network is the third-level feature F3, and the output is the third-level reconstruction feature R3. The input of the second-level decoding network is the concatenation of the second-level feature F2 and the third-level reconstruction feature R3, and the output is the second-level reconstruction feature R2. The input of the first-level decoding network is the concatenation of the first-level feature F1 and the second-level reconstruction feature R2, and the output is the first-level reconstruction feature R1, which corresponds to the final prediction result. (3) Train the twin optical flow generation module. This step is implemented through the following sub-steps: (3.1) Use the pre-trained optical flow dataset T to train the twin optical flow module and set hyperparameters such as the number of training rounds, learning rate, and batch size; (3.2) During the training process, EPEloss is used as the loss function: Where M×N is the number of pixels of the template image Tn; It is the prediction result output by the twin optical flow generation module, corresponding to the change from the template image Tn to the detection image Td, the lateral offset and longitudinal offset of each pixel point, and x, y are the actual lateral offset and longitudinal offset of each pixel point provided by the suffix flo file; (3.3) During the training process, the SGD optimizer is used to update the network weights of the twin optical flow generation module. After the model training reaches the set training round, the trained weights in the twin optical flow generation module are saved; (4) Test the twin optical flow generation module. This step is implemented through the following sub-steps: (4.1) Load the twin optical flow generation module and the trained weights, input the template image Tn and the detection image Td into the trained twin optical flow module for testing, predict the lateral and longitudinal offsets of each pixel, and generate a flo suffix file; (4.2) Normalize the offset data in the generated flo suffix file and visualize it according to the Munsell color system to generate an optical flow image; Defect change detection module: extracts and fuses features of forward optical flow images, reverse optical flow images, template images, and detection images, and uses rectangular frames to locate defect change areas at multiple scales; The defect change detection module includes the following steps: (1) Data preprocessing: This step is implemented through the following sub-steps: (1.1) Based on the monitoring image data of the actual substation equipment area, a substation equipment defect detection dataset I is constructed. The image pairs are used as samples. The samples include the template image In and the detection image Id. The label file is an XML suffix file. The camera movement, weather and lighting changes are ignored. The VOC rectangular box format is used to annotate the defect change area between the template image In and the detection image Id. The category is anomaly. (1.2) The template image In and the detection image Id are swapped in order, and the trained twin optical flow generation module is input twice for testing. After visualization, the forward optical flow image Mf and the reverse optical flow image Mb are generated to form the bidirectional optical flow dataset M; (1.3) Data enhancement is performed on the template image In, the detection image Id, and the bidirectional optical flow dataset M using random mirroring, random cutting, random noise, and other methods. The size of each image becomes M×N×3, and the size after splicing becomes M×N×12; (2) Build a defect change detection module. This step is implemented through the following sub-steps: (2.1) Build Darknet-53 to extract features. Darknet-53 consists of multiple DBL modules and res-n modules. The DBL module consists of a convolutional layer, a BN layer, and a ReLU activation function. The res-n module consists of multiple DBL modules and res residual units. Modify the input channel of Darknet-53 to 12 channels, remove the last fully connected layer, perform 5 downsamplings in sequence, and extract the downsampled features D1, D2, D3, D4, and D5. (2.2) Build a feature pyramid fusion feature. Use the extracted downsampled features D3, D4, and D5 to build a feature pyramid. D5 passes through the DBL module and is upsampled before being spliced ​​with D4 to generate fusion feature A1. D4 passes through the DBL module and is upsampled before being spliced ​​with D3 to generate fusion feature A2. (2.3) Use fused features for multi-scale prediction. Input the fifth down-sampled feature D5 and fused features A1 and A2 into the DBL module and convolution layer, and output the prediction values ​​of three scales respectively. The three scales correspond to different numbers of division units S and the number of anchors K at each scale. (3) Training the defect change detection module. This step is implemented through the following sub-steps: (3.1) Use the template image In, the detection image Id and the training set data of the bidirectional optical flow dataset M to train the defect detection module, and set hyperparameters such as training rounds, learning rate, batch size, etc. (3.2) During the training process, weighted loss is used As a loss function: Among them, x, y, w, h, C are the prediction results output by the defect change detection module, which correspond to the center coordinates, width, length, and confidence predicted based on a certain anchor of a certain unit, S×S is the number of units in which the image data is divided, and K is the number of anchors at each scale; if the j anchor of the i unit is responsible for the prediction, then If the jAnchor of unit i is not responsible for prediction, then Weighted Loss The first two items are the regression loss of the bounding box position, and the last two items are the cross entropy loss of the confidence level; (3.3) During the training process, the Adam optimizer is used to update the network weights of the defect change detection module. After the model training reaches the set number of iterations, the trained weights in the defect change detection module are saved; (4) Test the defect change detection module. This step is implemented through the following sub-steps: (4.1) Load the defect change detection module and the trained weights, use the test set data of the substation defect detection dataset I and the bidirectional optical flow dataset M as the input of the defect change detection module, and generate test results; (4.2) Post-process the test results and use NMS non-maximum suppression to post-process the prediction results of the three scales. Set the confidence threshold con_t and the interaction ratio threshold iou_t to filter out the prediction results with low confidence and large interaction ratio, and generate the final defect change detection results.

2. A multi-scenario substation defect change detection system based on optical flow feature fusion, applying the multi-scenario substation defect change detection method based on optical flow feature fusion according to claim 1, characterized in that: It includes a twin optical flow generation module and a defect change detection module; the twin optical flow generation module is used to extract general difference features between template images and detection images in multiple scenes, and output forward optical flow images and reverse optical flow images after visualization; The defect change detection module is used to extract and fuse features of the forward optical flow image, the reverse optical flow image, the template image and the detection image, and use a rectangular frame to locate the defect change area at multiple scales.

3. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements a multi-scenario substation equipment defect change detection method based on optical flow feature fusion as described in claim 1.

4. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements the multi-scenario substation equipment defect change detection method based on optical flow feature fusion as described in claim 1.

Images