A station building defect identification and detection method, system, device and storage medium

Abstract

The present invention discloses a method, system, device and storage medium for identifying and detecting defects in a station building. The method comprises the following steps: obtaining online video data of the station building; converting a video stream of the online video data into frames using OpenCV to obtain online frame images; preprocessing the online frame images; inputting the preprocessed online frame images into a trained defect recognition model to obtain defect recognition results. The present invention incorporates a BAM attention mechanism into the convolutional network feature extraction stage, and uses PSPNet global pooling to perform weighted fusion on feature channels after pyramid pooling to retain more important feature information. The method has a strong perception of details, improves the accuracy of station building anomaly detection, and has high research significance for real-time monitoring of station building safety.

Description

Technical Field

[0001] The present invention relates to a method, system, device and storage medium for identifying and detecting defects in a station building, and belongs to the technical field of image recognition. Background Art

[0002] With the continuous development and widespread adoption of power grid construction, the number of power stations is also increasing. Power station security is a key research focus and hotspot in power grid security. Power station video surveillance systems now cover most critical areas, providing monitoring centers with clear video resources and becoming a crucial basis for real-time monitoring, anomaly warnings, and post-incident verification. However, existing power station video surveillance systems are relatively inefficient, with content analysis primarily relying on manual observation. Some sites have simple anomaly detection functions, but the anomaly recognition algorithms suffer from low accuracy, and the primary task of video analysis is still performed manually. With the increasing number of video acquisition points and the increasing intensity of monitoring, the challenges facing video surveillance systems are becoming increasingly significant. Existing video monitoring and analysis models, due to high investment costs and low information utilization, are no longer sufficient for field requirements. To improve the effectiveness of video surveillance systems, automated video content analysis and anomaly detection using image processing technology are becoming increasingly critical. Therefore, in this highly information-rich era, a high-precision power station defect detection method based on computer vision is being designed to enable real-time monitoring of power station safety management. This approach holds significant implications for power grid planning, grid operation and maintenance, and control departments, as well as for power station tracking and resource optimization.

[0003] Research on station anomaly detection methods utilizes video information, image information, and textual information such as instrumentation, switch status, and indicator lights from robot inspections. These three modalities are highly beneficial for model detection. However, due to physical and environmental factors, as well as wear and tear on instrument dials over time, the accuracy of defect identification is currently low. Summary of the Invention

[0004] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a method, system, device and storage medium for identifying and detecting defects in a station building, so as to solve the technical problem that the prior art means have low accuracy in defect identification results.

[0005] To achieve the above object, the present invention is implemented by adopting the following technical solutions:

[0006] In a first aspect, the present invention provides a method for identifying and detecting station building defects, comprising:

[0007] Obtain online video data of the station building;

[0008] Convert the video stream of online video data into frames through OpenCV to obtain online frame images;

[0009] Preprocessing the online frame image;

[0010] The preprocessed online frame image is input into the trained defect recognition model to obtain the defect recognition result.

[0011] Optionally, the training of the defect recognition model includes:

[0012] Obtain offline video data of the station building;

[0013] Convert the video stream of offline video data into frames through OpenCV to obtain offline frame images;

[0014] Perform defect annotation and preprocessing on offline frame images;

[0015] Generate a dataset based on the defect annotations and pre-processed offline frame images, and divide the dataset into a training set and a validation set according to a preset ratio;

[0016] Integrate the BAM attention mechanism and PSPNet pyramid pooling into the ResNet-34 network structure to build a defect recognition model;

[0017] The defect recognition model is trained through the training set to optimize the model parameters, and the effect of the defect recognition model corresponding to each model parameter is verified through the validation set. The defect recognition model with the best effect is selected as the training result.

[0018] Optionally, the preprocessing includes pixel brightness transformation, geometric transformation, local neighborhood preprocessing and image restoration.

[0019] Optionally, the BAM attention mechanism is integrated into the ResNet-34 network structure for inputting images for feature extraction, wherein the feature extraction includes:

[0020] Extract features of the input image based on the convolutional network to generate a feature map F;

[0021] The feature map F is globally average pooled and max pooled respectively through the channel attention map;

[0022] The pooling results of the global average pooling and the maximum pooling of the feature map F are connected and input into the multi-layer perceptron for addition operation;

[0023] The channel weight coefficient Mc(F) is generated by processing the addition operation result through the Sigmoid activation function:

[0024] Mc(F)=σ(MLP(AvgPool(F))+MLP(MaxPool(F)))

[0025] Where AvgPool is average pooling, MaxPool is maximum pooling, MLP is multi-layer perceptron, and σ is Sigmoid activation function;

[0026] According to the channel weight coefficient Mc(F), the channel weight of the feature map F is adjusted to obtain the feature map F ′ :

[0027]

[0028] The feature map F is processed by spatial attention mapping ′ Perform average pooling and maximum pooling respectively;

[0029] The feature map F ′ The pooling results of average pooling and maximum pooling are processed by standard convolutional layers for dimensionality reduction;

[0030] The dimensionality reduction result is processed by the Sigmoid activation function to generate the spatial weight coefficient Ms(F):

[0031] Ms(F)=σ{f 7×7 [AvgPool(F ′ );MaxPool(F ′ )]}

[0032] Where: f 7×7 It is a standard convolution layer with a convolution kernel of 7×7;

[0033] According to the spatial weight coefficient Ms(F), the feature map F ′ Perform spatial weight adjustment to obtain the feature map F″:

[0034]

[0035] Optionally, the PSPNet pyramid pooling is integrated into the ResNet-34 network structure to perform different-scale fusion of input features; the different-scale fusion includes:

[0036] Apply pyramid pooling operations of different sizes to the input features to obtain feature maps of corresponding sizes;

[0037] 1×1 convolution is used to reduce the dimension of feature maps of each size;

[0038] The feature map after dimensionality reduction is upsampled using bilinear interpolation to make it the same size as the input feature;

[0039] The upsampled feature map is placed on the input feature map from bottom to top, and the feature points on each feature channel are averaged through global average pooling. The weight coefficient of the feature channel is obtained by multiplying the average result with the corresponding channel matrix;

[0040] Based on the weight coefficient, 1×1 convolution is used to accumulate and sum the input features to obtain the fused feature map:

[0041]

[0042] Where y i is the eigenvalue of the i-th feature channel on the fusion feature map, N is the number of feature channels, w i is the weight coefficient of the i-th feature channel, x i is the eigenvalue of the i-th feature channel on the input feature.

[0043] Optionally, a Softmax classifier is sampled on the fused feature map to perform classification prediction to obtain a defect prediction result.

[0044] In a second aspect, the present invention provides a station building defect identification and detection system, the system comprising:

[0045] The first data acquisition module is used to acquire online video data of the station building;

[0046] The first image extraction module is used to convert the video stream of the online video data into frames through OpenCV to obtain online frame images;

[0047] A first preprocessing module, used for preprocessing the online frame image;

[0048] The defect recognition module is used to input the preprocessed online frame image into the trained defect recognition model to obtain the defect recognition result.

[0049] Optionally, also include:

[0050] The second data acquisition module is used to acquire offline video data of the station building;

[0051] The second image extraction module is used to convert the video stream of the offline video data into frames through OpenCV to obtain offline frame images;

[0052] The second preprocessing module is used to mark defects and preprocess the offline frame images;

[0053] The dataset construction module is used to generate a dataset based on defect annotations and pre-processed offline frame images, and divide the dataset into a training set and a validation set according to a preset ratio;

[0054] A model building module, which integrates the BAM attention mechanism and PSPNet pyramid pooling into the ResNet-34 network structure to build a defect recognition model;

[0055] The model training module is used to train the defect recognition model through the training set, optimize the model parameters, and verify the effect of the defect recognition model corresponding to each model parameter through the verification set, and select the defect recognition model with the best effect as the training result.

[0056] In a third aspect, the present invention provides a station building defect identification and detection system, including a processor and a storage medium;

[0057] The storage medium is used to store instructions;

[0058] The processor is configured to operate according to the instructions to execute the steps of the above method.

[0059] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above method when executed by a processor.

[0060] Compared with the prior art, the present invention has the following beneficial effects:

[0061] The present invention provides a method, system, device and storage medium for identifying and detecting station building defects, which are used for high-precision detection of station building defects. In view of the fact that the existing semantic segmentation model of defect images has unsatisfactory segmentation accuracy of small target objects and the difficulty in segmenting small strip-shaped target areas, the model incorporates the BAM attention mechanism into the convolutional network feature extraction stage, and uses PSPNet global pooling to perform weighted fusion of feature channels after pyramid pooling to retain more important feature information. It has a strong perception of details, improves the accuracy of station building anomaly detection, and has high research significance for real-time monitoring of station building safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 This is a flow chart of a method for identifying and detecting station building defects provided by the first embodiment of the present invention;

[0063] Figure 2 This is a training flow chart of the defect recognition model provided in the first embodiment of the present invention;

[0064] Figure 3 This is a schematic diagram of the BAM attention mechanism provided in Example 1 of the present invention. DETAILED DESCRIPTION

[0065] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention.

[0066] Example 1:

[0067] like Figure 1 As shown, an embodiment of the present invention provides a method for identifying and detecting station building defects, comprising the following steps:

[0068] S101, obtaining online video data of the station building;

[0069] S102, converting the video stream of the online video data into frames through OpenCV to obtain an online frame image;

[0070] S103, pre-processing the online frame image;

[0071] S104: Input the pre-processed online frame image into the trained defect recognition model to obtain a defect recognition result.

[0072] like Figure 2 As shown in Figure 2, the training of the defect recognition model includes the following steps:

[0073] S201, obtaining offline video data of the station building;

[0074] S202, converting the video stream of the offline video data into frames through OpenCV to obtain an offline frame image;

[0075] S203, marking defects and preprocessing the offline frame image;

[0076] S204, generating a data set based on the defect annotations and pre-processed offline frame images, and dividing the data set into a training set and a validation set according to a preset ratio;

[0077] S205. Integrate the BAM attention mechanism and PSPNet pyramid pooling into the ResNet-34 network structure to build a defect recognition model;

[0078] S206 , training the defect recognition model through the training set, optimizing the model parameters, and verifying the effect of the defect recognition model corresponding to each model parameter through the validation set, and selecting the defect recognition model with the best effect as the training result.

[0079] In this embodiment, the preprocessing includes pixel brightness transformation, geometric transformation, local neighborhood preprocessing, and image restoration.

[0080] In this embodiment, if Figure 3 As shown in the figure, the BAM attention mechanism is integrated into the ResNet-34 network structure for feature extraction of input images. Feature extraction includes the following steps:

[0081] S301, extracting features from the input image based on a convolutional network to generate a feature map F;

[0082] S302, perform global average pooling and maximum pooling on the feature map F through channel attention mapping;

[0083] S303, connecting the pooling results of global average pooling and maximum pooling of the feature map F and inputting them into the multi-layer perceptron for addition operation;

[0084] S304. Process the addition operation result through the Sigmoid activation function to generate the channel weight coefficient Mc(F):

[0085] Mc(F)=σ(MLP(AvgPool(F))+MLP(MaxPool(F)))

[0086] Where AvgPool is average pooling, MaxPool is maximum pooling, MLP is multi-layer perceptron, and σ is Sigmoid activation function;

[0087] S305, adjust the channel weight of the feature map F according to the channel weight coefficient Mc(F) to obtain the feature map F ′ :

[0088]

[0089] S306, feature map F through spatial attention mapping ′ Perform average pooling and maximum pooling respectively;

[0090] S307, the feature map F ′ The pooling results of average pooling and maximum pooling are processed by standard convolutional layers for dimensionality reduction;

[0091] S308. Process the dimensionality reduction result through the Sigmoid activation function to generate a spatial weight coefficient Ms(F):

[0092] Ms(F)=σ{f 7×7 [AvgPool(F ′ );MaxPool(F ′ )]}

[0093] Where: f 7×7 It is a standard convolution layer with a convolution kernel of 7×7;

[0094] S309, according to the spatial weight coefficient Ms (F) feature map F ′ Perform spatial weight adjustment to obtain the feature map F″:

[0095]

[0096] In this embodiment, PSPNet pyramid pooling is integrated into the ResNet-34 network structure to fuse input features at different scales. Fusion at different scales includes the following steps:

[0097] S401, applying pyramid pooling operations of different sizes to input features to obtain feature maps of corresponding sizes;

[0098] Different sizes include 1×1, 2×2, 3×3, and 6×6, with a total of four layers. The first layer is a global average pooling operation to extract the global context features of the input image; the second, third, and fourth layers first divide the input into 2×2, 3×3, and 6×6 sub-regions respectively, and then use global average pooling in these sub-regions to extract local features.

[0099] S402, perform dimensionality reduction processing on the feature maps of each size using 1×1 convolution;

[0100] S403, upsampling the feature map after dimensionality reduction using bilinear interpolation to make it the same size as the input feature map;

[0101] S404: Place the upsampled feature map on the input feature map in order from bottom to top, calculate the average value of the feature points on each feature channel through global average pooling, and multiply the average value by the corresponding channel matrix to obtain the weight coefficient of the feature channel;

[0102] S405: Based on the weight coefficient, 1×1 convolution is used to accumulate and sum the input features to obtain a fused feature map:

[0103]

[0104] Where y i is the eigenvalue of the i-th feature channel on the fusion feature map, N is the number of feature channels, w i is the weight coefficient of the i-th feature channel, x i is the eigenvalue of the i-th feature channel on the input feature.

[0105] After obtaining the fused feature map, the Softmax classifier is sampled on the fused feature map for classification prediction to obtain the defect prediction result; in the offline stage, the loss is calculated by combining the defect prediction result with the defect labeling result to optimize the model parameters; in the online stage, the defect prediction result is used as the final defect recognition result.

[0106] Example 2:

[0107] An embodiment of the present invention provides a station building defect identification and detection system, the system comprising:

[0108] The first data acquisition module is used to acquire online video data of the station building;

[0109] The first image extraction module is used to convert the video stream of the online video data into frames through OpenCV to obtain online frame images;

[0110] A first preprocessing module, used for preprocessing the online frame image;

[0111] The defect recognition module is used to input the preprocessed online frame image into the trained defect recognition model to obtain the defect recognition result.

[0112] Specifically, it also includes:

[0113] The second data acquisition module is used to acquire offline video data of the station building;

[0114] The second image extraction module is used to convert the video stream of the offline video data into frames through OpenCV to obtain offline frame images;

[0115] The second preprocessing module is used to mark defects and preprocess the offline frame images;

[0116] The dataset construction module is used to generate a dataset based on defect annotations and pre-processed offline frame images, and divide the dataset into a training set and a validation set according to a preset ratio;

[0117] A model building module, which integrates the BAM attention mechanism and PSPNet pyramid pooling into the ResNet-34 network structure to build a defect recognition model;

[0118] The model training module is used to train the defect recognition model through the training set, optimize the model parameters, and verify the effect of the defect recognition model corresponding to each model parameter through the verification set, and select the defect recognition model with the best effect as the training result.

[0119] Example 3:

[0120] Based on the first embodiment, the embodiment of the present invention provides a station building defect identification and detection system, including a processor and a storage medium;

[0121] The storage medium is used to store instructions;

[0122] The processor is configured to operate according to the instructions to execute the steps of the above method.

[0123] Example 4:

[0124] Based on the first embodiment, the embodiment of the present invention provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the above method are implemented.

[0125] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0126] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0127] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0128] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0129] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for identifying and detecting station building defects, characterized in that: include: Obtain online video data of the station building; Convert the video stream of online video data into frames through OpenCV to obtain online frame images; Preprocessing the online frame image; Input the preprocessed online frame image into the trained defect recognition model to obtain the defect recognition result; Among them, the construction of the defect recognition model includes: integrating the BAM attention mechanism and PSPNet pyramid pooling into the ResNet-34 network structure; integrating the BAM attention mechanism into the ResNet-34 network structure for feature extraction of input images; integrating PSPNet pyramid pooling into the ResNet-34 network structure for fusing input features at different scales.

2. A method for identifying and detecting station building defects according to claim 1, characterized in that: The training of the defect recognition model includes: Obtain offline video data of the station building; Convert the video stream of offline video data into frames through OpenCV to obtain offline frame images; Perform defect annotation and preprocessing on offline frame images; Generate a dataset based on the defect annotations and pre-processed offline frame images, and divide the dataset into a training set and a validation set according to a preset ratio; The defect recognition model is trained through the training set to optimize the model parameters, and the effect of the defect recognition model corresponding to each model parameter is verified through the validation set. The defect recognition model with the best effect is selected as the training result.

3. A method for identifying and detecting station building defects according to claim 1, characterized in that: The preprocessing includes pixel brightness transformation, geometric transformation, local neighborhood preprocessing and image restoration.

4. A method for identifying and detecting station building defects according to claim 1, characterized in that: The feature extraction includes: Extract features of the input image based on the convolutional network to generate a feature map F; The feature map F is globally average pooled and max pooled respectively through the channel attention map; The pooling results of the global average pooling and the maximum pooling of the feature map F are connected and input into the multi-layer perceptron for addition operation; The channel weight coefficient Mc(F) is generated by processing the addition operation result through the Sigmoid activation function: Mc(F)=σ(MLP(AvgPool(F))+MLP(MaxPool(F))) Where AvgPool is average pooling, MaxPool is maximum pooling, MLP is multi-layer perceptron, and σ is Sigmoid activation function; According to the channel weight coefficient Mc(F), the channel weight of the feature map F is adjusted to obtain the feature map F′: The feature map F′ is average pooled and max pooled respectively through spatial attention mapping; The pooling results of average pooling and maximum pooling of feature map F′ are processed through standard convolutional layer for dimensionality reduction; The dimensionality reduction result is processed by the Sigmoid activation function to generate the spatial weight coefficient Ms(F): Ms(F)=σ{f 7×7 [AvgPool(F′);MaxPool(F′)]} Where: f 7×7 It is a standard convolution layer with a convolution kernel of 7×7; According to the spatial weight coefficient Ms(F), the spatial weight of the feature map F′ is adjusted to obtain the feature map F″:

5. The method for identifying and detecting station building defects according to claim 1, characterized in that: The different scale fusion includes: Apply pyramid pooling operations of different sizes to the input features to obtain feature maps of corresponding sizes; 1×1 convolution is used to reduce the dimension of feature maps of each size; The feature map after dimensionality reduction is upsampled using bilinear interpolation to make it the same size as the input feature; The upsampled feature map is placed on the input feature map from bottom to top, and the feature points on each feature channel are averaged through global average pooling. The weight coefficient of the feature channel is obtained by multiplying the average result with the corresponding channel matrix; Based on the weight coefficient, 1×1 convolution is used to accumulate and sum the input features to obtain the fused feature map: Where y i is the eigenvalue of the i-th feature channel on the fusion feature map, N is the number of feature channels, w i is the weight coefficient of the i-th feature channel, x i is the eigenvalue of the i-th feature channel on the input feature.

6. A method for identifying and detecting station building defects according to claim 5, characterized in that: The Softmax classifier is sampled on the fusion feature map for classification prediction to obtain the defect prediction result.

7. A station building defect identification and detection system, characterized in that: The system comprises: The first data acquisition module is used to acquire online video data of the station building; The first image extraction module is used to convert the video stream of the online video data into frames through OpenCV to obtain online frame images; A first preprocessing module, used for preprocessing the online frame image; The defect recognition module is used to input the pre-processed online frame image into the trained defect recognition model to obtain the defect recognition result; Among them, the construction of the defect recognition model includes: integrating the BAM attention mechanism and PSPNet pyramid pooling into the ResNet-34 network structure; integrating the BAM attention mechanism into the ResNet-34 network structure for feature extraction of input images; integrating PSPNet pyramid pooling into the ResNet-34 network structure for fusing input features at different scales.

8. A station building defect identification and detection system according to claim 7, characterized in that: Also includes: The second data acquisition module is used to acquire offline video data of the station building; The second image extraction module is used to convert the video stream of the offline video data into frames through OpenCV to obtain offline frame images; The second preprocessing module is used to mark defects and preprocess the offline frame images; The dataset construction module is used to generate a dataset based on defect annotations and pre-processed offline frame images, and divide the dataset into a training set and a validation set according to a preset ratio; The model training module is used to train the defect recognition model through the training set, optimize the model parameters, and verify the effect of the defect recognition model corresponding to each model parameter through the verification set, and select the defect recognition model with the best effect as the training result.

9. A station building defect identification and detection system, characterized in that: including processors and storage media; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the steps of the method according to any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images