Transformer substation inspection method and system based on image recognition

Abstract

The invention discloses a transformer substation inspection method and system based on image recognition, and relates to the technical field of image processing. Performing 1 * 1 convolution operation on the inspection image, and substituting the inspection image into an enhancement module to obtain a first inspection enhancement feature; downsampling the first inspection enhancement feature, and substituting the first inspection enhancement feature into an enhancement module to obtain a second inspection enhancement feature; substituting the first inspection enhancement feature and the second inspection enhancement feature into a fusion module to obtain an inspection fusion feature; fusing the inspection fusion feature and the inspection image, and substituting into an object recognition model to obtain a plurality of recognition objects; and carrying out behavior identification analysis on each identification object, and if the identification analysis is abnormal, giving an alarm prompt. The inspection images are enhanced and denoised through the enhancement module and then combined, the problem that the inspection images are poor in quality under the complex environment of the transformer substation is effectively solved, the abnormal state of equipment is accurately judged through behavior recognition analysis, an alarm is given in time, and the inspection efficiency of the transformer substation is improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing, and particularly relates to a substation inspection method and system based on image recognition. BACKGROUND

[0002] With the continuous development and upgrading of network communication technology and video monitoring technology, the conditions for carrying out remote equipment inspection through a high-definition video system are becoming increasingly mature, which will also become the development trend of substation operation and maintenance inspection. The traditional substation video system mainly aims at the overview and video review of security, fire protection, and the overall appearance of equipment, and the camera deployment position is mainly located at the passageway, entrance, and other parts. The corresponding point optimization and design for online intelligent inspection have not been carried out. The substations under the jurisdiction of the power supply company are all unattended substations, and the operation and maintenance personnel need to regularly monitor the operation status of the electrical equipment of the substation in order to discover abnormal conditions in a short time and timely handle accidents. Moreover, the substations under jurisdiction are built at the edge of the city or in remote places with few people, which brings heavy workload to the operation and maintenance personnel. The regular manual inspection operation and maintenance mode has problems that need to be solved urgently. Therefore, the traditional manual operation and maintenance mode cannot meet the development needs, and it is imperative to improve the daily operation and maintenance mode.

[0003] Patent CN116824481A discloses a substation inspection method and system based on image recognition. First, the state detection image and the reference image of the secondary system of the substation are respectively subjected to image blocking processing to obtain a sequence of state detection image blocks and a sequence of reference image blocks. Then, the sequence of state detection image blocks and the sequence of reference image blocks are respectively subjected to a twin detection model to obtain a context state detection semantic feature vector and a context reference image semantic feature vector. Then, a transition matrix between the context state detection semantic feature vector and the context reference image semantic feature vector is calculated as a classification feature matrix. Finally, after planning and strengthening the classification feature matrix, a classifier can obtain a classification result for representing whether the operation state of the secondary system of the substation is normal.

[0004] However, in the real scene, when promoting the intelligent inspection of substations based on image recognition technology, the image acquisition quality becomes a key bottleneck restricting the recognition accuracy and reliability of the entire system. Due to the complex and diverse on-site environment of substations, affected by multiple factors such as natural conditions, equipment operation environment, and performance of the acquisition equipment itself, the actual inspection images collected often have problems such as insufficient clarity, uneven illumination, blurred picture, missing details, and noise interference. These images with poor quality will directly affect the effect of subsequent feature extraction, target detection, and abnormality recognition algorithms, making it difficult for the intelligent inspection system to accurately judge the equipment state, and even leading to misjudgment and missed judgment, resulting in low efficiency of substation inspection. SUMMARY

[0005] The present application aims to solve the problem of low efficiency of substation inspection caused by image quality problems, and proposes a substation inspection method and system based on image recognition.

[0006] In the first aspect of the present application, a substation inspection method based on image recognition is first proposed, which comprises:

[0007] Collecting an inspection image, performing a 1x1 convolution operation on the inspection image, and then inputting the first inspection enhanced feature into the enhancement module to obtain the first inspection enhanced feature;

[0008] Down-sampling the first inspection enhanced feature and inputting it into the enhancement module to obtain the second inspection enhanced feature;

[0009] Inputting the first inspection enhanced feature and the second inspection enhanced feature into the fusion module to obtain the inspection fusion feature;

[0010] Fusing the inspection fusion feature and the inspection image to obtain an inspection enhanced image;

[0011] Inputting the inspection enhanced image into the object recognition model to obtain a plurality of recognized objects;

[0012] Performing behavior recognition analysis on each recognized object; the recognition analysis includes existence of abnormality and no abnormality;

[0013] If the recognition analysis is abnormal, an alarm prompt is issued.

[0014] Optionally, the working principle of the enhancement module comprises:

[0015] Obtaining an input feature, performing point-by-point convolution on the normalized output of the input feature to obtain a first convolution feature;

[0016] Performing 3x3 convolution operation and 5x5 convolution operation on the first convolution feature respectively, and then performing feature fusion to obtain a first convolution fusion feature;

[0017] Element-wise addition of the first convolution fusion feature and the input feature to obtain a fusion convolution feature;

[0018] Inputting the fusion convolution feature into the channel attention to obtain a channel attention feature, and performing element-wise addition of the channel attention feature and the fusion convolution feature to obtain the output feature of the enhancement module.

[0019] Optionally, inputting the first inspection enhanced feature and the second inspection enhanced feature into the fusion module to obtain the inspection fusion feature comprises:

[0020] The first inspection enhanced feature is substituted into a multilayer perception machine to obtain a first inspection perception feature;

[0021] The first inspection perception feature and the second inspection enhanced feature are element-wise added and then globally average-pooled to obtain a pooling feature;

[0022] The pooling feature is substituted into a multilayer perception machine, and segmentation is performed to obtain a first weight and a second weight;

[0023] The first inspection perception feature and the second inspection enhanced feature are fused according to the first weight and the second weight to obtain an inspection fusion feature.

[0024] Optionally, the enhanced image is substituted into an object recognition model to obtain a plurality of recognition objects, including:

[0025] Multi-scale feature extraction is performed on the enhanced image to obtain a scale feature set; the scale feature set includes high-scale features, medium-scale features, and low-scale features;

[0026] The scale feature set is substituted into a neck network in a YOLOv8 model to obtain first output features, second output features, and third output features;

[0027] The first output features and the second output features are fused to obtain first fused output features; and the second output features and the third output features are fused to obtain second fused output features;

[0028] Spatial attention is applied to the first fused output features to obtain fused spatial attention features, and channel attention is applied to the second fused output features to obtain fused channel attention features;

[0029] The fused spatial attention features and the fused channel attention features are fused and then substituted into a fully connected layer to obtain a plurality of recognition objects.

[0030] Optionally, behavior recognition analysis is performed on each recognition object, including:

[0031] For each recognition object, appearance feature extraction and behavior action extraction are performed on the object to obtain appearance features and action features;

[0032] Appearance feature time series and action feature time series of the recognition object within a preset time period are determined, and the appearance feature time series and the action feature time series are merged to obtain a comprehensive behavior feature sequence;

[0033] The comprehensive behavior feature sequence is substituted into a preset behavior feature recognition model to obtain a behavior recognition analysis result.

[0034] In the second aspect of the present application, a substation inspection system based on image recognition is provided, comprising:

[0035] A first inspection enhanced feature generation module is configured to collect an inspection image, perform a 1x1 convolution operation on the inspection image, and input the enhanced module to obtain a first inspection enhanced feature;

[0036] A second inspection enhanced feature generation module is configured to down-sample the first inspection enhanced feature and input the enhanced module to obtain a second inspection enhanced feature;

[0037] An inspection fusion feature generation module is configured to input the first inspection enhanced feature and the second inspection enhanced feature into a fusion module to obtain an inspection fusion feature;

[0038] An inspection enhanced image generation module is configured to fuse the inspection fusion feature and the inspection image to obtain an inspection enhanced image;

[0039] An object determination module is configured to input the inspection enhanced image into an object recognition model to obtain a plurality of recognition objects;

[0040] An object behavior recognition analysis module is configured to perform behavior recognition analysis on each recognition object; the recognition analysis includes existence of abnormality and no abnormality;

[0041] An alarm module is configured to issue an alarm prompt if the recognition analysis is existence of abnormality.

[0042] Optionally, the working principle of the enhanced module includes:

[0043] A first inspection enhanced feature generation module is configured to obtain an input feature, normalize the input feature, and perform point-by-point convolution to obtain a first convolution feature;

[0044] The first inspection enhanced feature generation module is configured to perform 3x3 convolution operation and 5x5 convolution operation on the first convolution feature respectively, and then perform feature fusion to obtain a first convolution fusion feature;

[0045] The first inspection enhanced feature generation module is configured to perform element-by-element addition on the first convolution fusion feature and the input feature to obtain a fusion convolution feature;

[0046] The first inspection enhanced feature generation module is configured to input the fusion convolution feature into a channel attention to obtain a channel attention feature, and perform element-by-element addition on the channel attention feature and the fusion convolution feature to obtain an output feature of the enhanced module.

[0047] Optionally, the inspection fusion feature generation module includes:

[0048] The first inspection perception feature generation module is used to substitute the first inspection enhancement feature into the multilayer perceptron to obtain the first inspection perception feature.

[0049] The pooling feature generation module is used to add the first inspection perception feature and the second inspection enhancement feature element by element, and then perform global average pooling to obtain the pooled feature.

[0050] The weight generation module is used to input the pooled features into a multilayer perceptron and then segment them to obtain the first weight and the second weight.

[0051] The inspection fusion feature generation module is used to fuse the first inspection perception feature and the second inspection enhancement feature according to the first weight and the second weight to obtain the inspection fusion feature.

[0052] Optionally, the object identification module includes:

[0053] A scale feature set generation module is used to extract multi-scale features from the enhanced image to obtain a scale feature set; the scale feature set includes high-scale features, medium-scale features, and low-scale features;

[0054] The output feature generation module is used to substitute the scale feature set into the neck network of the YOLOv8 model to obtain the first output feature, the second output feature and the third output feature;

[0055] An output feature fusion module is used to fuse the first output feature and the second output feature to obtain a first fused output feature; and to fuse the second output feature and the third output feature to obtain a second fused output feature.

[0056] The attention feature extraction module is used to apply spatial attention to the first fused output feature to obtain fused spatial attention features, and to apply channel attention to the second fused output feature to obtain fused channel attention features;

[0057] The object generation module is used to perform feature fusion on the fused spatial attention features and the fused channel attention features, and then input them into the fully connected layer to obtain multiple objects for recognition.

[0058] Optionally, the object behavior recognition and analysis module includes:

[0059] The appearance and action feature extraction module is used to extract appearance features and action features for each identified object.

[0060] The comprehensive behavioral feature sequence generation module is used to determine the appearance feature time sequence and action feature time sequence of the identified object within a preset time period, and merge the appearance feature time sequence and action feature time sequence to obtain a comprehensive behavioral feature sequence;

[0061] The behavior recognition analysis result determination module is used to substitute the comprehensive behavior feature sequence into a preset behavior feature recognition model to obtain the behavior recognition analysis result.

[0062] The beneficial effects of this invention are:

[0063] This invention proposes a substation inspection method based on image recognition. By enhancing and denoising the inspection images using an enhancement module and then combining them with the inspection images, the method effectively solves the problem of poor image quality in complex substation environments. This provides high-quality input for subsequent object recognition models, significantly improving the accuracy and reliability of target recognition. Furthermore, behavior recognition analysis accurately determines abnormal equipment conditions and issues timely alarms, thereby improving the efficiency of substation inspections. Attached Figure Description

[0064] The invention will now be further described with reference to the accompanying drawings.

[0065] Figure 1 A flowchart illustrating a substation inspection method based on image recognition, provided as an embodiment of the present invention;

[0066] Figure 2 A logic flowchart of an enhancement module provided in an embodiment of the present invention;

[0067] Figure 3 This is a framework diagram of a substation inspection system based on image recognition, provided as an embodiment of the present invention. Detailed Implementation

[0068] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0069] Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0070] This invention provides a substation inspection method based on image recognition. See also... Figure 1 , Figure 1 A flowchart illustrating a substation inspection method based on image recognition, provided in an embodiment of the present invention. The method includes the following steps:

[0071] S101, Collect inspection images, perform a 1×1 convolution operation on the inspection images, and then input them into the enhancement module to obtain the first inspection enhancement feature;

[0072] S102, the first inspection enhancement feature is downsampled and then input into the enhancement module to obtain the second inspection enhancement feature;

[0073] S103, Substitute the first inspection enhancement feature and the second inspection enhancement feature into the fusion module to obtain the inspection fusion feature;

[0074] S104, The inspection fusion features and the inspection image are fused to obtain the inspection enhancement image;

[0075] S105, Substitute the enhanced inspection image into the object recognition model to obtain multiple recognition objects;

[0076] S106, Perform behavioral recognition analysis on each identified object; the recognition analysis includes the presence of anomalies and the absence of anomalies;

[0077] S107 If the analysis indicates an anomaly, an alarm will be issued.

[0078] This invention provides a substation inspection method based on image recognition. By using 1×1 convolution and enhancement modules to precisely optimize the clarity of inspection images, suppress noise, and compensate for lost details, combined with dual-scale enhanced feature fusion and original image fusion techniques, it effectively solves the problem of poor image quality in complex substation environments. This provides high-quality input for subsequent object recognition models, significantly improving the accuracy and reliability of target recognition. Furthermore, behavior recognition analysis accurately determines abnormal equipment states and issues timely alarms. This method overcomes the bottleneck of image acquisition quality constraints from the source, reduces misjudgments and omissions in intelligent inspection systems, and significantly improves the efficiency and intelligence level of substation inspections.

[0079] In one implementation, 1×1 convolution is used to quickly compress redundant dimensions of the image and reduce computational complexity. Combined with an enhancement module, it is used to specifically optimize problems such as insufficient image clarity and noise interference, and to initially extract effective visual features, laying a high-quality foundation for subsequent processing. Downsampling focuses on key areas of the image and reduces interference from invalid information. Then, the enhancement module strengthens deep semantic features, making up for the loss of details caused by uneven lighting and image blur, forming multi-scale enhanced features that complement the first feature and improve the comprehensiveness of feature expression.

[0080] In one implementation, if the analysis indicates no anomalies, no action is taken.

[0081] In one implementation, dual-scale enhancement features are fused, integrating shallow detail information and deep semantic information to avoid the limitations of single-scale features, making the features more representative and providing more comprehensive feature support for subsequent image enhancement. By combining the fused features with the original inspection image, the basic information of the original image is preserved while incorporating the advantages of the optimized features, thoroughly improving image quality defects and outputting an enhanced image with high clarity, complete details, and noise suppression, providing high-quality input for recognition tasks.

[0082] In one implementation, high-quality inspection enhancement images significantly reduce the difficulty of feature extraction in the recognition model, improve the accuracy of target detection, ensure that substation equipment-related objects are fully and accurately identified, and reduce target omissions due to image quality issues; anomaly determination is carried out for each identified object to clearly distinguish between the presence and absence of anomalies, thereby achieving accurate judgment of equipment status and avoiding judgment bias caused by general recognition.

[0083] In one embodiment, see Figure 2 , Figure 2 A logic flowchart of an enhancement module provided in an embodiment of the present invention is shown. The working principle of the enhancement module includes:

[0084] The input features are obtained, and after normalization, they are convolved point by point to obtain the first convolutional features.

[0085] The first convolutional feature is obtained by performing 3×3 convolution and 5×5 convolution operations on the first convolutional feature, and then fusing the features.

[0086] The first convolutional fusion feature and the input feature are added element-wise to obtain the fused convolutional feature;

[0087] Substituting the fused convolutional features into the channel attention features yields the channel attention features. The channel attention features and the fused convolutional features are then added element-wise to obtain the output features of the enhancement module.

[0088] In one implementation, normalization is first used to eliminate scale differences in input features and avoid interference from extreme values; then, pointwise convolution is used to compress redundant channels and extract key information, laying a stable and efficient feature foundation for subsequent convolution operations.

[0089] In one implementation, normalization is achieved through LayerNormalization. Figure 2 A circle with a plus sign inside indicates element-wise addition and feature fusion, both of which involve adding features and calculating the average.

[0090] In one implementation, dual-scale convolutions capture local details of different ranges (3×3 focuses on fine features, while 5×5 covers a wider area of ​​information). The fusion of these features enriches the multi-scale expressive power of the features, effectively compensating for feature deficiencies caused by uneven illumination and loss of details. Residual connections preserve the basic information of the original input features, avoiding feature loss caused by convolution operations and enhancing the module's compatibility with the original features and the integrity of feature transfer.

[0091] In one implementation, the importance weights of the channel dimension are focused, key feature channels (such as device edges, abnormal regions, etc.) are focused, and irrelevant noise channels are suppressed to further improve the discriminativeness of the features. Finally, the attention-weighted features are combined with the fused convolutional features by adding elements one by one to strengthen effective information, weaken interference, and output more accurate and robust enhanced features, providing strong support for subsequent image quality optimization and recognition tasks.

[0092] In one embodiment, substituting the first inspection enhancement feature and the second inspection enhancement feature into the fusion module to obtain the inspection fusion feature includes:

[0093] Substitute the first inspection enhancement feature into the multilayer perceptron to obtain the first inspection perception feature.

[0094] After adding the first inspection perception features and the second inspection enhancement features element by element, global average pooling is then performed to obtain the pooled features.

[0095] After substituting the pooled features into the multilayer perceptron, the first weight and the second weight are obtained by segmentation.

[0096] The first inspection perception feature and the second inspection enhancement feature are fused according to the first weight and the second weight to obtain the inspection fusion feature.

[0097] In one implementation, the multilayer perceptron deeply mines the high-order semantic information of the first inspection enhancement features, strengthens the expressive ability of shallow detailed features, and makes the features that originally focused on basic optimization more discriminative, providing a high-quality adaptation foundation for fusion with deep features; element-wise addition initially integrates the complementary information of dual-scale features, while global average pooling extracts the global statistical information of features, weakens local redundant interference, condenses the core feature value, and provides a comprehensive and accurate feature basis for subsequent weight calculation.

[0098] In one implementation, the inspection fusion feature is obtained by fusing the first inspection perception feature and the second inspection enhancement feature according to the first weight and the second weight. Specifically, the first weight is multiplied by the first inspection perception feature to obtain the first fusion feature; the second weight is multiplied by the second inspection enhancement feature and then upsampled to obtain the second fusion feature; the first fusion feature and the second fusion feature are averaged to obtain the inspection fusion feature.

[0099] In one implementation, the importance ratio of dual-scale features is dynamically learned through a multilayer perceptron to generate targeted weights, avoiding the limitations of traditional fixed-weight fusion and making feature fusion more in line with the actual quality optimization needs of substation inspection images. The contribution of the two features is accurately allocated according to the dynamic weights, which not only fully preserves the detailed advantages of the first feature, but also deeply integrates the deep semantic value of the second feature, achieving comprehensive feature coverage of both details and the overall picture, greatly improving the integrity and representativeness of the fused features, and laying a solid core support for subsequent image enhancement and recognition tasks.

[0100] In one embodiment, substituting the enhanced image into an object recognition model to obtain multiple recognized objects includes:

[0101] Multi-scale feature extraction is performed on the enhanced image to obtain a scale feature set; the scale feature set includes high-scale features, medium-scale features, and low-scale features;

[0102] Substituting the scale feature set into the neck network of the YOLOv8 model yields the first output feature, the second output feature, and the third output feature.

[0103] The first output feature is obtained by fusing the first output feature and the second output feature; the second output feature and the third output feature are obtained by fusing the second output feature and the third output feature.

[0104] Spatial attention is applied to the first fused output feature to obtain fused spatial attention features, and channel attention is applied to the second fused output feature to obtain fused channel attention features;

[0105] After fusing spatial attention features and channel attention features, the resulting data is fed into a fully connected layer to obtain multiple recognition objects.

[0106] In one implementation, the system covers equipment targets and detailed information of different sizes. High-scale features capture the overall outline of the equipment, medium-scale features focus on core components, and low-scale features restore fine structures, avoiding target omissions due to a single scale and adapting to the identification needs of various equipment in substations. By fusing adjacent scale features in pairs, the system achieves complementary integration of global, local, detailed, and semantic aspects, making up for the information shortcomings of single-scale features, enhancing the feature coverage of equipment of different sizes, and reducing identification errors caused by differences in equipment size.

[0107] In one implementation, the scale features are set as follows: low-scale features are used as the output features of the second C2f module in the backbone network of the YOLOv8 model; medium-scale features are used as the output features of the third C2f module in the backbone network of the YOLOv8 model; high-scale features are used as the output features of the SPPF module in the backbone network of the YOLOv8 model; the first output feature is the output feature corresponding to the scale of the low-scale features; the second output feature is the output feature corresponding to the scale of the medium-scale features; and the third output feature is the output feature corresponding to the scale of the high-scale features.

[0108] In one implementation, spatial attention focuses on the key spatial region where the device is located, suppressing background redundancy and environmental interference, and increasing the feature weight of the target region; channel attention strengthens the feature channels related to device recognition, weakens the influence of invalid channels, and allows the model to more accurately capture the core features of the device.

[0109] In one embodiment, behavioral recognition analysis for each identified object includes:

[0110] For each identified object, appearance features and behavior / action features are extracted to obtain appearance features and action features.

[0111] Determine the time series of appearance features and action features of the object to be identified within a preset time period, and merge the time series of appearance features and action features to obtain a comprehensive behavioral feature sequence;

[0112] Substituting the comprehensive behavioral feature sequence into the preset behavioral feature recognition model yields the behavioral recognition analysis results.

[0113] In one implementation, the appearance attributes (such as device shape, color, damage marks, etc.) and action performance (such as abnormal displacement, loose parts, etc.) of the object are captured and identified simultaneously. This avoids the one-sidedness of a single feature dimension and provides comprehensive feature support for behavior analysis. By integrating the temporal changes of features through a preset time period (determined by technical personnel), the dynamic evolution process of the object's state is restored, rather than judging single frame information in isolation. At the same time, appearance and action temporal features are merged to form a comprehensive behavioral feature sequence that combines static attributes and dynamic changes, thereby improving the completeness and timeliness of the features.

[0114] In one implementation, a pre-set behavioral feature recognition model (which can be any one of Bi-LSTM, TimeSformer, 2s-AGCN, etc.) is used to accurately match the comprehensive behavioral feature sequence with normal and abnormal behavioral patterns. This avoids the subjectivity and lag of manual judgment, and realizes automated and standardized judgment of equipment status, ensuring that abnormal situations are accurately identified and that no abnormal status is stably confirmed.

[0115] Based on the same inventive concept, this invention also provides an image recognition-based substation inspection system. See also Figure 3 , Figure 3 A framework diagram of an image recognition-based substation inspection system provided in this embodiment of the invention includes:

[0116] The first inspection enhancement feature generation module is used to collect inspection images, perform a 1×1 convolution operation on the inspection images, and then input them into the enhancement module to obtain the first inspection enhancement feature.

[0117] The second inspection enhancement feature generation module is used to downsample the first inspection enhancement feature and input it into the enhancement module to obtain the second inspection enhancement feature.

[0118] The inspection fusion feature generation module is used to substitute the first inspection enhancement feature and the second inspection enhancement feature into the fusion module to obtain the inspection fusion feature.

[0119] The inspection enhancement image generation module is used to fuse inspection fusion features and inspection images to obtain inspection enhancement images;

[0120] The object identification module is used to input the enhanced inspection image into the object recognition model to obtain multiple objects to be identified.

[0121] The object behavior recognition and analysis module is used to perform behavior recognition and analysis on each object; the recognition and analysis includes the presence of anomalies and the absence of anomalies;

[0122] The alarm module is used to issue an alarm if the analysis identifies an anomaly.

[0123] The substation inspection system based on image recognition provided in this invention uses 1×1 convolution and enhancement modules to precisely optimize the clarity of inspection images, suppress noise, and compensate for lost details. Combined with dual-scale enhanced feature fusion and original image fusion technology, it effectively solves the problem of poor image quality in complex substation environments, providing high-quality input for subsequent object recognition models, significantly improving the accuracy and reliability of target recognition. Then, through behavior recognition analysis, it accurately determines abnormal equipment status and issues timely alarms, breaking through the bottleneck of image acquisition quality constraints from the source, reducing misjudgments and omissions in intelligent inspection systems, and significantly improving the efficiency and intelligence level of substation inspections.

[0124] In one embodiment, the enhancement module operates as follows:

[0125] The first inspection enhancement feature generation module is used to obtain input features, normalize the input features, and then perform point-by-point convolution to obtain the first convolution feature;

[0126] The first inspection enhancement feature generation module is used to perform 3×3 convolution operation and 5×5 convolution operation on the first convolution feature respectively, and then perform feature fusion to obtain the first convolution fused feature;

[0127] The first inspection enhancement feature generation module is used to add the first convolutional fusion feature and the input feature element by element to obtain the fusion convolutional feature;

[0128] The first inspection enhancement feature generation module is used to substitute the fused convolutional features into the channel attention to obtain the channel attention features, and to add the channel attention features and the fused convolutional features element by element to obtain the output features of the enhancement module.

[0129] In one embodiment, the inspection fusion feature generation module includes:

[0130] The first inspection perception feature generation module is used to input the first inspection enhancement feature into the multilayer perceptron to obtain the first inspection perception feature.

[0131] The pooling feature generation module is used to add the first inspection perception feature and the second inspection enhancement feature element by element, and then perform global average pooling to obtain the pooled feature.

[0132] The weight generation module is used to input pooled features into a multilayer perceptron and then segment them to obtain the first weight and the second weight.

[0133] The inspection fusion feature generation module is used to fuse the first inspection perception feature and the second inspection enhancement feature according to the first weight and the second weight to obtain the inspection fusion feature.

[0134] In one embodiment, the object identification module includes:

[0135] The scale feature set generation module is used to extract multi-scale features from the enhanced image to obtain a scale feature set; the scale feature set includes high-scale features, medium-scale features, and low-scale features;

[0136] The output feature generation module is used to substitute the scale feature set into the neck network of the YOLOv8 model to obtain the first output feature, the second output feature and the third output feature;

[0137] The output feature fusion module is used to fuse the first output feature and the second output feature to obtain the first fused output feature; and to fuse the second output feature and the third output feature to obtain the second fused output feature.

[0138] The attention feature extraction module is used to apply spatial attention to the first fused output feature to obtain fused spatial attention features, and to apply channel attention to the second fused output feature to obtain fused channel attention features.

[0139] The object generation module is used to perform feature fusion on the fused spatial attention features and fused channel attention features, and then input them into the fully connected layer to obtain multiple objects for recognition.

[0140] In one embodiment, the object behavior recognition and analysis module includes:

[0141] The appearance and action feature extraction module is used to extract appearance features and action features for each identified object.

[0142] The comprehensive behavioral feature sequence generation module is used to determine the appearance feature time sequence and action feature time sequence of the identified object within a preset time period, and merge the appearance feature time sequence and action feature time sequence to obtain the comprehensive behavioral feature sequence;

[0143] The behavior recognition analysis result determination module is used to input the comprehensive behavior feature sequence into the preset behavior feature recognition model to obtain the behavior recognition analysis result.

[0144] The foregoing has provided a detailed description of one embodiment of the present invention, but this description is merely a preferred embodiment and should not be construed as limiting the scope of the invention. All equivalent variations and modifications made within the scope of the claims of this invention should still fall within the patent coverage of this invention.

Claims

1. A substation inspection method based on image recognition, characterized in that, The method comprises: Collecting a patrol image, and inputting the patrol image into an enhancement module after 1*1 convolution operation to obtain a first patrol enhancement feature; Inputting the first patrol enhancement feature into the enhancement module after down-sampling to obtain a second patrol enhancement feature; Inputting the first patrol enhancement feature and the second patrol enhancement feature into a fusion module to obtain a patrol fusion feature; Fusing the patrol fusion feature and the patrol image to obtain a patrol enhancement image; Inputting the patrol enhancement image into an object recognition model to obtain a plurality of recognized objects; Performing behavior recognition analysis on each recognized object; the recognition analysis comprises existence of an abnormality and no abnormality; If the recognition analysis is existence of an abnormality, an alarm prompt is sent.

2. The image recognition based substation inspection method of claim 1, wherein, The working principle of the enhancement module comprises: Obtaining an input feature, performing point-by-point convolution on the input feature after normalization to obtain a first convolution feature; Performing 3*3 convolution operation and 5*5 convolution operation on the first convolution feature respectively, and then performing feature fusion to obtain a first convolution fusion feature; Performing element-by-element addition on the first convolution fusion feature and the input feature to obtain a fusion convolution feature; Inputting the fusion convolution feature into channel attention to obtain a channel attention feature, and performing element-by-element addition on the channel attention feature and the fusion convolution feature to obtain an output feature of the enhancement module.

3. The image recognition based substation inspection method of claim 1, wherein, Inputting the first patrol enhancement feature and the second patrol enhancement feature into a fusion module to obtain a patrol fusion feature comprises: Inputting the first patrol enhancement feature into a multi-layer perception machine to obtain a first patrol perception feature; Performing element-by-element addition on the first patrol perception feature and the second patrol enhancement feature, and then performing global average pooling processing to obtain a pooling feature; Inputting the pooling feature into a multi-layer perception machine to perform segmentation to obtain a first weight and a second weight; Fusing the first patrol perception feature and the second patrol enhancement feature according to the first weight and the second weight to obtain a patrol fusion feature.

4. The image recognition based substation inspection method of claim 1, wherein, Inputting the enhancement image into an object recognition model to obtain a plurality of recognized objects comprises: Performing multi-scale feature extraction on the enhancement image to obtain a scale feature set; the scale feature set comprises high-scale features, medium-scale features and low-scale features; Inputting the scale feature set into a neck network in a YOLOv8 model to obtain a first output feature, a second output feature and a third output feature; Fusing the first output feature and the second output feature to obtain a first fusion output feature; and fusing the second output feature and the third output feature to obtain a second fusion output feature; Applying spatial attention to the first fusion output feature to obtain a fusion spatial attention feature, and applying channel attention to the second fusion output feature to obtain a fusion channel attention feature; Performing feature fusion on the fusion spatial attention feature and the fusion channel attention feature, and then inputting the fusion spatial attention feature and the fusion channel attention feature into a fully connected layer to obtain a plurality of recognized objects.

5. The image recognition based substation inspection method of claim 1, wherein, Performing behavior recognition analysis on each recognized object comprises: For each recognized object, appearance feature extraction and behavior action extraction are performed on the object to obtain appearance features and action features; Determine appearance feature time series and action feature time series of the identified object in a preset time period, and combine the appearance feature time series and the action feature time series to obtain a comprehensive behavior feature sequence; Substitute the comprehensive behavior feature sequence into a preset behavior feature recognition model to obtain a behavior recognition analysis result.

6. An image recognition based substation inspection system, characterized in that, The system comprises: A first inspection enhanced feature generation module is configured to collect an inspection image, perform a 1×1 convolution operation on the inspection image, and substitute the inspection image into an enhanced module to obtain a first inspection enhanced feature; A second inspection enhanced feature generation module is configured to down-sample the first inspection enhanced feature and substitute the down-sampled first inspection enhanced feature into the enhanced module to obtain a second inspection enhanced feature; An inspection fusion feature generation module is configured to substitute the first inspection enhanced feature and the second inspection enhanced feature into a fusion module to obtain an inspection fusion feature; An inspection enhanced image generation module is configured to fuse the inspection fusion feature and the inspection image to obtain an inspection enhanced image; An identified object determination module is configured to substitute the inspection enhanced image into an object recognition model to obtain a plurality of identified objects; An object behavior recognition analysis module is configured to perform behavior recognition analysis on each identified object; the recognition analysis includes existence of an abnormality and absence of an abnormality; An alarm module is configured to issue an alarm prompt if the recognition analysis indicates existence of an abnormality.

7. The image recognition based substation inspection system of claim 6, wherein, The working principle of the enhanced module comprises: A first inspection enhanced feature generation module is configured to obtain an input feature, normalize the input feature, and perform point-by-point convolution to obtain a first convolution feature; A first inspection enhanced feature generation module is configured to perform a 3×3 convolution operation and a 5×5 convolution operation on the first convolution feature, respectively, and fuse the first convolution features to obtain a first convolution fusion feature; A first inspection enhanced feature generation module is configured to add the first convolution fusion feature and the input feature element by element to obtain a fusion convolution feature; A first inspection enhanced feature generation module is configured to substitute the fusion convolution feature into a channel attention to obtain a channel attention feature, add the channel attention feature and the fusion convolution feature element by element to obtain an output feature of the enhanced module.

8. The image recognition based substation inspection system of claim 6, wherein, The inspection fusion feature generation module comprises: A first inspection perception feature generation module is configured to substitute the first inspection enhanced feature into a multi-layer perception machine to obtain a first inspection perception feature; A pooling feature generation module is configured to add the first inspection perception feature and the second inspection enhanced feature element by element, and then perform global average pooling processing to obtain a pooling feature; A weight generation module is configured to substitute the pooling feature into a multi-layer perception machine, and then perform segmentation to obtain a first weight and a second weight; An inspection fusion feature generation module is configured to fuse the first inspection perception feature and the second inspection enhanced feature according to the first weight and the second weight to obtain an inspection fusion feature.

9. The image recognition based substation inspection system of claim 6, wherein, The identified object determination module comprises: A scale feature set generation module is configured to extract multi-scale features from the enhanced image to obtain a scale feature set; the scale feature set comprises high-scale features, medium-scale features, and low-scale features; The output feature generation module is configured to input the scale feature set into a neck network in the YOLOv8 model to obtain first output features, second output features, and third output features. The output feature fusion module is configured to fuse the first output features and the second output features to obtain first fused output features, and fuse the second output features and the third output features to obtain second fused output features. The attention feature extraction module is configured to apply spatial attention to the first fused output features to obtain fused spatial attention features, and apply channel attention to the second fused output features to obtain fused channel attention features. The recognition object generation module is configured to input the fused spatial attention features and the fused channel attention features into a fully connected layer after feature fusion to obtain a plurality of recognition objects.

10. The image recognition based substation inspection system of claim 6, wherein, The object behavior recognition analysis module includes: The appearance and action feature extraction module is configured to, for each recognition object, extract appearance features and action features of the object to obtain appearance features and action features. The comprehensive behavior feature sequence generation module is configured to determine an appearance feature time sequence and an action feature time sequence of the recognition object within a preset time period, and combine the appearance feature time sequence and the action feature time sequence to obtain a comprehensive behavior feature sequence. The behavior recognition analysis result determination module is configured to input the comprehensive behavior feature sequence into a preset behavior feature recognition model to obtain a behavior recognition analysis result.

Images