A method and terminal for detecting small targets in electrical equipment

Abstract

This invention specifically relates to a method and terminal for detecting small targets in electrical equipment. The detection method includes: S100, acquiring an original image; S200, inputting the original image into a target detection model; S300, the target detection model processing the original image to output a labeled image; the processing includes: preprocessing, localization processing, bounding box selection, and classification. This invention addresses the technical problems of unsatisfactory small target recognition performance, low accuracy, and slow processing speed in existing technologies, and possesses advantages such as reliable recognition, high accuracy, fast processing, and strong anti-interference capability.

Description

Technical Field

[0001] This invention relates to the field of target recognition technology, and specifically to a method and terminal for detecting small targets in electrical equipment. Background Technology

[0002] With the deployment of power systems, the inspection and maintenance of electrical equipment in the power Internet of Things (IoT) is becoming increasingly complex. Artificial intelligence (AI) provides a good solution for the autonomous analysis and maintenance of the power IoT. AI-powered inspection equipment generates a large number of images that are difficult to process manually in a timely manner. This necessitates automated processing using computer vision technology, leading to the emergence of increasingly sophisticated object detection and instance segmentation techniques for processing infrared images to achieve automated power system inspections. However, infrared image processing suffers from technical challenges such as low resolution, high noise, and blurred or missing edges. Therefore, improving the reliability of deep learning models and reducing manual intervention in intelligent substation operations have become key research technologies in the field of power detection.

[0003] In existing technologies, Convolutional Neural Networks (CNNs) can automatically learn and extract more features from images, thereby improving image recognition capabilities, reducing processing time, decreasing human resource consumption, and increasing accuracy. However, in many scenarios, images are mainly composed of background elements such as forests, rivers, fields, and mountains, which typically contain little useful information. Therefore, object detection algorithms that can effectively identify objects in such images are crucial.

[0004] Among these algorithms, widely used methods include two-stage methods such as Region-based CNN (R-CNN), Fast Region-based Convolutional Network (Fast R-CNN), and Faster Region-based Convolutional Network (Faster R-CNN); and single-stage methods such as Single Shot MultiBoxDetector (SSD) and You Only Look Once (YOLO). Faster R-CNN stands out for its speed and accuracy and is often used in conjunction with Visual Geometry Group (VGG) feature extraction networks.

[0005] However, as a feature extractor for detecting small objects, VGG-16's target extraction performance is not ideal, and the recognition of low-level features of small objects remains crucial. Furthermore, VGG-16 relies heavily on high-level features extracted from deep convolutional layers, which easily leads to insufficient utilization of shallow features, thus affecting extraction performance. In addition, the frequent downsampling operations in the VGG-16 network can lose information about small objects during feature extraction, resulting in unsatisfactory performance in detecting small targets. Summary of the Invention

[0006] This invention addresses the technical problems of unsatisfactory recognition effect, low accuracy, and slow processing speed of small targets in complex backgrounds in the prior art. It provides a method and terminal for detecting small targets in electrical equipment, which has at least the advantages of reliable recognition, high accuracy, fast processing, and strong anti-interference ability.

[0007] First aspect

[0008] This invention provides a method for detecting small targets in electrical equipment, comprising:

[0009] S100, Obtain the original image;

[0010] S200. Input the original image into the target detection model;

[0011] S300, The target detection model processes the original image to output a labeled image;

[0012] The processing includes: preprocessing, localization, bounding box selection, and classification.

[0013] Specifically, the main technical concept of this invention is to improve the recognition accuracy and reliability of small targets in the target detection model through processing operations such as preprocessing, localization, bounding box selection and classification.

[0014] Optionally, the target detection model is obtained by training the training set as the input set and the validation set as the control set.

[0015] And / or,

[0016] The target detection model iterates between the training set or the original images and the corresponding validation set or the labeled images.

[0017] Specifically, another technical concept of the present invention is to provide multiple methods for acquiring and iteratively upgrading the target detection model in order to improve the maintainability of the target detection model.

[0018] Furthermore, the preprocessing includes:

[0019] S311. Input the original image into the convolutional layer to expand its number of channels through convolution operations;

[0020] S312. Further input the maximum pooling layer to unify the format and size through pooling operations;

[0021] S313, Output processed image.

[0022] Specifically, another technical concept of the present invention is to eliminate image differences in the original image through preprocessing operations to improve the learning efficiency and generalization ability of the target detection model, while making the original image deeper and having more channels, reducing the difficulty of the target detection model to capture abstract features, thereby improving the classification efficiency of small targets.

[0023] Furthermore, the localization process includes:

[0024] S321. Input the processed image into the hend layer to extract texture and edge information through convolution operation;

[0025] S322. Feature extraction is performed sequentially through at least two residual layers. Each residual layer is sampled by an FPN network and multi-scale fusion is performed layer by layer to generate the corresponding feature map.

[0026] Specifically, another technical concept of the present invention is to reduce the loss rate of texture and edge information by using the head layer, while improving the accuracy of feature generation through multi-scale fusion of multiple residual layers.

[0027] Furthermore, the feature extraction includes:

[0028] S322a, Expand the dimension of feature information via the first residual convolutional block;

[0029] S322b, The feature information is extracted via a second residual convolutional block;

[0030] S322c, The dimension of the feature information is further expanded via the third residual convolution block;

[0031] S322d: Attention is paid to the feature information via the SE attention mechanism.

[0032] Specifically, another technical concept of the present invention is to optimize the feature extraction operation of residual blocks to improve the network representation capability of feature extraction.

[0033] Furthermore, the first residual convolutional block, the second residual convolutional block, and the third residual convolutional block all include backpropagation (BP) processing;

[0034] The BP processing includes: accumulating the portions of the original image corresponding to the feature information, calculated using the following formula:

[0035]

[0036] This represents the processed output value. The learning parameters represent the scaling operations. The learning parameters represent the offset operation. express Standardization of zero mean and unit variance, , It is the first One sample, To prevent zero denominator, The formula for calculating the variance of a sample is given. , Indicates the length of the batch. The formula for calculating the sample mean is shown below.

[0037] .

[0038] Specifically, another technical concept of the present invention is to use backpropagation (BP) processing to standardize and regularize each residual convolutional block in the residual layer, thereby improving the efficiency and stability of residual processing. At the same time, the intervention of the original image is used to prevent the technical problem of overfitting in residual processing.

[0039] Optionally, the selection box includes:

[0040] S331. Input the feature map into the RPN network to generate at least two candidate boxes, the candidate boxes including possible target objects;

[0041] S332. Perform classification and regression operations on each of the candidate boxes to obtain the labeled boxes;

[0042] S333. Merge the marked boxes into the feature map.

[0043] Specifically, another technical concept of the present invention utilizes candidate boxes in conjunction with classification and regression operations to select and screen possible electrical devices in the original image, thereby improving the accuracy of small target detection.

[0044] Furthermore, the classification includes:

[0045] S334. The marked box is evenly divided to generate sub-regions of a fixed size;

[0046] S335. The feature information of each sub-region is processed by pooling operation to generate sub-feature output;

[0047] S336. Connect the outputs of each of the sub-features to form a feature vector;

[0048] S337. Input the feature vector into the classifier and output the classification of the target.

[0049] Specifically, another technical concept of the present invention is to perform local and uniform processing on the marker box through segmentation and pooling operations, so that the present invention can be compatible with the generation of candidate boxes of different sizes, thereby improving the compatibility and reliability of the present invention.

[0050] Furthermore, the classification includes: linear processing and nonlinear processing;

[0051] The linear processing relationship is expressed as follows:

[0052]

[0053] It represents Linear relationship of layers express Layer weight coefficients, express Layer input, This represents the offset during the convolution process. Indicates the current convolutional layer number;

[0054] The relationship described in the nonlinear processing is expressed as follows:

[0055] ; Indicates the first The first layer Each feature mapping output, Indicates the receptive field of the input layer. express The first layer Each feature mapping output, Represents the convolution kernel. Indicates the first The first layer of feature mapping Bias parameters of each neuron This represents the activation function.

[0056] Specifically, another technical concept of the present invention is that by using linear and nonlinear processing, the target detection model can handle both simple linear relationships and complex nonlinear relationships, so that both linear and nonlinear features can be extracted, thereby improving the reliability of the present invention in small target detection.

[0057] Second aspect

[0058] This invention provides a terminal for implementing a method for detecting small targets in electrical equipment as described in any possible embodiment of the first aspect. The technical effects provided by the second aspect can be understood with reference to the first aspect.

[0059] In summary, the present invention provides a method and terminal for detecting small targets in electrical equipment, which has at least the following advantages:

[0060] 1. This invention improves the recognition accuracy and reliability of small targets in the target detection model through preprocessing, localization, bounding box selection, and classification operations;

[0061] 2. This invention provides multiple methods for acquiring and iteratively upgrading target detection models to improve the maintainability of target detection models;

[0062] 3. This invention eliminates image differences in the original image through preprocessing operations to improve the learning efficiency and generalization ability of the target detection model. At the same time, it makes the original image deeper and has more channels, reducing the difficulty for the target detection model to capture abstract features, thereby improving the classification efficiency of small targets.

[0063] 4. This invention utilizes the head layer to reduce the loss rate of texture and edge information, while improving the accuracy of feature generation through multi-scale fusion of multiple residual layers; the feature extraction operation of residual blocks is optimized to improve the network's expressive power for feature extraction.

[0064] 5. This invention optimizes the feature extraction operation of residual blocks, thereby improving the network's expressive power for feature extraction;

[0065] 6. This invention utilizes backpropagation (BP) processing to standardize and regularize each residual convolutional block in the residual layer, thereby improving the efficiency and stability of residual processing. At the same time, it uses the intervention of the original image to prevent overfitting in residual processing.

[0066] 7. This invention utilizes candidate boxes in conjunction with classification and regression operations to select and screen possible electrical devices in the original image, thereby improving the accuracy of small target detection;

[0067] 8. This invention performs local and uniform processing on the marker box through segmentation and pooling operations, which makes the invention compatible with the generation of candidate boxes of different sizes, thereby improving the compatibility and reliability of the invention.

[0068] 9. This invention utilizes linear and nonlinear processing to enable the target detection model to handle both simple linear relationships and complex nonlinear relationships, allowing both linear and nonlinear features to be extracted, thereby improving the reliability of this invention for small target detection. Attached Figure Description

[0069] The present invention will be further described in detail below with reference to the accompanying drawings and preferred embodiments. However, those skilled in the art will understand that these drawings are drawn only for the purpose of explaining the preferred embodiments and therefore should not be construed as limiting the scope of the invention. Furthermore, unless specifically indicated, the drawings are only schematic representations of the composition or structure of the described objects and may contain exaggerated depictions, and the drawings are not necessarily drawn to scale.

[0070] Figure 1 A schematic diagram of the operation of a method for detecting small targets in electrical equipment according to an embodiment of the present invention;

[0071] Figure 2 A flowchart of a method for detecting small targets in electrical equipment, provided by an embodiment of the present invention;

[0072] Figure 3 A schematic diagram of the residual layer structure provided in an embodiment of the present invention;

[0073] Figure 4 A schematic diagram of the attention mechanism provided in an embodiment of the present invention;

[0074] Figure 5 A schematic diagram of the labeling of the original image or training set provided in an embodiment of the present invention;

[0075] Figure 6 A schematic diagram of the training loss curve of the target detection model provided in an embodiment of the present invention. Detailed Implementation

[0076] The following is in conjunction with the appendix Figures 1 to 6 The present invention will be described in detail below.

[0077] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0078] The main technical concept of this invention is to provide a method for detecting small targets in electrical equipment. Through the coordination and optimization of various processing operations, the detection efficiency, accuracy and anti-fitting ability of small targets are improved.

[0079] For further details, please see Figure 1 The diagram shown is a schematic representation of a method for detecting small targets in electrical equipment according to an embodiment of the present invention.

[0080] Example 1

[0081] The main workflow of this invention is as follows: First, in step S100, the original image is acquired; then, in step S200, the original image is input into the target detection model; finally, in step S300, the target detection model processes the original image to output a labeled image. The processing includes: preprocessing, localization, bounding box selection, and classification.

[0082] Preprocessing refers to performing preliminary processing on the original image to ensure that the format of the processed image is uniform.

[0083] Localization refers to the extraction of features from a preprocessed image.

[0084] Box selection refers to the detection of target features in an image;

[0085] Classification refers to the process of filtering and distinguishing electrical devices identified in an image.

[0086] It is worth explaining that the original image is an image captured by an image acquisition device, and the marked image is an image that includes electrical equipment markings after processing the original image. This allows all electrical equipment in the power network to be identified by the detection method provided by this invention, greatly reducing the degree of manual intervention and improving the inspection efficiency of the power network.

[0087] Example 2

[0088] Based on Example 1, the target detection model is trained using a training set as the input group and a validation set as the control group; and / or, the target detection model iterates between the training set or the original image and the corresponding validation set or the labeled image.

[0089] Optionally, the target detection model is set on the terminal and trained using a training set as the input group and a validation set as the control group.

[0090] Optionally, the target detection model is set on the terminal, trained using the training set as the input group and the validation set as the control group, and iteratively updated using the original image and the labeled image;

[0091] Optionally, the target detection model is set on a server, trained using a training set as the input group and a validation set as the control group, and then distributed to the terminal for execution.

[0092] Optionally, the target detection model is set on the server side, trained using a training set as the input group and a validation set as the control group, and then distributed to the terminal. The target detection model is further trained using the training set and validation set to update and iterate the target detection model in the terminal; or, after the terminal uploads the original image and the labeled image to the server side, the server updates and iterates the target detection model using the original image and the labeled image, and then distributes the updated target detection model to the terminal for deployment.

[0093] Optionally, the initial object detection model is trained using training and validation sets. The training and validation sets can be derived from standard online databases, such as energy transmission lines from Furns or data extracted from operating power grids. Both the training and validation sets consist of images of electronic devices, with small targets within these devices labeled, including obstacles, bird nests, insulators, spacers, and cable trays. The training-to-validation ratio is 9:1.

[0094] For further details, please see Figure 2 The diagram shown is a processing flowchart of a method for detecting small targets in electrical equipment according to an embodiment of the present invention.

[0095] Example 3

[0096] Based on Example 1 or Example 2, after input, the original image undergoes preprocessing through steps S311-S313, including: S311, inputting the original image into a convolutional layer to expand its channel number through convolution operations; S312, further inputting it into a max pooling layer to unify its format and size through pooling operations; S313, outputting the processed image, i.e. Figure 2 The process described in yellow boxes (preprocessing) involves the original image undergoing steps S311-S313 to generate a processed image with a uniform format that can be processed by the object detection model. For example, assuming the original image is in one of several different formats such as 600*600*3, 700*700*3, or 800*800*3, after preprocessing, it will be uniformly transformed into a 300*300*64 processed image. This ensures a uniform format and a rich number of channels (64 channels in this example), eliminating differences in brightness and contrast in the original image. It also enriches the layers of the processed image, facilitating the object detection model's capture of abstract features and improving its subsequent object classification performance.

[0097] Furthermore, the processed image undergoes localization processing in steps S321-S322 to extract features of small targets in the electronic device, including: S321, inputting the processed image into a hend layer to extract texture and edge information through convolution operations; S322, sequentially extracting features through at least two residual layers, each of which is sampled via an FPN network and fused layer by layer at multiple scales to generate a corresponding feature map, corresponding to... Figure 2 The processing steps for the white-framed portions within the purple and green boxes are explained below. It's worth noting that steps S321-S322 are used to obtain feature maps from the processed image through residual convolution. Step S321 extracts texture and edge information from the processed image, preventing the loss of texture and edge features and improving the reliability of feature extraction in step S322. Furthermore, in step S322, each residual layer undergoes multi-scale fusion layer-by-layer through an FPN network to comprehensively utilize the feature information from each residual layer processing stage, avoiding distortion in feature extraction.

[0098] Furthermore, after completing the localization process, bounding box selection is performed through steps S331-S333, including: S331, inputting the feature map into the RPN network to generate at least two candidate bounding boxes, each candidate bounding box including a possible target object; S332, performing classification and regression operations on each candidate bounding box to obtain labeled bounding boxes; S333, merging the labeled bounding boxes into the feature map. Corresponding to... Figure 2 The processing of the dark green boxes within the medium green boxes. It's worth explaining that the box selection operation first uses candidate boxes to initially select possible electrical devices, and then uses feature merging to further select the boxes, thereby improving the confidence level of the final obtained bounding boxes.

[0099] Furthermore, after the bounding box is selected, classification is performed through steps S334-337, including: S334, uniformly dividing the bounding box to generate sub-regions of fixed size; S335, processing the feature information in each sub-region through pooling operations to generate sub-feature outputs; S336, concatenating the sub-feature outputs to form a feature vector; S337, inputting the feature vector into a classifier to output the target classification. It is worth noting that the classification operation is mainly used to filter and distinguish targets within electrical equipment, thus completing the task of automatically labeling electrical equipment in the acquired original image.

[0100] Based on Embodiment 3, the following optional technical solutions are included:

[0101] One possible implementation involves feature extraction comprising: S322a, expanding the dimension of the feature information via a first residual convolutional block; S322b, extracting the feature information via a second residual convolutional block; S322c, further expanding the dimension of the feature information via a third residual convolutional block; and S322d, focusing on the feature information via an SE attention mechanism.

[0102] The following is Figure 3 The schematic diagram of the residual layer provided in one embodiment of the present invention further explains the feature extraction: each residual layer includes a first residual convolutional block, a second residual convolutional block, and a third residual convolutional block connected in sequence to extract feature information sequentially, thereby increasing the dimensionality of the feature information. For example, the first residual convolutional block is a 1×1 convolutional layer, the second residual convolutional block is a 3×3 convolutional layer, and the third residual convolutional block is a 1×1 convolutional layer.

[0103] Optionally, the first residual convolutional block, the second residual convolutional block, and the third residual convolutional block all incorporate Batch Normalization (BN) processing. The BN processing includes: accumulating the portions of the original image corresponding to the aforementioned feature information, calculated using the following formula:

[0104] Please refer to the explanation above for the meaning of each character. It is worth explaining that Batch Normalization (BN) can standardize the input of residual convolutional blocks in any layer, thereby speeding up the training of object detection models and having a regularization effect, preventing overfitting of residual convolutional blocks during operation and ensuring the reliability of images.

[0105] Optionally, skip links are set between each residual convolutional block to add the feature information after multi-layer transformation, thereby alleviating gradient vanishing after processing by different residual convolutional blocks and simplifying the network training of the target detection model.

[0106] Another possible implementation involves including an SE attention mechanism after feature extraction, so as to... Figure 4 The diagram shown is a structural schematic of the attention mechanism provided in an embodiment of the present invention. The SE attention mechanism is used to focus on the channels of the feature map and suppress unnecessary information to improve the confidence of feature information extraction.

[0107] Another possible embodiment is that the bounding box selection includes: candidate boxes with different sizes and different aspect ratios to cover a variety of target shapes that may appear in the image, and these candidate boxes can slide on the feature map to better fit the actual position of the target.

[0108] Another possible embodiment is that the classification includes linear processing and nonlinear processing; the relationship of the linear processing is expressed as follows:

[0109] ;

[0110] The relationship described in the nonlinear processing is expressed as follows:

[0111] .

[0112] Please refer to the above text for an explanation of each character. It is worth explaining that linear processing and nonlinear processing are used together to improve the linear and nonlinear learning capabilities of the object detection model, enabling it to learn both linear relationships and complex nonlinear relationships, thus simultaneously ensuring the object detection model's ability to extract both linear and nonlinear features.

[0113] For further details, please see Figure 5 The image shown is a schematic diagram of the labeling of the original image or training set provided in an embodiment of the present invention.

[0114] according to Figure 5 As shown, the target detection model provided by this invention can identify targets such as insulation materials and bird nests of various shapes. It is worth explaining that the original images and training set mentioned in Example 2 are... Figure 5 Images without bounding boxes, while labeled images and control groups are... Figure 5 The image shown.

[0115] For further details, please see Figure 6 The figure shown is a schematic diagram of the training loss curve of the target detection model provided in an embodiment of the present invention.

[0116] As shown in the figure, the loss decreases continuously as the object detection model training progresses. However, the rate of loss reduction gradually flattens out as training continues.

[0117] The present invention has been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of the invention. The descriptions of the embodiments above are only for the purpose of helping to understand the invention and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to the invention without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

Claims

1. A method for detection of small objects in electrical equipment, characterized in that, The method comprises: S100, acquiring an original image; S200, inputting the original image into a target detection model; S300, processing the original image by the target detection model to output a labeled image; The processing comprises: preprocessing, localization processing, frame selection, and classification; Wherein, The localization processing comprises: S321, inputting the preprocessed image into a head layer to extract texture and edge information through convolution operation; S322, sequentially performing feature extraction through at least two residual layers, any residual layer being sampled through an FPN network and performing multi-scale fusion layer by layer to generate a corresponding feature map; The feature extraction comprises: S322a, expanding the dimension of the feature information through a first residual convolution block; S322b, extracting the feature information through a second residual convolution block; S322c, further expanding the dimension of the feature information through a third residual convolution block; S322d, focusing on the feature information through an SE attention mechanism.

2. The method for detection of small objects in electrical equipment as claimed in claim 1 wherein, The SE attention mechanism is used to focus on the channels of the feature map and suppress unnecessary information to improve the confidence of feature information extraction.

3. The method for detection of small objects in electrical equipment as claimed in claim 2 wherein, The target detection model is trained by taking a training set as an input group and a validation set as a control group; And / or, The target detection model iterates the training set or the original image with the corresponding validation set or labeled image.

4. The method for detection of small objects in electrical equipment as claimed in claim 3 wherein, The preprocessing comprises: S311, inputting the original image into a convolution layer to expand the number of channels thereof through convolution operation; S312, further inputting a max-pooling layer to unify the format and size through pooling operation; S313, outputting a processed image.

5. The method for detection of small objects in electrical equipment as claimed in claim 3 wherein, The first residual convolution block, the second residual convolution block, and the third residual convolution block all comprise BP processing; The BP processing comprises: accumulating the part of the original image corresponding to the feature information, and the calculation formula thereof is: ； represents the output value after processing, represents a learning parameter of a scale operation, represents a learning parameter of a shift operation, represents a standardization of zero mean and unit variance, , is the th sample, represents a guard against zero denominator, represents a variance calculation formula of a sample, , represents the length of a batch, represents a mean calculation formula of a sample, 。 6. The method for detection of small objects in electrical equipment as claimed in claim 1 wherein, The frame selection comprises: S331, inputting a feature map into an RPN network to generate at least two candidate frames, the candidate frames comprising target objects with possibility; S332, performing classification operation and regression operation on each candidate frame to obtain a labeled frame; S333, merging the labeled frame into the feature map.

7. The method for detection of small objects in electrical equipment as claimed in claim 6 wherein, The classification comprises: S334, uniformly dividing the labeled frame to generate sub-regions of fixed size; S335, processing the feature information in each sub-region through pooling operation to generate sub-feature output; S336, connecting each sub-feature output to form a feature vector; S337, inputting the feature vector into a classifier to output the classification of the target.

8. The method for detection of small objects in electrical equipment as claimed in claim 7 wherein, The classification comprises: linear processing and nonlinear processing; The relationship of the linear processing is represented as follows: ； represents a linear relationship of a layer, represents a weight coefficient of a layer, represents an input of a layer, represents an offset in a convolution process, represents a number of layers of a current convolution; The relationship of the nonlinear processing is represented as follows: ; Indicates the first The first layer Each feature mapping output, Indicates the receptive field of the input layer. express The first layer Each feature mapping output, Represents the convolution kernel. Indicates the first The first layer of feature mapping Bias parameters of each neuron This represents the activation function.

9. A terminal, characterized by comprising: The method is used to implement the method for detecting small targets in electrical equipment according to any one of claims 1-8.

Images