A small target recognition method and system based on cascade hierarchical detection and self-comparison

Abstract

The present invention discloses a small target recognition method and system based on cascaded hierarchical detection and self-comparison. The method includes: collecting aerial images of drone inspections and annotating the aerial images; inputting the annotated images into a device recognition module to perform device area detection and image cropping to obtain an area image containing only small targets; inputting the area image into a cascaded small target defect recognition module to perform defect recognition, and feature encoding the identified small target defects to obtain encoded defect features; calculating the Euclidean distance between the defect feature and the historical small target defect feature currently updated in the feature library. If the Euclidean distance is greater than a preset threshold, it means that the defect feature is a false detection and will be eliminated; if it is not greater than the preset threshold, it means that the defect feature is correct, outputting the corresponding small target defect recognition result, and caching the defect feature in the feature library. The present invention improves the recognition effect of small target defect detection.

Description

Technical Field

[0001] The present invention relates to the field of target detection technology, and in particular to a small target recognition method and system based on cascaded hierarchical detection and self-comparison. Background Art

[0002] With the continuous growth of electricity demand, distribution networks are expanding in scale and becoming increasingly complex. Ensuring the safe and stable operation of distribution networks is crucial for both social production and daily life. Small objects such as safety pins, cable clamp insulation covers, and screws are key protective components of distribution network equipment, and their condition is directly related to the safety of the equipment and the entire distribution network. In actual operation, due to various factors such as environmental factors (such as wind, rain, and vibration), equipment aging, and overload, some of these small objects may become detached. If these small objects become detached or lost, the relevant equipment may not function properly, potentially causing power outages, impacting power supply reliability, and even causing equipment damage and economic losses. Therefore, identifying these small objects is crucial. Traditional distribution network inspections rely primarily on manual labor, which is inefficient and labor-intensive. Furthermore, in complex environments, manual inspections can be difficult to fully and promptly detect problems such as detached safety pins. In recent years, drone inspection technology has been increasingly used for distribution network inspections, enabling rapid and efficient image capture of distribution network equipment.

[0003] There are two main types of mainstream object detection models. One directly inputs a full image and then uses the detection model to detect it directly. This can result in small objects occupying a very small portion of the image, making recognition and observation more difficult. The other employs a two-stage detection process (first coarse component detection, then fine-grained detection of small objects). For example, the prior art discloses a method, system, device, and storage medium for detecting small object defects in distribution lines. This solution uses a region-of-interest (ROI) detection network and an improved defect detection network to detect distribution line images, obtaining a ROI frame image, further obtaining a small object defect detection region image, and ultimately detecting small object defects. However, neither of these methods pays sufficient attention to small objects during feature extraction or feature fusion. This lack of a feature enhancement mechanism results in dilution of small object features in the deep network, leading to insufficient semantics. Consequently, the model discards or loses some shallow-layer feature information during feature extraction and feature fusion from shallow to deep layers. Crucial spatial details in an image are primarily contained in shallow layers, which help locate the ROI and identify small objects. A lack of this information can significantly impact overall recognition performance. Summary of the Invention

[0004] In order to address the deficiencies in the prior art, the present invention provides a small target recognition method and system based on cascaded hierarchical detection and self-comparison. Through the linkage of three parts: equipment recognition module, small target defect recognition module, and feature library caching and comparison, the types of small target defects in distribution network scenarios can be effectively identified.

[0005] The present invention adopts the following technical solutions.

[0006] In a first aspect, the present invention provides a small target recognition method based on cascaded hierarchical detection and self-alignment, the method comprising:

[0007] Collect and annotate aerial images from drone inspections;

[0008] The labeled image is input into the device recognition module to detect the device area and crop the image to obtain an image containing only the area where small objects exist.

[0009] Inputting the region image into a cascaded small target defect recognition module for defect recognition, and performing feature encoding on the identified small target defects to obtain encoded defect features;

[0010] Calculate the Euclidean distance between the defect feature and the historical small target defect feature currently updated in the feature library. If the Euclidean distance is greater than the preset threshold, it means that the defect feature is a false detection and will be eliminated; if it is not greater than the preset threshold, it means that the defect feature is correct, output its corresponding small target defect recognition result, and cache the defect feature in the feature library.

[0011] Optionally, the content of labeling the qualified image includes: adjusting the image size of the target frame, selecting the position frame, and category information of the device and small target.

[0012] Optionally, the device identification module includes a target detection module and a picture cropping module;

[0013] The target detection module detects the device area where small targets may appear in the input image to obtain a region of interest frame image; the image cropping module crops the detected region of interest image to obtain a region image containing only the small targets.

[0014] Optionally, the device identification module and the small target defect identification module are both composed of a YOLOv8s network;

[0015] Based on the annotated historical images, a device image sample set is constructed to train and optimize the YOLOv8s network to obtain the device recognition module, and the corresponding image dataset containing only the area where small objects exist is output;

[0016] A small target image sample set is constructed based on the regional image dataset to train and optimize the YOLOv8s network to obtain the small target defect recognition module.

[0017] Optionally, the YOLOv8s network is obtained by replacing the C2f feature fusion module of the neck network in the YOLOv8 network with a C2f-att module based on the attention mechanism;

[0018] The C2f-att module includes the first convolutional layer of Conv1, Segmentation layer, EMA attention layer, Bottleneck deep feature extraction module, Concat connection layer and Conv2 second convolution layer.

[0019] Optionally, the C2f-att module processes image features as follows:

[0020] For the feature T input to the C2f-att module, it is first input into the first convolutional layer of Conv1 to adjust the channel and obtain the adjusted feature ; The segmentation layer will adjust the features Divide the channel into two parts and get the feature and features ; The feature Input into the EMA attention layer to extract multi-scale feature information and output features ; Bottleneck deep feature extraction module uses multiple layers of Bottleneck to extract features Perform multiple stacking to obtain deep features ; The Concat layer connects the features and deep features After splicing, the features are obtained ; The second convolutional layer of Conv2 transforms the features Perform a convolution operation to adjust the number of channels back to output the final feature F.

[0021] Optionally, the EMA attention layer includes a third convolutional layer and a fourth convolutional layer, a Concat connection layer, a GAP pooling layer, a first fully connected layer FC1, a first fully connected layer FC2 and a Multiply weighted network layer;

[0022] The third convolutional layer and the fourth convolutional layer respectively Features output by the segmentation layer After performing feature transformations at different scales, the corresponding feature maps are obtained. and ; The Concat layer connects the feature map and After splicing, feature Y is obtained; channel attention is calculated on feature Y through the GAP pooling layer, the first fully connected layer FC1 and the first fully connected layer FC2 to obtain the channel attention weight vector ; Finally, the Multiply weighted network layer is used to re-weight the channel of feature Y to obtain the feature after attention enhancement .

[0023] Optionally, the channel attention weight vector The expression is as follows:

[0024]

[0025] Where GAP represents global average pooling; and Respectively represent the connection operations of the first fully connected layer FC1 and the first fully connected layer FC2; is the Sigmoid function; is the RELU function; Indicates that the shape of vector s is one-dimensional and the length of this dimension is C, where C represents the number of channels of the image.

[0026] Optionally, the loss function used when training and optimizing the YOLOv8s network is: DIoU loss function.

[0027] Optionally, a Transformer encoder based on the attention mechanism is used to feature encode the identified small object defects.

[0028] Optionally, the expression for calculating the Euclidean distance between the defect feature and the historical small target defect feature currently updated in the feature library is as follows:

[0029]

[0030] Where, Defect characteristics Historical small target defect characteristics with current updates The Euclidean distance of , , Represents the dimension of the feature, and Characteristics and features The data of the i-th dimension in .

[0031] Optionally, the step of updating the historical small target defect features includes:

[0032] Multiple small target images are obtained and encoded respectively to obtain their corresponding original defect features;

[0033] Perform mean extraction on the original defect features corresponding to each small target image, and use the features extracted from the mean as the initial features of the historical small target defect features in the feature library;

[0034] Each time a new defect feature is cached in the feature library, the initial feature and the new defect feature are weighted averaged to obtain an updated historical small target defect feature.

[0035] In a second aspect, the present invention provides a small target recognition system based on cascaded hierarchical detection and self-comparison, which executes the steps of any method described in the first aspect of the present invention, and the system includes:

[0036] The acquisition and annotation unit is used to collect and annotate aerial images of drone inspections;

[0037] The detection and cropping unit is used to input the labeled image into the device recognition module to perform device area detection and image cropping to obtain an image containing only the area where small objects exist;

[0038] an identification and encoding unit, configured to input the region image into a cascaded small target defect recognition module for defect recognition, and perform feature encoding on the identified small target defects to obtain encoded defect features;

[0039] The calculation and judgment unit is used to calculate the Euclidean distance between the defect feature and the historical small target defect feature currently updated in the feature library. If the Euclidean distance is greater than the preset threshold, it means that the defect feature is a false detection and will be eliminated; if it is not greater than the preset threshold, it means that the defect feature is correct, and the corresponding small target defect recognition result is output, and the defect feature is cached in the feature library.

[0040] In a third aspect, the present invention provides a terminal including a processor and a storage medium;

[0041] The storage medium is used to store instructions;

[0042] The processor is configured to operate according to the instructions to execute the steps of any one of the methods described in the first aspect of the present invention.

[0043] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any one of the methods described in the first aspect of the present invention.

[0044] The beneficial effects of the present invention are as follows:

[0045] 1. The present invention splits the detection process into two stages: coarse positioning and fine recognition through the linkage of three parts: the device identification module, the small target defect identification module, and the caching and comparison of the feature library. This avoids missing or misjudging small targets in one-time detection, and increases the proportion of small targets in the image through interest cropping and secondary detection. It can effectively identify the types of small target defects in distribution network scenarios and improve detection accuracy.

[0046] 2. The device identification module and small target defect identification module of the present invention both use the YOLOv8s network for detection and identification. Improvements are made to the original YOLOv8 network structure to enhance the model's perception of small targets. Mainly in the Neck feature enhancement part, the original C2f modules are completely replaced with C2f-att modules with an attention mechanism to enhance the ability to integrate cross-channel and spatial information, making the model more sensitive to small target boundaries, especially in low-resolution or dense scenes, and more effectively capturing fine-grained target information.

[0047] 3. The C2f-att module provided by the present invention further improves the structure of the EMA attention layer on the basis of the original C2f module to enhance the feature discrimination ability. The feature matching mechanism enhances the system's continuous learning and generalization capabilities. Through the improved structure of the EMA attention layer, a higher attention weight is assigned to small targets, so that it pays more attention to small targets during the feature processing and fusion stages than before, thereby retaining more shallow information, achieving more effective positioning and recognition, and having good recognition scalability.

[0048] 4. The present invention can significantly reduce the false detection rate of complex backgrounds and enhance the long-term stability of the system by dynamically updating the historical small target defect features in the feature library and further distinguishing the recognition results of the small target defect recognition module using the Euclidean distance.

[0049] 5. The present invention uses labeled images to construct a sample set for model training. By combining high-quality data construction with fine annotation, the efficiency and accuracy of model training are optimized. At the same time, the use of the DIoU loss function is more conducive to model convergence, improving the accuracy of small target detection and improving the stability and application effect in actual inspection scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 Schematic diagram of the process of the small target recognition method based on cascaded hierarchical detection and self-alignment in the present invention;

[0051] Figure 2 This is a flowchart of the device identification module and a schematic diagram of the structure of the target detection module in the present invention;

[0052] Figure 3 This is a schematic diagram of the traditional YOLOv8 network structure;

[0053] Figure 4 This is a schematic diagram of the YOLOv8s network structure improved by the present invention;

[0054] Figure 5 Schematic diagram of the C2f-att network structure in the present invention;

[0055] Figure 6 Schematic diagram of the improved EMA network structure in the present invention;

[0056] Figure 7 This is a schematic diagram of the structure of the detection head Detect of YOLOv8s in the present invention;

[0057] Figure 8 This is a comparison chart showing the recognition effects of YOLOv8 and YOLOv8s using the example of safety pin detachment recognition in the present invention;

[0058] Figure 9 This is a comparison chart showing the recognition effects of YOLOv8 and YOLOv8s using the example of screw breakage recognition in the present invention;

[0059] Figure 10 This is a block diagram of the structural principle of the small target recognition system based on cascaded hierarchical detection and self-comparison in the present invention. DETAILED DESCRIPTION

[0060] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. The embodiments described in the present invention are only part of the embodiments of the present invention, not all of the embodiments. Based on the spirit of the present invention, other embodiments obtained by ordinary technicians in this field without making any creative efforts are all within the scope of protection of the present invention.

[0061] Example 1:

[0062] Reference Figure 1 The embodiment of the present invention provides a small target recognition method based on cascaded hierarchical detection and self-comparison, which specifically includes the following steps:

[0063] Step 1: Collect and annotate aerial images of drone inspections;

[0064] Specifically, based on the original drone inspection aerial images, the images are annotated using manual recognition. The annotated content covers the adjustment of the image size of the target frame, the selection of the position frame, and the category information of the equipment and small targets.

[0065] Step 2: Input the labeled image into the device recognition module to detect the device area and crop the image to obtain an image containing only the area where small objects exist;

[0066] Reference Figure 2 As shown, the device recognition module includes a target detection module and an image cropping module;

[0067] Among them, the target detection module detects the device area where small targets may appear in the input image to obtain a frame image of the region of interest; the image cropping module crops the detected region of interest image to obtain an image of the area containing only the small targets.

[0068] In this embodiment, the image cropping module uses a sliding window mechanism to perform region cropping. In this embodiment, the window size is defined as 896×896, and the original image resolution is kept unchanged. Based on the marking information, a local device image containing a small target is cropped from the entire image. That is, the device area where the small target may appear is first identified in the original image, and then the device area is cropped to retain more small target information.

[0069] Step 3: Input the region image into the cascaded small target defect recognition module for defect recognition, and perform feature encoding on the identified small target defects to obtain encoded defect features;

[0070] Step 4: Calculate the Euclidean distance between the defect feature and the historical small target defect feature currently updated in the feature library. If the Euclidean distance is greater than the preset threshold, it means that the defect feature is a false detection and will be eliminated; if it is not greater than the preset threshold, it means that the defect feature is correct, and the corresponding small target defect recognition result is output, and the defect feature is cached in the feature library.

[0071] In a preferred but non-limiting embodiment, the device recognition module comprises a YOLOv8s network; a device image sample set is constructed based on annotated historical images to train and optimize the YOLOv8s network to obtain the device recognition module. Constructing the sample set includes extracting single frames from drone aerial video streams that match the recognition scene, dividing them into training and test sets as required, and correctly annotating the captured data. The image sample set is then obtained after the data is correctly selected and formatted.

[0072] In a preferred but non-limiting embodiment, the small object defect recognition module is also composed of a YOLOv8s network. Specifically, the labeled historical images are input into the trained device recognition module, which outputs a corresponding image dataset of regions containing only small objects. Based on this region image dataset, a small object image sample set is constructed to train and optimize the YOLOv8s network to obtain the small object defect recognition module.

[0073] In particular, the method provided by the present invention employs a more targeted data processing and modeling process. First, based on the original drone inspection images, manual recognition and annotation are used to directly annotate the original images. Specifically, the equipment areas and small object areas where small objects may be present are manually annotated in the original images, thereby constructing a high-quality image dataset containing small object labels. Subsequently, during the training phase, based on the annotated equipment area information, the original images are cropped to identify only the small objects. During training, both the original and cropped images are fed into the model. This annotation and training strategy not only helps the model accurately identify various small objects but also enhances its robustness and adaptability in complex scenarios. Furthermore, the present invention utilizes an annotated small object detection dataset specific to the distribution network equipment environment for model training. Because distribution network equipment in this environment is generally large (e.g., power towers), small objects in this environment (e.g., screws on power towers) are smaller in size and more challenging to identify. Therefore, annotating data for this scenario and applying it specifically to training allows the model to focus more on the feature extraction and recognition tasks of small objects, significantly improving the detection accuracy of small objects. By combining high-quality data construction with precise annotation, the efficiency and accuracy of model training are optimized, the convergence of the model is accelerated, and the stability and application effect in actual inspection scenarios are improved.

[0074] Reference Figure 3 and Figure 4 The YOLOv8s network provided by the present invention is obtained by replacing the C2f feature fusion module of the neck network in the YOLOv8 network with a C2f-att module based on the attention mechanism; wherein the C2f-att module includes the first convolutional layer of Conv1, Segmentation layer, EMA attention layer, Bottleneck deep feature extraction module, Concat connection layer and Conv2 second convolution layer.

[0075] Reference Figure 5 , the process of the C2f-att module in processing image features is as follows:

[0076] For the feature T input to the C2f-att module, it is first input into the first convolutional layer of Conv1 to adjust the channel and obtain the adjusted feature ; The segmentation layer will adjust the features Divide the channel into two parts and get the feature and features ; The feature Input into the EMA attention layer to extract multi-scale feature information and output features ; Bottleneck deep feature extraction module uses multiple layers of Bottleneck to extract features Perform multiple stacking to obtain deep features ; The Concat layer connects the features and deep features After splicing, the features are obtained ; The second convolutional layer of Conv2 transforms the features Perform a convolution operation to adjust the number of channels back to output the final feature F.

[0077] Specifically, for the input feature T, after inputting it into the C2f-att module, it is first input into the convolutional layer for channel adjustment to obtain the adjusted feature :

[0078]

[0079] Then the adjusted features Input The segmentation layer divides the channel into two parts and obtains the features and features :

[0080]

[0081] in, It is the main branch that is subsequently input to EMA. It is a shortcut branch that directly participates in the subsequent Concat connection layer.

[0082] The main branch Input to the subsequent EMA attention layer for attention calculation:

[0083]

[0084] in, is the output feature of the EMA attention layer;

[0085] The output features of EMA The Bottleneck deep feature extraction module input into the main branch is processed by multi-layer Bottleneck Perform multiple stacking to obtain deep features :

[0086] ;

[0087] in, Indicates the number of Bottleneck layers; in this embodiment,N The value of is 3.

[0088] Then, the Concat connection layer is used to splice the main branch and the shortcut branch to obtain the spliced ​​features :

[0089]

[0090] Finally, the result of the second convolutional layer of Conv2 is Perform convolution operation to adjust the number of channels back:

[0091]

[0092] Among them, F is the final output of the C2f-att module.

[0093] A preferred but non-limiting embodiment, EMA (Efficient Multi-scale Attention) is a lightweight multi-scale attention mechanism. The main idea is to fuse multi-scale feature information and perform channel attention modeling, allowing the network to automatically focus on more useful feature areas (especially small targets) while maintaining computational efficiency. The present invention makes structural improvements to the EMA structure that adapts to the model of the present invention, such as Figure 6 The figure shows the overall structure of the improved EMA attention layer, including the third and fourth convolutional layers, the Concat connection layer, the GAP pooling layer, the first fully connected layer FC1, the first fully connected layer FC2, and the Multiply weighted network layer;

[0094] Among them, the third convolutional layer and the fourth convolutional layer respectively Features output by the segmentation layer After performing feature transformations at different scales, the corresponding feature maps are obtained. and ; The Concat layer connects the feature map and After splicing, feature Y is obtained; channel attention is calculated on feature Y through the GAP pooling layer, the first fully connected layer FC1 and the first fully connected layer FC2 to obtain the channel attention weight vector ; Finally, the Multiply weighted network layer is used to re-weight the channel of feature Y to obtain the feature after attention enhancement .

[0095] Specifically, first The segmentation layer completes the feature map of channel division Input to the EMA attention layer, the input feature map Divided into multiple scales;

[0096] Then after two convolutional layers Perform scale-by-scale feature transformation, allowing each sub-channel to capture receptive fields of different scales through a separate convolution operation:

[0097]

[0098] in, Represents the feature maps corresponding to two branches of different scales, Indicates that the convolution kernel size is The convolution operation, Indicates that the convolution kernel size is The convolution operation.

[0099] The output is then passed to the Concat connection layer for splicing and channel attention weighting, and the feature maps of each scale are and After splicing, feature Y is obtained, and the result is input into the subsequent GAP layer and fully connected layer to calculate channel attention:

[0100]

[0101]

[0102] Where GAP represents global average pooling, and Respectively represent the connection operations of the first fully connected layer FC1 and the first fully connected layer FC2, is the Sigmoid function, is the RELU function, Indicates that the shape of vector s is one-dimensional and the length of this dimension is C. Similarly Indicates that the shape of vector Y is three-dimensional, and the lengths of the three dimensions are C, H, and W, corresponding to the number of channels, width, and height of the image, respectively.

[0103] Finally, it is input into the Multiply weighted network layer for channel reweighting, and we get Weighted features

[0104]

[0105] The attention-enhanced features are sent back to the C2f-att main path for further processing.

[0106] A preferred but non-limiting embodiment uses an attention-based Transformer encoder to feature encode the identified small object defects.

[0107] Refer to the following Figure 4To further illustrate the recognition process of the YOLOv8s network of the present invention, the image is first input into the backbone of YOLOv8s to extract image features. Figure 4 The main process shown is to input the original image Image into the first convolution layer of the backbone network for initial convolution operation, then in the second convolution layer, which is equivalent to downsampling operation, and then input it into the C2f layer for the first step of feature extraction operation. Then it is input into the third and fourth convolution layers for further downsampling, and the result is output to the second C2f layer to obtain higher-level features. Finally, it is input into the fifth convolution layer and the third C2f layer for downsampling and high-level feature acquisition, and the final result is input into the SPPF module for multi-scale context aggregation. Among them, the feature processing method inside the SPPF is:

[0108] Start by performing three consecutive pooling operations on the input to obtain three feature maps. The input feature map is X:

[0109]

[0110]

[0111]

[0112] Here is the feature map obtained after the pooling operation, represents the convolution kernel size, represents the step length, Indicates padding.

[0113] Then perform the splicing operation to splice the original input and the three pooling results along the channel dimension to obtain the feature :

[0114]

[0115] Finally, convolution fusion is performed to obtain multi-scale context information

[0116]

[0117] Image features processed by the backbone network are then fed into the neck network for further processing. The neck network's main process involves first upsampling the high-level semantic features output by the sppf layer through the Upsample layer, then concatenating them with the features output by the shallower C2f layer through the Concat layer. The combined vector is then fed into the C2f-att module for fusion. The addition of an attention mechanism to C2f allows for continuous learning and optimization of parameter weights, assigning higher weights to small target areas to increase the model's focus. After fusion, the features are further upsampled through the Upsample layer, merged with the shallowest layer features through Concat, and then fused again through the C2f-att module to form the top layer of the feature pyramid. Then the features here are divided into two outputs, one is directly input into the Detect detection head for prediction, and the other is input into the convolution layer, which is equivalent to downsampling and feature extraction. The features obtained here are merged with the features output by the previous C2f-att in the Concat module, and the merged features are input into the next layer of C2f-att. After this C2f-att processing, the output is divided into two again, one is input into the second Detect detection head, and the other continues to be output to the convolution layer below to obtain deeper feature information. This feature is continued to be input into the Concat layer below and spliced ​​with the output of SPPF to obtain new features. This part of the features is finally input into the C2f-att layer and finally input into the third Detect detection head for prediction.

[0118] The three-layer features from the neck network are respectively sent to the three Detects of the head detection head. The detection head of YOLOv8s is consistent with the original detection head structure of YOLOv8. Its main workflow is still to perform a convolution operation first, and process the input feature map through the convolution layer to obtain the category probability distribution of each position. Here, the sigmoid activation function is mainly used to normalize the category predicted by each grid, and then perform a bounding box regression operation. The model will regress the bounding box coordinates of the center of each grid, including the offset of the center point (phase, width and height). The sigmoid activation function is still used here to normalize the position of the box so that the output can be directly mapped to the original image coordinate system. Finally, each predicted box will generate a confidence value, which indicates the probability that the box contains the target, indicating whether the box contains the object. Each of the three Detect detection heads completes the prediction at different scales, and finally splices the multi-scale detection results into an output.

[0119] In a preferred but non-limiting embodiment, in terms of loss function processing, this model optimizes the loss function to DIoU. Compared with the traditional IoU Loss, which only focuses on the overlapping area of ​​​​two boxes, for two boxes that do not overlap but have very close centers, IoU is still 0 and cannot provide a gradient, which greatly affects the training effect of small target recognition. The DIoU in this embodiment introduces the Euclidean distance between the center point of the predicted box and the true box, encouraging the predicted box to converge to the true box faster, even if they do not overlap initially. The calculation formula of the DIoU loss function is:

[0120]

[0121] In the formula, the prediction box , real frame ; is the ratio of the intersection and union of the two boxes, where c is the diagonal length of the minimum bounding box containing the predicted box and the true box, It is the square of the diagonal length of the minimum enclosing rectangle containing the predicted box and the true box; It is the square of the Euclidean distance between the center of the predicted box and the center of the true box, and its calculation formula is:

[0122]

[0123] This formula takes the square of the Euclidean distance between the center points and introduces the DIoU loss function. By adding the center distance term, the predicted box is forced to quickly approach the center of the true box to prevent small targets from shifting. At the same time, since small targets are small in area and sensitive to position, this method will pay more attention to position errors and improve detection accuracy.

[0124] For the Detect detection head, its structure is as follows Figure 7 As shown, it mainly consists of two branches, one for category prediction and the other for bounding box regression and DIoU loss calculation. When the features output by the neck network are passed to the Detect detection head, the features will be divided into two parts. The ultimate goal is to perform category prediction and bounding box regression respectively. Both parts will first undergo two consecutive layers of Conv convolutional layers to further extract features and nonlinear transformations, thereby enhancing the expression ability. Finally, they will pass through Conv2D, which functions like a linear output layer to generate the final prediction result, and finally obtain the bounding box and calculate the DIoU loss optimization parameters, and obtain the category prediction cls.

[0125] After processing in the target detection module, the model obtains information about device regions where small targets may appear. This information is then fed into the image cropping module of the device identification module. This module uses a sliding window strategy for region cropping. Here, the window size is defined as 896×896, and the original image resolution is maintained. This means that device regions where small targets may appear are first identified in the original image, and then these regions are cropped to preserve more information about the small targets. Therefore, after passing through the device identification module, the image is first fed into the target detection module, specifically the YOLOv8s network, for detection. The YOLOv8s network detects regions containing small targets and outputs the labeled image as input to the image cropping module. This module uses a sliding window mechanism and, based on the labeled information, crops the local device image containing the small target from the entire image.

[0126] Subsequently, the device image is input into the small target defect recognition module, which is mainly composed of an improved YOLOv8s model. The device area image is sequentially passed through the backbone, neck, and head of YOLOv8s to finally obtain the image and annotation information of the small target defect. The effect of the small target defect recognition image obtained here is better than the existing common small target detection model and YOLOv8 model. This is mainly because small targets usually occupy a small image area and have weak features, which can easily be "submerged" during the downsampling process. In addition, when detecting small targets, the background or other large targets are prone to false detection. The present invention takes into account that the neck of YOLOv8 is the core of multi-scale feature fusion. The C2f-att module adds an attention mechanism to the C2f feature fusion module without increasing the computational complexity too much. The attention mechanism can automatically enhance the response strength of the small target area according to the context, allowing subsequent layers to pay more attention to the small target. At the same time, it can suppress irrelevant areas and improve the "signal-to-noise ratio" of the small target area, thereby improving the model recognition effect.

[0127] The detected small object defects will be encoded using an image encoder. This embodiment uses the Transformer encoder because it uses the self-attention mechanism to allow each position to interact with all other positions and capture global dependencies. At the same time, since the subsequent similarity calculation is also the calculation of similarity between small objects, the Transformer can better integrate contextual information into the encoding and improve the discriminative power. The calculation method of the Transformer encoder is as follows:

[0128] For a small target area corresponding to the feature patch , after linear mapping to token sequence ;

[0129] First, perform Patch Embedding and Positional Encoding, divide the image into multiple small intervals (patches), and then perform the following calculations:

[0130]

[0131]

[0132] Among them, the linear matrix , position encoding , the features represented by each patch , encoded features .

[0133] After passing through the multi-head attention and feedforward network of the Transformer encoder, the output for the kth layer is:

[0134]

[0135] in, is the output of the kth layer, FFN represents the feedforward neural network, and MSA is the multi-head attention mechanism, which is calculated as follows:

[0136]

[0137] in, Represents the three feature matrices of query, key, and value in the attention calculation process

[0138] For each head h we have:

[0139]

[0140] in, The matrix is ​​used to calculate the corresponding matrix.

[0141] Then multiple headers are concatenated and mapped:

[0142]

[0143] Finally, perform Global Pooling and Embedding Projection to generate the final code:

[0144]

[0145]

[0146] The dimensions that a Transformer encodes its output , the LayNorm layer is LN, where N is the number of patches.

[0147] The Euclidean distance between the feature vector encoded by the Transformer encoder and the feature vector in the pre-built target feature library is calculated. If the distance exceeds the set threshold, the target is considered to be a new target or a changed target, which means that the detected defect is a false detection and the target will be eliminated. If the recognition is successful and the confidence level is high (not less than the set threshold), the feature is added to the feature library and the recognition system is dynamically updated to enhance the model's memory and expansion capabilities. The expression of the Euclidean distance is as follows:

[0148]

[0149] Where, Defect characteristics Historical small target defect characteristics with current updates The Euclidean distance of , , Represents the dimension of the feature, and Characteristics and features The data of the i-th dimension in .

[0150] In a preferred but non-limiting embodiment, the step of updating the historical small target defect features in step 4 includes:

[0151] Step 4.1: Obtain multiple small target images and encode them separately to obtain their corresponding original defect features;

[0152] Step 4.2: Perform mean extraction on the original defect features corresponding to each small target image, and use the features extracted from the mean as the initial features of the historical small target defect features in the feature library;

[0153] Step 4.3: Each time a new defect feature is cached in the feature library, the initial feature and the new defect feature are weighted averaged to obtain an updated historical small target defect feature.

[0154] Specifically, the feature library is constructed as follows: in the feature library initialization stage, multiple small target images are manually selected and input into the encoder for encoding to obtain the corresponding feature vectors, which are then input into the feature library. This is equivalent to using these multiple feature vectors to initialize the feature library. Subsequently, as the number of input images and vectors increases during the training process, new vectors are continuously added to the feature library based on the Euclidean distance and threshold.

[0155] It is worth noting that the embodiment of the present invention introduces a multi-stage feature aggregation and dynamic weight fusion mechanism and a Euclidean distance dynamic threshold judgment mechanism for false detection suppression in the process of feature comparison and feature library establishment. The dynamic weight fusion mechanism is mainly reflected in the feature comparison module designed by the present invention. The feature library not only caches historical features, but also adopts a staged weighted averaging mechanism to aggregate features. The basic defect features and historical cached features are respectively averaged, and non-equal weighting coefficients are introduced during fusion (such as 1.0, 0.95, 0.95, that is, the weighting coefficient of the initial feature of the historical small target defect feature is 1.0, and the weighting coefficients of the first and second features of the subsequent cache are both 0.95). Finally, adaptive correction is performed through a normalization coefficient (such as 2.9). Compared with the existing simple feature averaging scheme, this structure can suppress the drift of dynamic cache features while maintaining the basic defect semantic expression capability, effectively improving the comparison stability and recognition robustness. The Euclidean distance dynamic thresholding mechanism for false detection suppression uses a fixed threshold to determine false detections when comparing defect features against a feature library. Instead of using a fixed threshold to determine false detections, the comparison threshold range is adaptively adjusted based on the current shift in the feature cluster center. If changes in the historical feature cluster center are detected, the threshold range is dynamically narrowed to avoid false positives caused by "feature drift." The calculation process relies primarily on a feature library that caches defect features for each small target. The feature dimension is 1 x (K+N) x 1024, where K is the base small target feature and N is the number of cached small targets subsequently added. Before each Euclidean distance calculation, the features of all small targets are merged into a cluster feature, representing the location of the small target feature cluster. If the distance between a newly detected feature and this cluster in feature space is less than a preset threshold, the newly detected feature belongs to this cluster. The average of the first K features and the average of the last N-1 features is taken during the merging process. This adaptive thresholding strategy can significantly reduce false detections in complex backgrounds and enhance long-term system stability in practical deployments.

[0156] In summary, this embodiment takes the example of the safety pin falling off to further illustrate the overall recognition process: after receiving the image from the drone, Figure 1As shown, the device identification module is first entered to obtain the device locations where small target areas may appear, such as crossarms and insulators. The detected device area is then cropped, and the cropped image is then input into the cascaded small target defect recognition module. This solution enables the small target defect recognition model to identify smaller targets at a larger resolution, and this targeted design is more efficient than outputting only the region of interest, which often contains a large number of useless or highly overlapping candidate areas. Next, the detected small target defects are encoded using a Transformer encoder. The encoded defect features are then compared with the historical small target defect features stored in the feature library using Euclidean distance. If the Euclidean distance is greater than the preset threshold, it means that the detected defect is a false detection and will be eliminated. If it is less than the preset threshold, it means that the detection is correct, the defect recognition result is output, and the defect features are cached in the feature library.

[0157] Figure 8 and Figure 9 The overall recognition renderings of two small target defect detection tasks are respectively demonstrated using traditional YOLOv8 and improved YOLOv8s, respectively showing the safety pin and screw breaking and falling off, identifying the region of interest and identifying defects in the region of interest. For the recognition renderings of the YOLOv8 model, the model will detect many regions of interest where small objects may exist in the picture, but in fact there are no small targets to be identified inside these regions. At the same time, the regional pictures containing small targets are separately passed into the YOLOv8 model, and the model will not identify the location of the small targets. For the model of the present invention, first, the YOLOv8s model will identify which areas may have small targets, and will separately identify the regions of interest where small targets may appear, thereby detecting the specific location of the small targets. By comparing the two model renderings, it can be seen that the model architecture and optimization method proposed in the present invention can obtain better results in small target defect detection.

[0158] The beneficial effects of the present invention are as follows:

[0159] 1. The present invention splits the detection process into two stages: coarse positioning and fine recognition through the linkage of three parts: the device identification module, the small target defect identification module, and the caching and comparison of the feature library. This avoids missing or misjudging small targets in one-time detection, and increases the proportion of small targets in the image through interest cropping and secondary detection. It can effectively identify the types of small target defects in distribution network scenarios and improve detection accuracy.

[0160] 2. The device identification module and small target defect identification module of the present invention both use the YOLOv8s network for detection and identification. Improvements are made to the original YOLOv8 network structure to enhance the model's perception of small targets. Mainly in the Neck feature enhancement part, the original C2f modules are completely replaced with C2f-att modules with an attention mechanism to enhance the ability to integrate cross-channel and spatial information, making the model more sensitive to small target boundaries, especially in low-resolution or dense scenes, and more effectively capturing fine-grained target information.

[0161] 3. The C2f-att module provided by the present invention further improves the structure of the EMA attention layer on the basis of the original C2f module to enhance the feature discrimination ability. The feature matching mechanism enhances the system's continuous learning and generalization capabilities. Through the improved structure of the EMA attention layer, a higher attention weight is assigned to small targets, so that it pays more attention to small targets during the feature processing and fusion stages than before, thereby retaining more shallow information, achieving more effective positioning and recognition, and having good recognition scalability.

[0162] 4. The present invention can significantly reduce the false detection rate of complex backgrounds and enhance the long-term stability of the system by dynamically updating the historical small target defect features in the feature library and further distinguishing the recognition results of the small target defect recognition module using the Euclidean distance.

[0163] 5. The present invention uses labeled images to construct a sample set for model training. By combining high-quality data construction with fine annotation, the efficiency and accuracy of model training are optimized. At the same time, the use of the DIoU loss function is more conducive to model convergence, improving the accuracy of small target detection and improving the stability and application effect in actual inspection scenarios.

[0164] Example 2:

[0165] like Figure 10 As shown, the present invention provides a small target recognition system based on cascaded hierarchical detection and self-comparison, which is used to implement the steps of the method in the above embodiment 1, and specifically includes:

[0166] The acquisition and annotation unit is used to collect and annotate aerial images of drone inspections;

[0167] The detection and cropping unit is used to input the labeled image into the device recognition module to perform device area detection and image cropping to obtain an image containing only the area where small objects exist;

[0168] an identification and encoding unit, configured to input the region image into a cascaded small target defect recognition module for defect recognition, and perform feature encoding on the identified small target defects to obtain encoded defect features;

[0169] The calculation and judgment unit is used to calculate the Euclidean distance between the defect feature and the historical small target defect feature currently updated in the feature library. If the Euclidean distance is greater than the preset threshold, it means that the defect feature is a false detection and will be eliminated; if it is not greater than the preset threshold, it means that the defect feature is correct, and the corresponding small target defect recognition result is output, and the defect feature is cached in the feature library.

[0170] The small target recognition system based on cascaded hierarchical detection and self-matching provided in the embodiment of the present invention and the small target recognition method based on cascaded hierarchical detection and self-matching provided in Example 1 are based on the same technical concept, and can produce the beneficial effects as described in Example 1. For the contents not fully described in this embodiment, please refer to Example 1.

[0171] Example 3:

[0172] An embodiment of the present invention provides a terminal comprising a processor and a storage medium, the terminal being an embedded computer system device. The terminal's storage medium is used to store instructions, and the memory comprises a non-volatile storage medium and an internal memory; wherein the non-volatile storage medium stores an operating system, a computer program, and a database, the internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium, and the database is used to store instruction data. The terminal's processor is used to operate according to the instructions provided by the storage medium to execute the steps of the small target recognition method based on cascaded hierarchical detection and self-comparison according to any one of the first embodiments.

[0173] Example 4:

[0174] An embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps of the method described in any one of the first embodiments are implemented.

[0175] The present invention may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.

[0176] A computer-readable storage medium can be a tangible device that can hold and store instructions for use by an instruction execution device. A computer-readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punched card or raised structure in a groove on which instructions are stored, and any suitable combination thereof. As used herein, a computer-readable storage medium is not to be construed as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse passing through a fiber optic cable), or an electrical signal transmitted through an electrical wire.

[0177] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in the computer-readable storage medium in each computing / processing device.

[0178] The computer program instructions for performing the operations of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, the state information of the computer-readable program instructions is used to personalize an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), so that the electronic circuit can execute the computer-readable program instructions, thereby implementing various aspects of the present invention.

[0179] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered by the scope of protection of the claims of the present invention.

Claims

1. A small target recognition method based on cascaded hierarchical detection and self-comparison, characterized in that the method include: Collect and annotate aerial images from drone inspections; The labeled image is input into the device recognition module to detect the device area and crop the image to obtain an image containing only the area with small objects. Inputting the region image into a cascaded small target defect recognition module for defect recognition, and encoding the identified small target defects to obtain encoded defect features; Calculate the Euclidean distance between the defect feature and the historical small target defect feature currently updated in the feature library. If the Euclidean distance is greater than a preset threshold, it means that the defect feature is a false detection and will be eliminated. If it is not greater than the preset threshold, it means that the defect feature is correct, and the corresponding small target defect recognition result is output, and the defect feature is cached in the feature library; The device recognition module and the small target defect recognition module are both composed of YOLOv8s networks; Based on the annotated historical images, a device image sample set is constructed to train and optimize the YOLOv8s network to obtain the device recognition module, and the corresponding image dataset containing only the area where small objects exist is output; Constructing a small target image sample set based on the regional image dataset to train and optimize the YOLOv8s network to obtain the small target defect recognition module; The YOLOv8s network is obtained by replacing the C2f feature fusion module of the neck network in the YOLOv8 network with the C2f-att module based on the attention mechanism; The C2f-att module includes the first convolutional layer of Conv1, Segmentation layer, EMA attention layer, Bottleneck deep feature extraction module, Concat connection layer and Conv2 second convolution layer; The process of the C2f-att module in processing image features is as follows: For the feature T input to the C2f-att module, it is first input into the first convolutional layer of Conv1 to adjust the channel and obtain the adjusted feature ; The segmentation layer will adjust the features Divide the channel into two parts and get the feature and features ; The features Input into the EMA attention layer to extract multi-scale feature information and output features ; Bottleneck deep feature extraction module uses multiple layers of Bottleneck to extract features Perform multiple stacking to obtain deep features ; The Concat layer connects the features and deep features After splicing, the features are obtained ; The second convolutional layer of Conv2 transforms the features Perform convolution operation to adjust the number of channels and output the final feature F; The EMA attention layer includes the third and fourth convolutional layers, the Concat connection layer, the GAP pooling layer, the first fully connected layer FC1, the first fully connected layer FC2 and the Multiply weighted network layer; The third convolutional layer and the fourth convolutional layer respectively Features output by the segmentation layer After performing feature transformations at different scales, the corresponding feature maps are obtained. and ; The Concat layer connects the feature maps and After splicing, feature Y is obtained; channel attention is calculated on feature Y through the GAP pooling layer, the first fully connected layer FC1 and the first fully connected layer FC2 to obtain the channel attention weight vector ; Finally, the Multiply weighted network layer is used to re-weight the channel of feature Y to obtain the feature after attention enhancement .

2. The small target recognition method based on cascaded hierarchical detection and self-comparison according to claim 1 is characterized in that: The contents of labeling for images that meet the conditions include: adjusting the image size of the target frame, selecting the position frame, and the category information of the device and small targets.

3. The small target recognition method based on cascaded hierarchical detection and self-comparison according to claim 1 is characterized in that: The device recognition module includes a target detection module and a picture cropping module; The target detection module detects the device area where small targets may appear in the input image to obtain a frame image of the region of interest; The image cropping module crops the detected region of interest image to obtain an image containing only the region where small targets exist.

4. The small target recognition method based on cascaded hierarchical detection and self-comparison according to claim 1, characterized in that: The channel attention weight vector The expression is as follows: Where GAP represents global average pooling; and Respectively represent the connection operations of the first fully connected layer FC1 and the first fully connected layer FC2; is the Sigmoid function; is the RELU function; Indicates that the shape of vector s is one-dimensional and the length of this dimension is C, where C represents the number of channels of the image.

5. The small target recognition method based on cascaded hierarchical detection and self-comparison according to claim 1 is characterized in that: The loss function used when training and optimizing the YOLOv8s network is: DIoU loss function.

6. The small target recognition method based on cascaded hierarchical detection and self-comparison according to claim 1, characterized in that: The Transformer encoder based on the attention mechanism is used to encode the features of the identified small object defects.

7. The small target recognition method based on cascaded hierarchical detection and self-comparison according to claim 1, characterized in that: The expression for calculating the Euclidean distance between the defect feature and the historical small target defect feature currently updated in the feature library is as follows: Where, Defect characteristics Historical small target defect characteristics with current updates The Euclidean distance of , , Represents the dimension of the feature, and Characteristics and features The data of the i-th dimension in .

8. The small target recognition method based on cascaded hierarchical detection and self-comparison according to claim 1 or 7, characterized in that: The updating steps of the historical small target defect features include: Multiple small target images are obtained and encoded respectively to obtain their corresponding original defect features; Perform mean extraction on the original defect features corresponding to each small target image, and use the features extracted from the mean as the initial features of the historical small target defect features in the feature library; Each time a new defect feature is cached in the feature library, the initial feature and the new defect feature are weighted averaged to obtain an updated historical small target defect feature.

9. A small target recognition system based on cascaded hierarchical detection and self-comparison, running the small target recognition method based on cascaded hierarchical detection and self-comparison according to any one of claims 1 to 8, characterized in that: The system includes: The acquisition and annotation unit is used to collect and annotate aerial images taken by drone inspections; The detection and cropping unit is used to input the labeled image into the device recognition module to perform device area detection and image cropping to obtain an image containing only the area where small objects exist; an identification and encoding unit, configured to input the area image into a cascaded small target defect recognition module for defect recognition, and perform feature encoding on the identified small target defects to obtain encoded defect features; The calculation and judgment unit is used to calculate the Euclidean distance between the defect feature and the historical small target defect feature currently updated in the feature library. If the Euclidean distance is greater than the preset threshold, it means that the defect feature is a false detection and will be eliminated; if it is not greater than the preset threshold, it means that the defect feature is correct, and the corresponding small target defect recognition result is output, and the defect feature is cached in the feature library.

10. A terminal comprising a processor and a storage medium; characterized in that: The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images