Robust attention-based small target object recognition precision improvement method and device

Abstract

The present application relates to the technical field of deep learning target detection, in particular to a small target object recognition precision improvement method and device based on robust attention. The small target object recognition precision improvement method based on robust attention comprises: based on the Hodges-Lehmann estimation method, using an attention module to construct a model to obtain a robust attention model based on the Hodges-Lehmann estimation; adding the robust attention model based on the Hodges-Lehmann estimation to a preset convolutional neural network detection model to obtain a to-be-trained detection precision improvement model; obtaining model training data; using the model training data to train the to-be-trained detection precision improvement model to obtain a detection precision improvement model; obtaining actual detection data; according to the actual detection data, performing target recognition through the detection precision improvement model to obtain a high-precision detection result. The present application is a small target object recognition precision improvement method based on robust attention with strong universality.

Description

Technical Field

[0001] This invention relates to the field of deep learning target detection technology, and in particular to a method and apparatus for improving the accuracy of small target object recognition based on robust attention. Background Technology

[0002] With the rapid development of artificial intelligence, deep learning-based visual recognition technology is gradually being applied to various fields. Commonly used visual technologies include image classification algorithms and object detection algorithms. The former is used to classify object images, such as face recognition, while the latter can realize tasks such as fire detection and human behavior detection based on cameras.

[0003] Both of these types of recognition algorithms are mostly built upon convolutional neural networks, such as classification models like ResNet and EfficientNet, and object detection models like YOLOv5, YOLOv8, and Faster R-CNN. These models are trained using 2D images, but small objects in images often cannot be learned by the models. Therefore, researchers are constantly developing more complex models, such as the ResNet series (ResNet18, ResNet50, ResNet101, etc.) and YOLOv5 (YOLOv5s, YOLOv5m, YOLOv5l, etc.), which rely on more parameters and network layers to learn all the features of objects within the image.

[0004] In different situations, users often cannot arbitrarily switch between and use various models. Therefore, some scholars have developed attention models that can improve model accuracy. Classic attention models include SENet, CBAM, and ECA. They mostly utilize convolutional and fully connected layers to reprocess feature maps and then reweight them to achieve efficient feature learning.

[0005] In the existing technology, there is a lack of a general method to improve the accuracy of small target object recognition based on robust attention. Summary of the Invention

[0006] To address the technical problems of existing technologies, such as the inability to learn small target features, the increased number of additional parameters in the detection model, and the lack of universality across multiple scenarios, this invention provides a method and apparatus for improving the accuracy of small target object recognition based on robust attention. The technical solution is as follows:

[0007] On the one hand, a method for improving the recognition accuracy of small target objects based on robust attention is provided. This method is implemented by a recognition accuracy improvement device and includes:

[0008] Based on the Hodges-Lehmann estimation method, an attention module is used to build the model, resulting in a robust attention model based on Hodges-Lehmann estimation.

[0009] The robust attention model based on Hodges-Lehmann estimation is added to a preset convolutional neural network detection model to obtain a model for improving detection accuracy to be trained.

[0010] Obtain model training data; use the model training data to train the detection accuracy improvement model to obtain the detection accuracy improvement model;

[0011] Obtain actual detection data; based on the actual detection data, perform target recognition through the detection accuracy improvement model to obtain high-precision detection results.

[0012] Optionally, training the detection accuracy improvement model using the model training data to obtain the detection accuracy improvement model includes:

[0013] Feature extraction is performed on the training data of the model to obtain a multi-dimensional feature map;

[0014] The multi-dimensional feature map is input into a robust attention model based on Hodges-Lehmann estimation for data fusion to obtain a small target attention-weighted feature map;

[0015] The small target attention-weighted feature map is used to train the preset convolutional neural network detection model to obtain a detection accuracy improvement model.

[0016] Optionally, the step of inputting the multi-dimensional feature map into a robust attention model based on Hodges-Lehmann estimation for data fusion to obtain a small target attention-weighted feature map includes:

[0017] The multi-dimensional feature map is input into the multi-layer information correction module to obtain a multi-layer channel information fusion feature map.

[0018] The multi-channel information fusion feature map is input into the multi-layer feature map correction module to obtain the small target attention weighted feature map.

[0019] Optionally, the step of inputting the multi-dimensional feature map into the multi-layer information correction module to obtain a multi-layer channel information fusion feature map includes:

[0020] Channel feature extraction is performed on the multi-dimensional feature map to obtain the first channel feature map;

[0021] Feature extraction is performed on the first channel feature map to obtain the second channel feature map;

[0022] Based on the second channel feature map, the data is filtered using Hodges-Lehmann estimation to obtain the third channel feature map;

[0023] The guiding value is obtained by calculating based on the multi-dimensional feature map;

[0024] Based on the guiding value, the Sigmoid function is used to perform channel attention weighting on the third channel feature map to obtain a multi-layer channel information fusion feature map.

[0025] Optionally, the step of inputting the multi-layer channel information fusion feature map into the multi-layer feature map correction module to obtain a small target attention-weighted feature map includes:

[0026] Spatial information features are extracted from the multi-layer channel information fusion feature map to obtain a first spatial feature map;

[0027] Based on the median method, specific channel spatial features are extracted from the first spatial feature map to obtain the second spatial feature map;

[0028] Spatial feature extraction is performed on the multi-dimensional feature map to obtain a third spatial feature map;

[0029] The median method and the sigmoid function are used to perform shallow spatial feature extraction on the multi-dimensional feature map to obtain the fourth spatial feature map.

[0030] Based on the second, third, and fourth spatial feature maps, data is filtered using Hodges-Lehmann estimation to obtain the fifth spatial feature map;

[0031] The Sigmoid function is used to perform spatial attention weighting on the fifth spatial feature map to obtain a small target attention-weighted feature map.

[0032] Optionally, the step of extracting specific channel spatial features from the first spatial feature map based on the median method to obtain the second spatial feature map includes:

[0033] The median method is used to filter channels in the first spatial feature map to obtain specific channels; the spatial information in the specific channels is used to represent the spatial information of the first spatial feature map.

[0034] Based on the specific channel, spatial features are extracted from the first spatial feature map to obtain a second spatial feature map.

[0035] The robust attention model based on Hodges-Lehmann estimation is a lightweight model.

[0036] On the other hand, a device for improving the accuracy of small object recognition based on robust attention is provided. This device is applied to a method for improving the accuracy of small object recognition based on robust attention. The device includes:

[0037] The model building module is used to build a robust attention model based on the Hodges-Lehmann estimation method using the attention module.

[0038] The model joint module is used to add the robust attention model based on Hodges-Lehmann estimation to a preset convolutional neural network detection model to obtain a model to be trained to improve detection accuracy.

[0039] The detection accuracy improvement module is used to acquire model training data; and to train the detection accuracy improvement model to be trained using the model training data to obtain the detection accuracy improvement model.

[0040] The detection result acquisition module is used to acquire actual detection data; based on the actual detection data, target recognition is performed through the detection accuracy improvement model to obtain high-precision detection results.

[0041] Optionally, the detection accuracy improvement module is further used for:

[0042] Feature extraction is performed on the training data of the model to obtain a multi-dimensional feature map;

[0043] The multi-dimensional feature map is input into a robust attention model based on Hodges-Lehmann estimation for data fusion to obtain a small target attention-weighted feature map;

[0044] The small target attention-weighted feature map is used to train the preset convolutional neural network detection model to obtain a detection accuracy improvement model.

[0045] Optionally, the detection accuracy improvement module is further used for:

[0046] The multi-dimensional feature map is input into the multi-layer information correction module to obtain a multi-layer channel information fusion feature map.

[0047] The multi-channel information fusion feature map is input into the multi-layer feature map correction module to obtain the small target attention weighted feature map.

[0048] Optionally, the detection accuracy improvement module is further used for:

[0049] Channel feature extraction is performed on the multi-dimensional feature map to obtain the first channel feature map;

[0050] Feature extraction is performed on the first channel feature map to obtain the second channel feature map;

[0051] Based on the second channel feature map, the data is filtered using Hodges-Lehmann estimation to obtain the third channel feature map;

[0052] The guiding value is obtained by calculating based on the multi-dimensional feature map;

[0053] Based on the guiding value, the Sigmoid function is used to perform channel attention weighting on the third channel feature map to obtain a multi-layer channel information fusion feature map.

[0054] Optionally, the detection accuracy improvement module is further used for:

[0055] Spatial information features are extracted from the multi-layer channel information fusion feature map to obtain a first spatial feature map;

[0056] Based on the median method, specific channel spatial features are extracted from the first spatial feature map to obtain the second spatial feature map;

[0057] Spatial feature extraction is performed on the multi-dimensional feature map to obtain a third spatial feature map;

[0058] The median method and the sigmoid function are used to perform shallow spatial feature extraction on the multi-dimensional feature map to obtain the fourth spatial feature map.

[0059] Based on the second, third, and fourth spatial feature maps, data is filtered using Hodges-Lehmann estimation to obtain the fifth spatial feature map;

[0060] The Sigmoid function is used to perform spatial attention weighting on the fifth spatial feature map to obtain a small target attention-weighted feature map.

[0061] Optionally, the detection accuracy improvement module is further used for:

[0062] The median method is used to filter channels in the first spatial feature map to obtain specific channels; the spatial information in the specific channels is used to represent the spatial information of the first spatial feature map.

[0063] Based on the specific channel, spatial features are extracted from the first spatial feature map to obtain a second spatial feature map.

[0064] The robust attention model based on Hodges-Lehmann estimation is a lightweight model.

[0065] On the other hand, a device for improving recognition accuracy is provided, the device comprising: a processor; a memory storing computer-readable instructions, wherein when the computer-readable instructions are executed by the processor, any one of the methods described above for improving the recognition accuracy of small target objects based on robust attention is implemented.

[0066] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored in the storage medium, the at least one instruction being loaded and executed by a processor to implement any of the above-described methods for improving the accuracy of small target object recognition based on robust attention.

[0067] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0068] This invention proposes a method to improve the accuracy of small target object recognition based on robust attention. It utilizes the proposed High-Level Attention Recognition Map (HLRA) to process feature maps generated at each stage of the model's computation, filtering these feature values ​​and extracting feature information in both channel and spatial dimensions through pooling operations. Redundant data is removed, maximizing the retention of key information and achieving attention processing in both spatial and channel dimensions. HLRA also uses the Sigmoid function to weight each feature value, assigning greater weight to important data and focusing the model's learning attention on the target object. Using a model with HLRA can enhance feature learning capabilities, further focusing on the target object and ultimately improving prediction accuracy. HLRA is an independent model structure that requires only a single, explicit feature map input. Therefore, HLRA can be inserted into most convolutional neural network-based image classification and object detection models, and can be inserted into different positions within different models, with the number of insertions controllable, thus achieving HLRA's versatility. HLRA uses pure numerical computations, such as max pooling and Hodges-Lehmann estimation, without relying on convolutional or fully connected layers for data filtering. Therefore, inserting this model does not increase the number of parameters in the original model, thereby minimizing the computational and deployment burden on the original model. This invention is a versatile method for improving the accuracy of small target object recognition based on robust attention. Attached Figure Description

[0069] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0070] Figure 1 This is a flowchart of a method for improving the accuracy of small target object recognition based on robust attention, provided by an embodiment of the present invention.

[0071] Figure 2 This is a schematic diagram illustrating three representation methods of Hodges-Lehmann estimation provided in an embodiment of the present invention;

[0072] Figure 3 This is a schematic diagram of the structure of a MIC module provided in an embodiment of the present invention;

[0073] Figure 4 This is a schematic diagram of the structure of an MFMC module provided in an embodiment of the present invention;

[0074] Figure 5 This is a schematic diagram of an HLRA use case provided by an embodiment of the present invention;

[0075] Figure 6 This is a block diagram of a device for improving the accuracy of small target object recognition based on robust attention, provided in an embodiment of the present invention.

[0076] Figure 7 This is a schematic diagram of the structure of a recognition accuracy improvement device provided in an embodiment of the present invention. Detailed Implementation

[0077] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0078] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0079] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, their intended meanings are consistent. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, their intended meanings are consistent.

[0080] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0081] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0082] This invention provides a method for improving the accuracy of small target object recognition based on robust attention. This method can be implemented by a recognition accuracy improvement device, which can be a terminal or a server. Figure 1 The flowchart shown is for a method to improve the accuracy of small target object recognition based on robust attention. The processing flow of this method may include the following steps:

[0083] S1. Based on the Hodges-Lehmann estimation method, an attention module is used to build the model, resulting in a robust attention model based on Hodges-Lehmann estimation.

[0084] Among them, the robust attention model based on Hodges-Lehmann estimation is a lightweight model.

[0085] In one feasible implementation, this invention designs a robust attention model based on Hodges-Lehmann estimation (Hodges-Lehmann-Based Robust Attention, HLRA). HLRA does not use convolutional layers and fully connected layers to process feature maps, but instead borrows the characteristics of Hodges-Lehmann to eliminate error samples in channel and spatial information, thereby improving the robustness of the model.

[0086] S2. The robust attention model based on Hodges-Lehmann estimation is added to the preset convolutional neural network detection model to obtain the detection accuracy improvement model to be trained.

[0087] In one feasible implementation, the preset convolutional neural network detection model is an image classification model or object detection model based on convolutional neural networks, such as a network model for object detection (You Only Look Once v5, YOLOv5, etc.). Depending on actual needs, a custom number of HLRA models can be added after any layer of the model.

[0088] S3. Obtain model training data; use the model training data to train the detection accuracy improvement model to obtain the detection accuracy improvement model.

[0089] Optionally, the detection accuracy improvement model is trained using the model training data to obtain the detection accuracy improvement model, including:

[0090] Feature extraction is performed on the model training data to obtain multi-dimensional feature maps;

[0091] Multi-dimensional feature maps are input into a robust attention model based on Hodges-Lehmann estimation for data fusion to obtain attention-weighted feature maps for small targets;

[0092] By using small target attention-weighted feature maps, a pre-defined convolutional neural network detection model is trained to obtain a model that improves detection accuracy.

[0093] In one feasible implementation, the HLRA model proposed in this invention does not require separate training. When improving the detection model's accuracy in recognizing small objects, it is only necessary to determine the input and output feature map sizes and number of channels before directly inserting it into an image classification and object detection model built on a convolutional neural network. The combined detection accuracy improvement model can be regarded as a complete new model, and data training and validation can be performed on the new model.

[0094] Optionally, the multi-dimensional feature maps are input into a robust attention model based on Hodges-Lehmann estimation for data fusion to obtain small-object attention-weighted feature maps, including:

[0095] The multi-dimensional feature map is input into the multi-layer information correction module to obtain the multi-layer channel information fusion feature map;

[0096] The multi-channel information fusion feature map is input into the multi-layer feature map correction module to obtain the small target attention weighted feature map.

[0097] In one feasible implementation, the HLRA consists of two parts: a multi-layer information correction module (MIC) that implements channel attention, and a multi-layer feature map correction module (MFMC) that implements spatial attention.

[0098] Optionally, the multi-dimensional feature map is input into the multi-layer information correction module to obtain a multi-layer channel information fusion feature map, including:

[0099] Channel features are extracted from the multi-dimensional feature map to obtain the first channel feature map;

[0100] Feature extraction is performed on the first channel feature map to obtain the second channel feature map;

[0101] Based on the second channel feature map, the data is filtered using Hodges-Lehmann estimation to obtain the third channel feature map;

[0102] The guiding value is calculated based on the multi-dimensional feature map;

[0103] Based on the guiding value, the Sigmoid function is used to perform channel attention weighting on the third channel feature map to obtain a multi-channel information fusion feature map.

[0104] In one feasible implementation, traditional channel feature extraction methods, such as average pooling, can only retain the feature information of the current layer, ignoring the deep feature information embedded in the feature map. The current feature layer may contain a large amount of irrelevant data, which hinders subsequent model inference and judgment. To address this issue, this invention proposes a feature extraction method based on the concept of feature fusion. Max pooling operations of different scales are applied to the input feature map to extract deep and shallow information. The max pooling operation is shown in Equation 1:

[0105] M(F) = MaxPool(F) (1)

[0106] Where F is the multi-dimensional feature map; MaxPool is a max pooling operation with different convolution sizes and different phase lengths; M is the first channel feature map after max pooling.

[0107] Based on the above steps, three feature maps of the same size but different information levels are obtained, as shown in Table 1 (feature maps of max pooling operations at different scales).

[0108] Table 1

[0109] M1 5×5 1 5 / / 2 M2 9×9 1 9 / / 2 M3 13×13 1 13 / / 2

[0110] Global average pooling is used to extract the feature information provided in the first channel feature map, as shown in Equation 2. The size of the feature map changes from H×W×C to 1×1×C.

[0111] MF(F)=AdaptiveAvgPool(F) (2)

[0112] Here, AdaptiveAvgPool is the global average pooling operation; MF is the second channel feature map after global average pooling.

[0113] Using methods such as cascading to fuse three feature maps representing different layers can compromise the realism of feature values ​​and fail to meet the requirements of feature extraction. Conversely, methods such as averaging typically include all features in the data but cannot eliminate potentially redundant data in each layer, thus reducing the accuracy of feature values. Based on Hodges-Lehmann estimation, this paper uses feature maps representing three layers of information as a dataset containing error samples to dilute excessively biased data.

[0114] The Hodges-Lehmann estimation can robustly obtain the center point of data in randomly sampled datasets. Unlike methods that use only the median and mean, the Hodges-Lehmann estimation uses grouped means and the overall median to mitigate the impact of outliers on the overall dataset. The formula is shown in Equation 3.

[0115]

[0116] Where θ is the center point of the sample; Median is the median operation; x i and x j It is data within the sample.

[0117] like Figure 2 As shown, given [x] i x j There are currently three methods to represent the Hodges-Lehmann estimate, depending on the different elements of the equation.

[0118] The first Hodges-Lehmann estimation representation considers all possible combinations, expanding each element horizontally and vertically and completing one-to-one correspondence and grouping. Because this combination method considers the combination of elements, it can theoretically select the best data, but it brings a huge computational burden, hindering the lightweight implementation of the model.

[0119] The second Hodges-Lehmann estimation representation reduces computational workload by nearly half compared to the first. This second method considers only the top-right and diagonal groupings, removing the bottom-left groupings that are similar to the top-right groupings and eliminating redundant data. However, in the second estimation representation, the selected diagonal groups contain invariant data generated by averaging, which may lead to fusion results that fail to highlight feature information.

[0120] The third Hodges-Lehmann estimation representation removes the diagonal combinations from the second representation. While this method removes original data samples from the result, it further emphasizes the role of the fused information in the model, and the subsequent attention weighting is based on the fused feature information. Based on the above discussion, to reduce the computational workload and improve efficiency, this invention adopts the second Hodges-Lehmann estimation representation method.

[0121] Based on the principle of Hodges-Lehmann estimation, the obtained multi-layer feature information is flattened and arranged as a m and a n Then pair them into [a] m ,an ] and placed in m ij Set, filter out m ij Elements in the set where m < n.

[0122] This invention will m ij The selected combinations are placed in set u ij The mean of the elements in each combination is calculated. The result is obtained by calculating the median of all elements, as shown in Equation 4:

[0123]

[0124] Among them, HL map This is the third channel feature map after Hodges-Lehmann estimation.

[0125] The magnitude of values ​​in different channels is considered a factor influencing attention determination. Therefore, this invention proposes a data activation formula to activate data in the feature map, scaling up or down based on the data's inherent size.

[0126] The specific principle of the data activation formula is as follows: This invention calculates the root mean square of the feature map to be processed, and calls it the guiding value. The feature value of each channel is divided by the guiding value, so that the feature values ​​greater than the guiding value increase to a certain extent, while the feature values ​​less than the guiding value are standardized. The specific formula is shown in Equation 5:

[0127]

[0128] Where A is the feature of each channel after activation; x i It is the characteristic of the i-th channel, 1≤i≤n; n is the number of channels.

[0129] The Sigmoid function is applied to the feature map for channel attention weighting, as shown in Equation 6:

[0130] x HL =x input ×Sigmoid(A(HL map (6)

[0131] Where, x input These are the initial input data; Sigmoid is the Sigmoid function; x HL It is a feature map of multi-layer channel information fusion after MIC processing.

[0132] The specific uses and structure of the MIC module are as follows: Figure 3 As shown. In principle, MICs can be added anywhere in the model. Figure 3The right side shows the structure after adding the MIC. In this invention, the MIC is used together with the MFMC and placed in multiple locations on the model.

[0133] Optionally, the multi-channel information fusion feature map is input into the multi-layer feature map correction module to obtain a small target attention-weighted feature map, including:

[0134] Spatial information features are extracted from the multi-layer channel information fusion feature map to obtain the first spatial feature map;

[0135] Based on the median method, specific channel spatial features are extracted from the first spatial feature map to obtain the second spatial feature map;

[0136] Spatial features are extracted from the multi-dimensional feature map to obtain the third spatial feature map;

[0137] The median method and the sigmoid function are used to perform shallow spatial feature extraction on the multi-dimensional feature map to obtain the fourth spatial feature map.

[0138] Based on the second, third, and fourth spatial feature maps, the data is filtered using Hodges-Lehmann estimation to obtain the fifth spatial feature map.

[0139] The Sigmoid function is used to perform spatial attention weighting on the fifth spatial feature map to obtain the small target attention-weighted feature map.

[0140] In one feasible implementation, unlike channel feature extraction, spatial information extraction should be determined for specific channels of the feature map. Dimensionality reduction and compression methods may compromise the realism and reliability of the feature map.

[0141] Therefore, spatial feature data of specific channels is chosen to represent the spatial information of the feature map. In this respect, the present invention uses global average pooling on the feature map, which is the output of the MIC, to extract the spatial information of each channel, i.e., the first spatial feature map. Based on the first spatial feature map, specific channel spatial features are extracted to obtain the second spatial feature map.

[0142] Spatial information extraction cannot be limited to a single level. Although the feature map processed by MIC achieves channel focus, the original spatial information may be compromised to some extent. To address this issue, this invention uses Equation 7 to process the initial input data to obtain a feature map of size H×W×1:

[0143] Map input =Median(AdaptivePool(x) input (7)

[0144] Among them, Mapinput It is the spatial feature map of the initial input.

[0145] Map HL Represents the spatial feature map after deep fusion. input This represents the third spatial feature map of the original training data. To broaden the coverage of spatial information, this invention uses Equations 8 to 9 to obtain a feature map (Map) representing shallow spatial information from the initial input feature map. Avg :

[0146] x Avg =x input ×Sigmoid(A(MF(x input ))) (8)

[0147] Map Avg =Median(AdaptivePool(x) Avg (9)

[0148] Where, x Avg It is the feature map after shallow channel attention processing; Map Avg It is a fourth spatial feature map of shallow space.

[0149] Map HL Map Avg and Map input Effective fusion of the three feature maps is crucial for spatial information feature extraction. This invention sets the spatial information of the three feature maps as a dataset with error samples, and uses the Hodges-Lehmann estimation method to dilute the error data, ultimately obtaining a more robust spatial feature map HL. spatial map The calculation method is shown in Equation 4 above.

[0150] The Sigmoid function is applied to the fifth spatial feature map for spatial attention weighting, as shown in Equation 10:

[0151] x output =x HL ×Sigmoid(HL spatial map (10)

[0152] Where, x output It is the small target attention weighted feature map after MFMC processing.

[0153] The specific uses and structure of the MFMC module are as follows: Figure 4 As shown.

[0154] Optionally, based on the median method, specific channel spatial features are extracted from the first spatial feature map to obtain a second spatial feature map, including:

[0155] The median method is used to filter channels in the first spatial feature map to obtain specific channels; the spatial information in the specific channels is used to represent the spatial information of the first spatial feature map.

[0156] Based on a specific channel, spatial features are extracted from the first spatial feature map to obtain the second spatial feature map.

[0157] In one feasible implementation, a median method is used to select a specific channel representing the spatial information of the feature map among all channels.

[0158] Compared to the maximum and minimum values, the median value is more representative of the general information of the feature map. The specific calculation formula is shown in Equation 11, where the feature map size changes from H×W×C to H×W×1:

[0159] Map HL =Median(AdaptivePool(x) HL (11)

[0160] Among them, Map HL It is the second spatial feature map after the fusion of multi-layer channel information.

[0161] S4. Obtain actual detection data; based on the actual detection data, use the detection accuracy improvement model to identify targets and obtain high-precision detection results.

[0162] In one feasible implementation, the HLRA proposed in this invention is a robust attention model that can eliminate error data and is plug-and-play. This model is an independent computational structure that can be adapted to various image classification and object detection models based on convolutional neural networks, and can be inserted and used in multiple quantities and locations. Because HLRA can process error data in image feature values ​​and reweight the values ​​within the feature map, it can enhance the overall feature learning ability of the model and improve the accuracy of the recognition results.

[0163] HLRA, as a standalone small model, can be inserted into various locations based on the actual structure of the baseline model. Before insertion, only the input and output feature map sizes and the number of channels of the HLRA need to be determined.

[0164] This invention uses the YOLOv5 object detection model as an example. This model has wide applications in the field of object detection and is recognized as an excellent detection model. The YOLOv5 model consists of three parts: a backbone network, a neck network, and a head network. The backbone network is composed of multiple convolutional layers (Conv), a multi-layer feature extraction module (C3), and a concatenated spatial pyramid (SPPF) layer for multi-layer information fusion. Its purpose is to extract features from the input image information and transform it into high-dimensional data. The neck network, immediately following the backbone network, is used to further learn and fuse the feature maps output by the backbone network that have already undergone feature extraction. The head network consists of convolutional layers and fully connected layers, used to ultimately predict the object's category and location information.

[0165] HLRA is inserted into the backbone network of YOLOv5 to further enhance the image learning and feature extraction capabilities of the YOLOv5 model, focusing more on the target object itself in feature learning. The model structure is as follows: Figure 5 As shown, the HLRA model is located between the 8th convolutional layer and the last SPPF layer in the backbone network.

[0166] according to Figure 5 The present invention provides a comparison of the specific network layer and channel number allocation of HLRA, as shown in Table 2 (HLRA-YOLOv5 model structure comparison table).

[0167] Table 2

[0168]

[0169]

[0170] In addition to the insertion method used in the above scheme, HLRA can also be inserted into the Neck layer in the YOLOv5 model. The position and number of insertions can be customized and selected according to actual needs.

[0171] This invention proposes a method to improve the accuracy of small target object recognition based on robust attention. It utilizes the proposed High-Level Attention Recognition Map (HLRA) to process feature maps generated at each stage of the model's computation, filtering these feature values ​​and extracting feature information in both channel and spatial dimensions through pooling operations. Redundant data is removed, maximizing the retention of key information and achieving attention processing in both spatial and channel dimensions. HLRA also uses the Sigmoid function to weight each feature value, assigning greater weight to important data and focusing the model's learning attention on the target object. Using a model with HLRA can enhance feature learning capabilities, further focusing on the target object and ultimately improving prediction accuracy. HLRA is an independent model structure that requires only a single, explicit feature map input. Therefore, HLRA can be inserted into most convolutional neural network-based image classification and object detection models, and can be inserted into different positions within different models, with the number of insertions controllable, thus achieving HLRA's versatility. HLRA uses pure numerical computations, such as max pooling and Hodges-Lehmann estimation, without relying on convolutional or fully connected layers for data filtering. Therefore, inserting this model does not increase the number of parameters in the original model, thereby minimizing the computational and deployment burden on the original model. This invention is a versatile method for improving the accuracy of small target object recognition based on robust attention.

[0172] Figure 6 This is a block diagram illustrating an apparatus for improving the accuracy of small target object recognition based on robust attention, according to an exemplary embodiment. The apparatus is used in a method for improving the accuracy of small target object recognition based on robust attention. (Refer to...) Figure 6 The device includes a model building module 610, a model integration module 620, a detection accuracy improvement module 630, and a detection result acquisition module 640. Among them:

[0173] The model building module 610 is used to build a robust attention model based on the Hodges-Lehmann estimation method using the attention module.

[0174] The model joint module 620 is used to add a robust attention model based on Hodges-Lehmann estimation to a preset convolutional neural network detection model to obtain a model to be trained to improve detection accuracy.

[0175] The detection accuracy improvement module 630 is used to acquire model training data; the model training data is used to train the detection accuracy improvement model to be trained, and the detection accuracy improvement model is obtained.

[0176] The detection result acquisition module 640 is used to acquire actual detection data; based on the actual detection data, target recognition is performed through the detection accuracy improvement model to obtain high-precision detection results.

[0177] Optionally, the detection accuracy improvement module 630 is further used for:

[0178] Feature extraction is performed on the model training data to obtain multi-dimensional feature maps;

[0179] Multi-dimensional feature maps are input into a robust attention model based on Hodges-Lehmann estimation for data fusion to obtain attention-weighted feature maps for small targets;

[0180] By using small target attention-weighted feature maps, a pre-defined convolutional neural network detection model is trained to obtain a model that improves detection accuracy.

[0181] Optionally, the detection accuracy improvement module 630 is further used for:

[0182] The multi-dimensional feature map is input into the multi-layer information correction module to obtain a multi-layer channel information fusion feature map.

[0183] The multi-channel information fusion feature map is input into the multi-layer feature map correction module to obtain the small target attention weighted feature map.

[0184] Optionally, the detection accuracy improvement module 630 is further used for:

[0185] Channel features are extracted from the multi-dimensional feature map to obtain the first channel feature map;

[0186] Feature extraction is performed on the first channel feature map to obtain the second channel feature map;

[0187] Based on the second channel feature map, the data is filtered using Hodges-Lehmann estimation to obtain the third channel feature map;

[0188] The guiding value is calculated based on the multi-dimensional feature map;

[0189] Based on the guiding value, the Sigmoid function is used to perform channel attention weighting on the third channel feature map to obtain a multi-channel information fusion feature map.

[0190] Optionally, the detection accuracy improvement module 630 is further used for:

[0191] Spatial information features are extracted from the multi-layer channel information fusion feature map to obtain the first spatial feature map;

[0192] Based on the median method, specific channel spatial features are extracted from the first spatial feature map to obtain the second spatial feature map;

[0193] Spatial features are extracted from the multi-dimensional feature map to obtain the third spatial feature map;

[0194] The median method and the sigmoid function are used to perform shallow spatial feature extraction on the multi-dimensional feature map to obtain the fourth spatial feature map.

[0195] Based on the second, third, and fourth spatial feature maps, the data is filtered using Hodges-Lehmann estimation to obtain the fifth spatial feature map.

[0196] The Sigmoid function is used to perform spatial attention weighting on the fifth spatial feature map to obtain the small target attention weighted feature map.

[0197] Optionally, the detection accuracy improvement module 630 is further used for:

[0198] The median method is used to filter channels in the first spatial feature map to obtain specific channels; the spatial information in the specific channels is used to represent the spatial information of the first spatial feature map.

[0199] Based on a specific channel, spatial features are extracted from the first spatial feature map to obtain the second spatial feature map.

[0200] Among them, the robust attention model based on Hodges-Lehmann estimation is a lightweight model.

[0201] This invention proposes a method to improve the accuracy of small target object recognition based on robust attention. It utilizes the proposed High-Level Attention Recognition Map (HLRA) to process feature maps generated at each stage of the model's computation, filtering these feature values ​​and extracting feature information in both channel and spatial dimensions through pooling operations. Redundant data is removed, maximizing the retention of key information and achieving attention processing in both spatial and channel dimensions. HLRA further weights feature values ​​through the Sigmoid function, assigning greater weight to important data and focusing the model's learning attention on the target object. Using a model with HLRA can enhance feature learning capabilities, further focusing on the target object and ultimately improving prediction accuracy. HLRA is an independent model structure that requires only a single, explicit feature map input. Therefore, HLRA can be inserted into most convolutional neural network-based image classification and object detection models, and can be inserted into different positions within different models, with the number of insertions controllable, thus achieving HLRA's versatility. HLRA uses pure numerical computations, such as max pooling and Hodges-Lehmann estimation, without relying on convolutional or fully connected layers for data filtering. Therefore, inserting this model does not increase the number of parameters in the original model, thereby minimizing the computational and deployment burden on the original model. This invention is a versatile method for improving the accuracy of small target object recognition based on robust attention.

[0202] Figure 7 This is a schematic diagram of the structure of a recognition accuracy improvement device provided in an embodiment of the present invention, as shown below. Figure 7 As shown, the recognition accuracy improvement device may include the above-mentioned Figure 6 The illustrated device is a small target object recognition accuracy improvement device based on robust attention. Optionally, the recognition accuracy improvement device 710 may include a first processor 2001.

[0203] Optionally, the recognition accuracy improvement device 710 may also include a memory 2002 and a transceiver 2003.

[0204] The first processor 2001, memory 2002, and transceiver 2003 can be connected via a communication bus.

[0205] The following is combined with Figure 7 A detailed introduction to each component of the recognition accuracy improvement device 710:

[0206] The first processor 2001 is the control center of the recognition accuracy improvement device 710. It can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0207] Optionally, the first processor 2001 can perform various functions of the recognition accuracy improvement device 710 by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.

[0208] In a specific implementation, as one example, the first processor 2001 may include one or more CPUs, for example... Figure 7 CPU0 and CPU1 are shown in the diagram.

[0209] In a specific implementation, as one example, the recognition accuracy improvement device 710 may also include multiple processors, for example... Figure 7 The first processor 2001 and the second processor 2004 are shown in the diagram. Each of these processors can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). Here, "processor" can refer to one or more devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).

[0210] The memory 2002 is used to store the software program that executes the present invention, and is controlled by the first processor 2001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.

[0211] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently, and may be connected to the interface circuit of the identification accuracy enhancement device 710. Figure 7 (Not shown in the figure) is coupled to the first processor 2001, and the embodiments of the present invention do not specifically limit this.

[0212] The transceiver 2003 is used to communicate with network devices or with terminal devices.

[0213] Alternatively, transceiver 2003 may include a receiver and a transmitter. Figure 7 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the sending function.

[0214] Optionally, the transceiver 2003 can be integrated with the first processor 2001 or exist independently, and its interface circuit can improve the recognition accuracy of the device 710. Figure 7 (Not shown in the figure) is coupled to the first processor 2001, and the embodiments of the present invention do not specifically limit this.

[0215] It should be noted that, Figure 7 The structure of the recognition accuracy enhancement device 710 shown in the diagram does not constitute a limitation on the router. Actual knowledge structure recognition devices may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0216] Furthermore, the technical effect of the recognition accuracy improvement device 710 can be referred to the technical effect of the small target object recognition accuracy improvement method based on robust attention described in the above method embodiments, and will not be repeated here.

[0217] It should be understood that the first processor 2001 in this embodiment of the invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0218] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0219] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0220] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0221] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0222] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0223] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0224] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0225] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0226] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0227] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0228] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0229] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for improving the accuracy of small target object recognition based on robust attention, characterized in that, The method includes: Based on the Hodges-Lehmann estimation method, an attention module is used to build the model, resulting in a robust attention model based on Hodges-Lehmann estimation. The robust attention model based on Hodges-Lehmann estimation is added to a preset convolutional neural network detection model to obtain a model for improving detection accuracy to be trained. Acquire model training data; train the detection accuracy improvement model to be trained using the model training data to obtain the detection accuracy improvement model, including: Feature extraction is performed on the training data of the model to obtain a multi-dimensional feature map; The multi-dimensional feature map is input into a robust attention model based on Hodges-Lehmann estimation for data fusion to obtain a small target attention-weighted feature map; The small target attention weighted feature map is used to train the preset convolutional neural network detection model to obtain a detection accuracy improvement model. The step of inputting the multi-dimensional feature map into a robust attention model based on Hodges-Lehmann estimation for data fusion to obtain a small target attention-weighted feature map includes: The multi-dimensional feature map is input into the multi-layer information correction module to obtain a multi-layer channel information fusion feature map. The multi-layer channel information fusion feature map is input into the multi-layer feature map correction module to obtain the small target attention weighted feature map; The step of inputting the multi-dimensional feature map into the multi-layer information correction module to obtain a multi-layer channel information fusion feature map includes: Channel feature extraction is performed on the multi-dimensional feature map to obtain the first channel feature map; Feature extraction is performed on the first channel feature map to obtain the second channel feature map; Based on the second channel feature map, the data is filtered using Hodges-Lehmann estimation to obtain the third channel feature map; The guiding value is obtained by calculating based on the multi-dimensional feature map; Based on the guiding value, the Sigmoid function is used to perform channel attention weighting on the third channel feature map to obtain a multi-layer channel information fusion feature map; The step of inputting the multi-layer channel information fusion feature map into the multi-layer feature map correction module to obtain the small target attention weighted feature map includes: Spatial information features are extracted from the multi-layer channel information fusion feature map to obtain a first spatial feature map; Based on the median method, specific channel spatial features are extracted from the first spatial feature map to obtain the second spatial feature map; Spatial feature extraction is performed on the multi-dimensional feature map to obtain a third spatial feature map; The median method and the Sigmoid function are used to perform shallow spatial feature extraction on the multi-dimensional feature map to obtain the fourth spatial feature map; Based on the second, third, and fourth spatial feature maps, data is filtered using Hodges-Lehmann estimation to obtain the fifth spatial feature map; The sigmoid function is used to perform spatial attention weighting on the fifth spatial feature map to obtain a small target attention-weighted feature map; Obtain actual detection data; based on the actual detection data, perform target recognition through the detection accuracy improvement model to obtain high-precision detection results.

2. The method for improving the accuracy of small target object recognition based on robust attention according to claim 1, characterized in that, The step of extracting specific channel spatial features from the first spatial feature map using the median method to obtain the second spatial feature map includes: The median method is used to filter channels in the first spatial feature map to obtain specific channels; the spatial information in the specific channels is used to represent the spatial information of the first spatial feature map. Based on the specific channel, spatial features are extracted from the first spatial feature map to obtain a second spatial feature map.

3. The method for improving the accuracy of small target object recognition based on robust attention according to claim 1, characterized in that, The robust attention model based on Hodges-Lehmann estimation is a lightweight model.

4. A device for improving the accuracy of small target object recognition based on robust attention, wherein the device is used to implement the method for improving the accuracy of small target object recognition based on robust attention as described in any one of claims 1-3, characterized in that, The device includes: The model building module is used to build a robust attention model based on the Hodges-Lehmann estimation method using the attention module. The model joint module is used to add the robust attention model based on Hodges-Lehmann estimation to a preset convolutional neural network detection model to obtain a model to be trained to improve detection accuracy. The detection accuracy improvement module is used to acquire model training data; and to train the detection accuracy improvement model to be trained using the model training data to obtain the detection accuracy improvement model. The detection result acquisition module is used to acquire actual detection data; based on the actual detection data, target recognition is performed through the detection accuracy improvement model to obtain high-precision detection results.

5. A device for improving recognition accuracy, characterized in that, The recognition accuracy improvement device includes: processor; A memory storing computer-readable instructions that, when executed by the processor, implement the method as described in any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1 to 3.

Images