A target detection method based on difference feature compensation under aggregated multi-scale information

Abstract

The application provides a target detection method based on differential feature compensation under polymeric multi-scale information, comprising constructing a multi-scale feature fusion module, selecting different receptive field convolution and various pooling methods to process input features to obtain differential features containing multi-scale information; constructing a parallel attention module to calculate spatial and channel attention maps in parallel, wherein various pooling methods are jointly used when inferring the attention map; combining the constructed multi-scale feature fusion module and parallel attention module in the best strategy to form a differential attention module for a feedforward convolutional neural network, and embedding the differential attention module into the network to obtain optimal performance. The application proposes a differential attention module (DAM) for a feedforward convolutional neural network, adopts a unique differential structure, and completes feature information compensation in the global through multi-scale feature fusion and attention mechanism to enhance the representation ability of features.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and more particularly to a target detection method based on difference feature compensation under aggregated multi-scale information. Background Technology

[0002] Convolutional Neural Networks (CNNs) have been widely used in image recognition. However, due to the explosive growth in depth and width of current CNNs, some image feature information is lost during training, affecting the image recognition results. Most existing techniques use attention mechanisms, which can enhance or suppress existing features at the pixel level, allowing the network to focus on more important regions. However, these mechanisms cannot reduce the impact of lost feature information on the network.

[0003] Image recognition aims to identify targets and objects in different patterns and has been widely applied in autonomous driving, machine vision, and biomedical image recognition. To effectively utilize CNNs for image recognition tasks, it's necessary to ensure the integrity of feature information during network flow. However, current mainstream networks typically evolve towards greater depth and breadth, inevitably leading to feature information loss and hindering accurate target category identification. Therefore, it's crucial to compensate for lost feature information at the pixel level to enhance the representational capabilities of the features.

[0004] Research indicates that the loss of feature information primarily stems from pooling layers and incomplete feature extraction. On one hand, deeper and wider networks contain excessive pooling layers. Pooling operations cause feature maps to lose some information, reducing the representational power of features and hindering the network's ability to better classify objects. Therefore, more training data is needed to compensate for the lost information. This compensation can be achieved by training interpolation attention without requiring additional training data. On the other hand, feature extraction layers in the network often fail to collect features effectively. Convolutions still miss features when processing them, leading to classifiers' inability to classify and regress objects effectively. While specialized convolutions are effective in this regard, most require fine-tuning and are therefore unsuitable for a wide range of feature extraction networks. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a target detection method based on difference feature compensation using aggregated multi-scale information. Specifically, it proposes a Differential Attention Module (DAM) for a feedforward convolutional neural network. Employing a unique difference structure, DAM utilizes multi-scale feature fusion and attention mechanisms to globally compensate for feature information, thereby enhancing feature representation capabilities.

[0006] The technical means employed in this invention are as follows:

[0007] A target detection method based on difference feature compensation under aggregated multi-scale information includes:

[0008] A multi-scale feature fusion module is constructed, and convolution with different receptive fields and multiple pooling methods are selected to process the input features to obtain difference features containing multi-scale information.

[0009] A parallel attention module is constructed to compute spatial and channel attention graphs in parallel. Multiple pooling methods are used in combination when inferring the attention graph.

[0010] The constructed multi-scale feature fusion module and parallel attention module are combined with the best strategy to form the difference attention module for feedforward convolutional neural networks. The difference attention module is then embedded into the network to obtain optimal performance.

[0011] Furthermore, the constructed multi-scale feature fusion module includes the use of 1×1 and 3×3 convolutions and Max & Avg joint pooling to increase the network's adaptability to scale and enhance the representation capability of features.

[0012] Furthermore, in the multi-scale feature fusion module, convolution with different receptive fields and various pooling methods are selected to process the input features, resulting in difference features containing multi-scale information, specifically including:

[0013] Given a local feature Local features are input into 1×1, 3×3, and 2*(3×3) convolutional layers and Max & Avg Pooling. The 2*(3×3) layer concatenates two 3×3 convolutions. A 1×1 convolutional layer is added before the 3×3 and 2*(3×3) convolutional layers. Max & Avg Pooling concatenates the results of max pooling and average pooling. A 1×1 convolutional layer is added after Max & Avg Pooling. This generates four new feature maps A, B, C, and D.

[0014] From F to F(i), the convolution operation is to... Mapped to In the transformation, V = [V1, V2, ..., V C ] represents the set of convolution kernels, where V c Let represent the parameters of the c-th filter; where:

[0015]

[0016] Where * denotes a convolution operation. This represents a 2D convolution kernel with an S-channel. It takes spatial features on one channel as input, learns spatial relationships between features, and sums the convolution results of each channel.

[0017] Simultaneously, the feature matrices of the generated receptive fields at different scales are concatenated along the channel dimension to obtain... As shown in formula (2):

[0018] E = Concatenate(A, B, C, D) (2)

[0019] Among them, E is more suitable for different scales while greatly reducing the number of weight parameters. Then, E is mapped to a 1×1 convolution. From equations (1) and (2), it can be inferred that G aggregates feature information at different scales and adaptively learns to assign weight values.

[0020] Furthermore, the construction of the parallel attention module computes spatial and channel attention maps in parallel, wherein multiple pooling methods are used in combination during attention map inference, specifically including:

[0021] Spatial information from feature maps is aggregated using average pooling, max pooling, and add operations to generate different spatial context descriptors: and These represent the average pooling feature, the max pooling feature, and the addition feature, respectively.

[0022] Three feature descriptors are input into a shared network, and channel attention maps are generated after fusion and activation functions. The shared network consists of a multilayer perceptron (MLP) with one hidden layer. To reduce parameter overhead, the hidden activation size is set to... Where r is the reduction rate;

[0023] After applying the shared network to each descriptor, the output feature vectors are merged using element-wise summation, and the channel attention is computed as follows:

[0024]

[0025] Where σ represents the Sigmoid function, W0 and W1 are the weights of the MLP shared by the two inputs;

[0026] To calculate spatial attention, Average pooling and max pooling operations are applied along the channel axis, then the Add operation is used to obtain the combined features, and finally the Concatenate operation is performed to connect them to generate an effective feature representation. Convolutional layers are applied to the feature description matrix of parallel connections to generate a spatial attention map M. s ∈R H×W Adaptive learning emphasizes or suppresses specific regions;

[0027] The channel information of the feature map is aggregated using convolution and Add operations to generate three 2D maps: and as well as Each channel represents the max-pooling feature, average-pooling feature, and combined feature; these are then concatenated and convolved through a standard convolutional layer to generate a 2D spatial attention map, where the spatial attention is calculated as follows:

[0028]

[0029] Where σ represents the Sigmoid function, f 7×7 This indicates a convolution operation with a filter size of 7×7.

[0030] Furthermore, the difference attention module includes a multi-scale feature fusion module and a parallel attention module connected in a serial manner, with the multi-scale feature fusion module having a higher priority than the parallel attention module. The multi-scale feature fusion module combines difference features of different scales and inputs them into the parallel attention module, which is responsible for generating the attention weights of the difference features.

[0031] Furthermore, the difference attention module can be flexibly embedded in existing CNN networks, that is, in operations between convolutions, using long-range dependencies to compensate for feature information. The difference attention module is plug-and-play and can function at any position in the CNN network.

[0032] Compared with the prior art, the present invention has the following advantages:

[0033] 1. The target detection method based on difference feature compensation under aggregated multi-scale information provided by the present invention uses multi-scale feature fusion and difference feature compensation mechanism to compensate for the lost detailed features in the network. First, features of different layers of multiple dimensions are aggregated and compressed. Aggregation allows the model to contain more scale information, and compression allows the model to adaptively learn the aggregated feature weights.

[0034] 2. The target detection method based on difference feature compensation under aggregated multi-scale information provided by this invention employs a difference structure and a parallel attention module structure for its difference feature compensation mechanism. The parallel attention module extracts more important feature weights at different scales from spatial and channel dimensions, and the difference structure performs weighted averaging of these features to achieve feature compensation, thereby enhancing the image recognition effect.

[0035] Based on the above reasons, this invention can be widely applied in fields such as image recognition. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 This is a flowchart of the difference attention module of the present invention.

[0038] Figure 2 This is a schematic diagram of the multi-scale feature fusion module of the present invention.

[0039] Figure 3 This is a schematic diagram of the Channel Attention Module and Spatial Attention Module in the differential attention module structure of this invention.

[0040] Figure 4 The Grad-CAM visualization results provided in the embodiments of the present invention.

[0041] Figure 5 The relationship between the performance of the difference structure and the network, as provided in this embodiment of the invention, compared with other technologies. Detailed Implementation

[0042] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0043] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the present invention or its application or use. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0044] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0045] Unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of the invention. It should also be understood that, for ease of description, the dimensions of the various parts shown in the drawings are not drawn to actual scale. Techniques, methods, and devices known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and devices should be considered part of the specification. In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters in the following figures denote similar items; therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0046] In the description of this invention, it should be understood that the orientation or positional relationship indicated by directional terms such as "front, back, up, down, left, right", "horizontal, vertical, horizontal" and "top, bottom" is generally based on the orientation or positional relationship shown in the accompanying drawings, and is only for the convenience of describing this invention and simplifying the description. Unless otherwise stated, these directional terms do not indicate or imply that the device or element referred to must have a specific orientation or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation on the scope of protection of this invention. The directional terms "inner" and "outer" refer to the inner and outer contours relative to the outline of each component itself.

[0047] For ease of description, spatial relative terms such as "above," "over," "on the upper surface of," "above," etc., are used herein to describe the spatial positional relationship of a device or feature as shown in the accompanying drawings to other devices or features. It should be understood that spatial relative terms are intended to encompass different orientations in use or operation besides the orientation of the device as described in the accompanying drawings. For example, if the device in the accompanying drawings were inverted, a device described as "above" or "above" other devices or structures would subsequently be positioned as "below" or "under" other devices or structures. Thus, the exemplary term "above" can include both "above" and "below." The device may also be positioned in other different ways (rotated 90 degrees or in other orientations), and the spatial relative descriptions used herein will be interpreted accordingly.

[0048] Furthermore, it should be noted that the use of terms such as "first" and "second" to define components is merely for the purpose of distinguishing the corresponding components. Unless otherwise stated, the above terms have no special meaning and therefore should not be construed as limiting the scope of protection of this invention.

[0049] This invention provides a target detection method based on difference feature compensation under aggregated multi-scale information, comprising:

[0050] A multi-scale feature fusion module is constructed, which selects convolution with different receptive fields and multiple pooling methods to process the input features and obtain difference features containing multi-scale information.

[0051] A parallel attention module is constructed to compute spatial and channel attention graphs in parallel. Multiple pooling methods are used in combination when inferring the attention graph.

[0052] The constructed multi-scale feature fusion module and parallel attention module are combined with an optimal strategy to form a difference attention module for feedforward convolutional neural networks. This difference attention module is then embedded into the network to achieve optimal performance. For example... Figure 1 The diagram shown is a flowchart of the difference attention module.

[0053] In specific implementation, as a preferred embodiment of the present invention, the method for difference feature compensation based on aggregated multi-scale information provided by the present invention can better compensate for lost feature information in the network, specifically including the following steps:

[0054] S1. The feature matrix flows through the network, constantly changing its weights. After passing through a certain convolutional layer, an N×N feature is obtained. This feature contains information from the previous layers of convolution, including color, texture, and other feature information. In this part, the present invention uses a mapping layer to map the N×N feature matrix obtained in the network, obtaining a new mapped feature layer. The size of this feature layer is fixed to ensure that the dimension of the feature matrix matches when performing interpolation calculations. It preserves more details without adding additional parameters.

[0055] S2. This model is located between any two convolutional layers L1 and L2 in the middle of the network model. Therefore, a difference structure is used to calculate the difference between matrices to obtain the difference information between L2 and L1. This information is used for subsequent multi-scale aggregation and adaptive difference attention. The method of directly calculating L1 and L2 avoids the loss of features before processing to the greatest extent, while obtaining rich difference feature information, which is beneficial to the next step of the model's calculation.

[0056] S3. After obtaining the difference information, the difference information is aggregated and compressed through a multi-scale aggregation module. In the multi-scale aggregation module, the input feature matrix H×W×C is first processed by convolutions of different sizes to obtain four different feature information matrices. The convolution sizes for the selected regions are 1×1, 3×3, 5×5, and Max&Avg, respectively. Then, the four different feature information matrices are concatenated, thus aggregating features from multiple scales to obtain a new high-dimensional feature matrix. Then, a 1×1 convolution is used to restore its dimension to the input size, i.e., H×W×C, for further processing.

[0057] S4. The restored H×W×C feature matrix enters the parallel attention module for adaptive learning of difference weights. The parallel attention module includes channel attention and spatial attention. Channel attention adaptively learns the weights of each dimension to obtain a 1×1×C weight matrix, where each weight represents information from each dimension. Spatial attention aggregates information from multiple spatial dimensions into one dimension to obtain an N×N×1 weight matrix. The two difference weight matrices obtained through the parallel attention module represent the weights of the lost feature information in the network, i.e., the importance of the lost information in different dimensions and spatial representations.

[0058] S5. After obtaining the channel and spatial difference weight matrices, the convolution L2 is weighted and summed with the weight matrix to compensate for the feature information lost during the transition from L1 to L2, thereby improving the image recognition effect.

[0059] According to the above scheme, when using the difference attention module in S2, the original network structure is not changed; that is, compensation for lost feature information is performed between L1 and L2. Mapping L1 to a matrix of a specific size is intended to enable direct matrix subtraction with the output L2 of the subsequent convolutional layer to obtain the difference in feature information, thus facilitating further optimization.

[0060] For an N×N feature matrix, which is the output of the L1 convolutional layer in the network model, the difference attention module of this invention performs loss feature compensation on the original basis. It can also select any two adjacent convolutional layers in the network model as L1 and L2. However, based on experience, the middle convolutional layer is generally selected because it contains more high-level features such as mid-to-high-level textures, which allows for better learning and compensation.

[0061] In a specific implementation, as a preferred embodiment of the present invention, according to the above scheme, step S3 specifically includes:

[0062] like Figure 2 As shown, given a local feature Local features are input into 1×1, 3×3, and 2*(3×3) convolutional layers and Max & Avg Pooling. The 2*(3×3) layer concatenates two 3×3 convolutions. A 1×1 convolutional layer is added before the 3×3 and 2*(3×3) convolutional layers. Max & Avg Pooling concatenates the results of max pooling and average pooling. A 1×1 convolutional layer is added after Max & Avg Pooling. This generates four new feature maps A, B, C, and D.

[0063] From F to F(i), the convolution operation is to... Mapped to In the transformation, V = [V1, V2, ..., V C ] represents the set of convolution kernels, where V c Let represent the parameters of the c-th filter; where:

[0064]

[0065] Where * denotes a convolution operation. This represents a 2D convolution kernel with an S-channel. It takes spatial features on one channel as input, learns the spatial relationships of features, and sums the convolution results of each channel. This method increases the network's adaptability to different scales.

[0066] Simultaneously, the feature matrices of the generated receptive fields at different scales are concatenated along the channel dimension to obtain... As shown in formula (2):

[0067] E = Concatenate(A, B, C, D) (2)

[0068] Among them, E is more suitable for different scales while greatly reducing the number of weight parameters. Then, E is mapped to a 1×1 convolution. Equations (1) and (2) suggest that G aggregates feature information at different scales and adaptively learns and assigns weight values. Therefore, it possesses multi-scale contextual information and selectively aggregates information based on its importance. The feature information at different scales achieves mutual benefit, thereby improving the network's adaptability to scale.

[0069] Channel Attention Module, for example Figure 3 As shown in Figure A, spatial information of the feature map is first aggregated using average pooling, max pooling, and Add operations to generate different spatial context descriptors: and These represent the average pooling feature, the max pooling feature, and the addition feature, respectively.

[0070] Three feature descriptors are input into a shared network, and channel attention maps are generated after fusion and activation functions. The shared network consists of a multilayer perceptron (MLP) with one hidden layer. To reduce parameter overhead, the hidden activation size is set to... Where r is the reduction rate;

[0071] After applying the shared network to each descriptor, the output feature vectors are merged using element-wise summation, and the channel attention is computed as follows:

[0072]

[0073] Where σ represents the Sigmoid function, W0 and W1 are the weights of the MLP shared by the two inputs;

[0074] To calculate spatial attention, Average pooling and max pooling operations are applied along the channel axis, then the Add operation is used to obtain the combined features, and finally the Concatenate operation is performed to connect them to generate an effective feature representation. Convolutional layers are applied to the feature description matrix of parallel connections to generate a spatial attention map M. s ∈R H×W Adaptive learning emphasizes or suppresses specific regions;

[0075] Spatial Attention Module, such as Figure 3 As shown in Figure B, this invention generates three 2D mappings by aggregating the channel information of the feature map through convolution and Add operations: as well as Each channel represents the max-pooling feature, average-pooling feature, and combined feature; these are then concatenated and convolved through a standard convolutional layer to generate a 2D spatial attention map, where the spatial attention is calculated as follows:

[0076]

[0077] Where σ represents the Sigmoid function, f 7×7 This indicates a convolution operation with a filter size of 7×7.

[0078] In a preferred embodiment of this invention, according to the above scheme, in step S5, the various modules are combined using the optimal strategy and embedded into the network to obtain optimal performance. The DAM consists of two sub-modules: a Multi-scale Feature Fusion Module and a parallel attention module. In this invention, the two sub-modules are connected serially, with the Multi-scale Feature Fusion Module having a higher priority than the parallel attention module. This connection method allows the network to adapt to multiple scales while reducing the number of additional parameters, and further refines features to obtain better attention weights. Furthermore, the parallel attention module includes channel attention and spatial attention, which are combined in parallel to obtain optimal performance. Overall, the Multi-scale Feature Fusion Module combines difference features at different scales and inputs them into the parallel attention module, which is responsible for generating the attention weights for the difference features.

[0079] Example

[0080] The effectiveness of the model design is demonstrated through experiments. In this ablation study, ResNet50 was used as the backbone network and evaluated on the ImageNet-1K dataset.

[0081] The ImageNet-1K classification dataset consists of 1.2 million images for training and 50,000 images representing 1,000 object classes for validation. Data augmentation was employed during training, and evaluation was performed using 224×224 images. The network in this invention had a learning rate of 1e-4, decreasing once every epoch, for a total of 230 epochs. The classification errors on the validation set are presented at the end.

[0082] The module design is divided into three parts: designing an effective structure for multi-scale fusion, the method of inference attention, and finally considering how to combine the Multi-scale Feature Fusion Module and the spatial attention module to achieve optimal performance. Details of each experiment will be described below.

[0083] The Multi-scale Feature Fusion Module demonstrates through experiments that combining average pooling and max pooling features achieves superior multi-scale feature fusion. This invention compares three variants of the Multi-scale Feature Fusion Module in experiments: average pooling, max pooling, and a combination of both. Notably, the Multi-scale Feature Fusion Module with average pooling is similar to the Inception-A module. Furthermore, when using joint pooling, the aggregated channel features reside in the same semantic space, and a bitwise addition operation is used to combine the two outputs to preserve parameters. In this experiment, ResNet50 is used as a benchmark, and different pooling operations are added to it to observe the impact on the experimental results. The experimental results of additional pooling are shown in Table 1.

[0084] Table 1: Comparison of different pooling methods in the Multi-scale Feature Fusion Module.

[0085]

[0086] As shown in Table 1, the method proposed in this invention outperforms other methods in the experiments. Both max pooling and average pooling features are meaningful, and their accuracy is improved compared to the baseline. However, in the Inception-A work, they only utilized average pooling features, without considering the importance of max pooling features. In this invention, using max pooling during multi-scale feature aggregation can effectively reduce the significant information loss caused by average pooling. Therefore, we attempted to use both methods simultaneously and apply these functions to a shared network, then output the feature connections through convolution. Experiments show that AvgPool & MaxPool is an effective method that can further improve the performance of the Multi-scale Feature Fusion Module without additional learnable parameters. In this section, this invention uses the AvgPool & MaxPool method in the Multi-scale Feature Fusion Module and continues to use it in subsequent experiments.

[0087] The Parallel Attention Module, building upon the Multi-scale Feature Fusion Module to enhance network scale adaptability, explores an efficient method for computational attention: the Parallel Attention Module. This module takes the same input features and simultaneously generates channel and spatial attention maps. For efficient attention map inference, a 2D descriptor is first computed, a matrix that encodes channel information at the pixel level for each location in the multi-dimensional space or channel. A convolutional layer is then applied to the 2D descriptor to obtain the original attention map. The final attention map is generated by normalization using the Sigmoid function.

[0088] The experiments compared several methods for generating 2D descriptors: average pooling, max pooling, and addition. Experiments were also designed to compare the priorities of the parallel attention module and the multi-scale feature fusion module, as the ultimate goal was to use these two modules in a reasonable combination. The comparative experiments are shown in Table 2.

[0089] Table 2: Comparison of different pooling methods and channel / spatial attention connection order based on Multi-scale Feature Fusion Module (MFFM)

[0090]

[0091] As shown in Table 2, the parallel generation of channel and spatial attention yields the best results, while the combined channel and spatial attention using three pooling methods performs even better. The parallel attention module has a lower priority than the Multi-scale Feature Fusion Module. The parallel attention generation module significantly reduces the error rate, indicating that explicit modeling of pooling leads to better attention inference, rather than relying on learnable weighted channel pooling. This invention compares methods using channel and spatial attention alone with methods combining parallel and sequential attention. It was found that, under the same conditions, using the parallel attention module produces better accuracy. This implies a need for a new attention computation method to select important regions of features. Considering this, a parallel channel and spatial dual attention module is used to compute attention. In short, the parallel attention module of this invention simultaneously computes channel and spatial dual attention and uses average pooling features, max pooling features, and combined features.

[0092] Arrangement of the Multi-scale Feature Fusion Module and Parallel Attention Module. This paper compares three different arrangements of the Multi-scale Feature Fusion Module and parallel attention sub-module: Multi-scale Feature Fusion Module-Parallel attention module, Parallel attention module-Multi-scale Feature Fusion Module, and serial and parallel connections of two attention modules. Each module has different functions, so different orders can lead to significant differences in overall performance. For example, from the perspective of parallel connection of two sub-modules, the final output cannot simultaneously inherit multi-scale features and the attention mechanism. Therefore, this invention proposes to choose an optimal strategy to combine the two modules to better capture the dependencies of the global context and compensate for lost feature information. In this case, a serial connection of the two sub-models is chosen. The difference features are enhanced by the Multi-scale Feature Fusion Module to improve the network's scale adaptability. Then, difference attention is generated in the parallel attention module and normalized using the Sigmoid function. In the experiments, using ResNet50+CBAM as a benchmark, the two sub-modules of DAM are arranged in different orders to obtain their optimal combination strategy.

[0093] Table 3: Combination methods of Multi-scale Feature Fusion Module (MFFM) and parallel attention module

[0094]

[0095] Table 3 summarizes the experimental results for different model permutation methods. It can be seen that using both structures is crucial, and the optimal combination strategy (i.e., sequential, channel-priority) further improves accuracy; the serial connection method performs better than the parallel one. Furthermore, the priority of the Multi-scale Feature Fusion Module is observed to be higher than that of the parallel attention module. Notably, all permutation methods outperform using only one model, indicating that utilizing both models is essential, and the optimal permutation strategy further improves performance.

[0096] Final module design. In ablation research, this invention designed a Multi-scale FeatureFusion Module and a parallel attention module, as well as the arrangement of these two modules. The final module of this invention is as follows: Figure 2 and Figure 3 As shown, the Multi-scale Feature Fusion Module selects average pooling and max pooling, and uses average pooling features, max pooling features, and addition features in the parallel attention module, arranging the DAM modules in a serial manner. The final module of this invention (i.e., ResNet50+DAM) achieves a top-1 error of 21.81%. After conducting the above three ablation experiments, the optimal strategy combination was selected, and the effect of ResNet50+DAM was visualized using Grad-CAM, as shown below. Figure 5 As shown. By Figure 5 Experimental results show that the visualization results of the DAM ensemble network cover the target object region better than those of ResNet50. In other words, the DAM ensemble network makes good use of the information in the target object region. Observations reveal that the feature compensation mechanism of DAM ultimately leads to the network acquiring richer feature information.

[0097] Table 4: Classification results on ImageNet-1K.

[0098]

[0099]

[0100] Table 4 presents an example of the results achieved by this invention on the ImageNet-1K dataset. As shown in Table 4, the DAM ensemble network significantly outperforms all baselines, indicating that DAM can better adapt to various large-scale datasets compared to other models. Furthermore, the accuracy of the DAM ensemble network is significantly improved compared to SimAM, one of the representative attention mechanisms published at ICML 2021. This means that the method proposed in this invention is powerful, demonstrating the effectiveness of the difference structure, which can better compensate for the detailed information lost and enhance the representational power of features.

[0101] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A target detection method based on difference feature compensation under aggregated multi-scale information, characterized in that, The target detection method belongs to the image recognition technical field, and comprises the following steps: A multi-scale feature fusion module is constructed, and different receptive field convolutions and multiple pooling methods are selected to process input features to obtain difference features containing multi-scale information; in the multi-scale feature fusion module, different receptive field convolutions and multiple pooling methods are selected to process input features to obtain difference features containing multi-scale information, and the method specifically comprises the following steps: Given a local feature , input 1 , input 3 , input 2 , input (3 ) is the convolution layer and Max&Avg Pooling of 3 , 2 , 3 , 3 , 3 , 2 , 3 , 1 , 1 , wherein, From F to F(i), where the convolution operation is to convolve with , and map onto , which represents a set of convolution kernels, where represents the parameters of the cth filter; where: （1） wherein, denotes a convolution operation, denotes a 2-D convolution kernel of an S Channel, inputting spatial features on one channel, learning feature spatial relationship, and summing convolution results of each channel; Meanwhile, the feature matrix of the generated different scale receptive fields is connected in the channel dimension to obtain a feature matrix of the generated different scale receptive fields As shown in formula (2): （2） wherein, More suitable for different scales while greatly reducing the number of weight parameters, then through a 1x1 convolution mapped to ; from equation (1), (2) deduced that G aggregates different scale feature information and adaptively learns the weight value; A parallel attention module is constructed to calculate spatial and channel attention maps in a parallel manner, wherein multiple pooling methods are jointly used when the attention map is inferred, and the method comprises the following steps: By using average pooling, max pooling and Add operation to aggregate the spatial information of the feature map, different spatial context descriptors are generated: , and , respectively represent the average pooling feature, the maximum pooling feature and the addition feature; The three feature descriptors are input into a shared network, and a channel attention map is generated through fusion and an activation function The shared network is composed of a multilayer perceptron with one hidden layer, and the size of hidden activation is set to , wherein is the reduction rate; After the shared network is applied to each descriptor, an element-wise summation is used to combine output feature vectors, and channel attention is calculated as: where, denotes a Sigmoid function, , , and are weights of the two-input shared MLP; To calculate the spatial attention, the , apply average pooling and max pooling operations along the channel axis, then use the Add operation to obtain the combined features, and concatenate them to generate an effective feature representation ; on the parallelly connected feature description matrix, apply a convolution layer to generate a spatial attention map , adaptive learning of the area to be emphasized or suppressed; The channel information of the feature map is aggregated using convolution and Add operation to generate three 2D maps: and and Each represents the maximum-pooled feature and the average-pooled feature in the channel and the combined feature; then they are connected and convolved by a standard convolution layer to generate a 2D spatial attention map, and the spatial attention is calculated as: wherein, denotes a Sigmoid function, denotes a convolution operation with a filter size of 7 7. The constructed multi-scale feature fusion module and parallel attention module are combined in an optimal strategy to form a difference attention module for a feedforward convolutional neural network, and the difference attention module is embedded in the network to obtain optimal performance.

2. The target detection method based on poly-scale information and difference feature compensation according to claim 1, characterized in that, The constructed multi-scale feature fusion module comprises the use of 1x1 and 3x3 convolutions and Max&Avg joint pooling, which increases the adaptability of the network to scales and enhances the representation ability of features.

3. The method of claim 1, wherein the method is based on poly-scale information and under-difference feature compensation for object detection. The difference attention module comprises a multi-scale feature fusion module and a parallel attention module connected in a serial manner, and the priority of the multi-scale feature fusion module is higher than that of the parallel attention module, the multi-scale feature fusion module combines difference features of different scales and inputs them to the parallel attention module, and the parallel attention module is responsible for generating attention weights of the difference features.

4. The method of claim 1, wherein the target detection is based on a difference feature compensation using aggregated multi-scale information. The difference attention module is flexibly embedded in the current CNN network, that is, between convolutions, and utilizes long-range dependencies to compensate for feature information, and the difference attention module is plug-and-play, and can play a role at any position in the CNN network.

Images