A local feature amplification target detection method and device based on an attention mechanism

Abstract

This invention discloses a method and apparatus for target detection with local feature magnification based on an attention mechanism. The method includes: determining the weight coefficients of each channel in the current detection image based on a pre-trained target detection model using a channel attention mechanism, and enhancing the feature map of the current detection image based on the weight coefficients; the spatial attention layer of the target detection model generates a weight matrix for each spatial location corresponding to the enhanced feature map, and converts the weight matrix into spatial coordinate feature information; normalizing the spatial coordinate feature information to obtain normalized spatial coordinate features and constructing an affine transformation matrix; performing a spatial transformation on the feature map based on the affine transformation matrix, and using the generated grid to perform linear interpolation on the spatially transformed feature map to obtain a magnified local image. This method can locally magnify the image, thus enabling accurate and comprehensive identification of targets in the captured image.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and apparatus for target detection based on attention mechanism and local feature magnification. Background Technology

[0002] Camera surveillance systems are an important component of modern security management. These systems use AI algorithms to analyze video content in real time, extracting useful information such as faces and license plate numbers for monitoring, alarm functions, and historical video playback. They can be widely used in public safety, traffic monitoring, and commercial settings, enabling real-time detection and identification of potential security risks.

[0003] Existing camera surveillance systems suffer from low camera resolution, complex shooting scenes, and some targets being too small in the images, leading to numerous false positives and false negatives during subsequent AI image recognition. Therefore, there is an urgent need for those skilled in the art to provide a target detection solution capable of accurately and comprehensively identifying targets in captured images. Summary of the Invention

[0004] The purpose of this invention is to provide a method and apparatus for target detection based on attention mechanism and local feature magnification, which can solve the problem in the prior art that it is impossible to accurately and comprehensively identify targets in captured images.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0006] This invention provides a method for target detection based on attention mechanism with local feature magnification, wherein the method includes:

[0007] Based on a pre-trained target detection model and a channel attention mechanism, the weight coefficients of each channel in the current detection image are determined, and the feature map of the current detection image is enhanced based on the weight coefficients.

[0008] The spatial attention layer of the target detection model generates a weight matrix for each spatial location corresponding to the enhanced feature map, and transforms the weight matrix into spatial coordinate feature information; wherein, the spatial coordinate feature information can be used to crop the current detection image;

[0009] The spatial coordinate feature information is normalized to obtain the normalized spatial coordinate features;

[0010] The affine transformation matrix is ​​constructed using the normalized spatial coordinate features.

[0011] The feature map is spatially transformed based on the affine transformation matrix, and the generated grid is used to perform linear interpolation on the spatially transformed feature map to obtain a magnified local image.

[0012] Optionally, the step of determining the weight coefficients of each channel in the current detection image based on a pre-trained object detection model using a channel attention mechanism, and then performing feature map enhancement processing on the current detection image based on the weight coefficients, includes:

[0013] Along the spatial dimension, the two-dimensional feature map of each channel of the current detection image is transformed into a single scalar through global average pooling and global max pooling layers, and the spatial information of each channel is aggregated into a global description;

[0014] Global average pooling and max pooling are performed on the feature map corresponding to the input current detection image;

[0015] Based on the nonlinear relationship between channels, a first weight is generated for each channel;

[0016] The first weights are normalized using the Sigmoid function to obtain a normalized weight vector.

[0017] The normalized weight vector is multiplied by the feature map corresponding to the current detection image to obtain the enhanced feature map.

[0018] Optionally, the step of the spatial attention layer of the target detection model generating a weight matrix for each spatial location corresponding to the enhanced feature map and converting the weight matrix into spatial coordinate feature information includes:

[0019] The enhanced feature maps are then subjected to max pooling and average pooling respectively to obtain average pooling feature maps and max pooling feature maps.

[0020] The average pooling feature map and the max pooling feature map are concatenated along the channel dimension to obtain a concatenated feature map.

[0021] The relationship between different locations in the spliced ​​feature map space is learned by using convolution operations to generate a preliminary spatial attention map;

[0022] The spatial attention map is normalized, and then a convolution operation is performed on the normalized spatial attention map to transform the weight features into spatial coordinate features.

[0023] Optionally, the normalized spatial coordinate features include the following parameter set:

[0024] X-axis translation, Y-axis translation, image width, and image height.

[0025] Optionally, the step of constructing the affine transformation matrix using the normalized spatial coordinate features includes:

[0026] Determine the scaling parameters;

[0027] Based on the normalized spatial coordinate characteristics and the scaling parameters, determine the scaling factor in the X direction and the scaling factor in the Y direction;

[0028] The translation amounts in the X and Y directions are determined based on the normalized spatial coordinate characteristics.

[0029] An affine transformation matrix is ​​constructed based on the normalized spatial coordinate characteristics, the X-direction scaling factor, the Y-direction scaling factor, the X-direction translation, and the Y-direction translation.

[0030] This invention also provides a target detection device based on attention mechanism with local feature magnification, wherein the device includes:

[0031] The determination module is used to determine the weight coefficients of each channel in the current detection image based on the channel attention mechanism of the pre-trained target detection model, and to perform enhancement processing on the feature map of the current detection image based on the weight coefficients.

[0032] The conversion module is used to call the spatial attention layer of the target detection model to generate a weight matrix for each spatial location corresponding to the feature map of the enhancement processing, and convert the weight matrix into spatial coordinate feature information; wherein, the spatial coordinate feature information can be used to crop the current detection image;

[0033] The normalization module is used to normalize the spatial coordinate feature information to obtain normalized spatial coordinate features.

[0034] The construction module is used to construct an affine transformation matrix using the normalized spatial coordinate features;

[0035] The processing module is used to perform spatial transformation on the feature map based on the affine transformation matrix, and to perform linear interpolation on the spatially transformed feature map using the generated grid to obtain a magnified local image.

[0036] Optionally, the determining module includes:

[0037] The first submodule is used to transform the two-dimensional feature map of each channel of the current detection image into a single scalar by passing through the global average pooling layer and the global max pooling layer along the spatial dimension, and to aggregate the spatial information of each channel into a global description.

[0038] The second submodule is used to perform global average pooling and max pooling on the feature map corresponding to the input current detection image;

[0039] The third submodule is used to generate a first weight for each channel based on the nonlinear relationship between the channels.

[0040] The fourth submodule is used to normalize each of the first weights using the Sigmoid function to obtain a normalized weight vector.

[0041] The fifth submodule is used to perform channel multiplication operation between the normalized weight vector and the feature map corresponding to the current detection image to obtain the enhanced feature map.

[0042] Optionally, the conversion module is specifically used for:

[0043] The enhanced feature maps are then subjected to max pooling and average pooling respectively to obtain average pooling feature maps and max pooling feature maps.

[0044] The average pooling feature map and the max pooling feature map are concatenated along the channel dimension to obtain a concatenated feature map.

[0045] The relationship between different locations in the spliced ​​feature map space is learned by using convolution operations to generate a preliminary spatial attention map;

[0046] The spatial attention map is normalized, and then a convolution operation is performed on the normalized spatial attention map to transform the weight features into spatial coordinate features.

[0047] Optionally, the normalized spatial coordinate features include the following parameter set:

[0048] X-axis translation, Y-axis translation, image width, and image height.

[0049] Optionally, the normalization module is specifically used for:

[0050] Determine the scaling parameters;

[0051] Based on the normalized spatial coordinate characteristics and the scaling parameters, determine the scaling factor in the X direction and the scaling factor in the Y direction;

[0052] The translation amounts in the X and Y directions are determined based on the normalized spatial coordinate characteristics.

[0053] An affine transformation matrix is ​​constructed based on the normalized spatial coordinate characteristics, the X-direction scaling factor, the Y-direction scaling factor, the X-direction translation, and the Y-direction translation.

[0054] This invention provides an electronic device, which includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the steps of any of the above-described attention-based local feature amplification target detection methods.

[0055] This invention provides a readable storage medium storing a program or instructions, which, when executed by a processor, implement the steps of any of the above-described attention-based local feature amplification target detection methods.

[0056] This application provides a local feature magnification target detection scheme based on an attention mechanism. It utilizes a pre-trained target detection model with a channel attention mechanism to determine the weight coefficients of each channel in the current detection image and enhances the feature map of the current detection image based on these weight coefficients. The spatial attention layer of the target detection model generates a weight matrix for each spatial location corresponding to the enhanced feature map and transforms the weight matrix into spatial coordinate feature information. The spatial coordinate feature information is then normalized to obtain normalized spatial coordinate features. An affine transformation matrix is ​​constructed using the normalized spatial coordinate features. A spatial transformation is performed on the feature map based on the affine transformation matrix, and the generated grid is used to perform linear interpolation on the spatially transformed feature map to obtain the magnified local image. This scheme, on the one hand, guides the model to magnify the spatial region through channel attention, enabling local magnification of the detection image. Because of this local magnification, accurate and comprehensive target identification in the captured image is possible. On the other hand, the modular design allows for flexible adjustment of the magnification module's position within the overall structure. Attached Figure Description

[0057] Figure 1 This is a flowchart illustrating the steps of a local feature amplification target detection method based on an attention mechanism according to an embodiment of this application;

[0058] Figure 2 This is a schematic diagram illustrating the structure of a target detection model according to an embodiment of this application;

[0059] Figure 3 This is a schematic diagram illustrating the hierarchical relationships within the Zoom module in an embodiment of this application;

[0060] Figure 4 This is a schematic diagram illustrating the structure of the Channel_Attention layer in an embodiment of this application;

[0061] Figure 5 This is a schematic diagram illustrating the SpatialLocator layer structure according to an embodiment of this application;

[0062] Figure 6 This is a block diagram illustrating the structure of a local feature magnification target detection device based on an attention mechanism according to an embodiment of this application. Detailed Implementation

[0063] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0064] The attention-based target detection scheme based on local feature magnification provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0065] As attached Figure 1 As shown, the local feature magnification target detection method based on attention mechanism in this application includes the following steps:

[0066] Step 101: Based on the pre-trained target detection model and the channel attention mechanism, determine the weight coefficients of each channel in the current detection image, and perform enhancement processing on the feature map of the current detection image based on the weight coefficients.

[0067] The object detection model used in this embodiment can be based on YOLO (You Only Look Once), a popular object detection and image segmentation model. In this embodiment, a local feature amplification module is added to the YOLO feature extraction module, based on YOLO, thereby enabling the through-channel attention-guided model to amplify the spatial region. A schematic diagram of the object detection model is shown below. Figure 2 As shown, the object detection model contains multiple Zoom modules. A schematic diagram of the hierarchical relationships within each Zoom module is shown below. Figure 3As shown, the Zoom module includes: a Channel_Attention layer, a SpatialLocator layer, a Normalize layer, an Affine layer, and a Grid layer. The Affine layer in the neural network performs matrix multiplication, which is called an "affine transformation" in geometry. During the forward propagation of the neural network, the Affine layer is responsible for linear transformations and translations, corresponding to weighted sums and bias operations. The computation of the Affine layer involves not only single data points but also batches of data. In actual implementation, step 101 can be executed by the Channel_Attention layer, step 102 by the SpatialLocator layer, step 103 by the Normalize layer, step 104 by the Affine layer, and step 105 by the Grid layer. However, this is not a limitation; the above layers can be integrated or further divided.

[0068] The attention-based local feature magnification target detection method provided in this application can be applied to electronic devices, which include a processor and a storage medium. The storage medium stores a computer program related to the attention-based local feature magnification target detection method, and the processor of the electronic device executes the computer program to implement the attention-based local feature magnification target detection process.

[0069] In an optional embodiment, the method of determining the weight coefficients of each channel in the current detection image based on a pre-trained object detection model using a channel attention mechanism, and then enhancing the feature map of the current detection image based on these weight coefficients, may include the following sub-steps:

[0070] Sub-step 1: Transform the two-dimensional feature map of each channel of the current detection image into a single scalar by passing through the global average pooling layer and the global max pooling layer along the spatial dimension, and aggregate the spatial information of each channel into a global description;

[0071] Sub-step 2: Perform global average pooling and max pooling on the feature map corresponding to the input current detection image;

[0072] Sub-step 3: Based on the nonlinear relationship between channels, generate a first weight for each channel;

[0073] Sub-step 4: Normalize each first weight using the Sigmoid function to obtain the normalized weight vector;

[0074] Sub-step 5: Perform channel multiplication on the normalized weight vector and the feature map corresponding to the current detection image to obtain the enhanced feature map.

[0075] This optional embodiment can be illustrated by a structural diagram of the Channel_Attention layer, as shown below. Figure 4 As shown, the Channel_Attention layer primarily determines which feature channels are important. The model needs to know which feature channels are important and which are secondary in the current task. The channel attention mechanism generates a weight coefficient for each channel; the larger the weight, the more important the feature of that channel. Feature channels can be understood as different filters; some are responsible for detecting edges, some for detecting textures, and some for detecting colors.

[0076] The specific execution steps of the Channel_Attention layer are as follows:

[0077] 1) Along the spatial dimension, through global average pooling and global max pooling, the two-dimensional feature map of each channel is transformed into a single scalar, and the spatial information of each channel is aggregated into a global description, representing the overall activation level of the channel.

[0078] 2) For the input feature map C×H×W, X∈R, perform global average pooling and max pooling:

[0079]

[0080] Where Xavg represents average pooling, Xmax represents max pooling; C represents the number of input channels of the feature map, H represents the height of the feature map, and W represents the width of the feature map.

[0081] 3) Multilayer perceptron (MLP) with shared parameters generates an initial importance weight for each channel by learning the nonlinear relationship between the channels.

[0082] MLP(X avg ) = W2·ReLU(W1·X avg )

[0083] MLP(X max ) = W2·ReLU(W1·X max )

[0084] Here, W1 and W2 represent the weights between different layers of the multilayer perceptron, and ReLU is an activation function.

[0085] 4) Normalize the value from the previous step to between 0 and 1 using a Sigmoid function.

[0086] Specifically, normalization can be performed using the following formula:

[0087] α=σ(MLP(X avg)+MLP(X max ))∈R C×1×1

[0088] 5) Multiply the normalized weight vector (of shape C, 1, 1) channel-by-channel with the original input feature map to enhance the feature responses of important channels and suppress the feature responses of unimportant channels. Specifically, the channel-by-channel multiplication can be calculated using the following formula:

[0089] X out =α⊙X

[0090] Where X is the original input feature map, X out Feature maps after enhancing the characteristic responses of important channels.

[0091] Step 102: The spatial attention layer of the object detection model generates a weight matrix for each spatial location corresponding to the enhanced feature map, and transforms the weight matrix into spatial coordinate feature information.

[0092] Among them, spatial coordinate feature information can be used to crop the currently detected image.

[0093] Step 102 can be executed by the SpatialLocator layer of the object detection module. The SpatialLocator layer primarily determines which regions are important after focusing on the important feature channels. The model then needs to determine which regions require attention in the spatial plane, i.e., the Height-Width plane. The spatial attention layer generates a weight matrix for each spatial location; a larger weight indicates greater importance. This matrix is ​​then converted into corresponding spatial coordinates for subsequent cropping operations. A schematic diagram of the SpatialLocator layer is shown below. Figure 5 As shown.

[0094] In one optional embodiment, the spatial attention layer of the object detection model generates a weight matrix for each spatial location corresponding to the enhanced feature map, and the method of converting the weight matrix into spatial coordinate feature information may include the following sub-steps:

[0095] Sub-step 1: Perform max pooling and average pooling on the enhanced feature maps to obtain average pooling feature maps and max pooling feature maps, respectively.

[0096] By employing max pooling and average pooling along the channel dimension, rich channel information is compressed into two 2D maps. These two 2D maps represent the average and maximum response intensities at each location across all channels, respectively, providing powerful clues for generating the spatial attention map. In practical implementation, the average pooling feature map and max pooling feature map can be calculated using the following formulas:

[0097]

[0098] in, Average pooling represents the spatial dimension; This represents max pooling in the spatial dimension; C is the number of channels in the feature map.

[0099] Sub-step 2: Concatenate the average pooling feature map and the max pooling feature map along the channel dimension to obtain the concatenated feature map.

[0100] Specifically, feature maps can be stitched together using the following formula:

[0101]

[0102] Concat means concatenating the pooling results from the above steps along the channel dimension.

[0103] Sub-step 3: Use convolution operations to learn the relationships between different locations in the spliced ​​feature map space and generate a preliminary spatial attention map.

[0104] In one feasible implementation, a preliminary spatial attention map can be generated using the following formula:

[0105] M = Conv(X) concat )∈R 1×R×W

[0106] Here, Conv represents the convolution operation.

[0107] Sub-step 4: Normalize the spatial attention map, and then perform a convolution operation on the normalized spatial attention map to transform the weight features into spatial coordinate features.

[0108] Normalization can be achieved using the Sigmoid function, which can be described as follows:

[0109] M = σ(M)

[0110] In the specific implementation, performing a convolution operation can transform the weighted features into n 4x1 coordinate features.

[0111] [cx,cy,w,h]=Conv coord (M)

[0112] Among them, Conv coord This represents a convolution operation that transforms the coordinates of the features; cx, cy, w, h represent the x and y coordinates, width, and height of the transformed center.

[0113] Step 103: Normalize the spatial coordinate feature information to obtain the normalized spatial coordinate features.

[0114] The normalized spatial coordinate features include the following parameter set: X-direction translation cx, Y-direction translation cy, image width w, and image height h.

[0115] In actual implementation, step 103 can be executed by the Normalize layer. This Normalize layer normalizes the coordinate features obtained from the SpatialLocator layer to obtain the parameters [cx,cy,w,h], which are the normalized spatial coordinate features. Specifically, the coordinate features can be normalized using the following formula:

[0116] cx,cy,w,h=Normalize([cx,cy,w,h])

[0117] Step 104: Construct an affine transformation matrix using the normalized spatial coordinate features.

[0118] Step 104 can be performed through the Affine layer. In an alternative embodiment, constructing the affine transformation matrix using normalized spatial coordinate features may include the following sub-steps:

[0119] Sub-step 1: Determine the scaling parameters;

[0120] Sub-step 2: Based on the normalized spatial coordinate characteristics and scaling parameters, determine the scaling factor in the X direction and the scaling factor in the Y direction;

[0121] Sub-step 3: Determine the translation amount in the X direction and the translation amount in the Y direction based on the normalized spatial coordinate characteristics;

[0122] Sub-step 4: Construct an affine transformation matrix based on the normalized spatial coordinate characteristics, the scaling factor in the X direction, the scaling factor in the Y direction, the translation in the X direction, and the translation in the Y direction.

[0123] One feasible implementation is to construct an affine transformation matrix using normalized coordinate features, where s = scale_factor.

[0124]

[0125] Set the scaling factor in the x-direction: w*scale_factor

[0126] y-axis scaling factor h*scale_factor

[0127] The scaling parameter, scale_factor, can be adjusted as needed.

[0128] x-direction translation amount cx

[0129] y-direction translation cy

[0130] Step 105: Perform spatial transformation on the feature map based on the affine transformation matrix, and use the generated grid to perform linear interpolation on the spatially transformed feature map to obtain the magnified local image.

[0131] Step 105 can be executed through the Grid layer. More specifically, it can be performed through the Affine_Grid layer, which transforms the feature map spatially based on the affine transformation matrix generated in the previous layer. This can be achieved using the following formula:

[0132] Grid = AffineGrid(θ)

[0133] Image magnification is achieved by linearly interpolating the input feature map using the generated grid through the Grid_Sample layer. This can be achieved using the following formula:

[0134] X zoomed =GridSample(X,Grid)

[0135] The changes in these two layers can create a differentiable image cropping and magnification module, enabling the model to perform backpropagation.

[0136] The local feature magnification target detection method based on attention mechanism provided in this application embodiment determines the weight coefficients of each channel in the current detection image based on a pre-trained target detection model using channel attention mechanism, and enhances the feature map of the current detection image based on the weight coefficients. The spatial attention layer of the target detection model generates a weight matrix for each spatial location corresponding to the enhanced feature map, and transforms the weight matrix into spatial coordinate feature information. The spatial coordinate feature information is normalized to obtain normalized spatial coordinate features. An affine transformation matrix is ​​constructed using the normalized spatial coordinate features. The feature map is spatially transformed based on the affine transformation matrix, and the generated grid is used to perform linear interpolation on the spatially transformed feature map to obtain the magnified local image. The solution provided in this application has two advantages: firstly, by guiding the model to magnify the spatial region through channel attention, the detection image can be locally magnified. Since the image can be locally magnified, the target in the captured image can be accurately and comprehensively identified; secondly, the modular design allows for flexible adjustment of the position of the magnification module in the entire structure.

[0137] Figure 6 The structural block diagram of a local feature magnification target detection device based on an attention mechanism according to an embodiment of this application is shown.

[0138] The local feature magnification target detection device based on attention mechanism provided in this application includes the following functional modules;

[0139] The determination module 601 is used to determine the weight coefficients of each channel in the current detection image based on the channel attention mechanism of the pre-trained target detection model, and to perform enhancement processing on the feature map of the current detection image based on the weight coefficients.

[0140] The conversion module 602 is used to call the spatial attention layer of the target detection model to generate a weight matrix for each spatial location corresponding to the feature map of the enhancement processing, and convert the weight matrix into spatial coordinate feature information; wherein, the spatial coordinate feature information can be used to crop the current detection image;

[0141] The normalization module 603 is used to normalize the spatial coordinate feature information to obtain normalized spatial coordinate features.

[0142] Construction module 604 is used to construct an affine transformation matrix using the normalized spatial coordinate features;

[0143] The processing module 605 is used to perform spatial transformation on the feature map based on the affine transformation matrix, and to perform linear interpolation on the spatially transformed feature map using the generated grid to obtain a magnified local image.

[0144] Optionally, the determining module includes:

[0145] The first submodule is used to transform the two-dimensional feature map of each channel of the current detection image into a single scalar by passing through the global average pooling layer and the global max pooling layer along the spatial dimension, and to aggregate the spatial information of each channel into a global description.

[0146] The second submodule is used to perform global average pooling and max pooling on the feature map corresponding to the input current detection image;

[0147] The third submodule is used to generate a first weight for each channel based on the nonlinear relationship between the channels.

[0148] The fourth submodule is used to normalize each of the first weights using the Sigmoid function to obtain a normalized weight vector.

[0149] The fifth submodule is used to perform channel multiplication operation between the normalized weight vector and the feature map corresponding to the current detection image to obtain the enhanced feature map.

[0150] Optionally, the conversion module is specifically used for:

[0151] The enhanced feature maps are then subjected to max pooling and average pooling respectively to obtain average pooling feature maps and max pooling feature maps.

[0152] The average pooling feature map and the max pooling feature map are concatenated along the channel dimension to obtain a concatenated feature map.

[0153] The relationship between different locations in the spliced ​​feature map space is learned by using convolution operations to generate a preliminary spatial attention map;

[0154] The spatial attention map is normalized, and then a convolution operation is performed on the normalized spatial attention map to transform the weight features into spatial coordinate features.

[0155] Optionally, the normalized spatial coordinate features include the following parameter set:

[0156] X-axis translation, Y-axis translation, image width, and image height.

[0157] Optionally, the normalization module is specifically used for:

[0158] Determine the scaling parameters;

[0159] Based on the normalized spatial coordinate characteristics and the scaling parameters, determine the scaling factor in the X direction and the scaling factor in the Y direction;

[0160] The translation amounts in the X and Y directions are determined based on the normalized spatial coordinate characteristics.

[0161] An affine transformation matrix is ​​constructed based on the scaling factor, the scaling factor in the X direction, the scaling factor in the Y direction, the translation in the X direction, and the translation in the Y direction.

[0162] The embodiments provided in this application Figure 6 The attention-based local feature magnification target detection device shown can achieve... Figure 1 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0163] The local feature magnification target detection device based on the attention mechanism provided in this application embodiment determines the weight coefficients of each channel in the current detection image based on a pre-trained target detection model using a channel attention mechanism, and enhances the feature map of the current detection image based on the weight coefficients. The spatial attention layer of the target detection model generates a weight matrix for each spatial location corresponding to the enhanced feature map, and transforms the weight matrix into spatial coordinate feature information. The spatial coordinate feature information is normalized to obtain normalized spatial coordinate features. An affine transformation matrix is ​​constructed using the normalized spatial coordinate features. The feature map is spatially transformed based on the affine transformation matrix, and the generated grid is used to perform linear interpolation on the spatially transformed feature map to obtain the magnified local image. The solution provided in this application, on the one hand, can magnify the spatial region by guiding the model through channel attention, thus enabling local magnification of the detection image. Since the image can be locally magnified, the target in the captured image can be accurately and comprehensively identified. On the other hand, the modular design allows for flexible adjustment of the position of the magnification module in the entire structure.

[0164] Optionally, embodiments of this application also provide an electronic device, including a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the processes executed by the aforementioned attention-based local feature amplification target detection device and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0165] Memory, used to store computer programs;

[0166] When the processor executes the program stored in the memory, it implements the local feature amplification target detection method based on the attention mechanism shown in the above method embodiments.

[0167] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0168] The communication interface is used for communication between the aforementioned terminal and other devices.

[0169] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0170] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0171] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0172] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A local feature amplification target detection method based on an attention mechanism, characterized in that, The method includes: Based on a pre-trained target detection model and a channel attention mechanism, the weight coefficients of each channel in the current detection image are determined, and the feature map of the current detection image is enhanced based on the weight coefficients. The enhanced feature maps are then subjected to max pooling and average pooling respectively to obtain average pooling feature maps and max pooling feature maps. The average pooling feature map and the max pooling feature map are concatenated along the channel dimension to obtain a concatenated feature map. The relationship between different locations in the spliced ​​feature map space is learned by using convolution operations to generate a preliminary spatial attention map; The spatial attention map is normalized, and the normalized spatial attention map is convolved to convert the weight features into spatial coordinate feature information. The spatial coordinate feature information can be used to crop the current detection image. The spatial coordinate feature information includes the x and y coordinates of the center after conversion, the image width and height. The spatial coordinate feature information is normalized to obtain the normalized spatial coordinate features; The affine transformation matrix is ​​constructed using the normalized spatial coordinate features. The feature map is spatially transformed based on the affine transformation matrix, and the generated grid is used to perform linear interpolation on the spatially transformed feature map to obtain a magnified local image.

2. The method according to claim 1, characterized in that, The steps of determining the weight coefficients of each channel in the current detection image based on a pre-trained object detection model using a channel attention mechanism, and enhancing the feature map of the current detection image based on the weight coefficients, include: Along the spatial dimension, the two-dimensional feature map of each channel of the current detection image is transformed into a single scalar through global average pooling and global max pooling layers, and the spatial information of each channel is aggregated into a global description; Global average pooling and max pooling are performed on the feature map corresponding to the input current detection image; Based on the nonlinear relationship between channels, a first weight is generated for each channel; The first weights are normalized using the Sigmoid function to obtain a normalized weight vector. The normalized weight vector is multiplied by the feature map corresponding to the current detection image to obtain the enhanced feature map.

3. The method according to claim 1, characterized in that, The normalized spatial coordinate features include the following parameter set: X-axis translation, Y-axis translation, image width, and image height.

4. The method according to claim 1, characterized in that, The steps for constructing the affine transformation matrix using the normalized spatial coordinate features include: Determine the scaling parameters; Based on the normalized spatial coordinate characteristics and the scaling parameters, determine the scaling factor in the X direction and the scaling factor in the Y direction; The translation amounts in the X and Y directions are determined based on the normalized spatial coordinate characteristics. An affine transformation matrix is ​​constructed based on the normalized spatial coordinate characteristics, the X-direction scaling factor, the Y-direction scaling factor, the X-direction translation, and the Y-direction translation.

5. A target detection device based on attention mechanism with magnified local features, characterized in that, The device includes: The determination module is used to determine the weight coefficients of each channel in the current detection image based on the channel attention mechanism of the pre-trained target detection model, and to perform enhancement processing on the feature map of the current detection image based on the weight coefficients. The transformation module is used to perform max pooling and average pooling on the enhanced feature maps to obtain average pooling feature maps and max pooling feature maps, respectively; concatenate the average pooling feature maps and max pooling feature maps along the channel dimension to obtain a concatenated feature map; learn the relationship between different positions in the space of the concatenated feature map using convolution operations to generate a preliminary spatial attention map; normalize the spatial attention map, and perform convolution operations on the normalized spatial attention map to convert the weight features into spatial coordinate feature information, wherein the spatial coordinate feature information can be used to crop the current detection image, and the spatial coordinate feature information includes the x and y coordinates of the transformed center, the image width, and the height; The normalization module is used to normalize the spatial coordinate feature information to obtain normalized spatial coordinate features. The construction module is used to construct an affine transformation matrix using the normalized spatial coordinate features; The processing module is used to perform spatial transformation on the feature map based on the affine transformation matrix, and to perform linear interpolation on the spatially transformed feature map using the generated grid to obtain a magnified local image.

6. The apparatus according to claim 5, characterized in that, The determining module includes: The first submodule is used to transform the two-dimensional feature map of each channel of the current detection image into a single scalar by passing through the global average pooling layer and the global max pooling layer along the spatial dimension, and to aggregate the spatial information of each channel into a global description. The second submodule is used to perform global average pooling and max pooling on the feature map corresponding to the input current detection image; The third submodule is used to generate a first weight for each channel based on the nonlinear relationship between the channels. The fourth submodule is used to normalize each of the first weights using the Sigmoid function to obtain a normalized weight vector. The fifth submodule is used to perform channel multiplication operation between the normalized weight vector and the feature map corresponding to the current detection image to obtain the enhanced feature map.

7. The apparatus according to claim 5, characterized in that, The normalized spatial coordinate features include the following parameter set: X-axis translation, Y-axis translation, image width, and image height.

8. The apparatus according to claim 5, characterized in that, The normalization module is specifically used for: Determine the scaling parameters; Based on the normalized spatial coordinate characteristics and the scaling parameters, determine the scaling factor in the X direction and the scaling factor in the Y direction; The translation amounts in the X and Y directions are determined based on the normalized spatial coordinate characteristics. An affine transformation matrix is ​​constructed based on the normalized spatial coordinate characteristics, the X-direction scaling factor, the Y-direction scaling factor, the X-direction translation, and the Y-direction translation.

Images