A target detection method and system based on cross-layer attention mechanism feature fusion

Abstract

The present invention discloses a target detection method and system based on feature fusion of a cross-layer attention mechanism. Using a cross-layer attention feature pyramid, the system first performs three feature attention interactions on different feature layers: self-level, top-down, and bottom-up. This not only effectively interacts with remote semantic information, but also captures the non-local context of objects at different scales, suppresses useless background information and emphasizes useful information, and concentrates the network's computing resources more on feature information that is more important to the output results. Secondly, for the feature maps output by the three interaction methods, an adaptive feature fusion method is used for feature fusion, which can learn the connections between different feature maps and search for the optimal fusion operation. For the same basic network and detector, the detection accuracy on a general image detection dataset is improved to a certain extent.

Description

Technical Field

[0001] The present invention belongs to the fields of target detection technology, machine learning technology, and attention mechanism technology, and specifically relates to a target detection method and system based on cross-layer attention mechanism feature fusion. Background Art

[0002] Object detection is a fundamental and crucial branch of computer vision. Its task is to determine whether an input image contains instances of a predefined category and, if so, to return the spatial location and category of the object. Object detection has a wide range of real-world applications, such as autonomous driving and security monitoring. With the advancement of deep learning, the performance of object detectors has greatly improved.

[0003] Object detection technology is primarily categorized into two schools: two-stage and one-stage. Initially, both detectors used a single feature map for object regression, localization, and classification. However, the scales of detected objects (whether within the same category or across different categories) can vary significantly, and using a single feature map for subsequent classification and regression cannot address this large scale variation. Consequently, object detection algorithms have evolved to output feature maps from different stages of the underlying backbone network to form a feature pyramid network, which represents objects of varying scales. Object detection is then performed based on this feature pyramid. Feature pyramid construction has evolved through various approaches, including no fusion (SSD), top-down fusion (FPN), simple bidirectional fusion (PANet), and complex bidirectional fusion (ASFF and NAS-FPN). The primary focus has been on finding better feature map fusion methods to achieve better feature representation. Therefore, exploring optimal feature fusion methods is one way to improve object detection accuracy.

[0004] The key point of feature fusion methods in target detection is how to better construct feature expressions, emphasize useful feature information, and suppress useless feature information. Although current feature fusion methods have achieved good results, there are still some problems:

[0005] 1. Failure to suppress useless information on the feature map and emphasize useful information will interfere with target detection;

[0006] 2. The connection between long-range information is not constructed, which is not conducive to strengthening the interaction between two positions and capturing long-range dependencies;

[0007] 3. When different feature maps are fused, the fusion method is not efficient enough. Summary of the Invention

[0008] The purpose of the present invention is to propose a technical solution for a target detection method and system based on cross-layer attention mechanism feature fusion to solve one or more technical problems existing in the prior art and at least provide a beneficial option or create conditions.

[0009] The present invention first extracts features from the input image through a pre-trained basic network (including convolutional layers, pooling layers, activation layers, etc.), and generates feature maps with different spatial resolutions and numbers of channels at each stage of the network. Then, for the feature maps obtained at each stage of the basic network, we use a cross-scale attention feature pyramid network to perform cross-layer fusion of these feature maps. Finally, we perform target classification and regression on multiple feature maps output by the pyramid to achieve target detection.

[0010] To achieve the above object, according to one aspect of the present invention, a method for target detection based on cross-layer attention mechanism feature fusion is provided, the method comprising the following steps:

[0011] S100, obtaining an input image;

[0012] S200, extracts features from the input image through the pre-trained basic network;

[0013] S300, generates feature maps of different spatial resolutions and numbers of channels at various stages of the base network;

[0014] S400 uses a cross-layer attention feature pyramid network to perform cross-layer fusion of feature maps obtained at each stage of the base network;

[0015] S500: Target classification and regression are performed on the output multiple feature maps to achieve target detection.

[0016] The basic network is any one of CNN, RNN, AlexNet, LeNet, and U-net networks.

[0017] Furthermore, in S400, the construction method of the cross-layer attention feature pyramid network is as follows: using cross-space and scale interaction methods, including self-level, top-down, and bottom-up interaction methods, to strengthen the feature interaction and fusion between each feature layer, wherein the interaction method adopts the attention mechanism, which is conducive to learning the global situation, capturing the non-local context of objects at different scales, suppressing useless background information, and emphasizing useful information; secondly, for the feature maps output by the three interaction methods, they are rearranged and grouped, and the feature maps of the same scale are grouped together, and each group of feature maps is subjected to feature fusion using an adaptive feature fusion method to obtain a feature map, wherein the adaptive feature fusion can learn the connection between different feature maps and search for the optimal fusion; finally, multiple fused feature maps constitute a cross-layer attention feature pyramid network.

[0018] Furthermore, in S400, cross-layer attention feature interaction mainly consists of three parts: self-level attention module, top-down attention module, and bottom-up attention module; it can not only strengthen the long-range connection within the feature map and realize cross-space information interaction, but also enable non-local information between feature maps of different scales to interact, that is, high-level semantic information is mapped to low-level feature maps, and low-level channel information is mapped to high-level feature maps.

[0019] Furthermore, in S400, the self-level attention module performs a self-attention mechanism operation on the input feature map. However, because the computational complexity of the original self-attention mechanism increases quadratically with the size of the input image, in order to reduce the computational complexity, we modify it. The modified self-attention mechanism is mainly divided into two parts: local grouping self-attention LSA and global subsampling self-attention GSA.

[0020] Locally-grouped self-attention (LSA)

[0021] For the input 2D feature map, the feature map is divided into m×n sub-windows, and then the following steps are performed in each sub-window:

[0022] For each sub-window, three vectors Q, K, and V are formed;

[0023] Q and K perform a dot product operation to calculate scores for each vector;

[0024] To prevent gradient explosion, the scores are normalized, that is, the value of scores is set to the value of scores divided by d k is the number of feature map channels;

[0025] Then scores are activated by the softmax function to obtain weights;

[0026] Finally, multiply the weight by V to get the size and original feature Figure 1 The feature map F after the change i .

[0027] Right now,

[0028] Among them, O, K, and V are three vectors Q, K, and V formed by selecting a row with the largest sum of grayscale values, a row with the smallest sum of grayscale values, and a row in the middle of the image matrix in the sub-window respectively.

[0029] Global sub-sampled attention (GSA)

[0030] First, the feature map F i It performs convolution operation to obtain the feature map F′ i ;

[0031] Construct three vectors Q1, K1, and V1, where Q1 is derived from F i Vectors, K1 and V1 are derived from F′ i Finally, Q1, K1, V1 are converted into the final feature map P by formula (2) i ;

[0032] in,

[0033] Among them, Q1, K1, V1 are the feature maps F′ respectively i The row with the largest sum of grayscale values in the image matrix, the row with the smallest sum of grayscale values, and the row in the middle of the image matrix form three vectors Q1, K1, and V1.

[0034] The purpose of performing global subsampling self-attention (GSA) is to enable the information in each independent small window to interact, expand the receptive field, reduce the amount of computation, and maintain the effect of the attention mechanism.

[0035] Self-level attention performs LSA and GSA on the feature maps output from each stage of the base network. This maintains the effectiveness of the attention mechanism, enabling the network to capture correlations within long-range features and weighting feature maps to emphasize useful information and suppress useless background information, which benefits subsequent classification and regression. It also reduces computational overhead compared to the original self-attention mechanism, making network training and inference faster.

[0036] Furthermore, the top-down attention module is a process in which high-level feature maps guide low-level feature maps to conduct global information interaction. It maps the semantic information of high-level feature maps to the pixels of low-level feature maps, enriching the semantic information of low-level feature maps.

[0037] Similar to self-level attention, top-down attention also uses a combination of LSA and GSA, but in order to solve the problem of different feature map scales, some optimizations are made. The specific process is as follows:

[0038] There are two feature maps required for the top-down attention module to be input, the high-level coarse-grained feature map C c and low-level fine-grained feature map C f ;

[0039] In the low-level fine-grained feature map C f Perform LSA, that is, C fDivide into m×n windows and perform self-attention mechanism operation in each sub-window to obtain the feature map F f ;

[0040] Similar to GSA, we perform high-level coarse-grained feature map C c Perform convolution operation, extract a representative value from each window to represent it, and obtain the feature map F c Similar to the original attention mechanism, we still need to construct three vectors Q, K, and V. Q is derived from F f vector, K and V are derived from F c Vector, perform self-attention operation on Q, K, and V to obtain the final feature map P f .

[0041] Furthermore, the bottom-up attention module uses global information interaction from bottom to top, mapping low-level channel information to high-level feature maps. Here, attention calculations are no longer performed on a pixel-by-pixel basis, but rather on the entire feature map. Because high-level feature maps are rich in semantic information, using self-attention to learn features and information about the current target from other targets is not very meaningful. Therefore, we use joint coordinate attention, a type of channel attention. The specific process is as follows:

[0042] The bottom-up attention module requires two input feature maps, the high-level coarse-grained feature map C c and low-level fine-grained feature map C f ;

[0043] For the low-level fine-grained feature map C f Perform average pooling along the two dimensions x and y of the space to obtain two tensors Z h =(1, y, C) and Z w =(x, 1, C), where x, y, and C are C f Image size and number of channels;

[0044] The two tensors are concatenated along the channel dimension to form a tensor Z(1, x+y, C), which is downsampled to Z′(1, x1+y1, C) through a convolutional layer (where x1, y1, and C are high-level coarse-grained feature maps C c The image size and number of channels are obtained), and then the final Z′ is obtained through the normalization layer and the activation layer;

[0045] Split the z′ feature map to obtain two tensors Z′ h (1, y1, C) and Z′ w (x1, 1, C), which pass through the activation layer respectively to obtain two attention weight tensors W h , W w ;

[0046] Finally, the high-level coarse-grained feature map C c With two attention weight tensors W h 、W w Multiply to get the final feature map P c .

[0047] Adaptive feature fusion:

[0048] After cross-layer attention feature interaction, the transformed feature maps are rearranged and grouped, and feature maps of the same size are grouped together. Each group of feature maps is input into the adaptive fusion module respectively, and the best fusion method is searched to obtain the fused feature map for subsequent target classification and regression.

[0049] make is the feature vector at position (x, y) along the channel on the feature map of the same scale. We assume that the fused first layer feature map is:

[0050]

[0051] Represents the feature vector at position (x, y) along the channel of the output first layer feature map, is the weight of each input feature map obtained by network adaptive learning. Note is a simple scalar variable that is shared between channels, and we let and have:

[0052]

[0053] Here we use Definition of the softmax function as a control parameter We use 1×1 convolutional layers to calculate The weight scalar mapping It can therefore be learned via standard back-propagation.

[0054] The present invention also provides a target detection system based on cross-layer attention mechanism feature fusion, the target detection system based on cross-layer attention mechanism feature fusion includes: a processor, a memory, and a computer program stored in the memory and executable on the processor, the processor implementing the steps of the target detection method based on cross-layer attention mechanism feature fusion when executing the computer program, the target detection system based on cross-layer attention mechanism feature fusion can be run on computing devices such as desktop computers, notebooks, PDAs, and cloud data centers, and the executable system may include, but is not limited to, a processor, a memory, and a server cluster, and the processor executes the computer program to run in the following system units:

[0055] An image acquisition unit, configured to acquire an input image;

[0056] Feature extraction unit, used to extract features from the input image through the pre-trained basic network;

[0057] The network feature map unit is used to generate feature maps of different spatial resolutions and numbers of channels at various stages of the basic network;

[0058] Pyramid fusion unit, used to use the cross-layer attention feature pyramid network to perform cross-layer fusion of feature maps obtained at each stage of the base network;

[0059] The target detection unit is used to perform target classification and regression on multiple output feature maps to achieve target detection.

[0060] The beneficial effects of the present invention are as follows: the present invention provides a target detection method and system based on feature fusion of a cross-layer attention mechanism, using a cross-layer attention feature pyramid, firstly, by performing three feature attention interactions on different feature layers, namely, self-level, top-down, and bottom-up, not only effectively performing remote semantic information interaction, but also capturing the non-local context of objects at different scales, suppressing useless background information and emphasizing useful information, and concentrating the network's computing resources more on feature information that is more important to the output results; secondly, for the feature maps output by the three interaction methods, an adaptive feature fusion method is used for feature fusion, which can learn the connections between different feature maps and search for the optimal fusion operation. For the same basic network and detector, the detection accuracy on the general image detection dataset is improved to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The above and other features of the present invention will become more apparent through a detailed description of the embodiments shown in conjunction with the accompanying drawings. In the drawings of the present invention, the same reference numerals represent the same or similar elements. Obviously, the drawings described below are only some embodiments of the present invention. It is possible for a person skilled in the art to derive other drawings based on these drawings without inventive effort. In the drawings:

[0062] Figure 1 The figure shows a flow chart of an object detection method based on cross-layer attention mechanism feature fusion;

[0063] Figure 2 The figure shows the structure of a target detection system based on cross-layer attention mechanism feature fusion. DETAILED DESCRIPTION

[0064] The following will be combined with the embodiments and drawings to clearly and completely describe the concept, specific structure and technical effects of the present invention so as to fully understand the purpose, scheme and effect of the present invention. It should be noted that the embodiments and features in the embodiments of this application can be combined with each other unless there is a conflict.

[0065] like Figure 1 As shown, Figure 1 The figure shows a flow chart of a target detection method based on cross-layer attention mechanism feature fusion. The present invention proposes a target detection method based on cross-layer attention mechanism feature fusion. The core module of this patent is the construction of a cross-layer attention feature pyramid. First, the module uses a cross-space and scale interaction method, including three interaction methods: self-level, top-down, and bottom-up, to strengthen the feature interaction and fusion between each feature layer. The interaction method adopts the attention mechanism, which is conducive to learning the global situation, capturing the non-local context of objects at different scales, suppressing useless background information, and emphasizing useful information. Secondly, for the feature maps output by the three interaction methods, we rearrange and group them, group the feature maps of the same scale into one group, and use the adaptive feature fusion method to fuse each group of feature maps to obtain a feature map. The adaptive feature fusion can learn the connection between different feature maps and search for the optimal fusion. Finally, multiple fused feature maps constitute our cross-layer attention feature pyramid.

[0066] S100, obtaining an input image;

[0067] S200, extracts features from the input image through the pre-trained basic network;

[0068] S300, generates feature maps of different spatial resolutions and numbers of channels at various stages of the base network;

[0069] S400 uses a cross-layer attention feature pyramid network to perform cross-layer fusion of feature maps obtained at each stage of the base network;

[0070] S500: Target classification and regression are performed on the output multiple feature maps to achieve target detection.

[0071] The basic network is any one of CNN, RNN, AlexNet, LeNet, and U-net networks.

[0072] Furthermore, in S400, the construction method of the cross-layer attention feature pyramid network is as follows: using cross-space and scale interaction methods, including self-level, top-down, and bottom-up interaction methods, to strengthen the feature interaction and fusion between each feature layer, wherein the interaction method adopts the attention mechanism, which is conducive to learning the global situation, capturing the non-local context of objects at different scales, suppressing useless background information, and emphasizing useful information; secondly, for the feature maps output by the three interaction methods, they are rearranged and grouped, and the feature maps of the same scale are grouped together, and each group of feature maps is subjected to feature fusion using an adaptive feature fusion method to obtain a feature map, wherein the adaptive feature fusion can learn the connection between different feature maps and search for the optimal fusion; finally, multiple fused feature maps constitute a cross-layer attention feature pyramid network.

[0073] Furthermore, in S400, cross-layer attention feature interaction mainly consists of three parts: self-level attention module, top-down attention module, and bottom-up attention module; it can not only strengthen the long-range connection within the feature map and realize cross-space information interaction, but also enable non-local information between feature maps of different scales to interact, that is, high-level semantic information is mapped to low-level feature maps, and low-level channel information is mapped to high-level feature maps.

[0074] Furthermore, in S400, the self-level attention module performs a self-attention mechanism operation on the input feature map. However, because the computational complexity of the original self-attention mechanism increases quadratically with the size of the input image, in order to reduce the computational complexity, we modify it. The modified self-attention mechanism is mainly divided into two parts: local grouping self-attention LSA and global subsampling self-attention GSA.

[0075] Locally-grouped self-attention (LSA)

[0076] For the input 2D feature map, the feature map is divided into m×n sub-windows, and then the following steps are performed in each sub-window:

[0077] For each sub-window, three vectors Q, K, and V are formed;

[0078] Q and K perform a dot product operation to calculate scores for each vector;

[0079] To prevent gradient explosion, the scores are normalized, that is, the value of scores is set to the value of scores divided by d k is the number of feature map channels;

[0080] Then scores are activated by the softmax function to obtain weights;

[0081] Finally, multiply the weight by V to get the size and original feature Figure 1 The feature map F after the change i .

[0082] Right now,

[0083] Among them, Q, K, and V are three vectors Q, K, and V formed by selecting a row with the largest sum of grayscale values, a row with the smallest sum of grayscale values, and a row in the middle of the image matrix in the sub-window respectively.

[0084] Global sub-sampled attention (GSA)

[0085] First, the feature map F i It performs convolution operation to obtain the feature map F′ i ;

[0086] Construct three vectors Q1, K1, and V1, where Q1 is derived from F i Vectors, K1 and V1 are derived from F′ i Finally, Q1, K1, V1 are converted into the final feature map P by formula (2) i ;

[0087] in,

[0088] Among them, Q1, K1, V1 are the feature maps F′ respectively i The row with the largest sum of grayscale values in the image matrix, the row with the smallest sum of grayscale values, and the row in the middle of the image matrix form three vectors Q1, K1, and V1.

[0089] The purpose of performing global subsampling self-attention (GSA) is to enable the information in each independent small window to interact, expand the receptive field, reduce the amount of computation, and maintain the effect of the attention mechanism.

[0090] Self-level attention performs LSA and GSA on the feature maps output from each stage of the base network. This maintains the effectiveness of the attention mechanism, enabling the network to capture correlations within long-range features and weighting feature maps to emphasize useful information and suppress useless background information, which benefits subsequent classification and regression. It also reduces computational overhead compared to the original self-attention mechanism, making network training and inference faster.

[0091] Furthermore, the top-down attention module is a process in which high-level feature maps guide low-level feature maps to conduct global information interaction. It maps the semantic information of high-level feature maps to the pixels of low-level feature maps, enriching the semantic information of low-level feature maps.

[0092] Similar to self-level attention, top-down attention also uses a combination of LSA and GSA, but in order to solve the problem of different feature map scales, some optimizations are made. The specific process is as follows:

[0093] There are two feature maps required for the top-down attention module to be input, the high-level coarse-grained feature map C c and low-level fine-grained feature map C f ;

[0094] In the low-level fine-grained feature map C f Perform LSA, that is, C f Divide into m×n windows and perform self-attention mechanism operation in each sub-window to obtain the feature map F f ;

[0095] Similar to GSA, we perform high-level coarse-grained feature map C c Perform convolution operation, extract a representative value from each window to represent it, and obtain the feature map F c Similar to the original attention mechanism, we still need to construct three vectors Q, K, and V. Q is derived from F f vector, K and V are derived from F c Vector, perform self-attention operation on Q, K, and V to obtain the final feature map P f .

[0096] Furthermore, the bottom-up attention module uses global information interaction from bottom to top, mapping low-level channel information to high-level feature maps. Here, attention calculations are no longer performed on a pixel-by-pixel basis, but rather on the entire feature map. Because high-level feature maps are rich in semantic information, using self-attention to learn features and information about the current target from other targets is not very meaningful. Therefore, we use joint coordinate attention, a type of channel attention. The specific process is as follows:

[0097] The bottom-up attention module requires two input feature maps, the high-level coarse-grained feature map C c and low-level fine-grained feature map C f ;

[0098] For the low-level fine-grained feature map C f Perform average pooling along the two dimensions x and y of the space to obtain two tensors Z h =(1, y, C) and Z w =(x, 1, C), where x, y, and C are C f Image size and number of channels;

[0099] The two tensors are concatenated along the channel dimension to form a tensor Z(1, x+y, C), which is downsampled to Z′(1, x1+y1, C) through a convolutional layer (where x1, y1, and C are high-level coarse-grained feature maps C c The image size and number of channels are obtained), and then the final Z′ is obtained through the normalization layer and the activation layer;

[0100] Split the Z′ feature map to obtain two tensors Z′ h (1, y1, C) and Z′ w (x1, 1, C), which pass through the activation layer respectively to obtain two attention weight tensors W h , W w ;

[0101] Finally, the high-level coarse-grained feature map C c With two attention weight tensors W h 、W w Multiply to get the final feature map P c .

[0102] Adaptive feature fusion

[0103] After cross-layer attention feature interaction, the transformed feature maps are rearranged and grouped, and feature maps of the same size are grouped together. Each group of feature maps is input into the adaptive fusion module respectively, and the best fusion method is searched to obtain the fused feature map for subsequent target classification and regression.

[0104] make is the feature vector at position (x, y) along the channel on the feature map of the same scale. We assume that the fused first layer feature map is:

[0105]

[0106] Represents the feature vector at position (x, y) along the channel of the output first layer feature map, is the weight of each input feature map obtained by network adaptive learning. Note is a simple scalar variable that is shared between channels, and we let and have:

[0107]

[0108] Here we use Definition of the softmax function as a control parameter We use 1×1 convolutional layers to calculate The weight scalar mapping It can therefore be learned via standard back-propagation.

[0109] The embodiment of the present invention provides a target detection system based on cross-layer attention mechanism feature fusion, such as Figure 2 As shown, this is a structural diagram of a target detection system based on cross-layer attention mechanism feature fusion of the present invention. The target detection system based on cross-layer attention mechanism feature fusion of this embodiment includes: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps in the above-mentioned target detection system embodiment based on cross-layer attention mechanism feature fusion are implemented.

[0110] The system includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to run in the following units of the system:

[0111] An image acquisition unit, configured to acquire an input image;

[0112] Feature extraction unit, used to extract features from the input image through the pre-trained basic network;

[0113] The network feature map unit is used to generate feature maps of different spatial resolutions and numbers of channels at various stages of the basic network;

[0114] Pyramid fusion unit, used to use the cross-layer attention feature pyramid network to perform cross-layer fusion of feature maps obtained at each stage of the base network;

[0115] The target detection unit is used to perform target classification and regression on multiple output feature maps to achieve target detection.

[0116] The target detection system based on cross-layer attention mechanism feature fusion can be run on computing devices such as desktop computers, laptops, PDAs, and cloud servers. The target detection system based on cross-layer attention mechanism feature fusion can be run on systems that include, but are not limited to, processors and memories. Those skilled in the art will appreciate that the example is merely an example of a target detection system based on cross-layer attention mechanism feature fusion and does not constitute a limitation on a target detection system based on cross-layer attention mechanism feature fusion. The system can include more or fewer components than the example, or a combination of certain components, or different components. For example, the target detection system based on cross-layer attention mechanism feature fusion can also include input and output devices, network access devices, buses, and the like.

[0117] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the target detection system based on cross-layer attention mechanism feature fusion operating system, and utilizes various interfaces and lines to connect the various parts of the target detection system based on cross-layer attention mechanism feature fusion operating system.

[0118] The memory can be used to store the computer programs and / or modules. The processor implements the various functions of the object detection system based on cross-layer attention mechanism feature fusion by running or executing the computer programs and / or modules stored in the memory and calling the data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function (such as a sound playback function, an image playback function, etc.); the data storage area may store data created based on the use of the mobile phone (such as audio data, a phone book, etc.). In addition, the memory may include high-speed random access memory and non-volatile memory, such as a hard disk, internal memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0119] Although the present invention has been described in considerable detail and with particularity with respect to several described embodiments, it is not intended to be limited to any of these details or embodiments or any particular embodiment, but rather should be construed as providing a broad possible interpretation of these claims in view of the prior art by reference to the appended claims, thereby effectively encompassing the intended scope of the invention. In addition, the invention has been described above in terms of embodiments foreseen by the inventors for the purpose of providing a useful description, and those insubstantial modifications of the invention that are not currently foreseen may still represent equivalent modifications of the invention.

Claims

1. A target detection method based on cross-layer attention mechanism feature fusion, characterized in that: The method comprises the following steps: S100, obtaining an input image; S200, extracts features from the input image through the pre-trained basic network; S300, generates feature maps of different spatial resolutions and numbers of channels at various stages of the base network; S400 uses a cross-layer attention feature pyramid network to perform cross-layer fusion of feature maps obtained at each stage of the base network; S500: performing target classification and regression on the output multiple feature maps to achieve target detection; In S400, the cross-layer attention feature interaction mainly consists of three parts: self-level attention module, top-down attention module, and bottom-up attention module; The bottom-up attention module performs the following steps: Select 2 feature maps as high-level coarse-grained feature maps and low-level fine-grained feature maps ; For low-level fine-grained feature maps Perform average pooling along the two dimensions x and y of the space to obtain two tensors and , where x, y, and C are Image size and number of channels; Concatenate the two tensors along the channel dimension to form a tensor , through a convolutional layer, it is downsampled into , and then pass the normalization layer and activation layer to get the final ; Among them, x1, y1, and C are high-level coarse-grained feature maps Image size and number of channels; Will The feature map is split to obtain two tensors and , pass them through the activation layer respectively to obtain two attention weight tensors ; Finally, the high-level coarse-grained feature map With two attention weight tensors Multiply to get the final feature map ; In S400, the method for performing cross-layer fusion is: After cross-layer attention feature interaction, the transformed feature maps are rearranged and grouped. Feature maps of the same size are grouped together. Each group of feature maps is input into the adaptive fusion module to obtain the fused feature map for subsequent target classification and regression. The fusion method of the adaptive fusion module is: make is the position along the channel on the feature map of the same scale The feature vector of , then the fused feature map of the lth layer is: (2) Indicates the position of the channel along the output layer l feature map The eigenvector of 、 、 is the weight of each input feature map obtained by adaptive learning of the basic network, 、 、 Be a scalar variable shared between channels, let and ,have: (3) Among them, use Definition of the softmax function as a control parameter 、 , use 1×1 convolutional layers to calculate The weight scalar mapping , and can therefore be learned via standard back-propagation.

2. The target detection method based on cross-layer attention mechanism feature fusion according to claim 1 is characterized in that: In S400, the method for constructing a cross-layer attention feature pyramid network is as follows: using cross-space and scale interaction methods, including self-level, top-down, and bottom-up interaction methods, rearrange and group the feature maps output by the three interaction methods, group feature maps of the same scale together, and use an adaptive feature fusion method to fuse each group of feature maps to obtain a feature map, wherein the adaptive feature fusion can learn the connection between different feature maps and search for the optimal fusion; finally, multiple fused feature maps constitute a cross-layer attention feature pyramid network.

3. The object detection method based on cross-layer attention mechanism feature fusion according to claim 1 is characterized in that In S400, the self-level attention module is divided into two parts: local grouping self-attention LSA and global subsampling self-attention GSA.

4. The target detection method based on cross-layer attention mechanism feature fusion according to claim 3 is characterized in that: Local group self-attention LSA includes the following steps: For the input 2D feature map, the feature map is divided into m×n sub-windows, and then the following steps are performed in each sub-window: For each sub-window, three vectors Q, K, and V are formed; By formula , get the feature map ; Among them, Q, K, and V are three vectors Q, K, and V formed by selecting the row with the largest sum of grayscale values, the row with the smallest sum of grayscale values, and the row in the middle of the image matrix in the sub-window, respectively. k is the number of feature map channels.

5. The target detection method based on cross-layer attention mechanism feature fusion according to claim 4 is characterized in that: Global subsampling self-attention GSA consists of the following steps: First, the feature map It performs convolution operation to obtain feature map ; Construct three vectors Q1, K1, and V1, where Q1 is derived from Vectors, K1 and V1 are derived from Finally, Q1, K1, V1 are transformed into vectors by the formula Get the final feature map ; Among them, Q1, K1, and V1 are feature maps selected respectively The row with the largest sum of grayscale values in the image matrix, the row with the smallest sum of grayscale values, and the row in the middle of the image matrix form three vectors Q1, K1, and V1, d k is the number of feature map channels.

6. The object detection method based on cross-layer attention mechanism feature fusion according to claim 5 is characterized in that The top-down attention module performs the following steps: Select 2 feature maps as high-level coarse-grained feature maps and low-level fine-grained feature maps ; In the low-level fine-grained feature map Perform local group self-attention LSA, that is, Division windows, and perform local grouping self-attention LSA in each sub-window to obtain the feature map ; For high-level coarse-grained feature maps Perform convolution operation to obtain feature map , construct three vectors Q, K, V, Q is derived from vector, K and V are derived from Vector, Q, K, V through the formula , and get the final feature map , d k is the number of feature map channels.

7. An object detection system based on cross-layer attention mechanism feature fusion, characterized in that: The target detection system based on cross-layer attention mechanism feature fusion includes: a processor, a memory and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of the target detection method based on cross-layer attention mechanism feature fusion in claim 1. The target detection system based on cross-layer attention mechanism feature fusion can run on desktop computers, notebooks, PDAs and computing devices in cloud data centers.

Images