Monocular 3D target detection method and device fusing attention mechanism
By introducing a fusion attention mechanism in monocular 3D object detection and combining 3D attention and spatial channel reconstruction attention module, the problem of low accuracy of monocular 3D object detection is solved, and the accuracy and real-time nature of target recognition are improved .
Patent Information
- Application Number
- CN202411456549.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-18
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-10-18
AI Technical Summary
In the prior art, the accuracy of monocular 3D object detection is low, resulting in limited accuracy and real-time 3D object detection in the fields of autonomous driving, virtual reality and robotics.
Using a monocular 3D object detection method with a fusion attention mechanism, the 3D gated attention module combines the 3D attention module and spatial channel reconstruction attention module to highlight important features and reduce the spatial and channel redundancy of the feature map, and improve the target Accuracy of recognition.
The accuracy of monocular 3D object detection is improved, non-important information interference is reduced, and the ability to identify 3D objects is enhanced, and the problem of low accuracy in the prior art is solved.
Smart Images

Figure CN118968043A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer vision technology, and in particular to a monocular 3D target detection method and device integrating an attention mechanism. Background Art
[0002] Accurate detection of 3D objects in various scenarios has wide applications in autonomous driving, virtual reality, and robotics. To obtain accurate perception of 3D information, many 3D object detection methods rely on depth sensors (such as LiDAR), which, while effective, often face the obstacles of high cost and relatively sparse data provided. In contrast, monocular 3D object detection has attracted increasing attention due to the use of monocular RGB cameras as a simpler and cheaper deployment setup.
[0003] However, with the rapid development of 3D object detection technology, many problems have emerged. Due to the large size of image input in practical applications, considering all input information during training will inevitably lead to a decrease in accuracy and real-time performance. Therefore, the existing technology has the problem of low accuracy when performing 3D object detection. Summary of the invention
[0004] In view of this, the purpose of the present invention is to provide a monocular 3D target detection method and device integrating an attention mechanism, aiming to solve the problem of low accuracy in identifying monocular 3D targets in the prior art.
[0005] An object of the present invention is to provide a monocular 3D target detection method integrating an attention mechanism, the method comprising: Obtain a target detection image to be detected; Inputting the target detection image into a target detection model trained by a neural network including a 3D gated attention module to obtain target detection information in the target detection image; Among them, the 3D gated attention module includes a 3D attention module and a spatial channel reconstruction attention module. The 3D attention module is used to highlight the important features of the received feature map, and the spatial channel reconstruction attention module is used to generate a feature map with more information from the received feature map. The 3D gated attention module is used to combine the feature maps processed by the 3D attention module and the spatial channel reconstruction attention module in parallel.
[0006] Furthermore, in the above-mentioned monocular 3D target detection method integrating the attention mechanism, the 3D attention module includes a channel attention module and a spatial attention module, and the processing formula of the channel attention module for the received feature map is: ; Among them, X is the feature map received by the channel attention module, X comes from the convolution layer with a fixed kernel size, , W, H and C are the width, height and channel size of the feature map, swish and Relu are activation functions, T1 and T2 are two one-dimensional convolutions with a kernel size of 3, and Z c is the output after squeezing the cth channel.
[0007] Furthermore, in the above-mentioned monocular 3D target detection method integrating the attention mechanism, the processing formula of the spatial domain attention module for the received feature map is: ; in, is the normalization factor, i is the index of the output position whose response is calculated, j is the index of all possible positions, and the pairwise function f is used to calculate the relationship between i and all j, using the dot product similarity To express, P n (X i ) represents the feature at position i, represents the feature at position j, , and g represent 1×1 convolution operations, and T is the weight matrix.
[0008] Furthermore, in the above-mentioned monocular 3D target detection method integrating the attention mechanism, the spatial channel reconstruction attention module includes a spatial reconstruction unit module and a channel reconstruction unit module, and the processing formula of the spatial reconstruction unit for the received feature map is: ; ; Among them, W1 and W2 represent the weighted matrices for different features, Swish is the activation function, is the normalized correlation weight indicating the importance of different feature maps, X represents the input feature map, and are features with more information and features with less information, respectively, 11 , X 22 Represents the features of X1 and X2 on different channels, X 21 and X 12 They represent the features of X1 and X2 fused together. is element-wise multiplication, is element-wise addition, is to find the union, It is the spatially refined feature map obtained by the spatial reconstruction unit module.
[0009] Furthermore, in the above-mentioned monocular 3D target detection method integrating the attention mechanism, the channel reconstruction unit module is used to refine the input spatial features. Use 1×1 convolution kernel for compression and get features respectively. and Features ; The characteristics As the input of the feature, GWC operation and PWC operation are performed respectively, and then the output is added , the feature As a supplement to the features, PWC operation is performed, and the results and features obtained are Take the union and get ; Use global average pooling technology to combine global spatial information and channel information to obtain pooled features and Features ; Features and Features Do SoftMax to get the feature weight vector and , according to the characteristics and Features and the feature weight vector and Get the output feature Y; in, .
[0010] Furthermore, in the above-mentioned monocular 3D target detection method integrating the attention mechanism, the step of the 3D gated attention module being used to parallelly combine the feature maps processed by the 3D attention module and the spatial channel reconstruction attention module respectively comprises: The feature map processed by the 3D attention module and the spatial channel reconstruction attention module is balanced with the representation capability of the adaptive adjustment model. blend together; Among them, X Z =λ t X 3D +(1-λ t )X 空 ,in, represents the final refined output, , where λ is a learnable training parameter, and the sigmoid function is used to project λ into the range of [0,1], X 3D and X 空 They are the feature maps processed by the 3D attention module and the spatial channel reconstruction attention module respectively.
[0011] Furthermore, in the above-mentioned monocular 3D target detection method integrating the attention mechanism, the neural network also includes a ResNet101 network with residual connection and an FPN network for generating multi-level predictions, the ResNet101 network is used to receive the target detection image, and the FPN network is used to receive the features output by the ResNet101 network to generate a corresponding feature map.
[0012] Another object of the present invention is to provide a monocular 3D object detection device integrating an attention mechanism, the device comprising: An acquisition module is used to acquire a detection image of a target to be detected; A detection module, configured to input the target detection image into a target detection model trained by a neural network including a 3D gated attention module to obtain target detection information in the target detection image; Among them, the 3D gated attention module includes a 3D attention module and a spatial channel reconstruction attention module. The 3D attention module is used to highlight the important features of the received feature map, and the spatial channel reconstruction attention module is used to generate a feature map with more information from the received feature map. The 3D gated attention module is used to combine the feature maps processed by the 3D attention module and the spatial channel reconstruction attention module in parallel.
[0013] Another object of the present invention is to provide a readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the steps of any one of the methods described above.
[0014] Another object of the present invention is to provide an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the program.
[0015] The present invention obtains a target detection image to be detected; inputs the target detection image into a target detection model obtained by training a neural network including a 3D gated attention module to obtain target detection information in the target detection image; wherein the 3D gated attention module includes a 3D attention module and a spatial channel reconstruction attention module, the 3D attention module is used to highlight the important features of the received feature map, the spatial channel reconstruction attention module is used to generate a feature map with more information from the received feature map, and the 3D gated attention module is used to combine the feature maps processed by the 3D attention module and the spatial channel reconstruction attention module in parallel, and by introducing the 3D gated attention module, the feature relationship between the airspace and the channel is considered, and the spatial and channel redundancy between the features in the convolutional neural network is reduced to provide an effective focus of the feature map and reduce interference from non-important information, thereby improving the accuracy of target recognition. The problem of low accuracy in recognizing monocular 3D targets in the prior art is solved. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] Figure 1 A flowchart of monocular 3D target detection with an attention mechanism integrated according to an embodiment of the present invention; Figure 2 This is a structural block diagram of a monocular 3D target detection device integrating an attention mechanism in the third embodiment of the present invention.
[0017] The following specific implementation manner will further illustrate the present invention in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION
[0018] In order to facilitate the understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. Several embodiments of the present invention are given in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive.
[0019] It should be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only.
[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.
[0021] How to improve the accuracy of 3D object detection will be described in detail below with reference to specific embodiments and drawings.
[0022] Embodiment 1 See also Figure 1 , shown is a monocular 3D target detection method integrating an attention mechanism in a first embodiment of the present invention, the method comprising steps S10 to S11.
[0023] Step S10, obtaining a detection image of a target to be detected.
[0024] The target detection image is a monocular image containing a 3D target. Specifically, the target detection image is obtained by acquiring an RGB image.
[0025] Step S11, inputting the target detection image into a target detection model trained by a neural network including a 3D gated attention module to obtain target detection information in the target detection image.
[0026] Among them, the target detection information includes the target category, depth distance, 2D bounding box and 3D bounding box. Specifically, a neural network for training the model is constructed. The neural network includes a ResNet101 network with residual connection, an FPN network for generating multi-level predictions, and a 3D gated attention module. The ResNet101 network is used to receive the target detection image, and the FPN network is used to receive the features output by the ResNet101 network to generate the corresponding feature map. The 3D gated attention module includes a 3D attention module and a spatial channel reconstruction attention module.
[0027] Specifically, the 3D gated attention module adaptively combines the 3D attention branch and the spatial channel reconstruction attention branch in parallel, learns which branch to pay more attention to through a parameter, and finally integrates the results for output. In the specific implementation, the feature map generated by the FPN network is used as the input of the module and enters the 3D attention module composed of the improved channel attention module and the spatial domain attention. At the same time, the feature map generated by the FPN network is used as the input to enter the spatial channel reconstruction attention module. After the feature map is adaptively fused by the above two attention modules, a new feature map is obtained.
[0028] More specifically, after building the neural network, the data set is used for model training. For example, the KITTI target detection data set is selected, which consists of 7481 images for training and validation sets, and 7518 images for test sets, including a total of 80256 labeled objects, the main categories of which are cars, bicycles, pedestrians, and trucks. According to the height, degree of occlusion, and degree of truncation of the object, the detection results are divided into three levels: simple, medium, and difficult. The training images are divided into 3712 training sets and 3769 validation sets. The model is trained and the corresponding network parameters are learned through the training data set, so that it can master the logic of 3D target recognition.
[0029] For example, in the specific training process, 8 heads are used for all attention modules. Multi-head attention enables the model to capture and process information in different subspaces, thereby enhancing the model's ability to express input data, and the number of queries N is set to 50. Setting N to 50 means that the model will perform 50 query operations on each input, which can enhance the model's ability to capture details. The channel and the potential feature dimensions of the FFN-based and MLP-based detection heads are set to 256. On a single GeForce RTX 4090 GPU, 48 epochs were trained, and the learning rate was set to , using weight decay The SGD optimizer is used with the momentum factor set to 0.9, the weight decay coefficient set to 0.0001, the number of warm-up iterations to 500, the warm-up ratio to 0.33, and the learning rates of 32 and 44 times are adjusted to the set learning rate values. The loss function formula is: ; The classification loss is the FocalLoss loss function, the geometric center point offset loss is the SmoothL1Loss loss function, size loss is the SmoothL1Loss loss function, rotation loss is the SmoothL1Loss loss function, speed loss is the SmoothL1Loss loss function, depth loss It is the SmoothL1Loss loss function.
[0030] In summary, the monocular 3D target detection method integrating the attention mechanism in the above-mentioned embodiment of the present invention obtains the target detection image to be detected; inputs the target detection image into the target detection model obtained by training the neural network including the 3D gated attention module to obtain the target detection information in the target detection image; wherein, the 3D gated attention module includes a 3D attention module and a spatial channel reconstruction attention module, the 3D attention module is used to highlight the important features of the received feature map, the spatial channel reconstruction attention module is used to generate the received feature map to obtain a feature map with more information, and the 3D gated attention module is used to combine the feature maps processed by the 3D attention module and the spatial channel reconstruction attention module in parallel, and by introducing the 3D gated attention module, the feature relationship between the airspace and the channel is considered, and the spatial and channel redundancy between the features in the convolutional neural network is reduced to provide an effective focus of the feature map and reduce the interference of non-important information, thereby improving the accuracy of target recognition. The problem of low accuracy in identifying monocular 3D targets in the prior art is solved.
[0031] Embodiment 2 This embodiment also proposes a monocular 3D target detection method integrating an attention mechanism. The difference between the monocular 3D target detection method integrating an attention mechanism in this embodiment and the monocular 3D target detection method integrating an attention mechanism in the first embodiment is that: The 3D attention module includes a channel attention module and a spatial attention module. The processing formula of the channel attention module for the received feature map is: ; Among them, X is the feature map received by the channel attention module, X comes from the convolution layer with a fixed kernel size, , W, H and C are the width, height and channel size of the feature map, swish and Relu are activation functions, T1 and T2 are two one-dimensional convolutions with a kernel size of 3, and Z c is the output after the c-th channel is squeezed. The squeezing operation makes it possible to collect global information.
[0032] Specifically, the formula for the extrusion operation is: ; Among them, X c (i, j) represents the value of the cth channel of the input feature map in space (i, j). i and j are indexes in the spatial dimension, representing the height and width of the feature map respectively. H and W represent the height and width of the input feature map.
[0033] The processing formula of the spatial attention module for the received feature map is: ; in, is the normalization factor, i is the index of the output position whose response is calculated, j is the index of all possible positions, and the pairwise function f is used to calculate the relationship between i and all j, using the dot product similarity To express, P n (X i ) represents the feature at position i, represents the feature at position j, , and g represent 1×1 convolution operations, and T is the weight matrix.
[0034] The spatial channel reconstruction attention module includes a spatial reconstruction unit module and a channel reconstruction unit module. The processing formula of the spatial reconstruction unit for the received feature map is: ; ; Among them, W1 and W2 represent the weighted matrices for different features, Swish is the activation function, is the normalized correlation weight indicating the importance of different feature maps, X represents the input feature map, and are features with more information and features with less information, respectively, 11 , X 22 Represents the features of X1 and X2 on different channels, X 21 and X 12 They represent the features of X1 and X2 fused together. is element-wise multiplication, is element-wise addition, is to find the union, It is the spatially refined feature map obtained by the spatial reconstruction unit module.
[0035] In addition, the channel reconstruction unit module is used to refine the input spatial features Use 1×1 convolution kernel for compression and get features respectively. and Features ; The characteristics As the input of the feature, GWC operation and PWC operation are performed respectively, and then the output is added , the feature As a supplement to the features, PWC operation is performed, and the results and features obtained are Take the union and get ; Use global average pooling technology to combine global spatial information and channel information to obtain pooled features and Features ; Features and Features Do SoftMax to get the feature weight vector and , according to the characteristics and Features and the feature weight vector and Get the output feature Y; in, .
[0036] Furthermore, the 3D gated attention module is used to parallelly combine the feature maps processed by the 3D attention module and the spatial channel reconstruction attention module respectively, including: The feature map processed by the 3D attention module and the spatial channel reconstruction attention module is balanced with the representation capability of the adaptive adjustment model. blend together; Among them, X Z =λ t X 3D +(1-λ t )X 空 ,in, represents the final refined output, , where λ is a learnable training parameter, and the sigmoid function is used to project λ into the range of [0,1], X 3D and X 空 They are the feature maps processed by the 3D attention module and the spatial channel reconstruction attention module respectively.
[0037] In summary, the monocular 3D target detection method integrating the attention mechanism in the above-mentioned embodiment of the present invention obtains the target detection image to be detected; inputs the target detection image into the target detection model obtained by training the neural network including the 3D gated attention module to obtain the target detection information in the target detection image; wherein, the 3D gated attention module includes a 3D attention module and a spatial channel reconstruction attention module, the 3D attention module is used to highlight the important features of the received feature map, the spatial channel reconstruction attention module is used to generate the received feature map to obtain a feature map with more information, and the 3D gated attention module is used to combine the feature maps processed by the 3D attention module and the spatial channel reconstruction attention module in parallel, and by introducing the 3D gated attention module, the feature relationship between the airspace and the channel is considered, and the spatial and channel redundancy between the features in the convolutional neural network is reduced to provide an effective focus of the feature map and reduce the interference of non-important information, thereby improving the accuracy of target recognition. The problem of low accuracy in identifying monocular 3D targets in the prior art is solved.
[0038] Embodiment 3 See also Figure 2 , shown is a monocular 3D target detection device integrating an attention mechanism proposed in the third embodiment of the present invention, the device comprising: The acquisition module 100 is used to acquire a detection image of a target to be detected; A detection module 200, configured to input the target detection image into a target detection model trained by a neural network including a 3D gated attention module to obtain target detection information in the target detection image; Among them, the 3D gated attention module includes a 3D attention module and a spatial channel reconstruction attention module. The 3D attention module is used to highlight the important features of the received feature map, and the spatial channel reconstruction attention module is used to generate a feature map with more information from the received feature map. The 3D gated attention module is used to combine the feature maps processed by the 3D attention module and the spatial channel reconstruction attention module in parallel.
[0039] Furthermore, in the above-mentioned monocular 3D target detection device integrating the attention mechanism, the 3D attention module includes a channel attention module and a spatial attention module, and the processing formula of the channel attention module for the received feature map is: ; Among them, X is the feature map received by the channel attention module, X comes from the convolution layer with a fixed kernel size, , W, H and C are the width, height and channel size of the feature map, swish and Relu are activation functions, T1 and T2 are two one-dimensional convolutions with a kernel size of 3, and Z c is the output after squeezing the cth channel.
[0040] Furthermore, in the above-mentioned monocular 3D target detection device integrating the attention mechanism, the processing formula of the spatial domain attention module for the received feature map is: ; in, is the normalization factor, i is the index of the output position whose response is calculated, j is the index of all possible positions, and the pairwise function f is used to calculate the relationship between i and all j, using the dot product similarity To express, P n (X i ) represents the feature at position i, represents the feature at position j, , and g represent 1×1 convolution operations, and T is the weight matrix.
[0041] Furthermore, in the above-mentioned monocular 3D target detection device integrating the attention mechanism, the spatial channel reconstruction attention module includes a spatial reconstruction unit module and a channel reconstruction unit module, and the processing formula of the spatial reconstruction unit for the received feature map is: ; ; Among them, W1 and W2 represent the weighted matrices for different features, Swish is the activation function, is the normalized correlation weight indicating the importance of different feature maps, X represents the input feature map, and are features with more information and features with less information, respectively, 11 , X 22 Represents the features of X1 and X2 on different channels, X 21 and X 12 They represent the features of X1 and X2 fused together. is element-wise multiplication, is element-wise addition, is to find the union, It is the spatially refined feature map obtained by the spatial reconstruction unit module.
[0042] Furthermore, in the above-mentioned monocular 3D target detection device integrating the attention mechanism, the channel reconstruction unit module is used to refine the input spatial features. Use 1×1 convolution kernel for compression and get features respectively. and Features ; The characteristics As the input of the feature, GWC operation and PWC operation are performed respectively, and then the output is added , the feature As a supplement to the features, PWC operation is performed, and the results and features obtained are Take the union and get ; Use global average pooling technology to combine global spatial information and channel information to obtain pooled features and Features ; Features and Features Do SoftMax to get the feature weight vector and , according to the characteristics and Features and the feature weight vector and Get the output feature Y; in, .
[0043] Furthermore, in the above-mentioned monocular 3D target detection device integrating the attention mechanism, the step of the 3D gated attention module for parallelly combining the feature maps processed by the 3D attention module and the spatial channel reconstruction attention module respectively comprises: The feature map processed by the 3D attention module and the spatial channel reconstruction attention module is balanced with the representation capability of the adaptive adjustment model. blend together; Among them, X Z =λ t X 3D +(1-λ t )X 空 ,in, represents the final refined output, , where λ is a learnable training parameter, and the sigmoid function is used to project λ into the range of [0,1], X 3D and X 空 They are the feature maps processed by the 3D attention module and the spatial channel reconstruction attention module respectively.
[0044] Furthermore, in the above-mentioned monocular 3D target detection device integrating the attention mechanism, the neural network also includes a ResNet101 network with residual connections and an FPN network for generating multi-level predictions, the ResNet101 network is used to receive the target detection image, and the FPN network is used to receive the features output by the ResNet101 network to generate a corresponding feature map.
[0045] The functions or operation steps implemented when the above modules are executed are substantially the same as those in the above method embodiments, and will not be repeated here.
[0046] Embodiment 4 Another aspect of the present invention further provides a readable storage medium having a computer program stored thereon, and when the program is executed by a processor, the steps of the method described in any one of the above-mentioned embodiments 1 to 2 are implemented.
[0047] Embodiment 5 Another aspect of the present invention provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in any one of the above-mentioned embodiments 1 to 2 when executing the program.
[0048] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0049] Those skilled in the art will appreciate that the logic and / or steps represented in the flowchart or otherwise described herein, for example, may be considered as an ordered list of executable instructions for implementing logical functions, and may be specifically implemented in any computer-readable storage medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For purposes of this specification, a "computer-readable storage medium" may be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.
[0050] More specific examples (a non-exhaustive list) of computer-readable storage media include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable storage medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.
[0051] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or a combination thereof: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.
[0052] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.
[0053] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.
Claims
1. A monocular 3D object detection method integrating attention mechanism, characterized in that: The method comprises: Obtain a target detection image to be detected; Inputting the target detection image into a target detection model trained by a neural network including a 3D gated attention module to obtain target detection information in the target detection image; The 3D gated attention module includes a 3D attention module and a spatial channel reconstruction attention module. The 3D attention module is used to highlight the important features of the received feature map. The spatial channel reconstruction attention module is used to generate a feature map with more information from the received feature map. The 3D gated attention module is used to combine the feature maps processed by the 3D attention module and the spatial channel reconstruction attention module in parallel. The step of the 3D gated attention module for combining the feature maps processed by the 3D attention module and the spatial channel reconstruction attention module in parallel comprises: The feature map processed by the 3D attention module and the spatial channel reconstruction attention module is balanced with the representation capability of the adaptive adjustment model. blend together; Among them, X Z =λ t X 3D + (1-λ t )X 空 ,in, represents the final refined output, , where λ is a learnable training parameter, and the sigmoid function is used to project λ into the range of [0,1], X 3D and X 空 They are the feature maps processed by the 3D attention module and the spatial channel reconstruction attention module respectively.
2. The monocular 3D target detection method integrating the attention mechanism according to claim 1, characterized in that: The 3D attention module includes a channel attention module and a spatial attention module. The processing formula of the channel attention module for the received feature map is: ; Among them, X is the feature map received by the channel attention module, X comes from the convolution layer with a fixed kernel size, , W, H and C are the width, height and channel size of the feature map, swish and Relu are activation functions, T1 and T2 are two one-dimensional convolutions with a kernel size of 3, and Z c is the output after squeezing the cth channel.
3. The monocular 3D target detection method integrating the attention mechanism according to claim 2, characterized in that: The processing formula of the spatial attention module for the received feature map is: ; in, is the normalization factor, i is the index of the output position whose response is calculated, j is the index of all possible positions, and the pairwise function f is used to calculate the relationship between i and all j, using the dot product similarity To express, P n (X i ) represents the feature at position i, represents the feature at position j, , and g represent 1×1 convolution operations, and T is the weight matrix.
4. The monocular 3D target detection method integrating attention mechanism according to claim 1, characterized in that: The spatial channel reconstruction attention module includes a spatial reconstruction unit module and a channel reconstruction unit module. The processing formula of the spatial reconstruction unit for the received feature map is: ; ; Among them, W1 and W2 represent the weighted matrices for different features, Swish is the activation function, is the normalized correlation weight indicating the importance of different feature maps, X represents the input feature map, and are features with more information and features with less information, respectively, 11 , X 22 Represents the features of X1 and X2 on different channels, X 21 and X 12 They represent the features of X1 and X2 fused together. is element-wise multiplication, is element-wise addition, is to find the union, It is the spatially refined feature map obtained by the spatial reconstruction unit module.
5. The monocular 3D target detection method integrating the attention mechanism according to claim 4, characterized in that: The channel reconstruction unit module is used to refine the input spatial features Use 1×1 convolution kernel for compression and get the features respectively. and Features ; The characteristics As the input of the feature, GWC operation and PWC operation are performed respectively, and then the output is added , the feature As a supplement to the features, PWC operation is performed, and the results and features obtained are Take the union and get ; Use global average pooling technology to combine global spatial information and channel information to obtain pooled features and Features ; Features and Features Do SoftMax to get the feature weight vector and , according to the characteristics and Features and the feature weight vector and Get the output feature Y; in, .
6. The monocular 3D target detection method integrating the attention mechanism according to claim 1, characterized in that: The neural network also includes a residual-connected ResNet101 network and an FPN network for generating multi-level predictions, wherein the ResNet101 network is used to receive a target detection image, and the FPN network is used to receive features output by the ResNet101 network to generate a corresponding feature map.
7. A monocular 3D object detection device integrating an attention mechanism, characterized in that: A monocular 3D target detection method for implementing the fusion attention mechanism according to any one of claims 1 to 6, the device comprising: An acquisition module is used to acquire a detection image of a target to be detected; A detection module, configured to input the target detection image into a target detection model trained by a neural network including a 3D gated attention module to obtain target detection information in the target detection image; Among them, the 3D gated attention module includes a 3D attention module and a spatial channel reconstruction attention module. The 3D attention module is used to highlight the important features of the received feature map, and the spatial channel reconstruction attention module is used to generate a feature map with more information from the received feature map. The 3D gated attention module is used to combine the feature maps processed by the 3D attention module and the spatial channel reconstruction attention module in parallel.
8. A readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
9. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the method according to any one of claims 1 to 6 are implemented.
Citation Information
Patent Citations
Lightweight target detection method
CN114120019A
Target detection method and device based on improved attention mechanism
CN114612791A
Image target detection method based on double attention mechanism
CN114863236A
Structure-preserving weak supervision multi-mode prostate image registration method
CN114897948A
Multistage feature aggregation system and method for land cover classification problem
CN115035402A
Cited By
Monocular 3D target detection method and device
CN119206196A
