Target detection method and device based on pyramid integration and attention enhancement

Abstract

The application discloses a target detection method and device based on pyramid integration and attention enhancement, and the method comprises the following steps: acquiring a to-be-detected image of a current scene; determining a corresponding target detection result in the to-be-detected image according to the to-be-detected image and a target detection model; wherein the target detection model comprises a feature extraction network, a pyramid integration network, an FPN network, an attention module and a detection module; the pyramid integration network comprises a max-pooling layer and a dilated convolution layer; and the max-pooling layer and the dilated convolution layer operate in parallel. In this way, small targets and large targets can be detected respectively, different scale features can be extracted, the diversity of receptive field features is improved, and the accuracy of small target and large target detection is improved, thereby improving the accuracy of the target detection result.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a target detection method and apparatus based on pyramid ensemble and attention enhancement. Background Technology

[0002] With the development of computer vision, object detection methods have been widely used in fields such as mask wearing detection, posture detection, robotic systems, medical image detection, and autonomous driving.

[0003] In existing technologies, after the feature extraction network, the feature maps after dimensionality reduction from different layers are usually directly fused.

[0004] In the above approach, the feature maps after dimensionality reduction at different layers have different receptive fields and semantic information. Direct fusion will weaken the accuracy of multi-scale features. In addition, the insufficient receptive field of the fused feature map will also lead to inaccurate detection results for small targets. Summary of the Invention

[0005] This application provides a target detection method and apparatus based on pyramid ensemble and attention enhancement, which improves the accuracy of small and large target detection and the accuracy of target detection results.

[0006] In a first aspect, embodiments of this application provide a target detection method based on pyramid ensemble and attention enhancement, the method comprising:

[0007] Obtain the image to be detected in the current scene;

[0008] Based on the image to be detected and the target detection model, determine the target detection result in the image to be detected;

[0009] The target detection model includes: a feature extraction network, a pyramid ensemble network, an FPN network, an attention module, and a detection module; the pyramid ensemble network includes a max pooling layer and a dilated convolutional layer; the max pooling layer and the dilated convolutional layer operate in parallel.

[0010] Optionally, determining the target detection result in the image to be detected based on the image to be detected and the target detection model includes:

[0011] Based on the feature extraction network, feature extraction is performed on the image to be detected to obtain a deep feature map;

[0012] Based on the pyramid ensemble network, multi-scale feature extraction is performed on the deep feature map to obtain a context information feature map;

[0013] Based on the FPN network, feature fusion is performed on the deep feature map and the context information feature map to obtain a first fused feature map, a second fused feature map, and a third fused feature map. The first fused feature map, the second fused feature map, and the third fused feature map have different sizes.

[0014] Based on the attention module, feature association is performed on the first fused feature map, the second fused feature map, and the third fused feature map to obtain the target feature map;

[0015] The target feature map is input into the detection network to determine the target detection result.

[0016] Optionally, the target detection model is trained in the following manner:

[0017] Acquire training images;

[0018] Construct a training sample set, wherein each training sample in the training sample set includes the training image;

[0019] The neural network model is trained using the training sample set to obtain the output result;

[0020] Determine the error between the output result and the target category;

[0021] If the error does not meet the requirements, adjust the parameters and retrain the adjusted model to obtain the target detection model.

[0022] Optionally, the feature extraction network includes an EfficientNet-B2 network, which is composed of MBConv convolutional modules.

[0023] Optionally, the pyramid ensemble network further includes an SE attention mechanism;

[0024] The step of performing multi-scale feature extraction on the deep feature map based on the pyramid ensemble network to obtain a context information feature map includes:

[0025] After processing the deep feature map through the max pooling layer and the dilated convolutional layer, the receptive field feature map is obtained.

[0026] The receptive field feature map and the deep feature map are processed by the SE attention mechanism to obtain the context information feature map.

[0027] Optionally, the attention module includes a channel attention module and a spatial attention module, wherein the channel attention module and the spatial attention module operate in series.

[0028] Based on the attention module, feature association is performed on the first fused feature map, the second fused feature map, and the third fused feature map to obtain the target feature map, including:

[0029] Based on the channel attention module, the first fused feature map, the second fused feature map, and the third fused feature map are processed to obtain the fourth fused feature map, the fifth fused feature map, and the sixth fused feature map.

[0030] The fourth, fifth, and sixth fused feature maps are unified in resolution using the cross-alignment module and then added together to obtain the seventh fused feature map.

[0031] The target feature map is obtained by processing the seventh fusion feature map based on the spatial attention module.

[0032] Optionally, the step of processing the seventh fused feature map based on the spatial attention module to obtain the target feature map includes:

[0033] The target feature map is obtained by processing the seventh fused feature map through asymmetric convolution.

[0034] Secondly, embodiments of this application provide a target detection device based on pyramid ensemble and attention enhancement, comprising:

[0035] The acquisition module is used to acquire the image to be detected in the current scene;

[0036] The determination module is used to determine the target detection result in the image to be detected based on the image to be detected and the target detection model;

[0037] The target detection model includes: a feature extraction network, a pyramid ensemble network, an FPN network, an attention module, and a detection module; the pyramid ensemble network includes a max pooling layer and a dilated convolutional layer; the max pooling layer and the dilated convolutional layer operate in parallel.

[0038] Thirdly, embodiments of this application provide an electronic device, including: a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the electronic device performs the method described in the first aspect above.

[0039] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer instructions, which, when executed by a processor, are used to implement the method described in the first aspect above.

[0040] In the above scheme, the target detection model can include a pyramid ensemble network, which can include max pooling layers and dilated convolutional layers. The max pooling layers and dilated convolutional layers run in parallel. The max pooling layers can obtain local features, while the dilated convolutional layers can increase the receptive field to obtain global features. In this way, small targets and large targets can be detected separately, realizing the extraction of features at different scales, improving the diversity of receptive field features, and thus improving the accuracy of small and large target detection, thereby improving the accuracy of target detection results. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a flowchart of the target detection method based on pyramid ensemble and attention enhancement provided in the embodiments of this application;

[0043] Figure 2 This is a schematic diagram of the target detection model provided in the embodiments of this application;

[0044] Figure 3 This is a schematic diagram of the EfficientNet-B2 module provided in an embodiment of this application;

[0045] Figure 4 This is a schematic diagram of the pyramid integrated network provided in the embodiments of this application;

[0046] Figure 5 A schematic diagram of the attention module provided in an embodiment of this application;

[0047] Figure 6 A schematic diagram of the channel attention module provided in an embodiment of this application;

[0048] Figure 7 A schematic diagram of the cross alignment module provided in an embodiment of this application;

[0049] Figure 8 A schematic diagram of the spatial attention module provided in an embodiment of this application;

[0050] Figure 9 This is a block diagram of a target detection method apparatus provided in an embodiment of this application.

[0051] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0053] Based on this, embodiments of this application provide a target detection method and apparatus based on pyramid ensemble and attention enhancement, which improves the accuracy of small and large target detection and improves the accuracy of target detection results.

[0054] This embodiment provides a target detection method based on pyramid ensemble and attention enhancement. Figure 1 This is a flowchart of a target detection method based on pyramid ensemble and attention enhancement provided in an embodiment of this application, as shown below. Figure 1 The method may include the following steps:

[0055] S101, Obtain the image to be detected in the current scene.

[0056] The current scene can be autonomous driving, or it can be other scenes such as a stadium. For example, a camera can be used to acquire an image of the current scene to be detected.

[0057] S102, Based on the image to be detected and the target detection model, determine the corresponding target detection result in the image to be detected.

[0058] An object detection model can be implemented based on a neural network and is used to detect objects in an image. For example, the image to be detected can be input into the object detection model to output the object detection result, such as a classification result.

[0059] The target detection model may include: a feature extraction network, a pyramid ensemble network, an FPN module, an attention module, and a detection module. The pyramid ensemble network may include a max pooling layer and a dilated convolutional layer, with the max pooling layer and the dilated convolutional layer operating in parallel.

[0060] Max pooling layers extract the maximum value from the local receptive field. By using different pooling sizes, different levels of feature information are extracted to obtain local features, thereby reducing noise interference and increasing the model's receptive field. Dilated convolutional layers (i.e., expanded convolutional layers) pad the convolutional kernel with zeros. This expands the receptive field while maintaining resolution, enabling multi-scale object detection. Furthermore, by setting the rate (dilation ratio) in dilated convolutional layers, global features can be obtained, thus fusing multi-scale information to obtain rich contextual information and achieve diversity of features across different receptive fields.

[0061] The image to be detected is first processed by a feature extraction network to extract features, resulting in a deep feature map. Since the receptive fields of the deep feature maps after dimensionality reduction are different at different layers, there are differences in semantic information. Based on this, this disclosure uses two different receptive field enhancement methods: max pooling layer and dilated convolutional layer, to obtain rich contextual information, thereby realizing the extraction of receptive field features of different sizes, improving the diversity of receptive field features, improving the detection effect of small and large targets, and improving the accuracy of target detection results.

[0062] In the above scheme, the target detection model can include a pyramid ensemble network, which can include max pooling layers and dilated convolutional layers. The max pooling layers and dilated convolutional layers run in parallel. The max pooling layers can obtain local features, while the dilated convolutional layers can increase the receptive field to obtain global features. In this way, small targets and large targets can be detected separately, realizing the extraction of features at different scales, improving the diversity of receptive field features, and thus improving the accuracy of small and large target detection, thereby improving the accuracy of target detection results.

[0063] In one possible embodiment, determining the target detection result in the image to be detected based on the image to be detected and the target detection model includes:

[0064] Based on the feature extraction network, feature extraction is performed on the image to be detected to obtain a deep feature map;

[0065] Based on the pyramid ensemble network, multi-scale feature extraction is performed on the deep feature map to obtain a context information feature map;

[0066] Based on the FPN network, feature fusion is performed on the deep feature map and the context information feature map to obtain a first fused feature map, a second fused feature map, and a third fused feature map, wherein the first fused feature map, the second fused feature map, and the third fused feature map have different sizes;

[0067] Based on the attention module, feature association is performed on the first fused feature map, the second fused feature map, and the third fused feature map to obtain the target feature map;

[0068] The target feature map is input into the detection network to determine the target detection result.

[0069] Figure 2 This is a schematic diagram of the target detection model provided in an embodiment of this application. For example... Figure 2 As shown, the feature extraction network in this disclosure is the EfficientNet-B2 network.

[0070] As shown in the figure, after multiple convolutions in the feature extraction network, a deep feature map is obtained from the image to be detected. This deep feature map is then processed by a pyramid ensemble network for multi-scale feature extraction, resulting in a contextual information feature map. The FPN network then fuses the deep feature map and the contextual information feature map to obtain a first fused feature map, a second fused feature map, and a third fused feature map. Next, these three fused feature maps are associated by an attention module to obtain a target feature map. This target feature map can then be input into the detection network to obtain the target detection result for the image to be detected, thus enabling accurate detection of small, medium, and large targets.

[0071] As shown in the figure, the specific operations of the FPN network are as follows: The context information feature map is first processed by CBL (convolution, normalization, activation) to obtain the first fused feature map. The first fused feature map is then processed by multiple CBLs and upsampling processes to obtain the feature map, which is then fused with the neighboring deep feature maps to obtain the second fused feature map. The second fused feature map is then processed by CBLs and upsampling processes to obtain the feature map, which is then fused with the neighboring deep feature maps to obtain the third fused feature map.

[0072] In one possible embodiment, the object detection model is trained in the following manner:

[0073] Acquire training images;

[0074] Construct a training sample set, wherein each training sample in the training sample set includes the training image;

[0075] The neural network model is trained using the training sample set to obtain the output result;

[0076] Determine the error between the output result and the target category;

[0077] If the error does not meet the requirements, adjust the parameters and retrain the adjusted model to obtain the target detection model.

[0078] For example, a neural network model can be trained using training images. This involves using training images from a training sample set as input to the neural network model and the target category as its output, thus training the model. For instance, by inputting training images into the neural network model, calculating the loss based on the corresponding target category and the output result, and then adjusting the model parameters according to the loss value to ensure the neural network model achieves the required accuracy.

[0079] The model training is considered complete when the model accuracy reaches the preset accuracy requirement or the number of training iterations reaches the preset threshold. Once training is complete, the object detection model is obtained.

[0080] Therefore, by using the above method, training images can be used as the training sample set for the object detection model. The model can then be trained based on this training sample set to obtain an output result that can accurately predict the image to be detected, thereby improving the detection accuracy and thus improving the object detection accuracy.

[0081] In one possible embodiment, the feature extraction network includes an EfficientNet-B2 network, which is composed of MBConv convolutional modules.

[0082] Figure 3 A schematic diagram of the EfficientNet-B2 network provided in the embodiments of this application is shown below. Figure 2 As shown and Figure 3 As shown, the EfficientNet-B2 network includes the MBConv convolutional module, which comprises 1x1 convolutions (for dimensionality upscaling), depthwise convolutions (depthwise separable convolutions), SENet (Squeeze-and-Excitation Networks, SE attention mechanism), 1x1 convolutions (for dimensionality reduction), and residual connections. Compared to other feature extraction networks, the EfficientNet-B2 network has a smaller number of parameters, which can improve the accuracy of object detection.

[0083] In one possible embodiment, the pyramid ensemble network further includes an SE attention mechanism;

[0084] The step of performing multi-scale feature extraction on the deep feature map based on the pyramid ensemble network to obtain a context information feature map includes:

[0085] After processing the deep feature map through the max pooling layer and the dilated convolutional layer, the receptive field feature map is obtained.

[0086] The receptive field feature map and the deep feature map are processed by the SE attention mechanism to obtain the context information feature map.

[0087] Figure 4 This is a schematic diagram of the pyramid integrated network provided in the embodiments of this application, as shown below. Figure 4 As shown, a deep feature map can be represented as x∈R H×W×C Where H, W, and C represent the height, width, and number of channels of the feature map, respectively. As shown in the figure, the void ratio R in this application... i ={5, 7, 9}, pooling size P i ={5, 7, 9}, with a dilated convolution kernel size of 3×3.

[0088] The specific operations in the pyramid ensemble network are as follows: First, the deep feature maps are passed through dilated convolution and max pooling layers in parallel. The specific operations of the dilated convolution and max pooling layers are shown in Equation (1) and Equation (2), respectively.

[0089] x i =Conv 3×3 (x)(i=1,2,3) Equation (1)

[0090] y i =Maxpooling 3×3 (x)(i=1,2,3) Equation (2)

[0091] Where, x i (i=1, 2, 3) and y i (i = 1, 2, 3) represents the input x ∈ R. H×W×C The branch output after the dilated convolutional layer and max pooling layer operations. For example,

[0092] Then we add the outputs of different branches element by element, as shown in Equation 3, to obtain the feature map of each branch.

[0093]

[0094] Among them, F i This represents the branch feature map.

[0095] Next, we fuse the three branch feature maps and the deep feature map to obtain the receptive field feature map.

[0096] Although the receptive field feature map integrates rich contextual information, it also contains redundant information. To eliminate the influence of redundant information, the SE attention mechanism is introduced into the pyramid ensemble network. The specific operation of the SE attention mechanism is as follows: the receptive field feature map is subjected to global average pooling to obtain the global statistical information of the feature map. The global statistical information is mapped into a vector called the "feature descriptor" through two fully connected layers (one compression layer and one activation layer). Then, the feature descriptor is passed through the sigmoid activation function to obtain an attention vector. Finally, the attention vector is multiplied with the original feature map to obtain the attention-weighted feature map, which is the context information feature map.

[0097] In this way, different receptive fields are obtained by using dilated convolutions with different dilation rates and pooling sizes, thereby obtaining features of different receptive fields and improving the accuracy of multi-scale object detection. The SE attention mechanism solves the information redundancy problem in multi-scale information fusion, provides data support for object detection, and improves the accuracy of object detection.

[0098] In one possible embodiment, the attention module includes a channel attention module and a spatial attention module, wherein the channel attention module and the spatial attention module operate in series.

[0099] Based on the attention module, feature association is performed on the first fused feature map, the second fused feature map, and the third fused feature map to obtain the target feature map, including:

[0100] Based on the channel attention module, the first fused feature map, the second fused feature map, and the third fused feature map are processed to obtain the fourth fused feature map, the fifth fused feature map, and the sixth fused feature map.

[0101] The fourth, fifth, and sixth fused feature maps are unified in resolution using the cross-alignment module and then added together to obtain the seventh fused feature map.

[0102] The target feature map is obtained by processing the seventh fusion feature map based on the spatial attention module.

[0103] In the FPN network, the context information feature map is upsampled and fused with the neighboring deep feature maps along the channel dimension. However, this simple and direct fusion of features at different scales introduces a large amount of conflicting and redundant information, failing to reflect the correlation and importance between the semantic and spatial information of each feature layer. Based on this, this application proposes an attention module.

[0104] Figure 5 The diagram illustrates the attention module provided in this embodiment. As shown, CAM is the channel attention module, Cross-align is the cross-alignment module, and SAM is the spatial attention module. The first, second, and third fused feature maps are processed by the channel attention module to obtain the fourth, fifth, and sixth fused feature maps, respectively. This suppresses the influence of redundant information in the FPN network and improves the accuracy of target detection. The fourth, fifth, and sixth fused feature maps are upsampled and downsampled by the cross-alignment module and then fused to obtain the seventh fused feature map. The seventh fused feature map is first subjected to a 1x1 convolution and then processed by the spatial attention module to obtain the target feature map. This strengthens the connection between channels, ensuring that the seventh fused feature map contains feature information from the other two fused feature maps, thereby improving the accuracy of target detection.

[0105] Figure 6The diagram illustrates the channel attention module provided in this embodiment. To extract richer features, we first use parallel max pooling and global average pooling to process the first, second, and third fused feature maps respectively, obtaining two 1×1×C feature maps. Then, we use two 1×1 convolutions to address the channel scale issue. Specifically, the pooled 1×1×C features are first fed into two 1×1 convolutions. The first 1×1 convolution reduces the dimensionality of the input features, compressing the number of channels to 1 / r. After passing through BatchNorm and ReLU (Rectified Linear Unit), the second 1×1 convolution expands and increases the dimensionality, restoring the number of channels to the same as the original input channel number, where r is the channel scaling ratio. Then, the two features after pooling are added, and after passing through the sigmoid activation function, adaptive channel-based weights are generated. Finally, multiplication is used to weight the original features channel by channel, completing the channel-guided fusion. Specifically, the first fused feature map, the second fused feature map, and the third fused feature map can be represented by P1, P2, P3 ∈ R, respectively. H×W×C Let represent the maximum pooling MAP(P) as follows: i ) and Global Average Pooling (GAP) i The features P′ refined through channel attention i It can be represented as:

[0106]

[0107]

[0108] Where σ represents the sigmoid activation function, F att (P i P' represents the weights of the attention vector generated after channel attention. i This represents the final refined features. This represents element-wise addition. This represents element-wise multiplication. This allows for full utilization of the spatial detail information of shallow features and the semantic information of deep features, enabling more efficient fusion of low-resolution and high-resolution feature maps of the entire image within the network, thereby improving the accuracy of object detection.

[0109] like Figure 7The diagram shown is a schematic of the cross-alignment module provided in this application embodiment. The specific operation of the cross-alignment module is as follows: The fourth fusion feature map is enlarged by upsampling using bilinear interpolation; the sixth fusion feature map is reduced by a factor of 's' by downsampling; and finally, the three feature maps of different scales are aligned and fused. For example, the sixth fusion feature map can be 52×52×48, the fifth fusion feature map can be 26×26×48, and the fourth fusion feature map can be 13×13×352. The feature map of size 52×52×48 can be downsampled to 26×26×48, and the feature map of size 13×13×352 can be upsampled to 26×26×352. For the feature map of size 26×26×120, its identity mapping remains unchanged. Then, they are added along the channel dimension to obtain the seventh fusion feature map of size 26×26×520. The formula is defined as follows:

[0110]

[0111] Similarly, for the other two feature maps, a similar sampling operation is performed to ensure resolution consistency, as defined in the following formula:

[0112]

[0113]

[0114] Where y l ,y l+1 ,y l+2 Representing different feature layers; This indicates element-wise addition.

[0115] In one possible embodiment, processing the seventh fusion feature map based on the spatial attention module to obtain the target feature map includes:

[0116] The target feature map is obtained by processing the seventh fused feature map through asymmetric convolution.

[0117] To further enhance object detection capabilities in the spatial dimension and reduce interference from complex background factors in road scenes, while considering that the same parts of features from different channels have the same spatial weights, and that the use of max pooling and global average pooling layers in channel attention generally smooths the global pixel information of the feature map, thus destroying the spatial pixel information and semantic features of the features, this application uses a spatial attention module to further process the seventh fused feature map.

[0118] For example, the seventh fused feature map can be represented as F∈R H×W×c , where H, W, and C represent the height, width, and number of channels of the feature map, respectively. Figure 8This is a schematic diagram of the spatial attention module provided in an embodiment of this application. As shown in the figure, the specific operation of the spatial attention module is as follows: First, two tensors of size H×W×1 are generated by cascaded convolutional layers of 1×3 convolution and 3×1 convolution and 3×1 convolution and 1×3 convolution, respectively. Then, they are concatenated by channel and stacked together. Then, the number of channels is changed to 1 while keeping H and W unchanged by convolution operation. Then, the learned spatial feature weights are mapped to the range [0, 1] by the Sigmoid activation function. Finally, the feature is multiplied pixel by pixel with the input feature to obtain the target feature map at the spatial level.

[0119] Thus, considering the small input of the seventh fusion feature map, asymmetric convolution is used to process each feature point differently. Different kernel sizes can be used in different directions, thereby better preserving the information of each channel and helping to generate more effective feature maps. Furthermore, asymmetric convolution can share parameters, which reduces the amount of computation to some extent and thus improves the training speed of the model.

[0120] Based on the same inventive concept, this embodiment provides a target detection device based on pyramid integration and attention enhancement. Figure 9 This is a block diagram of a target detection device based on pyramid ensemble and attention enhancement provided in an embodiment of this application, such as... Figure 9 The device may include:

[0121] The acquisition module 910 is used to acquire the image to be detected in the current scene;

[0122] The determining module 920 is used to determine the target detection result in the image to be detected based on the image to be detected and the target detection model;

[0123] The target detection model includes: a feature extraction network, a pyramid ensemble network, an FPN network, an attention module, and a detection module; the pyramid ensemble network includes a max pooling layer and a dilated convolutional layer; the max pooling layer and the dilated convolutional layer operate in parallel.

[0124] Optionally, the determining module 920 includes:

[0125] The first extraction submodule is used to extract features from the image to be detected based on the feature extraction network to obtain a deep feature map;

[0126] The second extraction submodule is used to perform multi-scale feature extraction on the deep feature map based on the pyramid ensemble network to obtain a context information feature map.

[0127] The fusion submodule is used to perform feature fusion on the deep feature map and the context information feature map based on the FPN network to obtain a first fused feature map, a second fused feature map, and a third fused feature map, wherein the first fused feature map, the second fused feature map, and the third fused feature map have different sizes;

[0128] The association submodule is used to perform feature association on the first fused feature map, the second fused feature map, and the third fused feature map based on the attention module to obtain the target feature map;

[0129] The input submodule is used to input the target feature map into the detection network and determine the target detection result.

[0130] Optionally, the target detection model is trained in the following manner:

[0131] Acquire training images;

[0132] Construct a training sample set, wherein each training sample in the training sample set includes the training image;

[0133] The neural network model is trained using the training sample set to obtain the output result;

[0134] Determine the error between the output result and the target category;

[0135] If the error does not meet the requirements, adjust the parameters and retrain the adjusted model to obtain the target detection model.

[0136] Optionally, the feature extraction network includes an EfficientNet-B2 network, which is composed of MBConv convolutional modules.

[0137] Optionally, the pyramid ensemble network further includes an SE attention mechanism; the second extraction submodule includes:

[0138] The first processing submodule is used to process the deep feature map through the max pooling layer and the dilated convolutional layer to obtain the receptive field feature map;

[0139] The second processing submodule is used to process the receptive field feature map and the deep feature map through the SE attention mechanism to obtain the context information feature map.

[0140] Optionally, the attention module includes a channel attention module and a spatial attention module, wherein the channel attention module and the spatial attention module operate in series.

[0141] The associated sub-modules include:

[0142] The third processing submodule is used to process the first fused feature map, the second fused feature map, and the third fused feature map based on the channel attention module to obtain the fourth fused feature map, the fifth fused feature map, and the sixth fused feature map.

[0143] The addition submodule is used to add the fourth fused feature map, the fifth fused feature map, and the sixth fused feature map after unifying their resolution based on the cross-alignment module, to obtain the seventh fused feature map;

[0144] The fourth processing submodule is used to process the seventh fusion feature map based on the spatial attention module to obtain the target feature map.

[0145] Optionally, the fourth processing submodule includes:

[0146] The fifth processing submodule is used to process the seventh fused feature map through asymmetric convolution to obtain the target feature map.

[0147] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 10 As shown, the electronic device 100 includes:

[0148] Processor 101 and memory 102;

[0149] The memory 102 stores computer instructions;

[0150] The processor 101 executes the computer instructions stored in the memory 102, causing the processor 101 to perform the above-described target detection method based on pyramid ensemble and attention enhancement.

[0151] The specific implementation process of processor 101 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0152] Optionally, the electronic device 100 also includes a communication component 103. The processor 101, memory 102, and communication component 103 can be connected via a bus 104.

[0153] This application also provides a computer-readable storage medium storing computer instructions, which, when executed by a processor, are used to implement the above-described target detection method based on pyramid ensemble and attention enhancement.

[0154] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0155] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A target detection method based on pyramid ensemble and attention enhancement, characterized in that, The method includes: Obtain the image to be detected in the current scene; Based on the image to be detected and the target detection model, determine the target detection result in the image to be detected; The target detection model includes: a feature extraction network, a pyramid ensemble network, an FPN network, an attention module, and a detection module; the pyramid ensemble network includes a max pooling layer, a dilated convolutional layer, and an SE attention mechanism; the max pooling layer and the dilated convolutional layer operate in parallel; The step of determining the target detection result in the image to be detected based on the image to be detected and the target detection model includes: Based on the feature extraction network, feature extraction is performed on the image to be detected to obtain a deep feature map; Multi-scale feature extraction is performed on the deep feature map based on the pyramid ensemble network to obtain a context information feature map; this includes: processing the deep feature map through the max pooling layer and the dilated convolutional layer to obtain a receptive field feature map; and processing the receptive field feature map and the deep feature map through the SE attention mechanism to obtain the context information feature map.

2. The method according to claim 1, characterized in that, The step of determining the target detection result in the image to be detected based on the image to be detected and the target detection model includes: Based on the FPN network, feature fusion is performed on the deep feature map and the context information feature map to obtain a first fused feature map, a second fused feature map, and a third fused feature map. The first fused feature map, the second fused feature map, and the third fused feature map have different sizes. Based on the attention module, feature association is performed on the first fused feature map, the second fused feature map, and the third fused feature map to obtain the target feature map; The target feature map is input into the detection network to determine the target detection result.

3. The method according to claim 1, characterized in that, The target detection model is trained in the following way: Acquire training images; Construct a training sample set, wherein each training sample in the training sample set includes the training image; The neural network model is trained using the training sample set to obtain the output result; Determine the error between the output result and the target category; If the error does not meet the requirements, adjust the parameters and retrain the adjusted model to obtain the target detection model.

4. The method according to claim 1, characterized in that, The feature extraction network includes the EfficientNet-B2 network, which is composed of MBConv convolutional modules.

5. The method according to claim 2, characterized in that, The attention module includes a channel attention module and a spatial attention module, which operate in series. Based on the attention module, feature association is performed on the first fused feature map, the second fused feature map, and the third fused feature map to obtain the target feature map, including: Based on the channel attention module, the first fused feature map, the second fused feature map, and the third fused feature map are processed to obtain the fourth fused feature map, the fifth fused feature map, and the sixth fused feature map. The fourth, fifth, and sixth fused feature maps are unified in resolution using the cross-alignment module and then added together to obtain the seventh fused feature map. The target feature map is obtained by processing the seventh fusion feature map based on the spatial attention module.

6. The method according to claim 5, characterized in that, The step of processing the seventh fused feature map based on the spatial attention module to obtain the target feature map includes: The target feature map is obtained by processing the seventh fused feature map through asymmetric convolution.

7. A target detection device based on pyramid ensemble and attention enhancement, characterized in that, include: The acquisition module is used to acquire the image to be detected in the current scene; The determination module is used to determine the target detection result in the image to be detected based on the image to be detected and the target detection model; The target detection model includes: a feature extraction network, a pyramid ensemble network, an FPN network, an attention module, and a detection module; the pyramid ensemble network includes a max pooling layer, a dilated convolutional layer, and an SE attention mechanism; the max pooling layer and the dilated convolutional layer operate in parallel; The determining module is specifically used to extract features from the image to be detected based on the feature extraction network to obtain a deep feature map; to extract multi-scale features from the deep feature map based on the pyramid ensemble network to obtain a context information feature map; further, it is used to process the deep feature map through the max pooling layer and the dilated convolutional layer to obtain a receptive field feature map; and to process the receptive field feature map and the deep feature map through the SE attention mechanism to obtain the context information feature map.

8. An electronic device comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it causes the electronic device to implement the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6.

Images