Target detection method and model based on semantic attention injection

Abstract

The invention discloses a target detection method and model based on semantic attention injection. The method comprises the following steps: preprocessing a target data set; obtaining a multi-scale basic feature map and a semantic feature map; and performing feature fusion to obtain a global feature fusion image, and performing decoding operation on the global target feature fusion image to obtain a target detection result. According to the method, the basic feature extraction and the semantic feature extraction are carried out on the input image, so that the local and semantic features in the image can be extracted more accurately; by means of the multi-scale feature pyramid, features of different scales can be considered at the same time, and detection of targets of different sizes is facilitated; through residual connection of the encoder and the decoder, semantic information of different scales can be better utilized, and the feature expression ability is improved; the mixed attention mechanism can pay attention to channels and space information in the feature map at the same time, and the utilization efficiency of the features is further improved; and non-maximum suppression processing is performed on the detection result, so that the accuracy of the detection result is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of target detection, and in particular relates to a target detection method based on injecting semantic attention. Background Art

[0002] In recent years, with the rapid development of artificial intelligence technology, computer vision has been widely used in various fields such as autonomous driving, security monitoring and robotics. Object detection is an important research direction in the field of computer vision. Its performance will directly affect the performance of subsequent tasks such as object tracking, action recognition and behavior understanding. Traditional object detection methods usually face challenges in accuracy and efficiency when dealing with complex scenes such as pedestrian detection. With the rapid development of GPU computing power and deep learning algorithms, object detection has been continuously developed and introduced detection methods based on deep learning, including the use of attention mechanisms to enhance feature representation. However, there are still problems that the original image provides less semantic information and the information of a single feature layer is not sufficient. The low-level feature map can present the target location but the semantic information is relatively scarce, and the high-level feature map has rich semantic information but is inaccurate, which makes it difficult for the network to fully learn the feature information of the target and there is a problem of insufficient recognition accuracy. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a target detection method based on injected semantic attention with high detection precision and accuracy.

[0004] The technical solution adopted to solve the above technical problems is: a target detection method based on injecting semantic attention, comprising the following steps:

[0005] Step 1. Target dataset preprocessing

[0006] Obtain a target data set, and perform scaling and image enhancement processing on the images in the data set to obtain a target detection data set;

[0007] Step 2. Obtain multi-scale basic feature maps and semantic feature maps

[0008] The images to be detected in the target detection dataset are subjected to feature extraction at different scales to obtain the first to fifth scale basic feature maps, and the deep first to third scale basic feature maps are selected and processed by additional convolutional layers to obtain multi-scale target basic feature maps;

[0009] At the same time, the images to be detected in the target detection dataset are processed by the position-aware mask attention module and the convolutional layer to obtain the target semantic feature maps of the first to third scales;

[0010] Step 3. Feature fusion to obtain the global feature fusion map

[0011] The multi-scale target basic feature map and the target semantic feature map are subjected to feature fusion processing by injecting an attention mechanism to obtain a feature fusion map. The feature fusion map is processed by convolution and batch normalization to obtain a global target feature fusion map.

[0012] The injection attention mechanism is:

[0013] Zi,c(x)=Conv{(1+Si,c(x))*Fi,c(x)}

[0014] Where Zi,c(x) is the global target feature fusion map, i is the feature space position, c is the index of the feature channel, S(x) is the attention information of the semantic information feature, and the value range is [0,1], Fi,c(x) is the multi-scale basic feature map, x is the feature, and Conv{} is the convolution process;

[0015] Step 4. Decode the global target feature fusion map to obtain the target detection result

[0016] Decoding the global target feature fusion map, mapping the bounding box coordinates of the global target feature fusion map back to the original image scale, and adjusting the bounding box position through the loss function to obtain the adjusted target feature map;

[0017] The object type and confidence level in the bounding box are determined through confidence loss and classification loss to obtain preliminary detection results.

[0018] The detection results are processed by non-maximum suppression to remove redundant bounding boxes and finally obtain the target detection results.

[0019] As a preferred technical solution, the method for jointly processing the position-aware mask attention module and the convolution layer in step 2 is: the image to be detected is downsampled at different scales to obtain downsampled first to third scale feature maps, the third scale feature map is downsampled to obtain a fourth scale feature map, the fourth scale feature map is upsampled to obtain a fifth scale upsampled feature map, the fifth scale upsampled feature map is upsampled to obtain a sixth scale upsampled feature map, the sixth scale upsampled feature map is upsampled to obtain a seventh scale upsampled feature map, the downsampled first, second, and third scale feature maps are respectively fused with the upsampled fifth, sixth, and seventh scale feature maps through residual connection processing, and the low-resolution, semantically strong features are combined with the high-resolution, semantically weak features to obtain a semantic feature map.

[0020] As a preferred technical solution, in step 3, when the scale of the semantic feature map is inconsistent with the scale of the multi-scale basic feature map, the scale of the semantic feature map is adjusted to the scale of the multi-scale basic feature map.

[0021] A target detection model based on injected semantic attention, the target detection model is composed of a basic target feature extraction module, a semantic target feature extraction module, a feature distribution space alignment module, a semantic injection attention module, and a result generation module;

[0022] The basic feature extraction module is used to extract the target basic features of the image to be detected and obtain a target basic feature map;

[0023] The semantic feature extraction module is used to extract the target semantic features of the image to be detected and obtain a target semantic feature map;

[0024] The feature distribution space alignment module is used to adjust the spatial structures of the target basic feature map and the target semantic feature map to be consistent;

[0025] The semantic injection attention module is used to inject the target semantic feature map into the target basic feature map through the attention mechanism to obtain a global target feature fusion map;

[0026] The result generation module uses a classifier to perform classification and regression processing on the target bounding box to generate a target detection result.

[0027] As a preferred technical solution, the basic feature extraction module is composed of a convolutional network module and a pooling module. The convolutional network module performs a spatial convolution operation on the input image and extracts local target features to obtain an initial target feature map. The pooling module performs multiple sampling rate operations on the initial target feature map, extracts feature information of different scale areas as a multi-scale feature pyramid, and selects the last three scales as the basic target feature map.

[0028] As a preferred technical solution, the semantic feature extraction module is composed of an encoder and a decoder. The encoder performs spatial convolution on the input image through a convolution operation to extract the target features of the layer image and obtain target semantic feature maps of different scales; the decoder maps the target semantic feature map back to the spatial structure of the original image. At the same time, the target semantic feature map in the encoder is residually connected with the corresponding feature map in the decoder, so that the decoder uses semantic feature information of different scales to form a semantic feature pyramid.

[0029] The beneficial effects of the present invention are as follows:

[0030] The present invention can more accurately extract local and semantic features in an image by performing basic feature extraction and semantic feature extraction on an input image; by utilizing a multi-scale feature pyramid, features of different scales can be considered simultaneously, which is helpful for detecting targets of different sizes; by using the residual connection of an encoder and a decoder, semantic information of different scales can be better utilized to improve the expressiveness of features; the hybrid attention mechanism can simultaneously focus on channel and spatial information in a feature map, further improving the utilization efficiency of features; and by performing non-maximum suppression processing on the detection results, the accuracy of the detection results is improved.

[0031] The invention has good flexibility and adaptability and can adapt to different image and target detection tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a flow chart of the target detection method based on injecting semantic attention of the present invention.

[0033] Figure 2 This is a schematic diagram of the feature injection operation process.

[0034] Figure 3 This is the structure diagram of the semantic injection attention target detection network. DETAILED DESCRIPTION

[0035] The present invention will be further described in detail below in conjunction with the accompanying drawings and examples, but the present invention is not limited to the following embodiments.

[0036] exist Figure 1 In the present embodiment, the object detection method based on injecting semantic attention includes the following steps:

[0037] Step 1. Target dataset preprocessing

[0038] Obtain a target data set, and perform scaling and image enhancement processing on the images in the data set to obtain a target detection data set;

[0039] Step 2. Obtain multi-scale basic feature maps and semantic feature maps

[0040] The images to be detected in the target detection dataset are subjected to feature extraction at different scales to obtain the first to fifth scale basic feature maps, and the deep first to third scale basic feature maps are selected and processed by additional convolutional layers to obtain multi-scale target basic feature maps;

[0041] At the same time, the images to be detected in the target detection dataset are processed by the position-aware mask attention module and the convolutional layer to obtain the target semantic feature maps of the first to third scales;

[0042] Among them, the joint processing method of the position-aware mask attention module and the convolutional layer is: the image to be detected is downsampled at different scales to obtain downsampled first to third scale feature maps, the third scale feature map is downsampled to obtain a fourth scale feature map, the fourth scale feature map is upsampled to obtain a fifth scale upsampled feature map, the fifth scale upsampled feature map is upsampled to obtain a sixth scale upsampled feature map, the sixth scale upsampled feature map is upsampled to obtain a seventh scale upsampled feature map, the downsampled first, second, and third scale feature maps are respectively fused with the upsampled fifth, sixth, and seventh scale feature maps through residual connection processing, and the low-resolution, semantically strong features are combined with the high-resolution, semantically weak features to obtain a semantic feature map.

[0043] Step 3. Feature fusion to obtain the global feature fusion map

[0044] The multi-scale target basic feature map and the target semantic feature map are subjected to feature fusion processing by injecting an attention mechanism to obtain a feature fusion map. When the scale of the semantic feature map is inconsistent with the multi-scale basic feature map, the scale of the semantic feature map is adjusted to the scale of the multi-scale basic feature map for feature fusion. The feature fusion map is subjected to convolution and batch normalization processing to obtain a global target feature fusion map.

[0045] like Figure 2 , the injection attention mechanism is:

[0046] Zi,c(x)=Conv{(1+Si,c(x))*Fi,c(x)}

[0047] Where Zi,c(x) is the global target feature fusion map, i is the feature space position, c is the index of the feature channel, S(x) is the attention information of the semantic information feature, and the value range is [0,1], Fi,c(x) is the multi-scale basic feature map, x is the feature, and Conv{} is the convolution process;

[0048] Step 4. Decode the global target feature fusion map to obtain the target detection result

[0049] Decoding the global target feature fusion map, mapping the bounding box coordinates of the global target feature fusion map back to the original image scale, and adjusting the bounding box position through the loss function to obtain the adjusted target feature map;

[0050] The object type and confidence level in the bounding box are determined through confidence loss and classification loss to obtain preliminary detection results.

[0051] The detection results are processed by non-maximum suppression to remove redundant bounding boxes and finally obtain the target detection results.

[0052] exist Figure 3 In this embodiment, the target detection model for injecting semantic attention is composed of a basic target feature extraction module, a semantic target feature extraction module, a feature distribution space alignment module, a semantic injection attention module, and a result generation module.

[0053] The basic feature extraction module is used to extract the target basic features of the image to be detected and obtain the target basic feature map.

[0054] The basic feature extraction module consists of a convolutional network module and a pooling module. The convolutional network module performs spatial convolution operations on the input image and extracts local target features to obtain an initial target feature map. The pooling module performs multiple sampling rate operations on the initial target feature map, extracts feature information of different scale areas as a multi-scale feature pyramid, and selects the last three scales as the basic target feature map.

[0055] A semantic feature extraction module is used to extract the target semantic features of the image to be detected and obtain a target semantic feature map;

[0056] The semantic feature extraction module consists of an encoder and a decoder. The encoder performs spatial convolution on the input image through convolution operations to extract the target features of the image layer and obtain target semantic feature maps of different scales; the decoder maps the target semantic feature map back to the spatial structure of the original image. At the same time, the target semantic feature map in the encoder is residually connected with the corresponding feature map in the decoder, so that the decoder uses semantic feature information of different scales to form a semantic feature pyramid.

[0057] The feature distribution space alignment module is used to adjust the spatial structures of the target basic feature map and the target semantic feature map to be consistent;

[0058] The semantic injection attention module is used to inject the target semantic feature map into the target basic feature map through the attention mechanism to obtain a global target feature fusion map;

[0059] The result generation module uses a classifier to perform classification and regression processing on the target bounding box to generate a target detection result.

Claims

1. A method for object detection based on injecting semantic attention, characterized in that: The following steps are involved: Step 1. Target dataset preprocessing Obtain a target data set, and perform scaling and image enhancement processing on the images in the data set to obtain a target detection data set; Step 2. Obtain multi-scale basic feature maps and semantic feature maps The images to be detected in the target detection dataset are subjected to feature extraction at different scales to obtain the first to fifth scale basic feature maps, and the deep first to third scale basic feature maps are selected and processed by additional convolutional layers to obtain multi-scale target basic feature maps; At the same time, the images to be detected in the target detection dataset are processed by the position-aware mask attention module and the convolutional layer to obtain the target semantic feature maps of the first to third scales; Step 3. Feature fusion to obtain the global feature fusion map The multi-scale target basic feature map and the target semantic feature map are subjected to feature fusion processing by injecting an attention mechanism to obtain a feature fusion map. The feature fusion map is processed by convolution and batch normalization to obtain a global target feature fusion map. The injection attention mechanism is: Zi,c(x)=Conv{(1+Si,c(x))*Fi,c(x)} Where Zi,c(x) is the global target feature fusion map, i is the feature space position, c is the index of the feature channel, S(x) is the attention information of the semantic information feature, and the value range is [0,1], Fi,c(x) is the multi-scale basic feature map, x is the feature, and Conv{} is the convolution process; Step 4. Decode the global target feature fusion map to obtain the target detection result Decoding the global target feature fusion map, mapping the bounding box coordinates of the global target feature fusion map back to the original image scale, and adjusting the bounding box position through the loss function to obtain the adjusted target feature map; The object type and confidence level in the bounding box are determined through confidence loss and classification loss to obtain preliminary detection results. The detection results are processed by non-maximum suppression to remove redundant bounding boxes and finally obtain the target detection results.

2. The object detection method based on injecting semantic attention according to claim 1, characterized in that: The method for jointly processing the position-aware mask attention module and the convolutional layer in the step 2 is as follows: the image to be detected is downsampled at different scales to obtain downsampled first to third scale feature maps, the third scale feature map is downsampled to obtain a fourth scale feature map, the fourth scale feature map is upsampled to obtain a fifth scale upsampled feature map, the fifth scale upsampled feature map is upsampled to obtain a sixth scale upsampled feature map, the sixth scale upsampled feature map is upsampled to obtain a seventh scale upsampled feature map, the downsampled first, second, and third scale feature maps are respectively fused with the upsampled fifth, sixth, and seventh scale feature maps through residual connection processing, and the low-resolution, semantically strong features are combined with the high-resolution, semantically weak features to obtain a semantic feature map.

3. The object detection method based on injecting semantic attention according to claim 1, characterized in that: In step 3, when the scales of the semantic feature map and the multi-scale basic feature map are inconsistent, the scale of the semantic feature map is adjusted to the scale of the multi-scale basic feature map.

4. An object detection model based on injected semantic attention, characterized in that: The target detection model is composed of a basic target feature extraction module, a semantic target feature extraction module, a feature distribution space alignment module, a semantic injection attention module, and a result generation module; The basic feature extraction module is used to extract the target basic features of the image to be detected and obtain a target basic feature map; The semantic feature extraction module is used to extract the target semantic features of the image to be detected and obtain a target semantic feature map; The feature distribution space alignment module is used to adjust the spatial structures of the target basic feature map and the target semantic feature map to be consistent; The semantic injection attention module is used to inject the target semantic feature map into the target basic feature map through the attention mechanism to obtain a global target feature fusion map; The result generation module uses a classifier to perform classification and regression processing on the target bounding box to generate a target detection result.

5. According to claim 4, a target detection model based on injected semantic attention is characterized in that: The basic feature extraction module is composed of a convolutional network module and a pooling module. The convolutional network module performs spatial convolution operations on the input image and extracts local target features to obtain an initial target feature map. The pooling module performs multiple sampling rate operations on the initial target feature map, extracts feature information of different scale areas as a multi-scale feature pyramid, and selects the last three scales as the basic target feature map.

6. According to claim 4, a target detection model based on injected semantic attention is characterized in that: The semantic feature extraction module is composed of an encoder and a decoder. The encoder performs spatial convolution on the input image through a convolution operation to extract the target features of the layer image and obtain target semantic feature maps of different scales. The decoder maps the target semantic feature map back to the spatial structure of the original image. At the same time, the target semantic feature map in the encoder is residually connected with the corresponding feature map in the decoder, so that the decoder uses semantic feature information of different scales to form a semantic feature pyramid.

Images