Method, apparatus and electronic device for object detection

Abstract

The application provides a target detection method, device and electronic equipment, wherein the method comprises: pre-processing a target image based on a spatial attention mechanism to obtain an identifiable image of the target image; performing layer-by-layer feature extraction on the identifiable image through a plurality of feature extraction layers connected in sequence to respectively obtain feature images extracted layer by layer; performing feature fusion on the feature images extracted layer by layer through upsampling to obtain a plurality of feature fusion images with different spatial sizes; and obtaining a prediction result of a target object with different sizes according to the plurality of feature fusion images with different spatial sizes. The application can improve target detection accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of digital image processing, and particularly relates to a target detection method and device and electronic equipment. BACKGROUND

[0002] Target detection has been one of the hot research topics in the field of computer vision. Target detection is widely used in various fields such as medicine, agriculture, industry, artificial intelligence, and has achieved good results, greatly improving people's quality of life. The task of target detection is to identify the category of the target in the image and locate the position of the target.

[0003] In the image target detection network, some researchers use the Faster R-CNN+FPN method to improve the detection effect. In the Faster R-CNN, ResNet50 is used as the feature extraction backbone network. In the feature extraction process of ResNet50 from bottom to top, many image information is lost through multiple convolution and pooling operations, resulting in poor target detection effect. In the FPN propagation path from top to bottom, the semantic information of the upper layer cannot be fully fused with the detailed information of the lower layer (especially between the cross-layer feature layers), and the input of the detection layer only comes from the same layer feature layer. These problems affect the detection effect to some extent. In the horizontal connection of ResNet50 and FPN, the existing network directly clips the channel number of the feature layer, resulting in a large amount of image information lost by the network, reducing the target detection accuracy, and causing the target detection network to be unable to effectively detect small objects in the image. SUMMARY

[0004] One purpose of the present application is to fully utilize the detailed information of the shallow layer and the semantic information of the deep layer, and solve the technical problem that small objects in the image cannot be effectively detected.

[0005] According to an aspect of an embodiment of the present application, a target detection method is provided, comprising:

[0006] Preprocessing a target image based on a spatial attention mechanism to obtain an identifiable image of the target image;

[0007] Performing layer-by-layer feature extraction on the identifiable image through a plurality of feature extraction layers connected in series to obtain feature images extracted layer by layer;

[0008] Performing feature fusion on the feature images extracted layer by layer through up-sampling to obtain a plurality of feature fusion images with different spatial sizes;

[0009] Obtaining a prediction result of a target object with different sizes according to the plurality of feature fusion images with different spatial sizes.

[0010] According to an aspect of the embodiment of the present application, the target image is preprocessed based on the spatial attention mechanism to obtain an identifiable image of the target image, comprising:

[0011] The global maximum pooling and the global average pooling are performed on the target image respectively and fused to obtain a pooling fusion feature;

[0012] The spatial attention is obtained by stacking the convolution layer on the pooling fusion feature;

[0013] The spatial attention is activated by the sigmoid function and applied to the target image to obtain the identifiable image of the target image.

[0014] According to an aspect of the embodiment of the present application, the identifiable image is extracted layer by layer through a plurality of feature extraction layers connected in series, the plurality of feature extraction layers connected in series include a first type of feature extraction layer and a plurality of second type of feature extraction layers, and the identifiable image is extracted layer by layer to obtain a feature image extracted layer by layer, comprising:

[0015] The first feature image is obtained by passing the identifiable image through a first type of feature extraction layer;

[0016] The first feature image is sequentially passed through a plurality of second type of feature extraction layers to obtain a feature image extracted layer by layer.

[0017] According to an aspect of the embodiment of the present application, after sequentially passing through a plurality of second type of feature extraction layers, the input image is passed through a channel attention mechanism to obtain a channel attention feature image, comprising:

[0018] The input image is subjected to global maximum pooling and global average pooling respectively to obtain global maximum channel attention and global average channel attention;

[0019] The global maximum channel attention and the global average channel attention are activated by the sigmoid function and applied to the input image respectively to obtain a channel attention feature image.

[0020] According to an aspect of the embodiment of the present application, the feature images extracted layer by layer are fused by upsampling to obtain a plurality of feature fusion images with different spatial sizes, comprising:

[0021] The feature images extracted layer by layer are kept consistent in dimension by convolution to obtain feature images corresponding to the same dimension;

[0022] The feature images of the same dimension are fused by upsampling to obtain a plurality of feature fusion images with different spatial sizes.

[0023] According to an aspect of the embodiment of the present application, the feature fusion of the feature images of the same dimension through upsampling obtains a feature fusion image, including:

[0024] The feature fusion is performed through nearest neighbor interpolation in upsampling to obtain a feature fusion image.

[0025] According to an aspect of the embodiment of the present application, the prediction result of the target object of different sizes is obtained according to the plurality of feature fusion images of different spatial sizes, including:

[0026] The prediction result of the target object of different sizes is obtained by respectively performing prediction on the plurality of feature fusion images of different spatial sizes.

[0027] According to an aspect of the embodiment of the present application, a target detection device is applied, including:

[0028] The image preprocessing module is configured to preprocess a target image based on a spatial attention mechanism to obtain an identifiable image of the target image.

[0029] The feature extraction module is configured to perform layer-by-layer feature extraction on the identifiable image through a plurality of feature extraction layers connected in series to obtain feature images extracted layer by layer.

[0030] The feature fusion module is configured to perform feature fusion on the feature images extracted layer by layer through upsampling to obtain a plurality of feature fusion images of different spatial sizes.

[0031] The target detection module is configured to obtain a prediction result of a target object of different sizes according to the plurality of feature fusion images of different spatial sizes.

[0032] According to an aspect of the embodiment of the present application, an electronic device is applied, including:

[0033] The electronic device includes a memory and a processor, and the memory stores a computer program, and the processor implements the method as described above when executing the computer program.

[0034] According to an aspect of the embodiment of the present application, a computer program medium is applied, and the computer program medium stores computer readable instructions, and when the computer readable instructions are executed by the processor of the computer, the computer executes the method as described above.

[0035] In the embodiments of the present application, the input image is preprocessed, the features of the input image are extracted, the input image is extracted layer by layer, the image size of the feature extraction is reduced in proportion, the image features extracted layer by layer are fused by upsampling, the semantic information and the detail information of the image after feature fusion are rich, the image after feature fusion is detected, the detection result of the image target is obtained, and the detection performance of the image target is improved.

[0036] Other characteristics and advantages of the present application will become apparent from the following detailed description, or will be learned by practice of the present application.

[0037] It should be understood that the foregoing general description and the following detailed description are only exemplary and are not restrictive of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0038] The above and other objects, features and advantages of the present application will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings in which:

[0039] Figure 1 A schematic diagram of an architecture to which a target detection method according to an embodiment of the present application is applied is shown.

[0040] Figure 2 A flowchart of a target detection method according to an embodiment of the present application is shown.

[0041] Figure 3 A flowchart of a target detection method according to another embodiment of the present application is shown.

[0042] Figure 4 A schematic diagram of an improved spatial attention mechanism structure according to an embodiment of the present application is shown.

[0043] Figure 5 A schematic diagram of an improved ResNet50 feature layer structure according to an embodiment of the present application is shown.

[0044] Figure 6 A flowchart of a target detection method according to another embodiment of the present application is shown.

[0045] Figure 7 A schematic diagram of a first type of feature extraction layer structure according to an embodiment of the present application is shown.

[0046] Figure 8 A schematic diagram of a second type of feature extraction layer structure according to an embodiment of the present application is shown.

[0047] Figure 9 A flowchart of a target detection method according to another embodiment of the present application is shown.

[0048] Figure 10 An improved channel attention mechanism structure diagram is shown according to one embodiment of the present application.

[0049] Figure 11 A target detection method flow chart is shown according to another embodiment of the present application.

[0050] Figure 12 An improved feature pyramid structure diagram is shown according to one embodiment of the present application.

[0051] Figure 13 A comparison diagram of implementation results before and after network structure improvement is shown according to one embodiment of the present application.

[0052] Figure 14 A hardware structure diagram of a target detection device is shown according to one embodiment of the present application. DETAILED DESCRIPTION

[0053] Example implementations will now be described more fully with reference to the accompanying drawings. Example implementations may, however, be implemented in many different forms and should not be construed as limited to the examples set forth herein; rather, these example implementations are provided so that this disclosure will be thorough and complete, and will fully convey the scope of example implementations to those skilled in the art. Like reference numerals refer to like elements throughout the description. Repeated use of illustrations indicates reusability of a drawing figure across one or more examples.

[0054] Moreover, described features, structures, or characteristics can be combined in any suitable manner in one or more example implementations. In the following description, numerous specific details are provided to give a thorough understanding of example implementations. One skilled in relevant art will recognize, however, that the

[0055] Some of the block diagrams shown in the drawings are functional entities that do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0056] Reference will be made to Figure 1 , Figure 1An architecture diagram to which a target detection method according to an embodiment of the present application is applied is shown. The architecture can include: an improved ResNet 50 module on the left, an improved feature pyramid (FNP) module in the middle, and a prediction layer network structure on the right. Among them, in the improved ResNet 50 on the left, an improved spatial attention mechanism (ISAM) and an improved channel attention mechanism (ICAM) are also added in the C2-C5 feature layers, an ISAM module is added between the input image and the C1 feature layer, and an ICAM module is added between the transverse connection of the C5 feature layer and the M5 feature layer. In the improved feature pyramid structure in the middle, the feature fusion between the cross-layer feature layers is enhanced by the method of upsampling.

[0057] It should be understood that, Figure 1 The number of C2-C5 feature layers, M2-M5 feature layers and P2-P5 prediction layers in the above table is only illustrative. According to the needs of implementation, there can be any number of feature layers and prediction layers.

[0058] Some technical solutions of the embodiments of the present application can be embodied based on the architecture as shown in Figure 1 or a variant architecture thereof.

[0059] Please refer to Figure 2 , Figure 2 A flowchart of a target detection method according to an embodiment of the present application is shown, which includes:

[0060] Step S210, pre-processing the target image based on a spatial attention mechanism to obtain a recognizable image of the target image;

[0061] Step S220, layer-by-layer feature extraction of the recognizable image through a plurality of feature extraction layers connected in series to obtain feature images extracted layer by layer;

[0062] Step S230, feature fusion of the feature images extracted layer by layer through upsampling to obtain a plurality of feature fusion images with different spatial sizes;

[0063] Step S240, obtaining prediction results of target objects of different sizes according to the plurality of feature fusion images with different spatial sizes.

[0064] The four steps are described in detail below.

[0065] In step S210, the target image is preprocessed by using a spatial attention mechanism before down-sampling, so as to improve the feature expression of the key area of the image, reduce the feature loss after maximum pooling, and thus obtain a recognizable image of the target image. The attention mechanism includes a soft attention mechanism, a strong attention mechanism and a self-attention mechanism, and the spatial attention mechanism belongs to the soft attention mechanism. The spatial attention mechanism gives all feature maps on a single feature layer the same attention weight, but the weight given to different positions on a single feature map is different. The preprocessing is mainly for high-resolution gray and color images, as well as images or feature maps with fewer channels.

[0066] Referring to Figure 3 , Figure 3 A flowchart of a target detection method according to another embodiment of the present application is shown. The embodiment provides step S210 of preprocessing the target image based on a spatial attention mechanism to obtain a recognizable image of the target image, which includes:

[0067] In step S211, global maximum pooling and global average pooling are respectively performed on the target image, and the pooled features are fused to obtain pooled fusion features.

[0068] In step S212, the pooled fusion features are obtained through a convolution layer to obtain spatial attention.

[0069] In step S213, the spatial attention is activated by a sigmoid function and applied to the target image to obtain a recognizable image of the target image.

[0070] The three steps are described in detail below.

[0071] In step S211, global maximum pooling and global average pooling are respectively performed on the target image, and the pooled features are fused to obtain pooled fusion features.

[0072] In step S212, the pooled fusion features are obtained through a convolution layer to obtain spatial attention.

[0073] In step S213, the spatial attention is activated by a sigmoid function and applied to the target image to obtain a recognizable image of the target image.

[0074] For example, referring to Figure 4 , Figure 4An improved spatial attention mechanism structure diagram according to one embodiment of the present application is shown. In the structure, the target image of input w x h x c is compressed into w x h x 1 in the channel dimension through global maximum pooling and global average pooling respectively, and the compressed features are fused through addition to obtain the pooling fusion features of w x h x 1. The pooling fusion features of w x h x 1 generate w x h x 1 spatial attention through 3 3 x 3 convolutions. After the spatial attention is activated by the sigmoid function, the extracted spatial attention parameters are applied to the target image through multiplication to obtain the recognizable image of the target image. It can be expressed as follows:

[0075] O = S(f 3x3(f 3x3(f 3x3(Mc(I) + Ac(I))))) x I (1)

[0076] In the formula, O represents the output image or feature layer; S represents the sigmoid activation function; f 3x3 represents 3x3 convolution; Mc represents global maximum pooling in the channel dimension; Ac represents global average pooling in the channel dimension; and I represents the input target image.

[0077] In step S220, the recognizable image passes through a plurality of characteristic extraction layers connected in series, and the feature extraction is performed through multiple convolution and pooling operations. It is worth noting that the image size extracted by each characteristic extraction layer is reduced in proportion, and the reduction ratio and the number of characteristic extraction layers passed can be freely set in advance. When the recognizable image passes through a characteristic extraction layer, the feature image corresponding to the characteristic extraction layer can be obtained, so that when the recognizable image passes through a plurality of characteristic extraction layers in sequence, a plurality of image features extracted by the characteristic extraction layers can be obtained.

[0078] Please refer to Figure 5 , Figure 5 An improved ResNet50 feature layer structure diagram according to one embodiment of the present application is shown. In this structure, C1 feature layer is the first type of feature extraction layer, and C2 feature layer is the second type of feature extraction layer.

[0079] For example, consider an image containing three different sized targets: large, medium, and small. Assume the image enters the first feature extraction layer. At this point, the image size is still large, and all three targets are still present. However, because the large target occupies a larger portion of the image, the first feature extraction layer performs better in detecting it. Then, in the second feature extraction layer, a convolution operation reduces the image size. With this reduction, the large target is cropped, so the second feature extraction layer performs better in detecting the medium target. This process continues until the image enters the next feature extraction layer, where the image size is further reduced, resulting in better detection of the small target. Therefore, when an image passes through three concatenated feature extraction layers, the feature images extracted by each of these three layers are obtained.

[0080] Please see Figure 6 , Figure 6 A flowchart of a target detection method according to another embodiment of this application is shown. This embodiment provides a step S220 of performing layer-by-layer feature extraction on an identifiable image through several sequentially connected feature extraction layers to obtain layer-by-layer extracted feature images, including:

[0081] Step S221: Obtain a first feature image by passing the identifiable image through a first type of feature extraction layer;

[0082] Step S222: The first feature image is passed through several second-type feature extraction layers in sequence to obtain the feature images extracted layer by layer.

[0083] The following is a detailed description of these two steps.

[0084] In step S221, the preprocessed identifiable image is first passed through a first type of feature extraction layer to obtain a first feature image. Considering that the feature map of the first type of feature extraction layer is relatively large and the processing speed is slow, and also considering the efficiency of network model detection, no spatial attention mechanism module and channel attention mechanism module are added in the first type of feature extraction layer.

[0085] To ensure that the identifiable image input to the first feature extraction layer can be processed across different feature layers, it is necessary to maintain consistency between the dimensions of the identifiable image and the dimensions of the processed feature layers. For example, an image processed by the C2 feature layer may be 100×100×256, but the C3 feature layer requires an input image dimension of 128. Therefore, the image needs to be reduced to 128 dimensions before it can be processed in the C3 feature layer. Furthermore, since the 3×3 convolution operation in the C3 feature layer increases the dimension, it is necessary to increase the image dimension to ensure that the dimension of the image processed by the C3 feature layer does not affect the input of the C4 feature layer.

[0086] Referring to Figure 7 , Figure 7 A first type of feature extraction layer structure diagram is shown according to an embodiment of the present application. After the input image enters the first type of feature extraction layer, the channel dimension of the input image is reduced through the left 1x1 convolution, and after being activated by the relu function, the feature is extracted by 3x3 convolution, and then activated by the relu function again, and finally the channel dimension of the input image is increased by 1x1 convolution. The input image changes its channel dimension by 1x1 convolution on the right side, so that the dimension of the output channel on the left side is the same, and then the fusion is performed by addition, and finally the output result is obtained after being activated by the relu function, and the first feature image is obtained.

[0087] Let the three convolution blocks (and the related relu functions) on the left side of the Bottleneck be a function F(x), and the right side 1 convolution block be G(x). Its structure can be represented by the following formula:

[0088] F(x) = f 1×1 (R(f 3×3 (R(f 1×1 (x))))) (2)

[0089] G(x) = f 1×1 (x)

[0090] O = F(x) + G(x)

[0091] Where F(x) is the output of the left branch of the Bottleneck; G(x) is the output of the right branch; f 1×1 represents 1x1 convolution; R represents the relu activation function; f 3×3 represents 3x3 convolution; x represents input; O represents output.

[0092] In step S222, the first feature image will be pooled before entering the next feature extraction layer, and the size of the first feature image will be reduced according to the pre-set proportion to reduce the amount of calculation, and then enter the next feature extraction layer. The recognizable image first passes through a first type of feature extraction layer, and then passes through a second type of feature extraction layer.

[0093] Since the second 1x1 convolution in the first type of feature layer structure increases the number of channels, the first 1x1 convolution in the second type of feature extraction layer structure will reduce the number of channels, which will cause the loss of the number of input image channels, and a lot of semantic information will be lost. At the same time, with the deepening of the network structure, the size of the input image is also constantly decreasing, for example, the image size of C2 feature layer is 56, and the image size of C3 feature layer is changed to 28, and part of the original input image details will be lost. Therefore, compared with the first type of feature extraction layer, the second type of feature extraction layer adds a channel attention mechanism module and a spatial attention mechanism module on the basis of the left branch of the first type of feature extraction layer.

[0094] Referring to Figure 8 , Figure 8 The second type of feature extraction layer structure according to an embodiment of the application is shown. After the input image enters the first type of feature extraction layer, the channel dimension of the input image is reduced by the left 1x1 convolution, and after the relu function is activated, the 3x3 convolution is used for feature extraction, and then the relu function is activated again, and the 1x1 convolution is used to increase the channel dimension of the input image. Then a channel attention mechanism module and a spatial attention mechanism module are added in sequence. Through the two added modules, the loss of target image details and semantic information caused by the network structure can be reduced. The input image changes its channel dimension by the right 1x1 convolution, so that the dimension of the output channel of the left side is the same, and then the fusion is performed by addition, and finally the output result is obtained after the relu function is activated. The feature image extracted by the second type of extraction layer is obtained.

[0095] After the second type of feature extraction layer adds a channel attention mechanism module and a spatial attention mechanism module, the overall improved structure can be represented by the following formula:

[0096] F(x)=IS(IC(f 1×1 (R(f 3×3 (R(f 1×1 (x))))))) (3)

[0097] In the formula, IC represents the ICAM module; IS represents the ISAM module; and the remaining symbols represent the same meaning as the F(x) formula in formula (2).

[0098] After the last layer of the second type of feature extraction layer in step S222, a channel attention mechanism module is added. In the original ResNet50+FPN structure, the transverse connection of feature layers C5 and M5 is directly processed by 1x1 convolution for dimension reduction, which loses a lot of high-level semantic information. Therefore, in this method, before the dimension reduction of feature layer C5, ICAM is used to process C5, which utilizes the dependence between channels to make the network pay more attention to key channel semantic information and reduce the loss of features caused by channel reduction.

[0099] Referring to Figure 9 , Figure 9 A flowchart of a target detection method according to another embodiment of the present application is shown. This embodiment provides a step S222 of obtaining a channel attention feature image of an input image through a channel attention mechanism, including:

[0100] In step S222a, global maximum pooling and global average pooling are performed on the input image, respectively, to obtain global maximum channel attention and global average channel attention.

[0101] In step S222b, after the global maximum channel attention and the global average channel attention are activated by a sigmoid function, they are respectively applied to the input image to obtain a channel attention feature image.

[0102] The two steps will be described in detail below.

[0103] In step S222a, global maximum pooling and global average pooling are performed on the input image, respectively, to obtain global maximum channel attention and global average channel attention. For example, an input image of w x h x c is compressed to 1 x 1 x c through global maximum pooling and global average pooling, respectively, and then 1 x 1 convolution is performed to generate global maximum channel attention and global average channel attention of 1 x 1 x c.

[0104] In step S222b, the global maximum channel attention and the global average channel attention are activated by a sigmoid function, and then applied to the input image, respectively, and then fused to obtain a channel attention feature image. For example, after the global maximum channel attention and the global average channel attention of 1 x 1 x c are activated by a sigmoid function, they are multiplied with the input image, respectively, and then added to obtain a channel attention feature image of w x h x c.

[0105] For example, referring to Figure 10 , Figure 10A schematic diagram of an improved channel attention mechanism according to an embodiment of this application is shown. In this structure, an input w×h×c image is compressed to 1×1×c using global max pooling and global average pooling, respectively. Then, it undergoes 1×1 convolution to generate 1×1×c global max channel attention and global average channel attention. Both are activated by the sigmoid function and then multiplied by the input image. Finally, they are fused by addition to obtain a w×h×c channel attention feature image. This can also be expressed by the following formula:

[0106] O = S(f 1×1 (Ms(I)))×I+S(f 1×1 (As(I)))×I (4)

[0107] In the formula, O represents the output image or feature layer; S represents the sigmoid activation function; f 1×1 Ms represents a 1×1 convolution; As represents a global max pooling operation in the spatial dimension; I represents a global average pooling operation in the spatial dimension; and I represents the input image or feature layer.

[0108] In step S230, the image features extracted layer by layer are fused by upsampling. The main purpose of upsampling is to enlarge the original image so that it can be displayed on a higher resolution display device. Upsampling the more abstract and semantically stronger high-level feature maps preserves as much of the original semantic information of the feature layers as possible, resulting in multiple feature fusion images with different spatial sizes.

[0109] Please see Figure 11 , Figure 11 A flowchart of a target detection method according to another embodiment of this application is shown. This embodiment provides a step S230 of fusing features extracted layer by layer through upsampling to obtain multiple feature fused images with different spatial sizes, including:

[0110] Step S231: The feature images extracted layer by layer are convolved to maintain the same dimension, so as to obtain feature images corresponding to the same dimension.

[0111] Step S232: Feature images of the same dimension are fused by upsampling to obtain multiple feature fused images with different spatial dimensions.

[0112] The following is a detailed description of these two steps.

[0113] In step S231, the proportionally scaled-down images are enlarged by the same ratio in this step. For example, if the images between adjacent feature extraction layers were scaled down by a factor of 0.5, then the adjacent feature images in this step are enlarged by a factor of 2. The image size extracted by the M5 feature layer is 4×4×256, the M4 feature layer is 8×8×256, the M3 feature layer is 16×16×256, and the M2 feature layer is 32×32×256. Before feature fusion, convolution is needed to ensure that the dimensions of the feature images are consistent, resulting in feature images of the same dimensions.

[0114] For example, the M5 feature is obtained by 1×1 convolution after passing through the channel attention mechanism module, the M4 feature is obtained by 1×1 convolution of the C4 feature and then fused with twice the M5 feature, the M3 feature is obtained by 1×1 convolution of the C3 feature and then fused with twice the M4 feature, and the M2 feature is obtained by 1×1 convolution of the C2 feature and then fused with twice the M3 feature.

[0115] In step S232, feature images of the same dimension are fused by upsampling. Since the feature images have the same dimension, they can be directly added together to perform feature fusion, resulting in multiple feature fused images with different spatial dimensions.

[0116] For example, please refer to Figure 12 , Figure 12 A schematic diagram of an improved feature pyramid structure according to an embodiment of this application is shown. In the diagram, M3 and M4 are processed by 2x upsampling and 4x upsampling (two 2x upsampling steps), respectively. The processed feature layers have the same spatial size as M2. Nearest neighbor interpolation in the upsampling method is used to preserve as much of the original semantic information of the M3 and M4 feature layers as possible. Finally, the above processing results are element-wise added to the M2 feature layer, and then processed by 3×3 convolution to obtain feature fusion images with different spatial sizes.

[0117] In step S240, multiple feature fusion images with different spatial dimensions are input into the prediction layer for prediction to obtain prediction results of the corresponding spatial dimensions, that is, prediction results of target objects of different sizes in the target image are obtained.

[0118] The following experiments, using the PASCAL VOC 2012 and MS COCO datasets as examples, demonstrate the performance improvement of this method in object detection.

[0119] The PASCAL VOC dataset is a classic dataset in the field of computer vision. There are two versions of this dataset, VOC 2007 and VOC 2012, each containing 20 categories (21 categories including background). The PASCAL VOC dataset can be used for tasks such as image classification, object detection, and image segmentation. This paper uses the VOC 2012 training dataset (containing 5717 images) to train the model and the val dataset (containing 5823 images) to test the model, for a total of 11540 images for training and testing.

[0120] The MS COCO dataset is an important dataset in the field of computer vision, and can be used for tasks such as image classification, object detection, and image segmentation. There are two versions of the MS COCO dataset: COCO 2014 and COCO 2017. This paper uses the COCO 2017 version for training and testing the model. The model was trained using the train2017 dataset, which contains 118,287 images; and tested using the val2017 dataset, which contains 5,000 images, for a total of 123,287 images for training and testing. MS COCO 2017 has 80 categories for object detection. Furthermore, to comprehensively evaluate the model's object detection performance, the MS COCO dataset further subdivides the evaluation metrics. In addition to evaluation metrics such as mAP, average precision (AP), and average recall (AR), the MS COCO dataset is divided according to intersection-union ratio (IOU) and object size. Based on different IOU thresholds, images are categorized into AP0.50 (APs with an IOU threshold of 0.50), AP0.75 (APs with an IOU threshold of 0.75), and AP0.50:0.95 (APs with an IOU threshold between 0.50 and 0.95). Based on different target sizes, images are categorized into APs (small target APs), ARs (small target ARs), APm (medium target APs), ARm (medium target ARs), APl (large target APs), and ARl (large target ARs).

[0121] To evaluate the performance of the improved model, a Faster R-CNN+FPN network with ResNet50 as the backbone was used for comparison. The comparison results are shown in Table 1.

[0122]

[0123] Table 1 shows the mAP values ​​on the PASCAL VOC 2012 dataset before and after the improvement.

[0124] Table 1 shows the mAP results of the unmodified Faster R-CNN object detection network and the improved network of our method on the PASCAL VOC2012 dataset. As can be seen from the table, the improved network's mAP value increased from 80.5% to 83.2%, a 2.7% improvement. Simultaneously, this experiment was also tested and compared on the MS COCO 2017 dataset, and the comparison results are shown in Table 2.

[0125]

[0126] Table 2 shows the mAP values ​​on the MS COCO 2017 dataset before and after the improvement.

[0127] As shown in Table 2, the mAP value of the network on the MS COCO 2017 dataset improved from 52.7% to 54.2%, an increase of 1.5%. To further verify the model's performance, the IOU threshold in the mAP evaluation metric on the MS COCO 2017 dataset was changed from between 0.50 and 0.95 to 0.75 to further ensure the rigor of the experiment. The results are shown in Table 3.

[0128]

[0129] Table 3 shows the mAP values ​​on the MS COCO 2017 dataset before and after the improvement (threshold is 0.75).

[0130] As shown in Table 3, the mAP on the MS COCO2017 dataset improved from 58.6% to 59.4%, an increase of 0.8%. The results in these three tables demonstrate that the improved network achieves better object detection performance on both the PASCAL VOC 2012 and MSCOCO 2017 datasets.

[0131] To evaluate the improved network's performance in detecting targets of different sizes, this experiment used three sizes for evaluation: small (APs), medium (APm), and large (APl). Small targets were defined as targets <32 pixels. 2 (Target length and width less than 32) for dimensions, medium for target >32 2 And <96 2 Size, target size >96 2 The dimensions are shown in Table 4.

[0132]

[0133] Table 4 shows the AP values ​​of targets of different sizes on the MS COCO 2017 dataset before and after the improvement.

[0134] As shown in Table 4, the improved network model demonstrates enhanced detection performance for targets of all sizes. The corresponding ARs, ARm, and ARl increased from 21.1%, 40.3%, and 50.8% to 23.0%, 41.9%, and 52.6%, respectively, representing improvements of 1.9%, 1.6%, and 1.8%. Table 5 shows the AR values ​​for targets of different sizes on the MS COCO 2017 dataset before and after the improvement.

[0135]

[0136] Table 5 shows the AR values ​​of targets of different sizes on the MS COCO 2017 dataset before and after the improvement.

[0137] As shown in Table 5, the improved network model significantly improves the AR values ​​for targets of all sizes, with ARs, ARm, and ARl increasing by 2.6%, 3.4%, and 6.7%, respectively. Tables 1-5 show that the improved network model enhances the detection accuracy for different datasets.

[0138]

[0139] Table 6 shows the AP values ​​of the network before and after improvement on the PASCAL VOC dataset for different categories.

[0140] Table 6 shows the AP values ​​for 20 categories in the PASCAL VOC dataset on Faster R-CNN and our network. As can be seen from the table, the AP values ​​for 17 out of the 20 categories are improved, indicating that the improved network structure performs better on the PASCAL VOC dataset compared to the Faster R-CNN+FPN structure.

[0141] Please see Figure 13 , Figure 13 A comparison diagram of the network structure improvement before and after an embodiment of this application is shown. Some visual detection results of Faster R-CNN (left) and this network (right) are compared. As can be seen from the diagram, this network model significantly improves the detection performance of small targets; at the same time, it also significantly improves the detection accuracy of large targets, and reduces some missed detections and false detections.

[0142] The target detection method according to the embodiments of this application can be derived from... Figure 14 This is achieved using target detection equipment. See below for reference. Figure 14 This describes a device for target detection according to embodiments of the present application. Figure 14The target detection device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0143] like Figure 14 As shown, the target detection device is represented in the form of a general-purpose computing device. The components of the target detection device may include, but are not limited to: at least one processing unit 810, at least one storage unit 820, and a bus 830 connecting different system components (including storage unit 820 and processing unit 810).

[0144] The storage unit stores program code that can be executed by the processing unit 810, causing the processing unit 810 to perform the steps described in the exemplary method description section of this specification according to various exemplary embodiments of the present invention. For example, the processing unit 810 can perform actions such as... Figure 2 The steps shown are as follows.

[0145] Storage unit 820 may include a readable medium in the form of a volatile storage unit, such as random access memory (RAM) 8201 and / or cache memory 8202, and may further include a read-only memory (ROM) 8203.

[0146] The storage unit 820 may also include a program / utility 8204 having a set (at least one) of program modules 8205, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0147] Bus 830 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0148] The target detection device can also communicate with one or more external devices 700 (e.g., keyboard, pointing device, Bluetooth device, etc.), one or more devices that enable users to interact with the target detection device, and / or any device that enables the target detection device to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 850. Furthermore, the target detection device can communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 860. As shown, network adapter 860 communicates with other modules of the target detection device via bus 830. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with the target detection device, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0149] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the method according to the embodiments of this application.

[0150] In an exemplary embodiment of this application, a computer program medium is also provided, on which computer-readable instructions are stored, which, when executed by a computer's processor, cause the computer to perform the methods described in the above method embodiments.

[0151] According to one embodiment of this application, a program product for implementing the methods in the above-described method embodiments is also provided. This product may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of this invention is not limited thereto. In this document, a readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0152] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0153] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0154] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0155] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0156] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0157] Furthermore, although the steps of the method in this application are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0158] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the method according to the embodiments of this application.

[0159] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the appended claims.

Claims

1. A method for target detection, characterized in that, The method includes: The target image is preprocessed based on a spatial attention mechanism to obtain a recognizable image of the target image; The identifiable image is subjected to layer-by-layer feature extraction through several sequentially connected feature extraction layers, resulting in layer-by-layer extracted feature images. These layers include a first-type feature extraction layer and several second-type feature extraction layers, with the size of the identifiable image extracted by each layer being proportionally reduced. Specifically, after the input image enters the first-type feature extraction layer, it passes through a 1×1 convolutional block on the left, is activated by the ReLU function, and then undergoes feature extraction using a 3×3 convolutional block. After reactivation using the ReLU function, it passes through a 1×1 convolutional block. After entering the first-type feature extraction layer, the input image passes through a 1×1 convolutional block on the right, and after passing through both 1×1 convolutional blocks, the layers are added and fused, and then activated by the ReLU function to obtain the first feature image. The second-type feature extraction layer adds a channel attention mechanism module and a spatial attention mechanism module to the left branch of the first-type feature extraction layer. The feature images extracted layer by layer are upsampled and fused to obtain multiple feature fused images with different spatial sizes; Based on the fusion of multiple feature images with different spatial dimensions, prediction results for target objects of different sizes are obtained; The step of extracting features from the identifiable image layer by layer through several sequentially connected feature extraction layers to obtain the extracted feature images layer by layer includes: The identifiable image is passed through a first-type feature extraction layer to obtain a first-feature image; The first feature image is passed through several second-type feature extraction layers in sequence to obtain the feature images extracted layer by layer; After sequentially passing through several second-type feature extraction layers, the input image is processed through a channel attention mechanism to obtain a channel attention feature image, including: The input image is processed by global max pooling and global average pooling respectively to obtain global max channel attention and global average channel attention; The global maximum channel attention and the global average channel attention are both activated by the sigmoid function and then applied to the input image respectively, and fused to obtain the channel attention feature image.

2. The method according to claim 1, characterized in that, The preprocessing of the target image based on the spatial attention mechanism to obtain a recognizable image of the target image includes: Global max pooling and global average pooling are performed on the target image respectively and then fused to obtain pooling fusion features; Spatial attention is obtained by passing the pooled fusion features through stacked convolutional layers; The spatial attention is activated by the sigmoid function and applied to the target image to obtain a recognizable image of the target image.

3. The method according to claim 1, characterized in that, The step of upsampling and fusing the extracted feature images layer by layer to obtain multiple fused feature images with different spatial dimensions includes: The feature images extracted layer by layer are convolved to maintain the same dimension, thus obtaining feature images corresponding to the same dimension. By upsampling, feature images of the same dimension are fused to obtain multiple feature fused images with different spatial dimensions.

4. The method according to claim 3, characterized in that, The step of fusing features from the same dimension through upsampling to obtain a feature-fused image includes: Feature fusion is achieved by performing nearest neighbor interpolation during upsampling to obtain a feature fused image.

5. The method according to claim 1, characterized in that, The step of obtaining prediction results for target objects of different sizes based on the fusion of feature images with different spatial dimensions includes: Prediction is performed on the multiple feature fusion images with different spatial dimensions to obtain prediction results for target objects of different sizes.

6. A target detection device, characterized in that, The device includes: Image preprocessing module: used to preprocess the target image based on spatial attention mechanism to obtain a recognizable image of the target image; Feature extraction module: This module extracts features from the identifiable image layer by layer through several sequentially connected feature extraction layers, obtaining feature images extracted layer by layer. These sequentially connected feature extraction layers include a first-type feature extraction layer and several second-type feature extraction layers. The size of the identifiable image extracted by each feature extraction layer is proportionally reduced. Specifically, after the input image enters the first-type feature extraction layer, it passes through a 1×1 convolutional block on the left, is activated by the ReLU function, and then undergoes feature extraction using a 3×3 convolutional block. After reactivation using the ReLU function, it passes through a 1×1 convolutional block. After entering the first-type feature extraction layer, the input image passes through a 1×1 convolutional block on the right, and after passing through both 1×1 convolutional blocks, the convolutions are added and fused, then activated by the ReLU function to obtain the first feature image. The second-type feature extraction layer adds a channel attention mechanism module and a spatial attention mechanism module to the left branch of the first-type feature extraction layer. Feature fusion module: used to perform feature fusion by upsampling on the feature images extracted layer by layer to obtain multiple feature fusion images with different spatial sizes; Target detection module: used to fuse images based on features with different spatial dimensions to obtain prediction results for target objects of different sizes; The feature extraction module is further configured to pass the identifiable image through a first type of feature extraction layer to obtain a first feature image; and to pass the first feature image through several second type of feature extraction layers in sequence to obtain feature images extracted layer by layer. Channel attention feature image: used to obtain global maximum channel attention and global average channel attention by performing global max pooling and global average pooling on the input image respectively; after activating the global maximum channel attention and global average channel attention through the sigmoid function, they are applied to the input image respectively and fused to obtain the channel attention feature image.

7. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1-5.

8. A computer program medium having computer-readable instructions stored thereon, characterized in that, When the computer-readable instructions are executed by the processor of a computer, the computer performs the method described in any one of claims 1-5.

Images