Method, device and equipment for multiscale small target detection of visual cortex simulation

Abstract

The application discloses a kind of visual cortex's multiscale small target detection method, device and equipment, the method includes: obtaining input image;Through the feature extraction of the input image by the preset imitated primary visual perception cortex model, obtain first feature map;The preset imitated primary visual perception cortex model includes VOneBlock layer, Conv layer and feature fusion layer;Through the target detection of the first feature map by the preset target detection model, obtain five different size target feature maps;The target classification and coordinate positioning of the five different size target feature maps are carried out, and target detection result is obtained.The application carries out target detection by five different size target feature maps, effectively improves the feature extraction capability to small target.Meanwhile join imitated primary visual perception cortex model, by simulating brain primary visual cortex perception mechanism, improve the anti-interference ability of model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target detection, and more particularly to a visual cortex-like multi-scale small target detection method, device and equipment. BACKGROUND

[0002] With the development of computer technology, a large number of target detection models have emerged, such as Faster-RCNN, SSD, CenterNet, Yolov1-Yolov5, etc. The existing target detection models have poor detection accuracy in the presence of environmental noise interference and for small targets, resulting in high missed detection rate and even detection failure of target detection, and cannot be applied to application scenarios with a large number of small targets and noise interference. SUMMARY

[0003] An object of the present application is to provide a new technical solution for small target detection.

[0004] According to a first aspect of the present application, a visual cortex-like multi-scale small target detection method is provided, the method comprising:

[0005] obtaining an input image;

[0006] extracting features of the input image by a preset visual cortex-like primary visual perception model to obtain a first feature map; the preset visual cortex-like primary visual perception model comprises a VOneBlock layer, a Conv layer and a feature fusion layer;

[0007] detecting targets in the first feature map by a preset target detection model to obtain five target feature maps of different sizes;

[0008] classifying and positioning the targets in the five target feature maps of different sizes to obtain a target detection result.

[0009] Optionally, the extracting features of the input image by the preset visual cortex-like primary visual perception model to obtain a first feature map comprises:

[0010] extracting features of the input image and compressing the size of the input image by the VOneBlock layer to obtain a second feature map;

[0011] compressing the size of the input image by the Conv layer to obtain a third feature map; the size of the third feature map is the same as that of the second feature map;

[0012] fusing feature values of the second feature map and the third feature map by the feature fusion layer to obtain the first feature map.

[0013] Optionally, the preset target detection model comprises a backbone network and a head network, the target detection on the first feature map is performed by using the preset target detection model, and five target feature maps of different sizes are obtained, comprising:

[0014] The first feature map is subjected to multiple times of size compression and feature extraction by using the backbone network, and a plurality of backbone feature maps of different sizes are obtained.

[0015] The first feature map and the plurality of backbone feature maps of different sizes are input into the head network, and the five target feature maps of different sizes are obtained.

[0016] Optionally, the head network comprises four FPN modules connected in series and four feature aggregation modules, the FPN module comprises a Conv layer, an Upsample layer, a Concat layer and a C3 layer connected in sequence, and the input of the first feature map and the plurality of backbone feature maps of different sizes into the head network to obtain the five target feature maps of different sizes comprises:

[0017] The four FPN modules generate a first target feature map according to the plurality of backbone feature maps of different sizes and the first feature map.

[0018] The first target feature map, a first backbone feature map and a feature map output by a Conv layer of a first FPN module are input into a first feature aggregation module, the first feature aggregation module performs size compression and channel aggregation on the plurality of input feature maps to obtain a second target feature map.

[0019] The second target feature map, a second backbone feature map and a feature map output by a Conv layer of a second FPN module are input into a second feature aggregation module, the second feature aggregation module performs size compression and channel aggregation on the plurality of input feature maps to obtain a third target feature map.

[0020] The third target feature map, a third backbone feature map and a feature map output by a Conv layer of a third FPN module are input into a third feature aggregation module, the third feature aggregation module performs size compression and channel aggregation on the plurality of input feature maps to obtain a fourth target feature map.

[0021] The fourth target feature map, a fourth backbone feature map and a feature map output by a Conv layer of a fourth FPN module are input into a fourth feature aggregation module, the fourth feature aggregation module performs size compression and channel aggregation on the plurality of input feature maps to obtain a fifth target feature map.

[0022] Optionally, the four FPN modules generate a first target feature map according to the plurality of feature maps of different sizes and the first feature map, comprising:

[0023] inputting the fourth backbone feature map and the third backbone feature map into a fourth FPN module to obtain an output feature map of the fourth FPN module;

[0024] inputting the output feature map of the fourth FPN module and the second backbone feature map into a third FPN module to obtain an output feature map of the third FPN module;

[0025] inputting the output feature map of the third FPN module and the first backbone feature map into a second FPN module to obtain an output feature map of the second FPN module;

[0026] inputting the output feature map of the second FPN module and the first feature map into the first FPN module to obtain a first target feature map.

[0027] Optionally, the backbone network comprises a first Conv layer, a first C3 layer, a second Conv layer, a second C3 layer, a third Conv layer, a third C3 layer, a fourth Conv layer, a fourth C3 layer and an SPP layer, and the first feature map is subjected to multiple feature map size compression and feature extraction through the backbone network to obtain multiple backbone feature maps of different sizes, comprising:

[0028] the first feature map is subjected to feature map size compression through the first Conv layer, and the compressed first feature map is subjected to feature extraction through the first C3 layer to obtain a first backbone feature map;

[0029] the first backbone feature map is subjected to feature map size compression through the second Conv layer, and the compressed first backbone feature map is subjected to feature extraction through the second C3 layer to obtain a second backbone feature map;

[0030] the second backbone feature map is subjected to feature map size compression through the third Conv layer, and the compressed second backbone feature map is subjected to feature extraction through the third C3 layer to obtain a third backbone feature map;

[0031] the third backbone feature map is subjected to feature map size compression through the fourth Conv layer, spatial information fusion of the compressed third backbone feature map is performed through the SPP layer, and feature extraction of a feature map output by the SPP layer is performed through the fourth C3 layer to obtain a fourth backbone feature map.

[0032] According to a second aspect of the present application, a visual cortex-like multi-scale small target detection device is provided, comprising:

[0033] an image acquisition module configured to acquire an input image;

[0034] The preprocessing module is configured to extract features of the input image by a preset primary visual perception cortex model to obtain a first feature map; the preset primary visual perception cortex model comprises a VOneBlock layer, a Conv layer and a feature fusion layer;

[0035] The target detection module is configured to detect targets by a preset target detection model to obtain five target feature maps of different sizes, and perform target classification and coordinate positioning on the five target feature maps of different sizes to obtain a target detection result.

[0036] Optionally, the target detection module comprises:

[0037] The backbone network module is configured to perform multiple times of size compression and feature extraction on the first feature map to obtain multiple backbone feature maps of different sizes.

[0038] The head network module is configured to receive the first feature map and the multiple backbone feature maps of different sizes to obtain the five target feature maps of different sizes.

[0039] Optionally, the target detection module comprises:

[0040] The detection head module is configured to perform target classification and coordinate positioning on the five target feature maps of different sizes to obtain a target detection result.

[0041] According to a third aspect of the present application, an electronic device is provided, comprising a processor and a memory, the memory storing a program executable on the processor, and the program is executed by the processor to implement the target detection method according to the first aspect of the present application.

[0042] According to one embodiment of the present disclosure, the present application effectively improves the feature extraction capability for small targets and the detection performance for small targets by generating five target feature maps of different sizes and detecting targets by the five target feature maps of different sizes. Meanwhile, the anti-interference capability of the model is improved by adding a primary visual perception cortex model to simulate the primary visual cortex perception mechanism of the brain.

[0043] Other features and advantages of the present application will become apparent from the following detailed description of illustrative embodiments thereof, which proceeds with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0044] The accompanying drawings incorporated in and forming a part of the specification, illustrate embodiments of the present application and, together with the description, serve to explain the principles of the application.

[0045] Figure 1 is a flowchart of a multi-scale small target detection method according to the present application.

[0046] Figure 2 This is a schematic diagram of the rudimentary visual perception cortex model of the present invention.

[0047] Figure 3 This is a schematic diagram of a multi-scale small target detection model that mimics the visual cortex according to the present invention.

[0048] Figure 4 This is a block diagram of a multi-scale small target detection device that mimics the visual cortex according to the present invention.

[0049] Figure 5 This is a block diagram of the electronic device of the present invention. Detailed Implementation

[0050] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the invention.

[0051] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the invention or its application or use.

[0052] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0053] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0054] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0055] Currently, there is no clear definition of small targets in the industry. Small targets can be defined from two perspectives: absolute scale and relative scale. For example, from an absolute scale perspective, a target area smaller than 32*32 pixels can be considered a small target. From a relative scale perspective, a target size where its length and width occupy 0.1 times the original image size is considered a small target.

[0056] like Figure 1 As shown in the figure, this embodiment of the invention introduces a multi-scale small target detection method that mimics the visual cortex, the method including steps S1-S4.

[0057] S1: Obtain the input image.

[0058] S2: performing feature extraction on the input image through a preset primary visual perception cortex model to obtain a first feature map; the preset primary visual perception cortex model comprises a VOneBlock layer, a Conv layer and a feature fusion layer.

[0059] S3: performing target detection on the first feature map through a preset target detection model to obtain five target feature maps of different sizes.

[0060] S4: performing target classification and coordinate positioning on the five target feature maps of different sizes to obtain a target detection result.

[0061] The input image is an image that needs to be subjected to target detection, which can be an image captured by a camera, an image frame extracted from a video or a partial region image cut from a complete image.

[0062] After the input image is obtained, the input image is preprocessed. In the preprocessing process, feature extraction is performed on the input image through a preset primary visual perception cortex model to obtain a first feature map. In the primary visual perception cortex model, a VOneBlock layer, a Conv (convolution) layer and a feature fusion layer are included. The VOneBlock layer is a neural network layer constructed according to the primary visual cortex of primates, adopts a Gabor filter as a core component and simulates the human visual perception cortex information processing mechanism to perform biomimetic visual feature extraction on the input image. After the feature extraction on the input image through the primary visual perception cortex model, the first feature map closer to the feature after the human brain visual processing can be obtained. The primary visual perception cortex model reflects the mapping relationship between the input image and the first feature map.

[0063] After the first feature map is obtained, target detection is performed on the first feature map through a preset target detection model to obtain five target feature maps of different sizes. Target classification and coordinate positioning are performed on the five target feature maps of different sizes to obtain a target detection result. The target detection result contains the position information and classification information of the target object to be detected in the input image, and the target detection model reflects the mapping relationship between the first feature map and the target detection result.

[0064] The present application generates five target feature maps of different sizes, performs target detection through the five target feature maps of different sizes, effectively improves the feature extraction capability for small targets and improves the detection performance for small targets. Meanwhile, the primary visual perception cortex model is added in the preprocessing process, the brain primary visual cortex perception mechanism is simulated, and the anti-interference capability of the model is improved.

[0065] In one embodiment of the present invention, step S2 includes S201-S203.

[0066] S201: The input image is subjected to feature extraction and size compression through the VOneBlock layer to obtain a second feature map.

[0067] S202: The input image is compressed through the Conv layer to obtain a third feature map; the size of the third feature map is the same as the size of the second feature map.

[0068] S203: The feature values ​​of the second feature map and the third feature map are fused through the feature fusion layer to obtain the first feature map.

[0069] like Figure 2 As shown, the simulated primary visual perception cortex model in this invention includes a VOneBlock layer, a Conv layer, and a feature fusion layer, wherein the VOneBlock layer and the Conv layer are connected in parallel. The input image serves as the input to the VOneBlock layer and the Conv layer. The VOneBlock layer extracts features from the input image and compresses its size to obtain a second feature map. The Conv layer compresses the input image to obtain a third feature map, and the second and third feature maps have the same size. The second and third feature maps serve as the input to the feature fusion layer, which fuses the feature values ​​of the second and third feature maps, finally outputting a first feature map.

[0070] For example, if the input image size is 640*640, after feature extraction and size compression in the VOneBlock layer, the resulting second feature map has a size of 320*320, which is half the size of the input image. After size compression in the Conv layer, the resulting third feature map also has a size of 320*320, which is also half the size of the input image and the same as the second feature map. Finally, the feature fusion layer fuses the feature values ​​of the second and third feature maps, resulting in a first feature map with a size of 320*320.

[0071] In this invention, the simulated primary visual perception cortex model simply connects the VOneBlock layer and a Conv layer in parallel, which not only makes the model simpler, but also makes the simulated primary visual perception cortex model more flexible in adjusting the feature map size compression ratio, thus improving the model's flexibility.

[0072] like Figure 3 As shown, in one embodiment of the present invention, the preset target detection model includes a backbone network 101 and a head network 102, and step S3 includes S301-S302.

[0073] S301: performing multiple times of size compression and feature extraction on the first feature map through the backbone network to obtain multiple backbone feature maps of different sizes.

[0074] S302: inputting the first feature map and the multiple backbone feature maps of different sizes into the head network to obtain the five target feature maps of different sizes.

[0075] The target detection model in the application is an improved model based on the Yolov5 model in the prior art, and the target detection model comprises a backbone network 101 and a head network 102. The backbone network is used for performing multiple times of size compression and feature extraction on a first feature map to obtain multiple backbone feature maps of different sizes. Then the head network receives the multiple backbone feature maps of different sizes obtained through the backbone network and the first feature map, and the head network outputs five target feature maps of different sizes. As shown in the figure, the target detection model further comprises a Detect layer 38, which performs target classification and coordinate positioning on the five target feature maps of different sizes to output a target detection result. Figure 3

[0076] In an embodiment of the application, the backbone network comprises a first Conv layer, a first C3 layer, a second Conv layer, a second C3 layer, a third Conv layer, a third C3 layer, a fourth Conv layer, a fourth C3 layer and an SPP (Spatial Pyramid Pooling) layer. As shown in the figure, the first Conv layer is a Conv layer 1 in the figure, the first C3 layer is a C3 layer 2 in the figure, the second Conv layer is a Conv layer 3 in the figure, the second C3 layer is a C3 layer 4 in the figure, the third Conv layer is a Conv layer 5 in the figure, the third C3 layer is a C3 layer 6 in the figure, the fourth Conv layer is a Conv layer 7 in the figure, the fourth C3 layer is a C3 layer 9 in the figure, and the SPP layer is an SPP layer 8 in the figure. Figure 3 Figure 3 Figure 3 Figure 3 Figure 3 Figure 3 Figure 3 Figure 3 Figure 3 Step S301 comprises S3011-S3014.

[0077] S3011: performing feature map size compression on the first feature map through the first Conv layer and performing feature extraction on the compressed first feature map through the first C3 layer to obtain a first backbone feature map.

[0078] ​​​​​​​​​S3012: performing feature map size compression on the first backbone feature map through the second Conv layer, performing feature extraction on the compressed first backbone feature map through the second C3 layer, and obtaining a second backbone feature map.

[0079] S3013: performing feature map size compression on the second backbone feature map through the third Conv layer, performing feature extraction on the compressed second backbone feature map through the third C3 layer, and obtaining a third backbone feature map.

[0080] S3014: performing feature map size compression on the third backbone feature map through the fourth Conv layer, performing spatial information fusion on the compressed third backbone feature map through the SPP layer, performing feature extraction on the feature map output by the SPP layer through the fourth C3 layer, and obtaining a fourth backbone feature map.

[0081] After obtaining the input image, the size of the first feature map obtained through the primary visual perception cortex model is 1 / 2 of the size of the input image. The first feature map is input into the backbone network, and the first Conv layer in the backbone network obtains the first feature map, performs size compression on the first feature map, and outputs a feature map with a size of 1 / 4 of the size of the input image. The feature map output by the first Conv layer is input into the first C3 layer for feature extraction, and the first C3 layer outputs a first backbone feature map. The second Conv layer performs size compression on the first backbone feature map, and the feature map output by the second Conv layer is input into the second C3 layer for feature extraction, and the second C3 layer outputs a second backbone feature map. The third Conv layer performs size compression on the second backbone feature map, and the feature map output by the third Conv layer is input into the third C3 layer for feature extraction, and the third C3 layer outputs a third backbone feature map. The fourth Conv layer performs size compression on the third backbone feature map, and the feature map output by the fourth Conv layer is input into the SPP layer for spatial information fusion, and the feature map output by the SPP layer is input into the fourth C3 layer for feature extraction, and the fourth C3 layer outputs a fourth backbone feature map.

[0082] For example, the size of the input image is 640*640, the primary visual perception cortex model performs size compression on the input image, and the output first feature map is 1 / 2 of the input image, and the size of the first feature map is 320*320. The first feature map is compressed four times in size by the backbone network, and a backbone feature map is obtained after feature extraction at each size compression. After the first size compression and feature extraction process by the first Conv layer and the first C3 layer, the first backbone feature map with a size of 160*160 is obtained, and the size of the first backbone feature map is 1 / 4 of the input image. After the second size compression and feature extraction process by the second Conv layer and the second C3 layer, the second backbone feature map with a size of 80*80 is obtained, and the size of the second backbone feature map is 1 / 8 of the input image size. After the third size compression and feature extraction process by the third Conv layer and the third C3 layer, the third backbone feature map with a size of 40*40 is obtained, and the size of the third backbone feature map is 1 / 16 of the input image size. After the fourth size compression and feature extraction process by the fourth Conv layer, the SPP layer and the fourth C3 layer, the fourth backbone feature map with a size of 20*20 is obtained, and the size of the fourth backbone feature map is 1 / 32 of the input image size.

[0083] As shown in the embodiment of the present application, Figure 3 the head network includes four FPN modules and four feature aggregation modules connected in series, the FPN module includes a Conv layer, an Upsample layer, a Concat layer and a C3 layer connected in sequence, and step S302 includes:

[0084] S3021: The four FPN modules generate a first target feature map according to the plurality of backbone feature maps with different sizes and the first feature map.

[0085] S3022: The first target feature map, the first backbone feature map, and the feature map output by the Conv layer of the first FPN module are input into the first feature aggregation module, and the first feature aggregation module performs size compression and channel aggregation on the input plurality of feature maps to obtain a second target feature map.

[0086] S3023: The second target feature map, the second backbone feature map, and the feature map output by the Conv layer of the second FPN module are input into the second feature aggregation module, and the second feature aggregation module performs size compression and channel aggregation on the input plurality of feature maps to obtain a third target feature map.

[0087] S3024: input the third target feature map, the third backbone feature map, and the feature map output by the Conv layer of the third FPN module into a third feature aggregation module, the third feature aggregation module performs size compression and channel aggregation on the input multiple feature maps to obtain a fourth target feature map.

[0088] S3025: input the fourth target feature map, the fourth backbone feature map, and the feature map output by the Conv layer of the fourth FPN module into a fourth feature aggregation module, the fourth feature aggregation module performs size compression and channel aggregation on the input multiple feature maps to obtain a fifth target feature map.

[0089] The Conv layer, the Upsample layer, the Concat layer, and the C3 layer in the FPN module are consistent with the existing Yolov5 model. The Conv layer is a basic convolution unit, which performs two-dimensional convolution, regularization, and activation operations on the input in sequence. The C3 layer is composed of a plurality of Bottleneck modules, and the Bottleneck is a classic residual structure. After the input is processed by two convolution layers, the original value is added to complete the residual feature transmission without increasing the output depth.

[0090] After the first feature map is input into the backbone network, the backbone network outputs four backbone feature maps of different sizes, namely the first backbone feature map, the second backbone feature map, the third backbone feature map, and the fourth backbone feature map. Five target feature maps of different sizes are the first target feature map, the second target feature map, the third target feature map, the fourth target feature map, and the fifth target feature map. Four feature aggregation modules are the first feature aggregation module, the second feature aggregation module, the third feature aggregation module, and the fourth feature aggregation module. Each aggregation module includes a Conv layer, a Concat layer, and a C3 layer. As shown in Figure 3 the first feature aggregation module includes a Conv layer 26, a Concat layer 27, and a C3 layer 28, the second feature aggregation module includes a Conv layer 29, a Concat layer 30, and a C3 layer 31, the third feature aggregation module includes a Conv layer 32, a Concat layer 33, and a C3 layer 34, and the fourth feature aggregation module includes a Conv layer 35, a Concat layer 36, and a C3 layer 37.

[0091] The four FPN modules in the application are the first FPN module, the second FPN module, the third FPN module, and the fourth FPN module, and the four FPN modules are connected in series. Figure 3As shown, the first FPN module includes a Conv layer 22, an Upsample layer 23, a Concat layer 24, and a C3 layer 25, the second FPN module includes a Conv layer 18, an Upsample layer 19, a Concat layer 20, and a C3 layer 21, the third FPN module includes a Conv layer 14, an Upsample layer 15, a Concat layer 16, and a C3 layer 17, and the fourth FPN module includes a Conv layer 10, an Upsample layer 11, a Concat layer 12, and a C3 layer 13.

[0092] The output of the fourth FPN module is taken as the input of the third FPN module, the output of the third FPN module is taken as the input of the second FPN module, and the output of the second FPN module is taken as the input of the first FPN module. Meanwhile, the fourth FPN module receives the fourth backbone feature map and the third backbone feature map as inputs, the third FPN module further receives the second backbone feature map as an input, the second FPN module further receives the first backbone feature map as an input, and the first FPN module further receives the first feature map as an input. Finally, the feature map output by the first FPN module is the first target feature map.

[0093] After obtaining the first target feature map, the first target feature map, the first backbone feature map, and the feature map output by the Conv layer of the first FPN module are input into the first feature aggregation module. The Conv layer in the first feature aggregation module performs size compression on the first target feature map, the Concat layer in the first feature aggregation module performs channel aggregation on the feature map output by the Conv layer in the first feature aggregation module, the first backbone feature map, and the feature map output by the Conv layer of the first FPN module, the C3 layer in the first feature aggregation module performs feature extraction on the feature map output by the Concat layer in the first feature aggregation module, and the feature map output by the C3 layer in the first feature aggregation module is the second target feature map.

[0094] After obtaining the second target feature map, the second target feature map, the second backbone feature map, and the feature map output by the Conv layer of the second FPN module are input into the second feature aggregation module. The Conv layer in the second feature aggregation module performs size compression on the second target feature map, the Concat layer in the second feature aggregation module performs channel aggregation on the feature map output by the Conv layer in the second feature aggregation module, the second backbone feature map, and the feature map output by the Conv layer of the second FPN module, the C3 layer in the second feature aggregation module performs feature extraction on the feature map output by the Concat layer in the second feature aggregation module, and the feature map output by the C3 layer in the second feature aggregation module is the third target feature map.

[0095] After the third target feature map is obtained, the third target feature map, the third backbone feature map and the feature map output by the Conv layer of the third FPN module are input into a third feature aggregation module. The Conv layer in the third feature aggregation module performs size compression on the third target feature map, the Concat layer in the third feature aggregation module performs channel aggregation on the feature map output by the Conv layer in the third feature aggregation module, the third backbone feature map and the feature map output by the Conv layer of the third FPN module, the C3 layer in the third feature aggregation module performs feature extraction on the feature map output by the Concat layer in the third feature aggregation module, and the feature map output by the C3 layer in the third feature aggregation module is the fourth target feature map.

[0096] After the fourth target feature map is obtained, the fourth target feature map, the fourth backbone feature map and the feature map output by the Conv layer of the fourth FPN module are input into a fourth feature aggregation module. The Conv layer in the fourth feature aggregation module performs size compression on the fourth target feature map, the Concat layer in the fourth feature aggregation module performs channel aggregation on the feature map output by the Conv layer in the fourth feature aggregation module, the fourth backbone feature map and the feature map output by the Conv layer of the fourth FPN module, the C3 layer in the fourth feature aggregation module performs feature extraction on the feature map output by the Concat layer in the fourth feature aggregation module, and the feature map output by the C3 layer in the fourth feature aggregation module is the fifth target feature map.

[0097] In one example, the size of the input image is 640*640, the size of the first feature map obtained through the primary visual cortex model is 320*320, and the size of the first target feature map output by the first FPN module is also 320*320. After the first feature map is input into the backbone network, the first backbone feature map with a size of 160*160, the second backbone feature map with a size of 80*80, the third backbone feature map with a size of 40*40 and the fourth backbone feature map with a size of 20*20 are obtained. The size of the finally obtained first target feature map is the same as that of the first feature map, and the size of the first target feature map is 320*320; the size of the second target feature map is the same as that of the first backbone feature map, and the size of the second target feature map is 160*160; the size of the third target feature map is the same as that of the second backbone feature map, and the size of the third target feature map is 80*80; the size of the fourth target feature map is the same as that of the third backbone feature map, and the size of the fourth target feature map is 40*40; and the size of the fifth target feature map is the same as that of the fourth backbone feature map, and the size of the fifth target feature map is 20*20.

[0098] In one embodiment of the present application, step S3021 comprises: inputting the fourth backbone feature map and the third backbone feature map into a fourth FPN module to obtain an output feature map of the fourth FPN module;

[0099] The output feature map of the fourth FPN module and the second backbone feature map are input into the third FPN module to obtain the output feature map of the third FPN module;

[0100] The output feature map of the third FPN module and the first backbone feature map are input into the second FPN module to obtain the output feature map of the second FPN module;

[0101] The output feature map of the second FPN module and the first feature map are input into the first FPN module to obtain the first target feature map.

[0102] like Figure 3 As shown, the fourth backbone feature map is input to the Conv layer of the fourth FPN module, and the third backbone feature map is input to the Concat layer of the fourth FPN module. The Conv layer of the fourth FPN module performs channel compression on the fourth backbone feature map. The Upsample layer of the fourth FPN module then expands the size of the feature map output from the Conv layer, ensuring that the size of the feature map output from the Upsample layer is the same as that of the third backbone feature map. The Concat layer of the fourth FPN module performs channel aggregation on the third backbone feature map and the feature map output from the Upsample layer. The C3 layer of the fourth FPN module extracts features from the feature map output from the Concat layer, and the feature map output from the C3 layer is the final feature map output by the fourth FPN module.

[0103] The feature map output from the fourth FPN module is input into the Conv layer of the third FPN module, and the second backbone feature map is input into the Concat layer of the third FPN module. The Conv layer of the third FPN module performs channel compression on the feature map output from the fourth FPN module. The Upsample layer of the third FPN module then expands the size of the feature map output from the Conv layer, ensuring that the size of the feature map output from the Upsample layer is the same as the size of the second backbone feature map. The Concat layer of the third FPN module performs channel aggregation on the feature map output from the second backbone feature map and the Upsample layer. The C3 layer of the third FPN module extracts features from the feature map output from the Concat layer, and the feature map output from the C3 layer is the final feature map output from the third FPN module.

[0104] The feature map output by the third FPN module is input to the Conv layer of the second FPN module, and the first backbone feature map is input to the Concat layer of the second FPN module. The Conv layer of the second FPN module performs channel compression on the feature map output by the third FPN module, the Upsample layer of the second FPN module performs size expansion on the feature map output by the Conv layer of the second FPN module, and the size of the feature map output by the Upsample layer of the second FPN module is the same as that of the first backbone feature map. The Concat layer of the second FPN module performs channel aggregation on the first backbone feature map and the feature map output by the Upsample layer of the second FPN module, the C3 layer of the second FPN module extracts features from the feature map output by the Concat layer of the second FPN module, and the feature map output by the C3 layer of the second FPN module is the feature map output by the second FPN module.

[0105] The feature map output by the second FPN module is input to the Conv layer of the first FPN module, and the first feature map is input to the Concat layer of the first FPN module. The Conv layer of the first FPN module performs channel compression on the feature map output by the second FPN module, the Upsample layer of the first FPN module performs size expansion on the feature map output by the Conv layer of the first FPN module, and the size of the feature map output by the Upsample layer of the first FPN module is the same as that of the first feature map. The Concat layer of the first FPN module performs channel aggregation on the first feature map and the feature map output by the Upsample layer of the first FPN module, the C3 layer of the first FPN module extracts features from the feature map output by the Concat layer of the first FPN module, and the feature map output by the C3 layer of the first FPN module is the first target feature map.

[0106] As shown in Figure 4 , the embodiment of the present application introduces a kind of multiscale small target detection device of visual cortex simulation, that is Figure 4 As shown in figure, the target detection device 200 is used to realize the multiscale small target detection method of visual cortex simulation of any embodiment of the present application, and the target detection device 200 includes:

[0107] Image acquisition module 201 is used to acquire input image;

[0108] Preprocessing module 202 is used to extract features from the input image by the preset simulation primary visual perception cortex model, and obtain the first feature map;The preset simulation primary visual perception cortex model includes VOneBlock layer, Conv layer and feature fusion layer;

[0109] The target detection module 203 is configured to perform target detection on the first feature map by using a preset target detection model to obtain five target feature maps of different sizes, and perform target classification and coordinate positioning on the five target feature maps of different sizes to obtain a target detection result.

[0110] The present application effectively improves the feature extraction capability for small targets and improves the detection performance for small targets by generating five target feature maps of different sizes and performing target detection on the five target feature maps of different sizes. Meanwhile, the anti-interference capability of the model is improved by adding a primary visual perception cortex model in the preprocessing process and simulating the primary visual cortex perception mechanism of the brain.

[0111] In an embodiment of the present application, the target detection module comprises:

[0112] The backbone network module is configured to perform multiple times of size compression and feature extraction on the first feature map to obtain a plurality of backbone feature maps of different sizes.

[0113] The head network module is configured to receive the first feature map and the plurality of backbone feature maps of different sizes to obtain the five target feature maps of different sizes.

[0114] In an embodiment of the present application, the target detection module comprises a detection head module configured to perform target classification and coordinate positioning on the five target feature maps of different sizes to obtain a target detection result.

[0115] As shown in Figure 5 An electronic device 300 is introduced in an embodiment of the present application, which comprises a processor 301 and a memory 302. The memory 302 stores a program that can be run on the processor 301. When the program is executed by the processor 301, the target detection method according to any embodiment of the present application is implemented.

[0116] An embodiment of the present application further introduces a computer readable storage medium, which stores a computer program. When the computer program is executed, the target detection method according to any embodiment of the present application is implemented.

[0117] The present application can be a system, a method, and / or a computer program product. The computer program product can include a computer readable storage medium having computer readable program instructions embodied therewith, which when executed by a processor, implement various aspects of the present application.

[0118] Computer readable storage media can be tangible storage media which can retain and store instructions for use by an instruction execution device. Computer readable storage media can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer readable storage media include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

[0119] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0120] Computer readable program instructions for carrying out operations of the present application can be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate array (FPGA), or programmable logic array (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present application.

[0121] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0122] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0123] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0124] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0125] Embodiments of the present application have been described above, and the description is intended to be illustrative, and not restrictive, of the disclosed embodiments. Many modifications and variations of the described embodiments are possible, and all such modifications and variations are intended to be within the scope of the described embodiments. The description used herein is intended to best explain the principles of the embodiments, the practical application, and the best mode of the present application, to make this disclosure understood in the various embodiments and many modifications are possible without departing from the spirit and scope of the described embodiments. The scope of the present application is defined by the appended claims.

Claims

1. A multi-scale small target detection method mimicking the visual cortex, characterized in that, The method includes: Obtain the input image; The input image is subjected to feature extraction using a pre-defined rudimentary visual perception cortex model to obtain a first feature map. The pre-defined rudimentary visual perception cortex model includes a VOneBlock layer, a Conv layer, and a feature fusion layer. The VOneBlock layer is connected in parallel with the Conv layer. The first feature map is subjected to target detection using a preset target detection model to obtain five target feature maps of different sizes. The target feature maps of the five different sizes are classified and their coordinates are located to obtain the target detection results. The step of extracting features from the input image using a preset simulated primary visual perception cortex model to obtain a first feature map includes: The input image is subjected to feature extraction and size compression through the VOneBlock layer to obtain a second feature map. The input image is compressed using the Conv layer to obtain a third feature map; the size of the third feature map is the same as the size of the second feature map. The first feature map is obtained by fusing the feature values ​​of the second feature map and the third feature map through the feature fusion layer.

2. The method according to claim 1, characterized in that, The preset target detection model includes a backbone network and a head network. The preset target detection model is used to perform target detection on the first feature map, resulting in five target feature maps of different sizes, including: The first feature map is subjected to multiple size compressions and feature extractions through the backbone network to obtain multiple backbone feature maps of different sizes. The first feature map and the multiple backbone feature maps of different sizes are input into the head network to obtain the five target feature maps of different sizes.

3. The method according to claim 2, characterized in that, The head network includes four cascaded FPN modules and four feature aggregation modules. Each FPN module includes a Conv layer, an Upsample layer, a Concat layer, and a C3 layer connected in sequence. The process of inputting the first feature map and the multiple backbone feature maps of different sizes into the head network to obtain the five target feature maps of different sizes includes: The four FPN modules generate a first target feature map based on the multiple backbone feature maps of different sizes and the first feature map; The first target feature map, the first backbone feature map, and the feature map output by the Conv layer of the first FPN module are input into the first feature aggregation module. The first feature aggregation module performs size compression and channel aggregation on the input multiple feature maps to obtain the second target feature map. The second target feature map, the second backbone feature map, and the feature map output from the Conv layer of the second FPN module are input into the second feature aggregation module. The second feature aggregation module performs size compression and channel aggregation on the input multiple feature maps to obtain the third target feature map. The third target feature map, the third backbone feature map, and the feature map output from the Conv layer of the third FPN module are input into the third feature aggregation module. The third feature aggregation module performs size compression and channel aggregation on the input multiple feature maps to obtain the fourth target feature map. The fourth target feature map, the fourth backbone feature map, and the feature map output from the Conv layer of the fourth FPN module are input into the fourth feature aggregation module. The fourth feature aggregation module performs size compression and channel aggregation on the input multiple feature maps to obtain the fifth target feature map.

4. The method according to claim 3, characterized in that, The four FPN modules generate a first target feature map based on the plurality of feature maps of different sizes and the first feature map, including: The fourth backbone feature map and the third backbone feature map are input into the fourth FPN module to obtain the feature map output by the fourth FPN module; The feature map output by the fourth FPN module and the second backbone feature map are input into the third FPN module to obtain the feature map output by the third FPN module. The feature map output by the third FPN module and the first backbone feature map are input into the second FPN module to obtain the feature map output by the second FPN module. The feature map output by the second FPN module and the first feature map are input into the first FPN module to obtain the first target feature map.

5. The method according to claim 2, characterized in that, The backbone network includes a first Conv layer, a first C3 layer, a second Conv layer, a second C3 layer, a third Conv layer, a third C3 layer, a fourth Conv layer, a fourth C3 layer, and an SPP layer. The first feature map is subjected to multiple feature map size compressions and feature extractions through the backbone network to obtain multiple backbone feature maps of different sizes, including: The first feature map is compressed by the first Conv layer, and the first feature map is extracted by the first C3 layer to obtain the first backbone feature map. The first backbone feature map is compressed by the second Conv layer, and the second backbone feature map is obtained by extracting features from the compressed first backbone feature map by the second C3 layer. The second backbone feature map is compressed by the third Conv layer, and the compressed second backbone feature map is extracted by the third C3 layer to obtain the third backbone feature map. The fourth Conv layer compresses the feature map size of the third backbone feature map, the SPP layer fuses the spatial information of the compressed third backbone feature map, and the fourth C3 layer extracts features from the feature map output by the SPP layer to obtain the fourth backbone feature map.

6. A multi-scale small target detection device that mimics the visual cortex, characterized in that, The device includes: The image acquisition module is used to acquire the input image; The preprocessing module is used to extract features from the input image using a preset simulating primary visual perception cortex model to obtain a first feature map; the preset simulating primary visual perception cortex model includes a VOneBlock layer, a Conv layer and a feature fusion layer, wherein the VOneBlock layer is connected in parallel with only the Conv layer; The target detection module is used to perform target detection on the first feature map using a preset target detection model to obtain five target feature maps of different sizes, and to perform target classification and coordinate localization on the five target feature maps of different sizes to obtain the target detection result. Specifically, the preprocessing module is used for: The input image is subjected to feature extraction and size compression through the VOneBlock layer to obtain a second feature map. The input image is compressed using the Conv layer to obtain a third feature map; the size of the third feature map is the same as the size of the second feature map. The first feature map is obtained by fusing the feature values ​​of the second feature map and the third feature map through the feature fusion layer.

7. The apparatus according to claim 6, characterized in that, The target detection module includes: The backbone network module is used to perform multiple size compressions and feature extractions on the first feature map to obtain multiple backbone feature maps of different sizes. The head network module is used to receive the first feature map and the multiple backbone feature maps of different sizes to obtain the five target feature maps of different sizes.

8. The apparatus according to claim 7, characterized in that, The target detection module includes: The detection head module is used to classify and locate the target coordinates of the five target feature maps of different sizes to obtain the target detection results.

9. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a program that can run on the processor, the program being executed by the processor to implement the target detection method as described in any one of claims 1-5.

Images