Object detection system based on multi-level primary visual cortex-like modules

Abstract

The present invention provides a target detection system based on a multi-level simulated primary visual cortex module, the system comprising: a simulated primary visual network module, used to obtain multiple simulated primary visual feature maps of different sizes and corresponding simulated primary visual feature maps of interest of an image to be detected; a backbone network module, used to obtain multiple backbone feature maps of corresponding sizes through channel aggregation and feature extraction based on multiple simulated primary visual feature maps of different sizes; a target detection module, used to perform channel aggregation based on multiple backbone feature maps and simulated primary visual feature maps of interest of corresponding sizes, to obtain multiple target feature maps of corresponding sizes; and to fuse multiple target feature maps to obtain target detection results. The present invention solves the problem that the target detection performance of the target detection method in the prior art is limited under noise interference such as strong light, weak light, rain, snow, fog and haze weather, and AI confrontation.

Description

Technical Field

[0001] The present invention belongs to the technical field of artificial intelligence target detection, and in particular relates to a target detection system based on a multi-level simulated primary visual cortex module. Background Art

[0002] Object detection is a technology for identifying and locating objects of interest in images. It solves the problem of what and where the objects are in the image, and plays a very important role in many fields. The YOLO object detection algorithm is a real-time object detection algorithm in general scenarios, and is widely used in daily learning and life. However, there are still some problems with the performance of the YOLO algorithm itself. For example, under strong light, weak light, rain, snow, fog, haze, AI confrontation and other noise interference, the object detection performance is significantly reduced, which hinders the actual application of the object detection system. Therefore, how to optimize the existing object detection system to have better performance indicators has become a problem that needs to be studied in the field of object detection. Summary of the invention

[0003] In view of the above analysis, the present invention aims to provide a target detection system based on a multi-level simulated primary visual cortex module, which is used to solve the problem that the target detection performance of the target detection method in the prior art is limited under noise interference such as strong light, weak light, rain, snow, fog and haze, and AI confrontation.

[0004] The purpose of the present invention is mainly achieved through the following technical solutions:

[0005] In one aspect, the present invention provides a target detection system based on a multi-level primary visual cortex-like module, the system comprising:

[0006] The simulated primary vision network module is used to obtain a plurality of simulated primary vision feature maps of different sizes and corresponding simulated primary vision feature maps of interest of the image to be detected;

[0007] A backbone network module is used to obtain a plurality of backbone feature maps of corresponding sizes through channel aggregation and feature extraction based on a plurality of simulated primary visual feature maps of different sizes;

[0008] The target detection module is used to perform channel aggregation based on the multiple backbone feature maps and the simulated primary visual feature maps of interest of corresponding sizes to obtain multiple target feature maps of corresponding sizes; and to fuse the multiple target feature maps to obtain the target detection result.

[0009] Further, the simulated primary visual network module includes n+1 simulated primary visual feature extraction networks, and the simulated primary visual feature extraction network includes a serially arranged simulated primary visual cortex module, a RepBlock module and a CA attention layer;

[0010] The simulated primary visual cortex module includes a VOneBlock layer and a Conv layer arranged in parallel, and a feature fusion layer for fusing the outputs of the VOneBlock layer and the Conv layer;

[0011] By setting different parameters for the VOneBlock layer and the Conv layer, the received image is compressed and feature extracted to different degrees to obtain simulated primary visual feature maps of different sizes.

[0012] Furthermore, it also includes an ImageSub layer, which is used to obtain the to-be-processed images in parallel, and input the to-be-processed images obtained in parallel into each simulated primary visual cortex module respectively.

[0013] Furthermore, the backbone network module includes n feature map compression extraction modules connected in sequence, each of the feature map compression extraction modules includes a Concat layer, and the Concat layer is used to perform channel aggregation based on an imitation primary visual feature map.

[0014] Furthermore, the i-th feature map compression extraction module includes a Conv layer, a Concat layer, a RepBlock layer and a CA attention layer connected in sequence, which are used to perform size compression, channel aggregation, re-parameterization feature extraction and feature extraction of interest on the feature map output by the previous module in sequence, wherein the feature map extracted by the i-th RepBlock layer is used as the i-th backbone feature map, and the feature map of interest extracted by the i-th CA attention layer is used as the input of the next feature map compression extraction module, where 1≤i <n;

[0015] The nth feature map compression extraction module includes a Conv layer, a Concat layer, a RepBlock layer, a SPPF layer and a CA attention layer connected in sequence; the feature map of interest extracted by the CA attention layer is the nth backbone feature map.

[0016] Further, the Concat layer of the first feature map compression extraction module performs channel aggregation based on the second simulated primary visual feature map and the feature map obtained by size compression of the first simulated primary visual feature map of interest;

[0017] The Concat layer of the j-th feature map compression advance module performs channel aggregation based on the j-th simulated primary visual feature map and the feature map obtained by size compression of the output of the j-1-th feature map compression extraction module, where 1 <j≤n。

[0018] Furthermore, the target detection module includes a target feature map extraction module and a Detect module;

[0019] The target feature map extraction module includes n-1 FPN modules and n-1 feature aggregation modules arranged in sequence, and is used to generate multiple target feature maps based on multiple simulated primary visual feature maps of interest and backbone feature maps;

[0020] The Detect module is used to fuse multiple target feature maps of different sizes, perform target classification and coordinate positioning, and obtain target detection results.

[0021] Furthermore, the FPN module includes a Conv layer, an Upsample layer, a Concat layer and a RepBlock layer connected in sequence; and the feature aggregation module includes a Conv layer, a Concat layer and a RepBlock layer connected in sequence.

[0022] Furthermore, the n is 3; multiple channel aggregations are performed based on the backbone feature map and the simulated primary visual feature map to obtain n target feature maps of corresponding sizes, including:

[0023] Based on the three backbone feature maps of different sizes, two FPN modules are used to perform size compression, size expansion, channel aggregation and re-parameterized feature extraction to obtain the first target feature map;

[0024] After the first target feature map and the feature map output by the second FPN module are compressed in size, they are aggregated with the third imitation primary visual feature map through the Concat layer in the first feature aggregation module, and re-parameterized feature extraction is performed through the RepBlock layer to obtain the second target feature map;

[0025] After the second target feature map and the feature map output by the first FPN module are compressed in size, they are aggregated with the fourth imitation primary visual feature map through the Concat layer in the second feature aggregation module, and re-parameterized feature extraction is performed through the RepBlock layer to obtain the third target feature map;

[0026] The first target feature map, the second target feature map and the third target feature map are subjected to feature fusion through the Detect layer to obtain a target detection result.

[0027] Furthermore, it also includes an image acquisition module; the image acquisition module publishes video stream data or image sequences through the ImagesPub layer, and subscribes to the image to be detected at the current moment through the ImageSub layer.

[0028] Beneficial effects of this technical solution:

[0029] 1. The target detection system of the present invention adopts a multi-level simulated primary visual cortex module, and obtains simulated primary visual feature maps of different sizes by adjusting parameters such as the convolution kernel of the simulated primary visual cortex module; and adds a channel aggregation layer in the backbone network module to integrate the feature map output by the simulated primary visual cortex module into the backbone network, thereby increasing the useful information of the simulated primary visual cortex in the backbone network, that is, the feature information processed by the human visual mechanism more closely resembles that of the human visual mechanism, and having a higher ability to resist noise interference.

[0030] 2. The present invention also transmits the beneficial information of the simulated primary visual cortex to the end of the target detection module across layers over long distances, avoiding the performance impact caused by information attenuation at the end of the network model, ensuring the effectiveness of the beneficial information of the simulated primary visual cortex in the entire network model, and further ensuring the sustainability of the effect of the simulated visual cortex module. The target detection performance of the target detection system is improved under noise interference such as strong light, weak light, rain, snow, fog and haze, and AI confrontation.

[0031] 3. The present invention introduces the RepBlock layer of the YOLO6 model, and reduces the number of parameters and calculations through the re-parameterization method during the feature extraction process, thereby improving the model reasoning efficiency.

[0032] Other features and advantages of the present invention will be described in the following description, and part of them will become obvious from the description, or will be understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The drawings are only for the purpose of illustrating specific embodiments and are not to be considered limiting of the present invention. Like reference symbols denote like components throughout the drawings.

[0034] Figure 1 It is a structural block diagram of a target detection system based on a multi-level simulated primary visual cortex module according to an embodiment of the present invention;

[0035] Figure 2 It is a structural diagram of the simulated primary visual cortex module according to an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the implementation cases of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.

[0037] One embodiment of the present invention provides a target detection system based on a multi-level primary visual cortex-like module, such as Figure 1 As shown, the system includes:

[0038] The simulated primary vision network module is used to obtain a plurality of simulated primary vision feature maps of different sizes and corresponding simulated primary vision feature maps of interest of the image to be detected;

[0039] A backbone network module is used to obtain a plurality of backbone feature maps of corresponding sizes through channel aggregation and feature extraction based on a plurality of simulated primary visual feature maps of different sizes;

[0040] The target detection module is used to perform channel aggregation based on multiple backbone feature maps and simulated primary visual feature maps of interest of corresponding sizes to obtain multiple target feature maps of corresponding sizes; and to fuse multiple target feature maps to obtain target detection results.

[0041] Specifically, the image to be detected can be an image captured by a camera in real time or an image stored in a hard disk; this embodiment obtains the image to be detected through an image acquisition module; the image acquisition module publishes video stream data or image sequences through the ImagesPub layer, and subscribes to the image to be detected at the current moment through the ImageSub layer. This embodiment can perform multi-threaded operations through the ImageSub layer, and each ImageSub layer belongs to one thread; the ImagesPub layer publishes data to the memory, and each ImageSub layer can concurrently subscribe to the data, thereby improving data acquisition efficiency.

[0042] Preferably, the simulated primary visual network module includes n+1 simulated primary visual feature extraction networks, and the simulated primary visual feature extraction network includes a serially arranged simulated primary visual cortex module, a RepBlock module and a CA attention layer; Figure 2 As shown, the simulated primary visual cortex module includes a VOneBlock layer and a Conv layer set in parallel, and a feature fusion layer for fusing the outputs of the VOneBlock layer and the Conv layer; and an ImageSub layer is set in front of each simulated primary visual cortex module to obtain the image to be processed in parallel, and the parallel obtained images to be processed are respectively input into each simulated primary visual cortex module for simulated primary visual cortex feature extraction. By setting different parameters for the VOneBlock layer and the Conv layer, the received image is compressed and feature extracted to different degrees to obtain simulated primary visual feature maps of different sizes, which are used for feature extraction and channel aggregation in the backbone network module and the target detection module to obtain the backbone feature map and the target feature map with useful information of the simulated primary visual cortex.

[0043] Specifically, the backbone network module includes n feature map compression extraction modules connected in sequence, each feature map compression extraction module includes a Concat layer, and the Concat layer is used to perform channel aggregation based on the simulated primary visual feature map.

[0044] The i-th feature map compression and extraction module includes a Conv layer, a Concat layer, a RepBlock layer, and a CA attention layer connected in sequence, which are used to perform size compression, channel aggregation, re-parameterization feature extraction, and feature extraction of interest on the feature map output by the previous module in sequence, wherein the feature map extracted by the i-th RepBlock layer is used as the i-th backbone feature map, and the feature map of interest extracted by the i-th CA attention layer is used as the input of the next feature map compression and extraction module, where 1≤i <n;

[0045] The nth feature map compression extraction module includes a Conv layer, a Concat layer, a RepBlock layer, a SPPF layer and a CA attention layer connected in sequence; the feature map of interest extracted by the CA attention layer is the nth backbone feature map.

[0046] More specifically, the Concat layer of the first feature map compression extraction module performs channel aggregation based on the second simulated primary visual feature map and the feature map obtained by size compression of the first simulated primary visual feature map of interest;

[0047] The Concat layer of the j-th feature map compression advance module performs channel aggregation based on the j-th simulated primary visual feature map and the feature map obtained by size compression of the output of the j-1-th feature map compression extraction module, where 1 <j≤n。

[0048] Furthermore, the target detection module includes a target feature map extraction module and a Detect module; wherein the target feature map extraction module includes n-1 FPN modules and n-1 feature aggregation modules arranged in sequence, which are used to generate multiple target feature maps based on multiple simulated primary visual interest feature maps and backbone feature maps; the Detect module is used to fuse multiple target feature maps of different sizes, perform target classification and coordinate positioning, and obtain target detection results.

[0049] Specifically, the FPN module includes a Conv layer, an Upsample layer, a Concat layer, and a RepBlock layer connected in sequence; the feature aggregation module includes a Conv layer, a Concat layer, and a RepBlock layer connected in sequence.

[0050] As a specific embodiment, in this embodiment, n is set to 3; the imitation primary visual network module includes 4 imitation primary visual cortex modules; by adjusting parameters such as the convolution kernel of the imitation primary visual cortex module, the 4 imitation primary visual cortex modules respectively output imitation primary visual feature maps of four sizes of H / 4*W / 4, H / 8*W / 8, H / 16*W / 16, and H / 32*W / 32.

[0051] The backbone network includes three feature map compression and extraction modules connected in sequence. The Conv layer in the first feature map compression and extraction module receives the first imitation primary visual feature map of interest with a size of H / 4*W / 4 and compresses the size to obtain a feature map with a size of H / 8*W / 8. The compressed feature map is sequentially passed through the corresponding Concat layer and the second imitation primary visual feature map for channel aggregation, and then re-parameterized feature extraction is performed through the RepBlock layer to obtain the first backbone feature map with a size of H / 8*W / 8; and the first backbone feature map is subjected to feature extraction of interest through the CA attention layer to obtain the feature map of interest output by the first feature map compression and extraction module.

[0052] The second to third feature map compression and extraction modules respectively receive the feature maps of interest output by the previous feature map compression and extraction module, and after corresponding size compression, channel aggregation and feature extraction, obtain a third backbone feature map with a size of H / 16*W / 16 and a third backbone feature map with a size of H / 32*W / 32 respectively.

[0053] Further, through the target detection module, multiple channel aggregations are performed based on the backbone feature map and the simulated primary visual feature map to obtain n target feature maps of corresponding sizes, including:

[0054] Based on three backbone feature maps of different sizes, two FPN modules are used to perform size compression, size expansion, channel aggregation and feature extraction to obtain a first target feature map; specifically, the method includes: inputting the third backbone feature map into the Conv layer of the first FPN module for feature extraction and performing size expansion through the Upsamle layer to obtain a feature map with a size of H / 16*W / 16, then performing channel aggregation on the feature map after size expansion and the second backbone feature map through the Concat layer of the first FPN module, and performing re-parameterized feature extraction through the corresponding RepBlock layer to obtain the output of the first FPN module; inputting the feature map output by the first FPN module into the second FPN module, performing feature extraction through the Conv layer of the second FPN module and performing size expansion through the Upsamle layer to obtain a feature map with a size of H / 8*W / 8, then performing channel aggregation on the feature map after size expansion and the first backbone feature map through the Concat layer of the second FPN module, and performing re-parameterized feature extraction through the corresponding RepBlock layer to obtain the first target feature map. Among them, the Concat layer of the second FPN module is used to perform channel aggregation on the feature map with a size of H / 8*W / 8 output by the Upsample layer in the second FPN module, the first backbone feature map and the second simulated primary visual feature map of interest; the feature map output by the Concat layer of the second FPN module is subjected to feature extraction by the corresponding RepBlock layer to obtain the first target feature map with a size of H / 8*W / 8.

[0055] After the first target feature map and the feature map output by the first FPN module are compressed in size respectively, they are aggregated with the third imitation primary visual feature map through the Concat layer in the first feature aggregation module, and feature extraction is performed through the corresponding RepBlock layer to obtain a second target feature map with a size of H / 16*W / 16;

[0056] After the second target feature map and the feature map output by the first FPN module are compressed in size, they are aggregated with the fourth imitation primary visual interest feature map through the Concat layer in the second feature aggregation module, and feature extraction is performed through the corresponding RepBlock layer to obtain a third target feature map with a size of H / 32*W / 32;

[0057] The first target feature map, the second target feature map and the third target feature map are fused through the Detect layer to obtain the target detection result.

[0058] In this embodiment, the RepBlock layer is consistent with the corresponding module in YOLOv6; the Detect layer can use the detection layer of YOLOv5 or YOLOv6, and this example uses the detection layer of YOLOv6.

[0059] In summary, the embodiment of the present invention provides a target detection system based on a multi-level simulated primary visual cortex module, which adopts a multi-level simulated primary visual cortex module, and obtains simulated primary visual feature maps of different sizes by adjusting the convolution kernel and other parameters of the simulated primary visual cortex module; and a channel aggregation layer is added to the backbone network module, and the feature map output by the simulated primary visual cortex module is integrated into the backbone network, which increases the useful information of the simulated primary visual cortex in the backbone network, that is, the feature information processed by the human visual mechanism is more similar, and has a higher ability to resist noise interference. In addition, the present invention introduces the RepBlock layer of the YOLO6 model, and reduces the amount of parameters and calculations by the re-parameterization method during the feature extraction process, thereby improving the model reasoning efficiency; and the present invention transmits the useful information of the simulated primary visual cortex to the end of the target detection module across layers over a long distance, avoiding the performance impact caused by the information attenuation at the end of the network model, ensuring the useful information of the simulated primary visual cortex in the entire network model. The information validity, further ensuring the continuity of the effect of the simulated visual cortex module. The target detection performance of the target detection system under noise interference such as strong light, weak light, rain, snow, fog and haze weather, and AI confrontation is improved.

[0060] Those skilled in the art will appreciate that all or part of the processes of the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0061] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A target detection system based on a multi-level primary visual cortex-like module, characterized in that: include: The simulated primary vision network module is used to obtain a plurality of simulated primary vision feature maps of different sizes and corresponding simulated primary vision feature maps of interest of the image to be detected; The simulated primary visual network module includes n+1 simulated primary visual feature extraction networks, and the simulated primary visual feature extraction network includes a simulated primary visual cortex module, a RepBlock module and a CA attention layer arranged in series; the simulated primary visual cortex module includes a VOneBlock layer and a Conv layer arranged in parallel, and a feature fusion layer for fusing the outputs of the VOneBlock layer and the Conv layer; by setting different parameters for the VOneBlock layer and the Conv layer, the received image is compressed and feature extracted to different degrees to obtain simulated primary visual feature maps of different sizes; A backbone network module is used to obtain a plurality of backbone feature maps of corresponding sizes through channel aggregation and feature extraction based on a plurality of simulated primary visual feature maps of different sizes; the backbone network module includes n feature map compression extraction modules connected in sequence, each of which includes a Concat layer, and the Concat layer is used to perform channel aggregation based on the simulated primary visual feature map; the Concat layer of the first feature map compression extraction module performs channel aggregation based on the second simulated primary visual feature map and the feature map obtained by size compression of the first simulated primary visual feature map of interest; the Concat layer of the jth feature map compression extraction module performs channel aggregation based on the jth simulated primary visual feature map and the feature map obtained by size compression of the output of the j-1th feature map compression extraction module, wherein, ; The target detection module is used to perform channel aggregation based on the multiple backbone feature maps and the simulated primary visual interest feature maps of corresponding sizes to obtain multiple target feature maps of corresponding sizes; and to fuse the multiple target feature maps to obtain the target detection result.

2. The target detection system based on the multi-level simulated primary visual cortex module according to claim 1 is characterized in that: It also includes an ImageSub layer, which is used to obtain the to-be-processed images in parallel, and input the to-be-processed images obtained in parallel into each simulated primary visual cortex module respectively.

3. The target detection system based on the multi-level simulated primary visual cortex module according to claim 1 is characterized in that: The i-th feature map compression and extraction module includes a Conv layer, a Concat layer, a RepBlock layer, and a CA attention layer connected in sequence, which are used to perform size compression, channel aggregation, re-parameterization feature extraction, and feature extraction of interest on the feature map output by the previous module in sequence, wherein the feature map extracted by the i-th RepBlock layer is used as the i-th backbone feature map, and the feature map of interest extracted by the i-th CA attention layer is used as the input of the next feature map compression and extraction module, wherein ; The nth feature map compression extraction module includes a Conv layer, a Concat layer, a RepBlock layer, a SPPF layer and a CA attention layer connected in sequence; the feature map of interest extracted by the CA attention layer is the nth backbone feature map.

4. The target detection system based on the multi-level simulated primary visual cortex module according to claim 1 is characterized in that: The target detection module includes a target feature map extraction module and a Detect module; The target feature map extraction module includes n-1 FPN modules and n-1 feature aggregation modules arranged in sequence, and is used to generate multiple target feature maps based on multiple simulated primary visual feature maps of interest and backbone feature maps; The Detect module is used to fuse multiple target feature maps of different sizes, perform target classification and coordinate positioning, and obtain target detection results.

5. The target detection system based on the multi-level simulated primary visual cortex module according to claim 4 is characterized in that: The FPN module includes a Conv layer, an Upsample layer, a Concat layer and a RepBlock layer connected in sequence; the feature aggregation module includes a Conv layer, a Concat layer and a RepBlock layer connected in sequence.

6. The target detection system based on the multi-level simulated primary visual cortex module according to claim 4 or 5, characterized in that: The n is 3; Based on the backbone feature map and the simulated primary visual feature map, multiple channel aggregations are performed to obtain n target feature maps of corresponding sizes, including: Based on the three backbone feature maps of different sizes, two FPN modules are used to perform size compression, size expansion, channel aggregation and re-parameterized feature extraction to obtain the first target feature map; After the first target feature map and the feature map output by the second FPN module are compressed in size, they are aggregated with the third imitation primary visual feature map through the Concat layer in the first feature aggregation module, and re-parameterized feature extraction is performed through the RepBlock layer to obtain the second target feature map; After the second target feature map and the feature map output by the first FPN module are compressed in size, they are aggregated with the fourth imitation primary visual feature map through the Concat layer in the second feature aggregation module, and re-parameterized feature extraction is performed through the RepBlock layer to obtain the third target feature map; The first target feature map, the second target feature map and the third target feature map are subjected to feature fusion through the Detect layer to obtain a target detection result.

7. The target detection system based on the multi-level simulated primary visual cortex module according to claim 1 is characterized in that: It also includes an image acquisition module; the image acquisition module publishes video stream data or image sequences through the ImagesPub layer, and subscribes to the image to be detected at the current moment through the ImageSub layer.

Images