Target object detection and processing method and related product

Through the improved YOLOv8 model, the application of anchor-free frames and layered attention treatment is adopted to solve the problem of accuracy and time balance in garbage detection, and high-precision and real-time detection of multiple types of garbage are achieved, which expands the application scenarios.

CN118887584APending Publication Date: 2024-11-01XUZHOU COLLEGE OF INDAL TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410963486.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-18
Publication Date
2024-11-01

AI Technical Summary

Technical Problem

The existing target detection technology has difficulties in balancing detection accuracy and time in the field of garbage detection, and at the same time, there are fewer categories to limit the scope of application and scenarios.

Method used

An improved YOLOv8 model is proposed, using an anchor-free frame detection method, combining image feature extraction, layered attention processing and spatial pyramid pooling, enhancing feature extraction capabilities, supporting multi-type domestic waste detection, and through feature fusion and prediction The header determines the target category and location information.

Benefits of technology

It realizes high-precision and real-time multi-type domestic waste detection, improves detection accuracy and application range, and meets the needs of real-time detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118887584A_ABST
    Figure CN118887584A_ABST
Patent Text Reader

Abstract

The invention relates to a target object detection and processing method and a related product, and relates to the technical field of target detection and processing. The detection method comprises the following steps: acquiring a to-be-detected target image, and performing feature extraction on the to-be-detected target image to obtain a target image convolution feature map; performing down-sampling processing and layered attention processing on the target image convolution feature map in sequence to obtain a down-sampling layered attention feature map; performing spatial pyramid pooling processing on the down-sampling layered attention feature map to obtain a pyramid pooling feature map; performing fusion processing on the target image convolution feature map, the down-sampling layered attention feature map and the pyramid pooling feature map to obtain fusion feature maps of different scales; and determining category and / or position information corresponding to a set target object in the to-be-detected target image based on the fused feature maps of different scales and the corresponding detection layers. And detection and processing of a target object can be realized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the technical field of target object detection and processing, and in particular to a target object detection and processing method and related products. Background Art

[0002] With the continuous development and progress of computer hardware computing power and deep learning technology, deep learning-based object detection technology is gradually being implemented in all aspects of life, such as smart cities, smart transportation, industrial manufacturing and other fields. The object detection task is to input an image and find all the target objects that contain the category to be detected and their corresponding location information. If the input object is a video, the object detection task needs to detect the target object and its location information frame by frame.

[0003] Accurate detection and identification of domestic waste is a basic task and research object in the construction of smart cities. With the rapid development of deep learning and target detection technology, technology-enabled cameras can complete waste detection tasks. With the continuous development of smart city construction, surveillance cameras will enable corresponding detection needs in the future. Therefore, it is necessary and basic to study high-precision real-time domestic waste target detection algorithms, which is an important basic guarantee for the successful construction of smart cities.

[0004] Nowadays, target detection technology has been successfully applied to garbage target detection. Cheng Hao et al. proposed a ground garbage detection model that improves Faster RCNN, using ResNet-50 as the feature extraction network, adding the channel attention model SENet to weight the channel dimension information of the feature map, using the feature pyramid structure to fuse the feature maps of different stages, and proposed to improve the intersection-over-union threshold parameter in two stages to train the model, thereby improving the quality of the candidate frame. Yao Lintao et al. proposed a lightweight garbage detection system that improves the YOLOv4 model, using GhostNet as the feature extraction network, and introducing the coordinate attention mechanism to capture the spatial feature information of the target. Lei Jianyun et al. proposed a recyclable garbage detection method that improves the YOLOv5s model, and proposed to use the channel attention model SENet to weight the feature map so that the target features are highlighted, and proposed an enhanced multi-scale receptive field structure based on SENet to express the image features more richly. The above methods are all limited by the balance between detection accuracy and detection time. At the same time, the small number of target detection categories limits the scope and scenarios of application. Summary of the invention

[0005] The present invention provides a method for detecting and processing a target object and a related product technical solution.

[0006] According to one aspect of the present disclosure, there is provided a method for obtaining a target object, comprising:

[0007] Acquire a target image to be detected, and perform feature extraction on the target image to be detected to obtain a convolution feature map of the target image;

[0008] Downsampling and hierarchical attention processing are sequentially performed on the convolution feature map of the target image to obtain a downsampled hierarchical attention feature map;

[0009] Performing spatial pyramid pooling processing on the downsampled hierarchical attention feature map to obtain a pyramid pooling feature map;

[0010] The target image convolution feature map, the downsampled hierarchical attention feature map and the pyramid pooling feature map are fused to obtain fused feature maps of different scales;

[0011] Based on the fused feature maps of different scales and their corresponding detection layers, the category and / or position information corresponding to the set target object in the target image to be detected is determined.

[0012] Preferably, the method of extracting features from the target image to be detected to obtain a convolution feature map of the target image comprises: performing convolution, batch normalization, activation function processing, downsampling, and convolution processing on the target image to be detected for a first set number of times in sequence to obtain a convolution feature map of the target image.

[0013] Preferably, the method of sequentially performing downsampling processing and layered attention processing on the target image convolution feature map to obtain a downsampled layered attention feature map includes: sequentially performing downsampling processing on the target image convolution feature map to obtain a first downsampled target image convolution feature map; performing a carrier marking operation on the first downsampled target image convolution feature map to obtain a first carrier convolution feature map; performing layered attention processing on the first downsampled target image convolution feature map and the first carrier convolution feature map to obtain a first downsampled layered attention feature map; performing downsampling processing on the first downsampled layered attention feature map to obtain a second downsampled target image convolution feature map; performing a carrier marking operation on the second downsampled target image convolution feature map to obtain a second carrier convolution feature map; performing layered attention processing on the second downsampled target image convolution feature map and the second carrier convolution feature map to obtain a final downsampled layered attention feature map.

[0014] Preferably, the target image convolution feature map is sequentially downsampled and layered-attention processed to obtain a downsampled layered-attention feature map, including: sequentially downsampling the target image convolution feature map to obtain a first downsampled target image convolution feature map; performing a carrier marking operation on the first downsampled target image convolution feature map to obtain a first carrier convolution feature map; performing layered-attention processing on the first downsampled target image convolution feature map and the first carrier convolution feature map to obtain a first downsampled layered-attention feature map; downsampling the first downsampled layered-attention feature map to obtain a second downsampled target image convolution feature map; performing a carrier marking operation on the second downsampled target image convolution feature map to obtain a second carrier convolution feature map; performing layered-attention processing on the second downsampled target image convolution feature map and the second carrier convolution feature map to obtain a final downsampled layered-attention feature map.

[0015] Preferably, the method of performing a carrier marking operation on the first down-sampled target image convolution feature map to obtain a first carrier convolution feature map includes: dividing the first down-sampled target image convolution feature map according to a set window size to obtain multiple first local window feature maps of the same size; sequentially using convolution layers and pooling layers corresponding to convolution kernels of the same size as the set window size to initialize the carrier mark to obtain a corresponding first carrier convolution feature map.

[0016] Preferably, the method of performing hierarchical attention processing on the first down-sampled target image convolution feature map and the first carrier convolution feature map to obtain a first down-sampled hierarchical attention feature map includes: normalizing the first carrier convolution feature map (the first carrier label after initialization) to obtain a first normalized carrier label; processing the first normalized carrier label using a first multi-layer perceptron and a first attention model respectively to obtain a corresponding first global information feature map and a second global information feature map; splicing the first local window feature map, the first global information feature map and the second global information feature map corresponding to the first down-sampled target image convolution feature map to obtain a first local global feature map; The invention relates to a method for preparing an image of a first downsampled hierarchical attention feature map and a method for preparing an image of a first downsampled hierarchical attention feature map. The method comprises: normalizing the local global feature map to obtain a first normalized local global feature map; processing the first normalized local global feature map using a second multilayer perceptron and a second attention model respectively to obtain a corresponding first information feature map and a second information feature map; splicing the first local global feature map, the first information feature map and the second information feature map to obtain a first feature map to be processed; segmenting the features corresponding to the carrier mark and the non-carrier mark of the first feature map to be processed to obtain a first carrier mark feature and a first non-carrier mark feature; upsampling and dimensionally transforming the first carrier mark feature and the first non-carrier mark feature respectively, and then splicing and merging them to obtain a first downsampled hierarchical attention feature map.

[0017] Preferably, the carrier marking operation is performed on the second down-sampled target image convolution feature map to obtain the second carrier convolution feature map, including: dividing the second down-sampled target image convolution feature map according to a set window size to obtain multiple second local window feature maps of the same size; sequentially using convolution layers and pooling layers corresponding to convolution kernels of the same size as the set window size to initialize the carrier mark to obtain the corresponding second carrier convolution feature map.

[0018] Preferably, the method of performing hierarchical attention processing on the second down-sampled target image convolution feature map and the second carrier convolution feature map to obtain the final down-sampled hierarchical attention feature map includes: normalizing the second carrier convolution feature map (the second carrier label after initialization) to obtain a second normalized carrier label; using a third multi-layer perceptron and a third attention model to process the second normalized carrier label to obtain a corresponding third global information feature map and a fourth global information feature map; splicing the second local window feature map, the third global information feature map and the fourth global information feature map corresponding to the second down-sampled target image convolution feature map to obtain a second local global feature map; The method comprises the steps of: normalizing the local global feature map to obtain a second normalized local global feature map; processing the second normalized local global feature map using a fourth multilayer perceptron and a fourth attention model respectively to obtain a corresponding third information feature map and a fourth information feature map; splicing the second local global feature map, the third information feature map and the fourth information feature map to obtain a second feature map to be processed; segmenting the features corresponding to the carrier mark and the non-carrier mark of the second feature map to be processed to obtain a second carrier mark feature and a second non-carrier mark feature; upsampling and dimensionally transforming the second carrier mark feature and the second non-carrier mark feature respectively, and then splicing and merging them to obtain a final downsampled hierarchical attention feature map.

[0019] Preferably, the method of fusing the target image convolution feature map, the downsampled hierarchical attention feature map and the pyramid pooling feature map to obtain fused feature maps of different scales comprises:

[0020] Performing upsampling processing on the pyramid pooling feature map to obtain an upsampled pyramid pooling feature map; splicing the upsampled pyramid pooling feature map and the downsampled layered attention feature map to obtain a first spliced ​​feature map; performing target feature information highlighting processing on the first spliced ​​feature map to obtain a first salient target feature map; performing upsampling processing on the first salient target feature map to obtain a first upsampled salient target feature map; splicing the first upsampled salient target feature map and the target image convolution feature map to obtain a second spliced ​​feature map; performing target feature information highlighting processing on the second spliced ​​feature map to obtain a second salient target feature map; configuring the second salient target feature map as a fusion feature map of the first scale;

[0021] The second salient target feature map is sequentially subjected to convolution, batch normalization, and activation function processing for a first set number of times to obtain a first salient target convolution feature map; the first salient target feature map and the first salient target convolution feature map are concatenated to obtain a third concatenated feature map; the third concatenated feature map is subjected to target feature information salient processing to obtain a third salient target feature map; the third salient target feature map is configured as a fusion feature map of the second scale;

[0022] The third salient target feature map is sequentially subjected to convolution, batch normalization, and activation function processing for a first set number of times to obtain a second salient target convolution feature map; the second salient target convolution feature map and the pyramid pooling feature map are spliced ​​to obtain a fourth concatenated feature map; the fourth concatenated feature map is subjected to target feature information salient processing to obtain a fourth salient target feature map; and the fourth salient target feature map is configured as a fusion feature map of a third scale.

[0023] Preferably, the method of performing target feature information highlighting processing on the first spliced ​​feature map to obtain a first salient target feature map includes: performing attention processing on the first spliced ​​feature map to obtain a first attention spliced ​​feature map; using multiple convolution blocks with residual connections to enhance and fuse the first attention spliced ​​feature map to obtain multiple first enhanced fused spliced ​​feature maps; sequentially using a first connection layer, a first convolution layer, a first batch normalization layer, and a first activation function layer to process the multiple first enhanced fused spliced ​​feature maps to obtain a first salient target feature map; and / or, before sequentially using the first connection layer, the first convolution layer, the first batch normalization layer, and the first activation function layer to process the multiple first enhanced fused spliced ​​feature maps to obtain the first salient target feature map, it also includes: sequentially performing pooling processing for a third set number of times on each first enhanced fused spliced ​​feature map in the multiple first enhanced fused spliced ​​feature maps, and inputting the first enhanced fused spliced ​​feature map after each pooling processing into the first connection layer.

[0024] Preferably, the method of performing target feature information highlighting processing on the second connection feature map to obtain a second salient target feature map includes: performing attention processing on the second splicing feature map to obtain a second attention splicing feature map; using multiple convolution blocks with residual connections to enhance and fuse the second attention splicing feature map to obtain multiple second enhanced fused splicing feature maps; sequentially using a second connection layer, a second convolution layer, a second batch normalization layer, and a second activation function layer to process the multiple second enhanced fused splicing feature maps to obtain a second salient target feature map; and / or, before sequentially using a second connection layer, a second convolution layer, a second batch normalization layer, and a second activation function layer to process the multiple second enhanced fused splicing feature maps to obtain a second salient target feature map, it also includes: sequentially performing pooling processing for a third set number of times on each second enhanced fused splicing feature map in the multiple second enhanced fused splicing feature maps, and inputting the second enhanced fused splicing feature map after each pooling processing into the second connection layer.

[0025] Preferably, the method of performing target feature information highlighting processing on the third connection feature map to obtain a third salient target feature map includes: performing attention processing on the third splicing feature map to obtain a third attention splicing feature map; using multiple convolution blocks with residual connections to enhance and fuse the third attention splicing feature map to obtain multiple third enhanced fused splicing feature maps; sequentially using a third connection layer, a third convolution layer, a third batch normalization layer, and a third activation function layer to process the multiple third enhanced fused splicing feature maps to obtain a third salient target feature map; and / or, before sequentially using a third connection layer, a third convolution layer, a third batch normalization layer, and a third activation function layer to process the multiple third enhanced fused splicing feature maps to obtain the third salient target feature map, it also includes: sequentially performing pooling processing for a third set number of times on each third enhanced fused splicing feature map in the multiple third enhanced fused splicing feature maps, and inputting the third enhanced fused splicing feature map after each pooling processing into the third connection layer.

[0026] Preferably, the method of performing target feature information highlighting processing on the fourth connection feature map to obtain a fourth salient target feature map includes: performing attention processing on the fourth splicing feature map to obtain a fourth attention splicing feature map; using multiple convolution blocks with residual connections to enhance and fuse the fourth attention splicing feature map to obtain multiple fourth enhanced fused splicing feature maps; sequentially using a fourth connection layer, a fourth convolution layer, a fourth batch normalization layer, and a fourth activation function layer to process the multiple fourth enhanced fused splicing feature maps to obtain a fourth salient target feature map; and / or, before sequentially using a fourth connection layer, a fourth convolution layer, a fourth batch normalization layer, and a fourth activation function layer to process the multiple fourth enhanced fused splicing feature maps to obtain the fourth salient target feature map, it also includes: sequentially performing pooling processing for a third set number of times on each of the multiple fourth enhanced fused splicing feature maps, and inputting the fourth enhanced fused splicing feature map after each pooling processing into the fourth connection layer.

[0027] Preferably, before acquiring the target image to be detected, the type corresponding to the source image of the target image to be detected is determined; if the type corresponding to the source image of the target image to be detected is a video image, the multi-moment video frame images corresponding to the video image are configured as the target image to be detected; if the type corresponding to the source image of the target image to be detected is a non-video image at a certain moment, the non-moment video image at a certain moment is configured as the target image to be detected.

[0028] Preferably, the target objects set in the target image to be detected are configured as multiple types of domestic waste.

[0029] According to one aspect of the present disclosure, a method for processing a target object is provided, comprising: applying the above-mentioned detection method to obtain the category and / or position information corresponding to the set target object in the target image to be detected;

[0030] Using a preset target object segmentation model, segmenting the set target object in the detection target image to obtain a corresponding set target object mask image;

[0031] Based on the set target object mask image, detection and correction are performed on the category and / or position information corresponding to the set target object in the target image to be detected to obtain corresponding correction category and / or correction position information.

[0032] Preferably, the method of detecting and correcting the category corresponding to the set target in the target image to be detected based on the set target mask image to obtain the corresponding corrected category includes: configuring the preset target segmentation model as a multi-type target segmentation model; wherein, using the multi-type target segmentation model, segmenting multiple types of set targets in the detection target image to obtain corresponding multi-type set target mask images; determining the category corresponding to the segmented set target based on different mask values ​​corresponding to the set target mask image; reconfiguring the category corresponding to the segmented set target based on the category corresponding to the segmented set target to complete the category correction corresponding to the set target; or,

[0033] Preferably, the method of detecting and correcting the category corresponding to the set target in the target image to be detected based on the set target mask image to obtain the corresponding corrected category includes: configuring the preset target segmentation model as a single target segmentation model; wherein, using the single target segmentation model, segmenting the set target in the detection target image to obtain the corresponding set target mask image; wherein, the set target may include multiple types of set targets, and the multiple types of set targets have the same mask values ​​in the set target mask image; using the set target mask image and the corresponding target image to be detected, determining the corresponding set target image; performing image convolution feature and image texture feature extraction on the set target image respectively to obtain a plurality of corresponding image convolution features and image texture features; and respectively extracting the image convolution features and image texture features based on the plurality of image convolution features. The method of extracting image convolution features from the set target image comprises: extracting features from the set target image to obtain a target convolution feature map; performing downsampling and layered attention processing on the target convolution feature map in sequence to obtain a downsampled layered attention target feature map; performing spatial pyramid pooling processing on the downsampled layered attention target feature map to obtain a pyramid pooling target feature map; and performing fusion processing on the target convolution feature map, the downsampled layered attention target feature map and the pyramid pooling target feature map to obtain a plurality of image convolution features of different scales.

[0034] Preferably, the method of determining the corresponding set target object image by using the set target object mask image and the corresponding target image to be detected includes: configuring the mask value corresponding to the set target object mask image to 1; performing a dot product operation on the configured set target object mask image and the corresponding target image to be detected to determine the corresponding set target object image.

[0035] Preferably, the method for correcting the category corresponding to the set target object based on the classification result and the category corresponding to the set target object includes: performing regression processing on the probability value corresponding to the result to obtain the corrected category; using the corrected category to reconfigure the erroneous category corresponding to the set target object to complete the correction of the category corresponding to the set target object; or, performing frequency statistics on the classification results according to the same category, configuring the category with the highest frequency as the corrected category; using the corrected category to reconfigure the erroneous category corresponding to the set target object to complete the correction of the category corresponding to the set target object.

[0036] Preferably, the method of classifying the set target object using a preset classifier based on each image convolution feature and image texture feature of the multiple image convolution features, comprises: respectively splicing each image convolution feature and the image texture feature to obtain multiple groups of classification splicing features; and respectively classifying the set target object using a preset classifier and each group of classification splicing features in the multiple groups of classification splicing features.

[0037] Preferably, the method of detecting and correcting the position information corresponding to the set target object in the target image to be detected based on the set target object mask image to obtain the corresponding corrected position information includes: determining a first number corresponding to the detected set target object based on the position information corresponding to the set target object; determining a second number corresponding to the segmented set target objects based on different mask values ​​corresponding to the set target object mask image; if the first number is less than the second number, determining the target image to be corrected corresponding to the missing target; performing target detection on the target image to be corrected to obtain the corresponding missing position information; and detecting and correcting the position information corresponding to the set target object in the target image to be detected using the missing position information to obtain the corresponding corrected position information.

[0038] Preferably, after using the missing position information to detect and correct the position information corresponding to the set target object in the target image to be detected and obtaining the corresponding corrected position information, it also includes: extracting a plurality of image convolution features and / or image texture features corresponding to the target image to be corrected corresponding to the missing target object; and classifying the set target object using a preset classifier based on each image convolution feature and / or image texture feature of the plurality of image convolution features corresponding to the target image to be corrected.

[0039] Preferably, the method for extracting multiple image convolution features corresponding to the target image to be corrected corresponding to the missing target object includes: performing feature extraction on the target image to be corrected corresponding to the missing target object to obtain a convolution feature map of the target to be corrected; performing downsampling processing and layered attention processing on the convolution feature map of the target to be corrected in sequence to obtain a layered attention feature map of the target to be corrected; performing spatial pyramid pooling processing on the layered attention feature map of the target to be corrected to obtain a pyramid pooling feature map of the target to be corrected; performing fusion processing on the convolution feature map of the target to be corrected, the layered attention feature map of the target to be corrected and the pyramid pooling feature map of the target to be corrected to obtain multiple image convolution features of different scales corresponding to the target image to be corrected.

[0040] According to one aspect of the present disclosure, there is provided an object detection device, comprising:

[0041] A first feature extraction unit is used to obtain a target image to be detected, and perform feature extraction on the target image to be detected to obtain a convolution feature map of the target image;

[0042] A first feature processing unit is used to sequentially perform downsampling processing and layered attention processing on the target image convolution feature map to obtain a downsampled layered attention feature map; wherein, the sequentially performing downsampling processing and layered attention processing on the target image convolution feature map to obtain a downsampled layered attention feature map includes: sequentially performing downsampling processing on the target image convolution feature map to obtain a first downsampled target image convolution feature map; performing a carrier marking operation on the first downsampled target image convolution feature map to obtain a first carrier convolution feature map; performing layered attention processing on the first downsampled target image convolution feature map and the first carrier convolution feature map to obtain a first downsampled layered attention feature map; performing downsampling processing on the first downsampled layered attention feature map to obtain a second downsampled target image convolution feature map; performing a carrier marking operation on the second downsampled target image convolution feature map to obtain a second carrier convolution feature map; performing layered attention processing on the second downsampled target image convolution feature map and the second carrier convolution feature map to obtain a final downsampled layered attention feature map; and / or,

[0043] Among them, the method of performing a carrier marking operation on the first down-sampled target image convolution feature map to obtain a first carrier convolution feature map includes: dividing the first down-sampled target image convolution feature map according to a set window size to obtain multiple first local window feature maps of the same size; sequentially using convolution layers and pooling layers corresponding to convolution kernels of the same size as the set window size to initialize the carrier mark to obtain a corresponding first carrier convolution feature map; and / or,

[0044] Among them, the method of performing hierarchical attention processing on the first down-sampled target image convolution feature map and the first carrier convolution feature map to obtain a first down-sampled hierarchical attention feature map includes: normalizing the first carrier convolution feature map (the first carrier label after initialization) to obtain a first normalized carrier label; using a first multi-layer perceptron and a first attention model to process the first normalized carrier label to obtain a corresponding first global information feature map and a second global information feature map; splicing the first local window feature map, the first global information feature map and the second global information feature map corresponding to the first down-sampled target image convolution feature map to obtain a first local global feature map; The local feature map is normalized to obtain a first normalized local global feature map; the first normalized local global feature map is processed by a second multilayer perceptron and a second attention model respectively to obtain a corresponding first information feature map and a second information feature map; the first local global feature map, the first information feature map and the second information feature map are spliced ​​to obtain a first feature map to be processed; the features corresponding to the carrier mark and the non-carrier mark of the first feature map to be processed are segmented to obtain a first carrier mark feature and a first non-carrier mark feature; the first carrier mark feature and the first non-carrier mark feature are upsampled and dimensionally transformed respectively, and then spliced ​​and merged to obtain a first downsampled hierarchical attention feature map; and / or,

[0045] Among them, the performing of a carrier marking operation on the second down-sampled target image convolution feature map to obtain a second carrier convolution feature map includes: dividing the second down-sampled target image convolution feature map according to a set window size to obtain a plurality of second local window feature maps of the same size; sequentially using a convolution layer and a pooling layer corresponding to a convolution kernel of the same size as the set window size to initialize the carrier mark to obtain a corresponding second carrier convolution feature map; and / or,

[0046] Among them, the method of performing hierarchical attention processing on the second down-sampled target image convolution feature map and the second carrier convolution feature map to obtain the final down-sampled hierarchical attention feature map includes: normalizing the second carrier convolution feature map (the second carrier label after initialization) to obtain a second normalized carrier label; using a third multi-layer perceptron and a third attention model to process the second normalized carrier label to obtain a corresponding third global information feature map and a fourth global information feature map; splicing the second local window feature map, the third global information feature map and the fourth global information feature map corresponding to the second down-sampled target image convolution feature map to obtain a second local global feature map; The first embodiment of the present invention relates to a method for processing a first local global feature map to obtain a second normalized local global feature map; the second normalized local global feature map is processed by a fourth multilayer perceptron and a fourth attention model respectively to obtain a corresponding third information feature map and a fourth information feature map; the second local global feature map, the third information feature map and the fourth information feature map are spliced ​​to obtain a second feature map to be processed; the features corresponding to the carrier mark and the non-carrier mark of the second feature map to be processed are segmented to obtain a second carrier mark feature and a second non-carrier mark feature; the second carrier mark feature and the second non-carrier mark feature are upsampled and dimensionally transformed respectively, and then spliced ​​and merged to obtain a final downsampled hierarchical attention feature map;

[0047] A second feature processing unit is used to perform spatial pyramid pooling processing on the downsampled hierarchical attention feature map to obtain a pyramid pooling feature map;

[0048] A feature fusion unit, used to fuse the target image convolution feature map, the downsampled hierarchical attention feature map and the pyramid pooling feature map to obtain fused feature maps of different scales;

[0049] The detection unit is used to determine the category and / or position information corresponding to the set target object in the target image to be detected based on the fused feature maps of different scales and their corresponding detection layers.

[0050] According to one aspect of the present disclosure, there is provided an object detection device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to call the instructions stored in the memory to execute the above-mentioned detection method.

[0051] According to one aspect of the present disclosure, there is provided a target object detection device, comprising: a computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions implement the above-mentioned detection method when executed by a processor.

[0052] According to one aspect of the present disclosure, there is provided a target object processing device, comprising: the target object detection device as described above; and

[0053] A preset target object segmentation unit, used to segment the set target object in the detection target image using a preset target object segmentation model to obtain a corresponding set target object mask image;

[0054] The correction unit is used to detect and correct the category and / or position information corresponding to the set target object in the target image to be detected based on the set target object mask image to obtain corresponding correction category and / or correction position information.

[0055] According to one aspect of the present disclosure, there is provided a target object processing device, comprising: a target object detection device as described above; and a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to call the instructions stored in the memory to execute the above-mentioned processing method.

[0056] According to one aspect of the present disclosure, there is provided a target object processing device, comprising: the target object detection device as described above; and a computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions implement the above processing method when executed by a processor.

[0057] In the embodiments of the present disclosure, a method for detecting and processing a target object and a related product technical solution are proposed to solve the technical problems that current methods are limited by the balance between detection accuracy and detection time, and the small number of target detection categories limits the scope of application and scenarios.

[0058] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

[0059] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used to illustrate the technical solutions of the present disclosure together with the specification.

[0061] Figure 1 A flow chart showing a method for detecting a target object according to an embodiment of the present disclosure;

[0062] Figure 2 A network structure diagram corresponding to the improved YOLOv8 model according to an embodiment of the present disclosure is shown;

[0063] Figure 3A network structure diagram corresponding to a hierarchical attention module of an image feature extraction structure block (image feature extraction model) in an improved YOLOv8 model according to an embodiment of the present disclosure is shown;

[0064] Figure 4 A network structure diagram corresponding to a target feature information highlighting processing module of a feature fusion structure block (feature fusion model) in an improved YOLOv8 model according to an embodiment of the present disclosure is shown;

[0065] Figure 5 The figure shows the index result of the improved YOLOv8 model (method) on the MS COCO2017 dataset according to the embodiment of the present disclosure;

[0066] Figure 6 A comparison chart of indicators of the improved YOLOv8 model (method) according to an embodiment of the present disclosure and the YOLOv8 model (method) before improvement on the MS COCO2017 dataset is shown;

[0067] Figure 7 The indicator result diagram of the improved YOLOv8 model (method) according to the embodiment of the present disclosure on the self-built data set is shown;

[0068] Figure 8 A comparison chart of indicators of the improved YOLOv8 model (method) on a self-built data set according to an embodiment of the present disclosure is shown;

[0069] Fig. 9 A comparison chart of loss indicators of the improved YOLOv8 model (method) according to an embodiment of the present disclosure and the YOLOv8 model (method) before improvement on the MS COCO2017 dataset is shown;

[0070] Fig.10 The image detection results of the improved YOLOv8 model (method) according to the embodiment of the present disclosure on the self-built domestic waste verification set are shown;

[0071] Fig.11 is a block diagram of an electronic device 800 according to an exemplary embodiment;

[0072] Fig.12 is a block diagram of an electronic device 1900 according to an exemplary embodiment. DETAILED DESCRIPTION

[0073] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0074] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0075] The term "and / or" herein is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set consisting of A, B, and C.

[0076] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0077] It can be understood that the above-mentioned detection and processing method embodiments of each target object mentioned in the present disclosure can be combined with each other to form a combined embodiment without violating the principle logic. Due to space limitations, the present disclosure will not go into details.

[0078] In addition, the present disclosure also provides a target object detection device, a processing device, and a computer program product, all of which can be used to implement any target object detection and processing method provided by the present disclosure. The corresponding technical solutions and descriptions are referred to the corresponding records in the method part and will not be repeated here.

[0079] Figure 1 FIG. 1 is a flow chart showing a method for detecting a target object according to an embodiment of the present disclosure. Figure 1As shown, the target object detection method includes: step S101: obtaining a target image to be detected, and extracting features from the target image to be detected to obtain a target image convolution feature map; step S102: sequentially performing downsampling processing and layered attention processing on the target image convolution feature map to obtain a downsampled layered attention feature map; step S103: performing spatial pyramid pooling processing on the downsampled layered attention feature map to obtain a pyramid pooling feature map; step S104: fusing the target image convolution feature map, the downsampled layered attention feature map and the pyramid pooling feature map to obtain fused feature maps of different scales; step S105: based on the fused feature maps of different scales and their corresponding detection layers, determining the category and / or position information corresponding to the set target object in the target image to be detected. This is to solve the technical problem that the current methods are limited by the balance between detection accuracy and detection time, and the small number of target detection categories limits the scope of application and scenarios.

[0080] In the embodiments disclosed herein and other possible embodiments, in response to the above-mentioned problem, the present invention designs a target detection algorithm for target objects (e.g., domestic waste) of an improved YOLOv8 model based on the YOLOv8 model (target detection model) that is lightweight but has rich feature extraction capabilities, which has fewer model parameters while ensuring efficient target image extraction capabilities to be detected. At the same time, the present invention also constructs a dataset of domestic waste images of six types of targets (target images to be detected corresponding to multiple types of domestic waste), and richer samples bring more accurate algorithm models. At the same time, in order to facilitate deployment in actual scenarios, the target detection algorithm designed by the present invention is simple in structure and efficient, and an end-to-end method is used to input a target image to be detected, which can directly output the target detection result (the category and / or location information corresponding to the target object set in the target image to be detected), which can enable the camera to achieve high-precision domestic waste detection in real scenarios and meet the needs of real-time detection.

[0081] In the embodiments of the present disclosure and other possible embodiments, YOLOv8 continues to use the idea of ​​improving the detection speed of images by converting the target detection task into a regression problem, and studies a new feature extraction module based on the latest technology. At the same time, the algorithm removes the anchor frame design of the original algorithm and proposes a detection method without an anchor frame. In terms of the overall model structure design, YOLOv8 includes: an image feature extraction structure block (image feature extraction model), a feature fusion structure block (feature fusion model), and a detection prediction head structure block (prediction module).

[0082] Figure 2The network structure diagram corresponding to the improved YOLOv8 model according to the embodiment of the present disclosure is shown. On the basis of the above, the present disclosure improves the image feature extraction structure block (image feature extraction model) and the feature fusion structure block (feature fusion model), and then designs a target detection algorithm for target objects (e.g., domestic garbage) of the improved YOLOv8 model with a lightweight but rich feature extraction capability.

[0083] In the embodiments of the present disclosure and other possible embodiments, the image feature extraction structure block is used to extract features from the target image to be detected to obtain a target image convolution feature map; the target image convolution feature map is sequentially downsampled and layered attention processed to obtain a downsampled layered attention feature map; the downsampled layered attention feature map is spatially pyramid pooled to obtain a pyramid pooling feature map; at the same time, the feature fusion structure block is used to fuse the target image convolution feature map, the downsampled layered attention feature map and the pyramid pooling feature map to obtain fused feature maps of different scales; the detection prediction head structure block is used to determine the category and / or position information corresponding to the set target object in the target image to be detected based on the fused feature maps of different scales and their corresponding detection layers.

[0084] Step S101: Acquire a target image to be detected, and perform feature extraction on the target image to be detected to obtain a convolution feature map of the target image.

[0085] In an embodiment of the present disclosure, before acquiring the target image to be detected, the type corresponding to the source image of the target image to be detected is determined; if the type corresponding to the source image of the target image to be detected is a video image, the multi-moment video frame images corresponding to the video image are configured as the target image to be detected; if the type corresponding to the source image of the target image to be detected is a non-video image at a certain moment, the non-moment video image at a certain moment is configured as the target image to be detected.

[0086] In the embodiments of the present disclosure and other possible embodiments, the source image of the target image to be detected can be a picture, a video or a camera input. If the input source (source image) is a picture (a non-video image at a certain moment), the present invention directly configures the picture as the target image to be detected. If the input source is a video or a camera, the present invention first extracts the video or camera to obtain frame-by-frame pictures (multi-moment video frame images), and configures the frame-by-frame pictures as the target image to be detected.

[0087] In an embodiment of the present disclosure, the target objects in the target image to be detected are configured as multiple types of domestic waste. For example, the multiple types of domestic waste can be configured as at least one or more of the six types of domestic waste: batteries, bricks and ceramics, metal cans, cigarettes, food waste, and plastic bottles.

[0088] Step S102: downsampling and hierarchical attention processing are sequentially performed on the convolution feature map of the target image to obtain a downsampled hierarchical attention feature map.

[0089] In the embodiments of the present disclosure, Figure 2 As shown, the method for extracting features from the target image to be detected and obtaining a convolution feature map of the target image includes: performing convolution for a first set number of times greater than 1, batch normalization, activation function processing, downsampling for a second set number of times greater than 1, and convolution (convolution module / convolution layer / residual convolution block) processing on the target image to be detected in sequence to obtain a convolution feature map of the target image.

[0090] In the embodiment of the present disclosure and other possible embodiments, the convolution block CBS includes: 1 convolution (convolution module / convolution layer), 1 batch normalization (layer), and 1 activation function (layer) connected in sequence. The activation function can be configured as a SiLU activation function. The number of the first set times can be configured as 2 times; similarly, the number of the second set times can be configured as 2 times.

[0091] In the embodiments of the present disclosure, Figure 2 As shown, the method of sequentially performing downsampling processing and layered attention processing on the target image convolution feature map to obtain a downsampled layered attention feature map includes: sequentially performing downsampling processing on the target image convolution feature map to obtain a first downsampled target image convolution feature map; performing a carrier marking operation on the first downsampled target image convolution feature map to obtain a first carrier convolution feature map; performing layered attention processing on the first downsampled target image convolution feature map and the first carrier convolution feature map to obtain a first downsampled layered attention feature map; performing downsampling processing on the first downsampled layered attention feature map to obtain a second downsampled target image convolution feature map; performing a carrier marking operation on the second downsampled target image convolution feature map to obtain a second carrier convolution feature map; performing layered attention processing on the second downsampled target image convolution feature map and the second carrier convolution feature map to obtain a final downsampled layered attention feature map.

[0092] Figure 3 The network structure diagram corresponding to the hierarchical attention module of the image feature extraction structure block (image feature extraction model) in the improved YOLOv8 model according to the embodiment of the present disclosure is shown. Figure 3As shown, in the embodiments of the present disclosure and other possible embodiments, the input image (target image to be detected) first passes through two convolution blocks (one convolution, one batch normalization, and one activation function connected in sequence) to reduce the resolution of the target image to be detected / set target image to obtain a convolution feature map of the target image. Subsequently, four stages are performed to extract features. Before each stage, the feature map will first undergo downsampling, convolution layer, and layer normalization to reduce the size of the feature map. In the first two stages, the input feature map size is still high, so the features are still extracted through the residual convolution block; in the subsequent two stages, the proposed hierarchical attention module is used to establish the spatial position dependency between the shallow features and the deep features in the first downsampled target image convolution feature map, and the global features and local window features of the deep first downsampled target image convolution feature map. At the same time, a carrier token (CTs) module is proposed, which is used to characterize the window features and interact with the overall features of the image and the local window information.

[0093] In the embodiments of the present disclosure and other possible embodiments, the method of performing a carrier marking operation on the first down-sampled target image convolution feature map to obtain a first carrier convolution feature map includes: dividing the first down-sampled target image convolution feature map according to a set window size to obtain multiple first local window feature maps of the same size; sequentially using the convolution layer and pooling layer corresponding to the convolution kernel of the same size as the set window size to initialize the carrier mark to obtain the corresponding first carrier convolution feature map. The set window size can be configured as 3×3; similarly, the size corresponding to the convolution kernel of the same size as the set window size is also 3×3. The pooling layer can be configured as a global average pooling layer.

[0094] In the embodiments of the present disclosure and other possible embodiments, the method of performing hierarchical attention processing on the first down-sampled target image convolution feature map and the first carrier convolution feature map to obtain a first down-sampled hierarchical attention feature map includes: normalizing the first carrier convolution feature map (the first carrier label after initialization) to obtain a first normalized carrier label; using a first multi-layer perceptron and a first attention model to process the first normalized carrier label to obtain a corresponding first global information feature map and a second global information feature map; splicing the first local window feature map, the first global information feature map and the second global information feature map corresponding to the first down-sampled target image convolution feature map to obtain a first local global feature map. feature map; normalize the first local global feature map to obtain a first normalized local global feature map; use a second multilayer perceptron and a second attention model to process the first normalized local global feature map to obtain a corresponding first information feature map and a second information feature map; splice the first local global feature map, the first information feature map and the second information feature map to obtain a first feature map to be processed; segment the features corresponding to the carrier mark and the non-carrier mark of the first feature map to be processed to obtain a first carrier mark feature and a first non-carrier mark feature; upsample the first carrier mark feature and the first non-carrier mark feature, respectively, and then splice and merge them to obtain a first downsampled hierarchical attention feature map.

[0095] In the embodiments of the present disclosure and other possible embodiments, the carrier marking operation is performed on the second downsampled target image convolution feature map to obtain the second carrier convolution feature map, including: dividing the second downsampled target image convolution feature map according to the set window size to obtain multiple second local window feature maps of the same size; sequentially using the convolution layer and pooling layer corresponding to the convolution kernel of the same size as the set window size to initialize the carrier mark to obtain the corresponding second carrier convolution feature map. The set window size can be configured as 3×3; similarly, the size corresponding to the convolution kernel of the same size as the set window size is also 3×3. The pooling layer can be configured as a global average pooling layer.

[0096] In the embodiments of the present disclosure and other possible embodiments, the method of performing hierarchical attention processing on the second down-sampled target image convolution feature map and the second carrier convolution feature map to obtain the final down-sampled hierarchical attention feature map includes: normalizing the second carrier convolution feature map (the second carrier label after initialization) to obtain a second normalized carrier label; using a third multi-layer perceptron and a third attention model to process the second normalized carrier label to obtain a corresponding third global information feature map and a fourth global information feature map; splicing the second local window feature map corresponding to the second down-sampled target image convolution feature map, the third global information feature map and the fourth global information feature map to obtain a second local global feature map. feature map; normalize the second local global feature map to obtain a second normalized local global feature map; use a fourth multi-layer perceptron and a fourth attention model to process the second normalized local global feature map to obtain a corresponding third information feature map and a fourth information feature map; splice the second local global feature map, the third information feature map and the fourth information feature map to obtain a second feature map to be processed; segment the features corresponding to the carrier mark and the non-carrier mark of the second feature map to be processed to obtain a second carrier mark feature and a second non-carrier mark feature; upsample the second carrier mark feature and the second non-carrier mark feature, respectively, and then splice and merge them to obtain a final downsampled hierarchical attention feature map.

[0097] like Figure 4 As shown, the hierarchical attention module includes: two branches; one branch is a local window feature extraction branch, and the other branch is a global feature extraction branch of CTs operation. The first down-sampled target image convolution feature map / the second down-sampled target image convolution feature map is subjected to a carrier labeling operation, and CTs summarize the information in each local window in the first down-sampled target image convolution feature map / the second down-sampled target image convolution feature map, obtained through pooling and convolution operations, and they provide the first carrier convolution feature map / the second carrier convolution feature map corresponding to the summary information of the respective local windows; wherein each local window has a unique CT.

[0098] Subsequently, self-attention (an attention model) and multi-layer perceptron (Attention+MLP) are applied in the CTs branch to summarize and transfer the global information of the first carrier convolution feature map / the second down-sampled target image convolution feature map (the first information feature map and the second information feature map / the third global information feature map and the fourth global information feature map), and then the first local window feature map, the first global information feature map and the second global information feature map corresponding to the local window feature and the global image feature are fused (spliced) / the second local window feature map, the third global information feature map and the fourth global information feature map corresponding to the second down-sampled target image convolution feature map are spliced ​​to obtain the first local global feature map / the second local global feature map.

[0099] Afterwards, self-attention is performed again on the fused feature map, which further promotes the information exchange and extraction of the fused features. Specifically, the first local global feature map is normalized to obtain a first normalized local global feature map; the first normalized local global feature map is processed using a second multilayer perceptron and a second attention model respectively to obtain a corresponding first information feature map and a second information feature map; the first local global feature map, the first information feature map and the second information feature map are spliced ​​to obtain a first feature map to be processed. Specifically, the second local global feature map is normalized to obtain a second normalized local global feature map; the second normalized local global feature map is processed using a fourth multilayer perceptron and a fourth attention model respectively to obtain a corresponding third information feature map and a fourth information feature map; the second local global feature map, the third information feature map and the fourth information feature map are spliced ​​to obtain a second feature map to be processed.

[0100] Finally, the fused features are separated, and the global attention mechanism and the local attention mechanism are used alternately. The model can be further grouped, and the window has a higher level of feature information to better capture information at different levels. Specifically, the features corresponding to the carrier mark and the non-carrier mark of the first feature map to be processed are segmented to obtain the first carrier mark feature and the first non-carrier mark feature; the first carrier mark feature and the first non-carrier mark feature are upsampled and dimensionally transformed, and then spliced ​​and merged to obtain the first downsampled hierarchical attention feature map. Specifically, the features corresponding to the carrier mark and the non-carrier mark of the second feature map to be processed are segmented to obtain the second carrier mark feature and the second non-carrier mark feature; the second carrier mark feature and the second non-carrier mark feature are upsampled and dimensionally transformed, and then spliced ​​and merged to obtain the final downsampled hierarchical attention feature map.

[0101] In the specific implementation of the hierarchical attention module, given an input feature map x (the first downsampled target image convolution feature map / the second downsampled target image convolution feature map), whose width, height and channel are H, W and C respectively, the input feature x is first split into local windows to obtain x1, as shown in the following formula (1).

[0102] x1=Split 3×3 (x) (1)

[0103] Among them, 3×3 is the window size; then a convolution layer with a convolution kernel of 3×3 and a global average pooling layer are used to initialize CTs to obtain the first normalized vector label / second normalized vector label x CTs , the specific calculation is shown in the following formula (2).

[0104] x CTs =AvgPool(Conv 3×3 (x1)) (2)

[0105] Conv3×3 is a convolution layer with a convolution kernel size of 3×3, and AvgPool is a global average pooling operation. CTs Finally, self-attention and multi-layer perceptron are used to summarize and transfer global information to obtain the first local global feature map / the second local global feature map, as shown in the following formula (3).

[0106] x' CTs =x CTs +γ1·MHSA(LN(x CTs ))+γ2·MLP(LN(x CTs )) (3)

[0107] Among them, LN represents layer normalization, MHSA represents multi-head self-attention (an attention model), γ1 and γ2 are learnable parameters, and MLP is a multi-layer perceptron using the GeLU activation function;. Subsequently, the obtained output feature map is fused with the feature map obtained by the local window feature extraction branch, as shown in the following formula (4).

[0108] x C =Concat(x1+x' CTs ) (4)

[0109] After obtaining the fused feature map, self-attention is used to further promote the information exchange and extraction of the fused features. The overall operation is shown in equation (5).

[0110] x w =x C +γ3·MHSA(LN(x C))+γ4·MLP(LN(x C )) (5)

[0111] Among them, LN represents layer normalization, MHSA represents multi-head self-attention (an attention model), γ3 and γ4 are learnable parameters, and MLP is a multi-layer perceptron with GeLU activation function. The subsequent output is further split to represent the extracted image global information and local feature information, and the separated features are processed separately to obtain the final output feature expression.

[0112] In addition, note that an absolute position bias vector and a logarithmic space relative position bias vector are added to the hierarchical attention module respectively. In implementation, the absolute two-dimensional position of each marker is mapped to the feature dimension by using a multi-layer perceptron. In this way, the model can encode the two-dimensional position information of each marker through the absolute position bias vector and take these absolute position bias information into account during the attention calculation process. By introducing the absolute position bias vector, the model is able to better capture the position correlation of the markers in the spatial dimension when performing self-attention. The logarithmic space relative position bias vector uses a two-layer multi-layer perceptron MLP to calculate the relative position bias information of each marker. The logarithmic space calculation is used because it can better handle relative positions in different distance ranges and treat the relative position relationship between long and short distances more balanced.

[0113] Step S103: performing spatial pyramid pooling processing on the downsampled hierarchical attention feature map to obtain a pyramid pooling feature map.

[0114] In the embodiments of the present disclosure and other possible embodiments, the spatial pyramid pooling model that performs spatial pyramid pooling on the downsampled hierarchical attention feature map can be configured as an SPPF (Spatial Pyramid Pooling-Fast) module.

[0115] In the embodiments of the present disclosure and other possible embodiments, a SPPF (Spatial PyramidPooling-Fast) module is used to further fuse the downsampled hierarchical attention feature map. SPPF changes the parallel maximum pooling operation into a serial and parallel synchronous mode, which can make better use of the feature stream information on the one hand, and on the other hand, the synchronous serial and parallel mode can make up for the defect of target feature information loss caused by the single serial maximum pooling operation.

[0116] Step S104: fusing the target image convolution feature map, the downsampled hierarchical attention feature map and the pyramid pooling feature map to obtain fused feature maps of different scales.

[0117] In an embodiment of the present disclosure, the method of fusing the target image convolution feature map, the downsampled hierarchical attention feature map and the pyramid pooling feature map to obtain fused feature maps of different scales includes: upsampling the pyramid pooling feature map to obtain an upsampled pyramid pooling feature map; splicing the upsampled pyramid pooling feature map and the downsampled hierarchical attention feature map to obtain a first spliced ​​feature map; performing target feature information highlighting processing on the first spliced ​​feature map to obtain a first salient target feature map; upsampling the first salient target feature map to obtain a first upsampled salient target feature map; splicing the first upsampled salient target feature map and the target image convolution feature map to obtain a second spliced ​​feature map; performing target feature information highlighting processing on the second spliced ​​feature map to obtain a second salient target feature map; configuring the second salient target feature map as a first The method comprises the following steps: a) performing convolution, batch normalization, and activation function processing on the second salient target feature map for a first set number of times in sequence to obtain a first salient target convolution feature map; b) concatenating the first salient target feature map and the first salient target convolution feature map to obtain a third salient target feature map; b) performing target feature information highlighting processing on the third salient target feature map to obtain a third salient target feature map; c) configuring the third salient target feature map as a fused feature map of the second scale; c) performing convolution, batch normalization, and activation function processing on the third salient target feature map for a first set number of times in sequence to obtain a second salient target convolution feature map; c) concatenating the second salient target convolution feature map and the pyramid pooling feature map to obtain a fourth salient target feature map; c) performing target feature information highlighting processing on the fourth salient target feature map to obtain a fourth salient target feature map; and c) configuring the fourth salient target feature map as a fused feature map of the third scale.

[0118] In an embodiment of the present disclosure, the method of performing target feature information highlighting processing on the first spliced ​​feature map to obtain a first prominent target feature map includes: performing attention processing on the first spliced ​​feature map to obtain a first attention spliced ​​feature map; using multiple convolution blocks with residual connections to enhance and fuse the first attention spliced ​​feature map to obtain multiple first enhanced fused spliced ​​feature maps; and sequentially using a first connection layer, a first convolution layer, a first batch normalization layer, and a first activation function layer to process the multiple first enhanced fused spliced ​​feature maps to obtain a first prominent target feature map.

[0119] In an embodiment of the present disclosure, before the first connection layer, the first convolutional layer, the first batch normalization layer, and the first activation function layer are used to process the multiple first enhanced fused spliced ​​feature maps in sequence to obtain the first salient target feature map, it also includes: performing pooling processing for a third set number of times on each of the multiple first enhanced fused spliced ​​feature maps in sequence, and inputting the first enhanced fused spliced ​​feature map after each pooling processing into the first connection layer.

[0120] In an embodiment of the present disclosure, the method of performing target feature information highlighting processing on the second connection feature map to obtain a second prominent target feature map includes: performing attention processing on the second splicing feature map to obtain a second attention splicing feature map; using multiple convolution blocks with residual connections to enhance and fuse the second attention splicing feature map to obtain multiple second enhanced fused splicing feature maps; sequentially using a second connection layer, a second convolution layer, a second batch normalization layer, and a second activation function layer to process the multiple second enhanced fused splicing feature maps to obtain a second prominent target feature map.

[0121] In an embodiment of the present disclosure, before the second connection layer, the second convolution layer, the second batch normalization layer, and the second activation function layer are used in sequence to process the multiple second enhanced fused spliced ​​feature maps to obtain the second salient target feature map, it also includes: performing pooling processing for a third set number of times on each second enhanced fused spliced ​​feature map in the multiple second enhanced fused spliced ​​feature maps in sequence, and inputting the second enhanced fused spliced ​​feature map after each pooling processing into the second connection layer.

[0122] In an embodiment of the present disclosure, the method of performing target feature information highlighting processing on the third connection feature map to obtain a third salient target feature map includes: performing attention processing on the third splicing feature map to obtain a third attention splicing feature map; using multiple convolution blocks with residual connections to enhance and fuse the third attention splicing feature map to obtain multiple third enhanced fused splicing feature maps; sequentially using a third connection layer, a third convolution layer, a third batch normalization layer, and a third activation function layer to process the multiple third enhanced fused splicing feature maps to obtain a third salient target feature map.

[0123] In an embodiment of the present disclosure, before the third connection layer, the third convolutional layer, the third batch normalization layer, and the third activation function layer are used in sequence to process the multiple third enhanced fused spliced ​​feature maps to obtain the third salient target feature map, it also includes: performing pooling processing for a third set number of times on each of the multiple third enhanced fused spliced ​​feature maps in sequence, and inputting the third enhanced fused spliced ​​feature map after each pooling processing into the third connection layer.

[0124] In an embodiment of the present disclosure, the method of performing target feature information highlighting processing on the fourth connection feature map to obtain a fourth salient target feature map includes: performing attention processing on the fourth splicing feature map to obtain a fourth attention splicing feature map; using multiple convolution blocks with residual connections to enhance and fuse the fourth attention splicing feature map to obtain multiple fourth enhanced fused splicing feature maps; sequentially using a fourth connection layer, a fourth convolution layer, a fourth batch normalization layer, and a fourth activation function layer to process the multiple fourth enhanced fused splicing feature maps to obtain a fourth salient target feature map.

[0125] In an embodiment of the present disclosure, before the fourth connection layer, the fourth convolutional layer, the fourth batch normalization layer, and the fourth activation function layer are used in sequence to process the multiple fourth enhanced fused spliced ​​feature maps to obtain the fourth salient target feature map, it also includes: performing pooling processing for a third set number of times on each of the multiple fourth enhanced fused spliced ​​feature maps in sequence, and inputting the fourth enhanced fused spliced ​​feature map after each pooling processing into the fourth connection layer.

[0126] Figure 4 The network structure diagram corresponding to the target feature information highlighting processing module of the feature fusion structure block (feature fusion model) in the improved YOLOv8 model according to the embodiment of the present disclosure is shown. Figure 4 As shown, in the embodiments of the present disclosure and other possible embodiments, in the feature fusion module, the present invention proposes a new enhanced fusion module C2-S module. The input feature map first passes through a parameter-free attention mechanism SimAM (attention processing) to focus on the features of the target area of ​​interest, and is subsequently divided into three branches to fuse the context features of the image through bottleneck blocks, wherein the bottleneck block contains two convolution blocks and a residual connection layer, which is used to enhance the fused feature map, and then calculate the important feature map channels through the parameter-free attention mechanism SimAM, and then fuse with subsequent features to enhance the context information around the target of interest, suppress invalid background features, and more obviously highlight the target feature information.

[0127] like Figure 4As shown, in the embodiments of the present disclosure and other possible embodiments, the method of performing target feature information highlighting processing on the first spliced ​​feature map to obtain a first prominent target feature map includes: using the SimAM module to perform attention processing on the first spliced ​​feature map to obtain an attention-weighted first attention spliced ​​feature map (a number of 3 first attention spliced ​​feature maps); using a plurality of convolution blocks (3 convolution blocks) with residual connections to enhance and fuse the first attention spliced ​​feature maps (3 identical first attention spliced ​​feature maps) to obtain a plurality of first enhanced fused spliced ​​feature maps (3 first enhanced fused spliced ​​feature maps); using a first connection layer, a first convolution layer, a first batch normalization layer, and a first activation function layer in sequence to process the plurality of first enhanced fused spliced ​​feature maps (3 first enhanced fused spliced ​​feature maps) to obtain a first prominent target feature map.

[0128] like Figure 4 As shown, in the embodiments of the present disclosure and other possible embodiments, the method of performing target feature information highlighting processing on the second spliced ​​feature map to obtain a second prominent target feature map includes: using the SimAM module to perform attention processing on the second spliced ​​feature map to obtain an attention-weighted second attention spliced ​​feature map (a number of 3 second attention spliced ​​feature maps); using a plurality of residual connected convolution blocks (3 convolution blocks) to enhance and fuse the second attention spliced ​​feature map (3 identical first attention spliced ​​feature maps) to obtain a plurality of second enhanced fused spliced ​​feature maps (3 second enhanced fused spliced ​​feature maps); using a second connection layer, a second convolution layer, a second batch normalization layer, and a first activation function layer in sequence to process the plurality of second enhanced fused spliced ​​feature maps (3 second enhanced fused spliced ​​feature maps) to obtain a second prominent target feature map.

[0129] like Figure 4 As shown, in the embodiments of the present disclosure and other possible embodiments, the method of performing target feature information highlighting processing on the third concatenated feature map to obtain a third salient target feature map includes: using the SimAM module to perform attention processing on the third concatenated feature map to obtain an attention-weighted third attention concatenated feature map (a number of 3 first attention concatenated feature maps); respectively using a plurality of residual-connected convolution blocks (3 convolution blocks) to enhance and fuse the third attention concatenated feature map (3 identical first attention concatenated feature maps) to obtain a plurality of third enhanced fused concatenated feature maps (3 third enhanced fused concatenated feature maps); sequentially using a third connection layer, a third convolution layer, a third batch normalization layer, and a third activation function layer to process the plurality of third enhanced fused concatenated feature maps (3 third enhanced fused concatenated feature maps) to obtain a third salient target feature map.

[0130] like Figure 4 As shown, in the embodiments of the present disclosure and other possible embodiments, the method of performing target feature information highlighting processing on the fourth concatenated feature map to obtain a fourth salient target feature map includes: using the SimAM module to perform attention processing on the fourth concatenated feature map to obtain an attention-weighted fourth attention concatenated feature map (a number of 3 first attention concatenated feature maps); respectively using a plurality of residual-connected convolution blocks (3 convolution blocks) to enhance and fuse the fourth attention concatenated feature map (3 identical first attention concatenated feature maps) to obtain a plurality of fourth enhanced fused concatenated feature maps (3 fourth enhanced fused concatenated feature maps); sequentially using a fourth connection layer, a fourth convolution layer, a fourth batch normalization layer, and a fourth activation function layer to process the plurality of fourth enhanced fused concatenated feature maps (3 fourth enhanced fused concatenated feature maps) to obtain a fourth salient target feature map.

[0131] Step S105: Based on the fused feature maps of different scales and their corresponding detection layers, determine the category and / or position information corresponding to the set target object in the target image to be detected.

[0132] In the embodiments of the present disclosure and other possible embodiments, in the prediction module, the present invention is consistent with the YOLOv8 model, and uses the convolution layer to respectively predict the category and target position information of the model, and then uses the binary cross entropy to calculate the target category loss, and uses CIoU Loss (Intersection over Union Loss) and DFL Loss (Distribution Focal Loss) to calculate the target border loss.

[0133] In the embodiments of the present disclosure and other possible embodiments, the present invention uses the PyTorch framework to train the model in the model training stage, and uses a server equipped with an Nvidia GeForce RTX 3090 graphics card for training. The server's central processor is Inter i9-12900HK and the memory is 32G.

[0134] In the embodiments of the present disclosure and other possible embodiments, firstly, in the model training phase, a proposed improved YOLOv8 model of domestic waste target detection algorithm is used to train on a self-built domestic waste target detection data set to obtain the optimal model parameters; secondly, in the model testing phase, the obtained model parameters are used to detect new images to achieve high-precision and real-time detection of waste targets in real monitoring scenarios. The model training phase and the model testing phase proposed in the present invention are described in detail below.

[0135] In the embodiments of the present disclosure and other possible embodiments, in order to adapt to garbage target detection in real scenes, the present invention first collects image data, including six categories: batteries, bricks and ceramics, metal cans, cigarettes, food waste and plastic bottles, and specifically collects them in the following ways: (1) regularly recording videos in scenes such as canteens, residences, shopping malls, etc., and then obtaining images by extracting frames; (2) using search engines or image databases to search for the above six categories of garbage images, covering as many images of different scenes and different perspectives as possible; (3) applying data enhancement methods such as rotation, flipping, cropping, changing the brightness and contrast of the image to the collected images to expand the data set and increase the diversity of samples. After obtaining enough images, the present invention uses the LabelImg tool to annotate the collected images to ensure that each image is marked with the category and location coordinate information of the garbage target. Through these methods, a sufficient number and diversity of garbage target image samples can be accumulated to provide a data basis for model training.

[0136] In the model testing phase, based on the output results obtained by the proposed improved YOLOv8 model household garbage target detection algorithm, this step will first decode these output results and convert them into target categories and coordinate information predicted by the original image size. After that, the present invention first filters out invalid and low-quality detection target frames, and then uses a non-maximum suppression post-processing algorithm to filter overlapping predicted target frames, and finally obtains the final target prediction result after the input image is passed through an improved YOLOv8 model household garbage target detection algorithm.

[0137] In the target detection task, target bounding box regression loss is necessary and important. It can make the network predict the output as close as possible to the true label in the training phase, and then optimize the parameters to complete accurate target position prediction in the inference phase. Previous algorithms often use IoU (Intersection over Union) Loss to measure target position loss, but IoU Loss has some problems. For example, when the two predicted boxes do not overlap with the true target box, their IoU with the true target is zero. At this time, the gradient of IoU Loss is zero and cannot be optimized; at the same time, IoU Loss cannot distinguish between different alignments between two objects. In order to solve these problems, CIoU (Complete Intersection over Union) Loss was proposed. This method considers the three major geometric factors of the target box, namely the center point distance, overlapping area, and aspect ratio, and calculates the loss based on these geometric factors. In CIoU Loss, the loss of the center point distance is normalized, and the losses of the overlapping area and aspect ratio are constrained by consistency, which can make the algorithm converge quickly. Its overall calculation method is shown in the following formula (6).

[0138]

[0139] Where IoU is the intersection over union ratio between the predicted target box and the true target box, ρ represents the Euclidean distance between the two rectangular boxes, c is the diagonal distance between the closure areas of the two target boxes, b and b gt Represent the center points of the two rectangular boxes, w gt ,h gt , w and h represent the width and height of the real target box and the width and height of the predicted box respectively.

[0140] Although CIoU Loss can more comprehensively measure the position deviation between the predicted box and the true target box than IoU Loss, its calculation is more complicated and the aspect ratio loss describes a relative value, which is somewhat ambiguous. This paper uses WIoU (Wise-IoU) Loss as the target box loss function and uses a dynamic non-monotonic focusing mechanism to mine common samples and difficult samples to improve the overall performance of the model.

[0141] In addition, it is noted that the dataset contains some low-quality target annotation boxes. The measurement methods used by CIoU Loss, such as the target center point distance and the target width-height aspect ratio, will accelerate the penalty of the loss on low-quality annotated targets and affect the performance of the model. WIoU Loss will construct a target border loss based on distance attention to reduce precision loss according to the target distance measurement. The overall expression of WIoU Loss is shown in the following formula (8).

[0142] L WIoU =R WIoU (1-IoU) (8)

[0143]

[0144] where x gt ,y gt are the horizontal and vertical coordinates of the center point of the real target frame, x and y are the horizontal and vertical coordinates of the center point of the predicted frame, respectively. g and H g are the width and height of the smallest rectangle enclosing the predicted box and the true box, respectively. IoU is the intersection over union ratio between the predicted box and the true target box. * indicates separating this item from the computational graph. When the difference between the predicted box and the target true box is large, the RIoU item will amplify the loss of low-quality anchor boxes.

[0145] This paper uses WIoU Loss to replace CIoU Loss used in YOLOv8 on the target bounding box loss. On the one hand, it reduces the amount of calculation. On the other hand, WIoU Loss mines difficult samples, so that the model can be steadily reduced during the training process. It focuses on the optimization of difficult samples, so that the detection accuracy of the model can be further improved.

[0146] Similar to YOLOv8, the improved YOLOv8s proposed in this paper is consistent with the YOLOv8s model in terms of the number of feature map channels and the number of modules. Some important hyperparameter settings used in the training phase are as follows: the batch and total number of training rounds of the model training dataset are 64 and 100 respectively; the initial learning rate, momentum and weight decay are set to 0.01, 0.937 and 0.0005 respectively, and the weighting coefficients of target loss, target category loss and target border loss are 1.5, 0.5 and 7.5 respectively. In addition, in terms of dataset enhancement, the probabilities of image hue, saturation and value transformation are set to 0.015, 0.7 and 0.4 respectively, and the probability of Mosaic data enhancement is 1.0. Other parameters are consistent with the YOLOv8s model. Table 1 summarizes some hyperparameter information set in the training phase of this algorithm.

[0147] Table 1 Some important hyperparameter information used in the training phase of this algorithm

[0148]

[0149] The datasets used in this paper include the MS COCO2017 version of the target detection benchmark dataset and the self-built household garbage image dataset. The MS COCO2017 dataset is a large-scale benchmark dataset for target detection released by Microsoft in 2017. The image annotation information includes target boxes, target object masks, image information descriptions, and key points of human postures. In the target detection task, MS COCO2017 contains 118,287 training images, the validation set contains 5,000 images, and the test set also contains 5,000 images. The self-built seven-category household garbage target detection dataset includes seven types of targets: battery, brick and tile ceramics, no_rubbish, cans, cigarettes, food_waste, and plastic. The training set and validation set of the dataset contain 7,319 and 1,796 image samples, respectively.

[0150] The current experimental evaluation indicators used for target detection tasks mainly include accuracy (Precision), recall (Recall), average precision (Average Precision, AP), and mean average precision (mean Average Precision, mAP). The calculation methods of Precision and Recall are shown in the following equations (10) and (11) respectively.

[0151]

[0152] Among them, TP, FN, FP and TN respectively indicate that the model predicts that the target is positive and the target category is also positive, the model predicts that the target is positive and the target category is negative, the model predicts that the target is negative and the target category is positive, and the model predicts that the target is negative and the target category is also negative.

[0153] The calculation methods of AP and mAP are shown in equations (12) and (13) respectively.

[0154]

[0155]

[0156] Where n is the category, N is the total category, and p(r) represents the PR curve with the recall rate as the horizontal axis and the precision rate as the vertical axis. In terms of specific calculation indicators, mAP is further divided into mAP50 and mAP50-95, which respectively represent the detection results under the Intersection over Union (IoU) threshold of 0.5 and the detection results under the IoU threshold from 0.5 to 0.95 (every 0.05 interval).

[0157] Figure 5The indicator result graph of the improved YOLOv8 model (method) according to the embodiment of the present disclosure on the MS COCO2017 dataset is shown. On the MS COCO2017 dataset, the indicator results of the YOLOv8s model reproduced in this paper are as follows: During the training process of the model, the target category loss, border loss and DFL dropped to 0.893, 0.964 and 1.09 respectively. On the validation set, the target category loss, border loss and DFL of the YOLOv8s model can be reduced to 1.014, 1.008 and 1.105 respectively. In terms of specific accuracy result indicators, the YOLOv8s model achieved 67.6%, 54.7%, 59.9% and 43.9% in the four indicators of Precision, Recall, mAP50 and mAP50-95, respectively. Compared with other single-stage methods with the same number of parameters, such as YOLOv5s, YOLOX-s, YOLOv6-s, and YOLOv7-tiny, the YOLOv8s model has improved these four indicators. Compared with some other two-stage methods such as Faster RCNN, FCN, and Grid RCNN, the YOLOv8s model has obvious advantages in training time, stability of the training process, and reasoning time. The training process and verification indicators of the improved YOLOv8s proposed in this paper on MS COCO. The four indicators of Precision, Recall, mAP50, and mAP50-95 reached 71.2%, 57.2%, 62.8%, and 46.0%, respectively. Compared with the baseline YOLOv8s model, they have increased by 3.6%, 2.5%, 2.9%, and 2.1%, respectively. In order to more clearly compare the method in this paper with the YOLOv8s model.

[0158] Figure 6 The following is a comparison chart of the indicators of the improved YOLOv8 model (method) and the YOLOv8 model (method) before improvement on the MS COCO2017 dataset according to the disclosed embodiment. The comparison chart of the mAP50 and mAP50-95 indicators of the two methods on the MS COCO2017 validation set. Benefiting from the proposed method, the feature extraction network has richer target detail information extraction for the image. The method in this paper can bring a certain detection accuracy in the early stage of training. With the continuous learning of the attention module in the C2-S module, the target features and background information are further distinguished. Therefore, the method in this paper maintains a 2-3 point improvement in the mAP50 and mAP50-95 indicators. At the same time, thanks to the introduction of WIoU Loss, the target border loss of the method in this paper decreases faster and more steadily in both the training stage and the verification stage.

[0159] Figure 7The indicator result diagram of the improved YOLOv8 model (method) according to the embodiment of the present disclosure on the self-built data set is shown. The experimental results of the improved model proposed in this article on the domestic waste data set are shown in Figure 7. It can be seen from the figure that the target bounding box, target category and DFL loss of the improved YOLOv8s on the training set can converge to 0.550, 0.377 and 1.01. The four indicators of Precision, Recall, mAP50 and mAP50-95 can reach 92.0%, 88.0%, 92.2% and 80.1% respectively.

[0160] Figure 8 The following figure shows the comparison of the indicators of the improved YOLOv8 model (method) on the self-built data set according to the embodiment of the present disclosure; the following figures (1) and (2) respectively draw the comparison of the mAP50 and mAP50-95 indicators of the YOLOv8s model and the improved YOLOv8s on the domestic waste validation set. It can be seen from the figure that, on the one hand, the improved OLOv8s can have a higher detection accuracy in the early, middle and late stages of training. In the final mAP50 and mAP50-95 indicators, the improved YOLOv8s is 2.0% and 2.6% higher than YOLOv8s respectively; on the other hand, the method in this paper is more stable in training, which is due to the effective combination of CNN's fast local representation learning and ViT's global feature modeling to achieve efficient extraction of image features.

[0161] Fig. 9 A comparison chart of the loss indicators of the improved YOLOv8 model (method) according to the embodiment of the present disclosure and the YOLOv8 model (method) before improvement on the MS COCO2017 dataset is shown; (1), (2), (3), (4), (5) and (6) respectively draw comparison charts of the target border loss, classification loss and DFL of the improved YOLOv8s and the YOLOv8 model in the training stage and the target border loss, classification loss and DFL in the verification stage. It can be seen from the figure that, whether in the training stage or in the verification stage, the improved YOLOv8s can converge to a smaller value in target border loss, classification loss and DFL compared with the YOLOv8s model, and the converged value is also more stable. This is because the method in this paper improves the model's ability to extract features from images, and can highlight and characterize target objects more accurately. On the other hand, WIoU Loss prompts the model to learn more difficult samples in the training stage, providing direction for subsequent model parameter optimization. In terms of specific values, the improved YOLOv8s reduces the target bounding box loss, classification loss and DFL by 0.34, 0.39 and 0.20 respectively in the training phase compared to the YOLOv8s model, and reduces the target bounding box loss, classification loss and DFL by 0.25, 0.36 and 0.22 respectively in the verification phase.

[0162] Fig.10 The image detection results of the improved YOLOv8 model (method) according to the embodiment of the present disclosure on the self-built domestic waste verification set are shown. It can be seen from the figure that the improved YOLOv8s model can accurately identify and detect domestic waste, and has a high detection confidence. In particular, for small targets such as cigarette butts, the method proposed in this article can also accurately identify and detect them.

[0163] Table 2 is a comparison table of the indicators (Precision, Recall, mAP50 and mAP50-95) of different models (YOLOv5s, YOLOv6s, YOLOv7-tiny, YOLOv8s and improved YOLOv8s) on the validation set of the self-built domestic waste data set. As can be seen from the table, the improved YOLOv8s of the proposed method has reached the highest level in all four indicators, leading the YOLOv5s model by 4.9% in Precision, 1.9% in Recall, 3.3% in mAP50, and 2.6% in mAP50-95. Note that the YOLO series methods all use convolutional blocks to extract features in the image feature extraction network, and the proposed method shows obvious advantages.

[0164] Table 2 Comparison of indicators of different models on the self-built domestic waste validation set

[0165]

[0166] In addition, an embodiment of the present disclosure also proposes a method for processing a target object, comprising: applying the detection method described in any one of claims 1 to 5 to obtain the category and / or position information corresponding to the set target object in the target image to be detected; using a preset target object segmentation model to segment the set target object in the detection target image to obtain a corresponding set target object mask image; based on the set target object mask image, detecting and correcting the category and / or position information corresponding to the set target object in the target image to be detected to obtain the corresponding corrected category and / or corrected position information.

[0167] In an embodiment of the present disclosure, the method for detecting and correcting the category corresponding to the set target in the target image to be detected based on the set target mask image to obtain the corresponding corrected category includes: the preset target segmentation model is configured as a multi-type target segmentation model; wherein, using the multi-type target segmentation model, multiple types of set targets in the detection target image are segmented to obtain corresponding multiple types of set target mask images; based on different mask values ​​corresponding to the set target mask image, the category corresponding to the segmented set target is determined; based on the category corresponding to the segmented set target, the category corresponding to the set target is reconfigured to complete the correction of the category corresponding to the set target; or, the method for detecting and correcting the category corresponding to the set target in the target image to be detected based on the set target mask image to obtain the corresponding corrected category includes: the preset target segmentation model is configured as a single target segmentation model; wherein, using the single target segmentation model, the set target in the detection target image is segmented to obtain the corresponding set target mask image; wherein, the set target may include multiple types of set targets, and the multiple types of set targets are in The mask values ​​in the set target mask image are the same; the corresponding set target image is determined by using the set target mask image and the corresponding target image to be detected; image convolution features and image texture features are extracted from the set target image to obtain corresponding multiple image convolution features and image texture features; based on each image convolution feature and image texture feature of the multiple image convolution features, the set target is classified using a preset classifier; based on the classification result and the category corresponding to the set target, the category correction corresponding to the set target is completed; wherein, the A method for setting a target object image for image convolution feature extraction includes: performing feature extraction on the set target object image to obtain a target object convolution feature map; performing downsampling processing and layered attention processing on the target object convolution feature map in sequence to obtain a downsampled layered attention target object feature map; performing spatial pyramid pooling processing on the downsampled layered attention target object feature map to obtain a pyramid pooling target object feature map; performing fusion processing on the target object convolution feature map, the downsampled layered attention target object feature map and the pyramid pooling target object feature map to obtain a plurality of image convolution features of different scales.

[0168] Among them, in the embodiments of the present disclosure and other possible embodiments, the target object convolution feature map, the downsampled layered attention target object feature map and the pyramid pooling target object feature map are fused to obtain a variety of image convolution features of different scales. The specific method can refer to the processing method of fusing the target image convolution feature map, the downsampled layered attention feature map and the pyramid pooling feature map to obtain fused feature maps of different scales.

[0169] In the embodiments of the present disclosure and other possible embodiments, the method of determining the corresponding set target object image by using the set target object mask image and the corresponding target image to be detected includes: configuring the mask value corresponding to the set target object mask image to 1; performing a dot product operation on the configured set target object mask image and the corresponding target image to be detected to determine the corresponding set target object image.

[0170] In the embodiments of the present disclosure and other possible embodiments, the method for correcting the category corresponding to the set target object based on the classification results and the category corresponding to the set target object includes: performing regression processing on the probability value corresponding to the result to obtain the corrected category; using the corrected category to reconfigure the erroneous category corresponding to the set target object to correct the category corresponding to the set target object; or, performing frequency statistics on the classification results according to the same category, configuring the category with the highest frequency as the corrected category; using the corrected category to reconfigure the erroneous category corresponding to the set target object to correct the category corresponding to the set target object.

[0171] In the embodiments of the present disclosure and other possible embodiments, the method of classifying the set target object using a preset classifier based on each image convolution feature and image texture feature of the multiple image convolution features, includes: splicing each image convolution feature and the image texture feature, respectively, to obtain multiple groups of classification splicing features; and classifying the set target object using a preset classifier and each group of classification splicing features of the multiple groups of classification splicing features, respectively.

[0172] In an embodiment of the present disclosure, the method of detecting and correcting the position information corresponding to the set target object in the target image to be detected based on the set target object mask image to obtain the corresponding corrected position information includes: determining a first number corresponding to the detected set target object based on the position information corresponding to the set target object; determining a second number corresponding to the segmented set target object based on different mask values ​​corresponding to the set target object mask image; if the first number is less than the second number, determining the target image to be corrected corresponding to the missing target object; performing target detection on the target image to be corrected to obtain the corresponding missing position information; and detecting and correcting the position information corresponding to the set target object in the target image to be detected using the missing position information to obtain the corresponding corrected position information.

[0173] In an embodiment of the present disclosure, after detecting and correcting the position information corresponding to the set target object in the target image to be detected by using the missing position information to obtain the corresponding corrected position information, it also includes: extracting a plurality of image convolution features and / or image texture features corresponding to the target image to be corrected corresponding to the missing target object; and classifying the set target object by using a preset classifier based on each image convolution feature and / or image texture feature of the plurality of image convolution features corresponding to the target image to be corrected.

[0174] In the embodiments of the present disclosure and other possible embodiments, the method for extracting multiple image convolution features corresponding to the target image to be corrected corresponding to the missing target object includes: performing feature extraction on the target image to be corrected corresponding to the missing target object to obtain a convolution feature map of the target to be corrected; performing downsampling processing and layered attention processing on the convolution feature map of the target to be corrected in sequence to obtain a layered attention feature map of the target to be corrected; performing spatial pyramid pooling processing on the layered attention feature map of the target to be corrected to obtain a pyramid pooling feature map of the target to be corrected; performing fusion processing on the convolution feature map of the target to be corrected, the layered attention feature map of the target to be corrected and the pyramid pooling feature map of the target to be corrected to obtain multiple image convolution features of different scales corresponding to the target image to be corrected.

[0175] Among them, in the embodiments of the present disclosure and other possible embodiments, the convolution feature map of the target object to be corrected, the layered attention feature map of the target object to be corrected and the pyramid pooling feature map of the target object to be corrected are fused to obtain a plurality of image convolution features of different scales corresponding to the target image to be corrected. The specific method can refer to the processing method of fusing the convolution feature map of the target image, the downsampled layered attention feature map and the pyramid pooling feature map to obtain fused feature maps of different scales.

[0176] The subject of executing the method for detecting and processing the target object may be a detection and processing device for the target object, for example, the method for detecting and processing the target object may be executed by a terminal device or a server or other processing device, wherein the terminal device may be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementations, the method for detecting and processing the target object may be implemented by a processor calling a computer-readable instruction stored in a memory.

[0177] Those skilled in the art will understand that, in the above-mentioned target object detection and processing methods of the specific implementation methods, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0178] The disclosed embodiment also proposes a target object detection device, including: a first feature extraction unit, used to obtain a target image to be detected, and perform feature extraction on the target image to be detected to obtain a target image convolution feature map; a first feature processing unit, used to perform downsampling processing and layered attention processing on the target image convolution feature map in sequence to obtain a downsampled layered attention feature map; a second feature processing unit, used to perform spatial pyramid pooling processing on the downsampled layered attention feature map to obtain a pyramid pooling feature map; a feature fusion unit, used to perform fusion processing on the target image convolution feature map, the downsampled layered attention feature map and the pyramid pooling feature map to obtain fused feature maps of different scales; a detection unit, used to determine the category and / or position information corresponding to the set target object in the target image to be detected based on the fused feature maps of different scales and their corresponding detection layers.

[0179] Among them, in the embodiments of the present disclosure and other possible embodiments, the target image convolution feature map is sequentially downsampled and layered attentive to obtain a downsampled layered attentive feature map, including: downsampling the target image convolution feature map in sequence to obtain a first downsampled target image convolution feature map; performing a carrier marking operation on the first downsampled target image convolution feature map to obtain a first carrier convolution feature map; performing layered attentive processing on the first downsampled target image convolution feature map and the first carrier convolution feature map to obtain a first downsampled layered attentive feature map; downsampling the first downsampled layered attentive feature map to obtain a second downsampled target image convolution feature map; performing a carrier marking operation on the second downsampled target image convolution feature map to obtain a second carrier convolution feature map; performing layered attentive processing on the second downsampled target image convolution feature map and the second carrier convolution feature map to obtain a final downsampled layered attentive feature map.

[0180] Among them, in the embodiments of the present disclosure and other possible embodiments, the method of performing a carrier marking operation on the first down-sampled target image convolution feature map to obtain a first carrier convolution feature map includes: dividing the first down-sampled target image convolution feature map according to a set window size to obtain multiple first local window feature maps of the same size; sequentially using convolution layers and pooling layers corresponding to convolution kernels of the same size as the set window size to initialize the carrier mark to obtain a corresponding first carrier convolution feature map.

[0181] Among them, in the embodiments of the present disclosure and other possible embodiments, the method of performing hierarchical attention processing on the first down-sampled target image convolution feature map and the first carrier convolution feature map to obtain a first down-sampled hierarchical attention feature map includes: normalizing the first carrier convolution feature map (the first carrier label after initialization) to obtain a first normalized carrier label; using a first multi-layer perceptron and a first attention model to process the first normalized carrier label to obtain a corresponding first global information feature map and a second global information feature map; splicing the first local window feature map, the first global information feature map and the second global information feature map corresponding to the first down-sampled target image convolution feature map to obtain a first local global feature map. feature map; normalize the first local global feature map to obtain a first normalized local global feature map; use a second multilayer perceptron and a second attention model to process the first normalized local global feature map to obtain a corresponding first information feature map and a second information feature map; splice the first local global feature map, the first information feature map and the second information feature map to obtain a first feature map to be processed; segment the features corresponding to the carrier mark and the non-carrier mark of the first feature map to be processed to obtain a first carrier mark feature and a first non-carrier mark feature; upsample and dimensionally transform the first carrier mark feature and the first non-carrier mark feature, respectively, and then splice and merge them to obtain a first downsampled hierarchical attention feature map. ,

[0182] Among them, in the embodiments of the present disclosure and other possible embodiments, the carrier marking operation is performed on the second down-sampled target image convolution feature map to obtain the second carrier convolution feature map, including: dividing the second down-sampled target image convolution feature map according to a set window size to obtain multiple second local window feature maps of the same size; sequentially using convolution layers and pooling layers corresponding to convolution kernels of the same size as the set window size to initialize the carrier mark to obtain the corresponding second carrier convolution feature map.

[0183] Among them, in the embodiments of the present disclosure and other possible embodiments, the method of performing hierarchical attention processing on the second down-sampled target image convolution feature map and the second carrier convolution feature map to obtain the final down-sampled hierarchical attention feature map includes: normalizing the second carrier convolution feature map (the second carrier label after initialization) to obtain a second normalized carrier label; using a third multi-layer perceptron and a third attention model to process the second normalized carrier label to obtain a corresponding third global information feature map and a fourth global information feature map; splicing the second local window feature map corresponding to the second down-sampled target image convolution feature map, the third global information feature map and the fourth global information feature map to obtain a second local global feature map. feature map; normalize the second local global feature map to obtain a second normalized local global feature map; use a fourth multi-layer perceptron and a fourth attention model to process the second normalized local global feature map to obtain a corresponding third information feature map and a fourth information feature map; splice the second local global feature map, the third information feature map and the fourth information feature map to obtain a second feature map to be processed; segment the features corresponding to the carrier mark and the non-carrier mark of the second feature map to be processed to obtain a second carrier mark feature and a second non-carrier mark feature; upsample the second carrier mark feature and the second non-carrier mark feature, respectively, and then splice and merge them to obtain a final downsampled hierarchical attention feature map.

[0184] The disclosed embodiment also proposes a target object detection device, comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to call the instructions stored in the memory to execute the above-mentioned detection method.

[0185] The embodiment of the present disclosure further proposes a target object detection device, including: a computer-readable storage medium on which computer program instructions are stored, and the computer program instructions implement the above-mentioned detection method when executed by a processor.

[0186] The disclosed embodiment also proposes an object processing device, comprising: an object detection device as described above; and a preset object segmentation unit, used to segment the set object in the detection target image using a preset object segmentation model to obtain a corresponding set object mask image; and a correction unit, used to detect and correct the category and / or position information corresponding to the set object in the target image to be detected based on the set object mask image to obtain a corresponding correction category and / or correction position information.

[0187] The embodiment of the present disclosure also proposes a target object processing device, comprising: a target object detection device as described above; and a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to call the instructions stored in the memory to execute the above-mentioned processing method.

[0188] The disclosed embodiment also proposes a target object processing device, comprising: the target object detection device as described above; and a computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions implement the above processing method when executed by a processor.

[0189] The disclosed embodiment also proposes a computer program product, including a computer program / instruction, characterized in that the computer program / instruction implements the above-mentioned detection method and / or the above-mentioned processing method when executed by a processor.

[0190] In some embodiments, the functions or modules included in the device provided in the embodiments of the present disclosure can be used to execute the method described in the above target object detection and processing method embodiments. The specific implementation can refer to the description of the above target object detection and processing method embodiments, and for the sake of brevity, it will not be repeated here.

[0191] The embodiment of the present disclosure also provides a computer-readable storage medium on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the above-mentioned target object detection and processing method is implemented. The computer-readable storage medium may be a non-volatile computer-readable storage medium.

[0192] The embodiment of the present disclosure also provides an electronic device, comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to detect and process the above-mentioned target object. The electronic device can be provided as a terminal, a server or other forms of equipment.

[0193] Fig.11 8 is a block diagram of an electronic device 800 according to an exemplary embodiment. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, or the like.

[0194] Reference Fig.11 , the electronic device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0195] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0196] The memory 804 is configured to store various types of data to support operations on the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0197] The power supply component 806 provides power to the various components of the electronic device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 800.

[0198] The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP).

[0199] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0200] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0201] The sensor component 814 includes one or more sensors for providing various aspects of status evaluation for the electronic device 800. For example, the sensor component 814 can detect the open / closed state of the electronic device 800 and the relative positioning of the components.

[0202] The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof.

[0203] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0204] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 804 including computer program instructions, which can be executed by a processor 820 of an electronic device 800 to perform the above method.

[0205] Fig.12 1 is a block diagram of an electronic device 1900 according to an exemplary embodiment. For example, the electronic device 1900 may be provided as a server. Fig.12 , the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions executable by the processing component 1922, such as an application. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

[0206] The electronic device 1900 may also include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output (I / O) interface 1958. The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™ or the like.

[0207] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the electronic device 1900 to perform the above method.

[0208] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for detecting a target object, characterized in that: include: Acquire a target image to be detected, and perform feature extraction on the target image to be detected to obtain a convolution feature map of the target image; Downsampling and hierarchical attention processing are sequentially performed on the convolution feature map of the target image to obtain a downsampled hierarchical attention feature map; Performing spatial pyramid pooling processing on the downsampled hierarchical attention feature map to obtain a pyramid pooling feature map; The target image convolution feature map, the downsampled hierarchical attention feature map and the pyramid pooling feature map are fused to obtain fused feature maps of different scales; Based on the fused feature maps of different scales and their corresponding detection layers, the category and / or position information corresponding to the set target object in the target image to be detected is determined.

2. The detection method according to claim 1, characterized in that: The method of extracting features from the target image to be detected to obtain a convolution feature map of the target image comprises: sequentially performing convolution, batch normalization, activation function processing, downsampling, and convolution processing for a first set number of times on the target image to be detected to obtain a convolution feature map of the target image; and / or, The method of sequentially performing downsampling processing and layered attention processing on the target image convolution feature map to obtain a downsampled layered attention feature map includes: sequentially performing downsampling processing on the target image convolution feature map to obtain a first downsampled target image convolution feature map; performing a carrier marking operation on the first downsampled target image convolution feature map to obtain a first carrier convolution feature map; performing layered attention processing on the first downsampled target image convolution feature map and the first carrier convolution feature map to obtain a first downsampled layered attention feature map; performing downsampling processing on the first downsampled layered attention feature map to obtain a second downsampled target image convolution feature map; performing a carrier marking operation on the second downsampled target image convolution feature map to obtain a second carrier convolution feature map; performing layered attention processing on the second downsampled target image convolution feature map and the second carrier convolution feature map to obtain a final downsampled layered attention feature map.

3. The detection method according to any one of claims 1 to 2, characterized in that: The method of fusing the target image convolution feature map, the downsampled hierarchical attention feature map and the pyramid pooling feature map to obtain fused feature maps of different scales includes: Performing upsampling processing on the pyramid pooling feature map to obtain an upsampled pyramid pooling feature map; splicing the upsampled pyramid pooling feature map and the downsampled layered attention feature map to obtain a first spliced ​​feature map; performing target feature information highlighting processing on the first spliced ​​feature map to obtain a first salient target feature map; performing upsampling processing on the first salient target feature map to obtain a first upsampled salient target feature map; splicing the first upsampled salient target feature map and the target image convolution feature map to obtain a second spliced ​​feature map; performing target feature information highlighting processing on the second spliced ​​feature map to obtain a second salient target feature map; configuring the second salient target feature map as a fusion feature map of the first scale; The second salient target feature map is sequentially subjected to convolution, batch normalization, and activation function processing for a first set number of times to obtain a first salient target convolution feature map; the first salient target feature map and the first salient target convolution feature map are concatenated to obtain a third concatenated feature map; the third concatenated feature map is subjected to target feature information salient processing to obtain a third salient target feature map; the third salient target feature map is configured as a fusion feature map of the second scale; The third salient target feature map is sequentially subjected to convolution, batch normalization, and activation function processing for a first set number of times to obtain a second salient target convolution feature map; the second salient target convolution feature map and the pyramid pooling feature map are spliced ​​to obtain a fourth concatenated feature map; the fourth concatenated feature map is subjected to target feature information salient processing to obtain a fourth salient target feature map; and the fourth salient target feature map is configured as a fusion feature map of a third scale.

4. The detection method according to claim 3, characterized in that: include: The method for performing target feature information highlighting processing on the first spliced ​​feature map to obtain a first salient target feature map includes: performing attention processing on the first spliced ​​feature map to obtain a first attention spliced ​​feature map; using multiple convolution blocks with residual connections to enhance and fuse the first attention spliced ​​feature map to obtain multiple first enhanced fused spliced ​​feature maps; sequentially using a first connection layer, a first convolution layer, a first batch normalization layer, and a first activation function layer to process the multiple first enhanced fused spliced ​​feature maps to obtain a first salient target feature map; and / or, before sequentially using the first connection layer, the first convolution layer, the first batch normalization layer, and the first activation function layer to process the multiple first enhanced fused spliced ​​feature maps to obtain the first salient target feature map, it also includes: sequentially performing pooling processing for a third set number of times on each first enhanced fused spliced ​​feature map in the multiple first enhanced fused spliced ​​feature maps, and inputting the first enhanced fused spliced ​​feature map after each pooling processing into the first connection layer; and / or, The method for performing target feature information highlighting processing on the second connection feature map to obtain a second salient target feature map includes: performing attention processing on the second splicing feature map to obtain a second attention splicing feature map; using multiple convolution blocks with residual connections to enhance and fuse the second attention splicing feature map to obtain multiple second enhanced fused splicing feature maps; sequentially using a second connection layer, a second convolution layer, a second batch normalization layer, and a second activation function layer to process the multiple second enhanced fused splicing feature maps to obtain a second salient target feature map; and / or, before sequentially using a second connection layer, a second convolution layer, a second batch normalization layer, and a second activation function layer to process the multiple second enhanced fused splicing feature maps to obtain a second salient target feature map, it also includes: sequentially performing pooling processing for a third set number of times on each second enhanced fused splicing feature map in the multiple second enhanced fused splicing feature maps, and inputting the second enhanced fused splicing feature map after each pooling processing into the second connection layer; and / or, The method for performing target feature information highlighting processing on the third joint feature map to obtain a third prominent target feature map includes: performing attention processing on the third splicing feature map to obtain a third attention splicing feature map; using multiple convolution blocks with residual connections to enhance and fuse the third attention splicing feature map to obtain multiple third enhanced fused splicing feature maps; sequentially using a third connection layer, a third convolution layer, a third batch normalization layer, and a third activation function layer to process the multiple third enhanced fused splicing feature maps to obtain a third prominent target feature map; and / or, before sequentially using the third connection layer, the third convolution layer, the third batch normalization layer, and the third activation function layer to process the multiple third enhanced fused splicing feature maps to obtain the third prominent target feature map, it also includes: sequentially performing pooling processing for a third set number of times on each third enhanced fused splicing feature map in the multiple third enhanced fused splicing feature maps, and inputting the third enhanced fused splicing feature map after each pooling processing into the third connection layer; and / or, The method of performing target feature information highlighting processing on the fourth connection feature map to obtain a fourth salient target feature map includes: performing attention processing on the fourth splicing feature map to obtain a fourth attention splicing feature map; using multiple convolution blocks with residual connections to enhance and fuse the fourth attention splicing feature map to obtain multiple fourth enhanced fused splicing feature maps; sequentially using a fourth connection layer, a fourth convolution layer, a fourth batch normalization layer, and a fourth activation function layer to process the multiple fourth enhanced fused splicing feature maps to obtain a fourth salient target feature map; and / or, before sequentially using a fourth connection layer, a fourth convolution layer, a fourth batch normalization layer, and a fourth activation function layer to process the multiple fourth enhanced fused splicing feature maps to obtain a fourth salient target feature map, it also includes: sequentially performing pooling processing for a third set number of times on each of the multiple fourth enhanced fused splicing feature maps, and inputting the fourth enhanced fused splicing feature map after each pooling processing into the fourth connection layer.

5. The detection method according to any one of claims 1 to 4, characterized in that: include: Before acquiring the target image to be detected, determining the type corresponding to the source image of the target image to be detected; If the type corresponding to the source image of the target image to be detected is a video image, the multi-moment video frame images corresponding to the video image are configured as the target image to be detected; if the type corresponding to the source image of the target image to be detected is a non-video image at a certain moment, the non-moment video image at the certain moment is configured as the target image to be detected; and / or, the target object set in the target image to be detected is configured as multiple types of domestic waste.

6. A method for processing a target object, comprising: Applying the detection method according to any one of claims 1 to 5 to obtain the category and / or position information corresponding to the set target object in the target image to be detected, characterized in that it also includes: Using a preset target object segmentation model, segmenting the set target object in the detection target image to obtain a corresponding set target object mask image; Based on the set target object mask image, detection and correction are performed on the category and / or position information corresponding to the set target object in the target image to be detected to obtain corresponding correction category and / or correction position information.

7. The processing method according to claim 6, characterized in that: The method for detecting and correcting the category corresponding to the set target in the target image to be detected based on the set target mask image to obtain the corresponding corrected category includes: configuring the preset target segmentation model as a multi-type target segmentation model; wherein, using the multi-type target segmentation model, segmenting multiple types of set targets in the detection target image to obtain corresponding multi-type set target mask images; determining the category corresponding to the segmented set target based on different mask values ​​corresponding to the set target mask image; reconfiguring the category corresponding to the segmented set target based on the category corresponding to the segmented set target to complete the category correction corresponding to the set target; and / or, The method for detecting and correcting the position information corresponding to the set target in the target image to be detected based on the set target mask image to obtain the corresponding corrected position information includes: determining a first number corresponding to the detected set target based on the position information corresponding to the set target; determining a second number corresponding to the segmented set target based on different mask values ​​corresponding to the set target mask image; if the first number is less than the second number, determining the target image to be corrected corresponding to the missing target; performing target detection on the target image to be corrected to obtain the corresponding missing position information; detecting and correcting the position information corresponding to the set target in the target image to be detected using the missing position information to obtain the corresponding corrected position information; and / or, After using the missing position information to detect and correct the position information corresponding to the set target object in the target image to be detected to obtain the corresponding corrected position information, it also includes: extracting multiple image convolution features and / or image texture features corresponding to the target image to be corrected corresponding to the missing target object; and classifying the set target object using a preset classifier based on each image convolution feature and / or image texture feature of the multiple image convolution features corresponding to the target image to be corrected.

8. A target object detection device, characterized in that: include: A first feature extraction unit is used to obtain a target image to be detected, and perform feature extraction on the target image to be detected to obtain a convolution feature map of the target image; A first feature processing unit is used to sequentially perform downsampling processing and layered attention processing on the target image convolution feature map to obtain a downsampled layered attention feature map; wherein, the sequentially performing downsampling processing and layered attention processing on the target image convolution feature map to obtain a downsampled layered attention feature map includes: sequentially performing downsampling processing on the target image convolution feature map to obtain a first downsampled target image convolution feature map; performing a carrier marking operation on the first downsampled target image convolution feature map to obtain a first carrier convolution feature map; performing layered attention processing on the first downsampled target image convolution feature map and the first carrier convolution feature map to obtain a first downsampled layered attention feature map; performing downsampling processing on the first downsampled layered attention feature map to obtain a second downsampled target image convolution feature map; performing a carrier marking operation on the second downsampled target image convolution feature map to obtain a second carrier convolution feature map; performing layered attention processing on the second downsampled target image convolution feature map and the second carrier convolution feature map to obtain a final downsampled layered attention feature map; and / or, Among them, the performing of a carrier marking operation on the first down-sampled target image convolution feature map to obtain a first carrier convolution feature map includes: dividing the first down-sampled target image convolution feature map according to a set window size to obtain a plurality of first local window feature maps of the same size; sequentially using convolution layers and pooling layers corresponding to convolution kernels of the same size as the set window size to obtain a corresponding first carrier convolution feature map; and / or, Among them, the performing of a carrier marking operation on the second down-sampled target image convolution feature map to obtain a second carrier convolution feature map includes: dividing the second down-sampled target image convolution feature map according to a set window size to obtain a plurality of second local window feature maps of the same size; sequentially using convolution layers and pooling layers corresponding to convolution kernels of the same size as the set window size to obtain a corresponding second carrier convolution feature map; and / or, A second feature processing unit is used to perform spatial pyramid pooling processing on the downsampled hierarchical attention feature map to obtain a pyramid pooling feature map; A feature fusion unit, used to fuse the target image convolution feature map, the downsampled hierarchical attention feature map and the pyramid pooling feature map to obtain fused feature maps of different scales; A detection unit, used to determine the category and / or position information corresponding to the set target object in the target image to be detected based on the fused feature maps of different scales and their corresponding detection layers; or, comprising: a processor; a memory for storing processor executable instructions; wherein the processor is configured to call the instructions stored in the memory to execute the detection method according to any one of claims 1 to 5; or, The invention comprises: a computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions implement the detection method according to any one of claims 1 to 5 when executed by a processor.

9. A target object processing device, comprising: The target object detection device according to claim 8, further comprising: A preset target object segmentation unit, used to segment the set target object in the detection target image using a preset target object segmentation model to obtain a corresponding set target object mask image; a correction unit, configured to detect and correct the category and / or position information corresponding to the set target object in the target image to be detected based on the set target object mask image, so as to obtain corresponding correction category and / or correction position information; or comprising: a processor; a memory for storing processor executable instructions; wherein the processor is configured to call the instructions stored in the memory to execute the processing method according to any one of claims 6 to 7; or, The invention comprises: a computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions, when executed by a processor, implement the processing method described in any one of claims 6 to 7.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the detection method described in any one of claims 1 to 5 and / or the processing method described in any one of claims 6 to 7 are implemented.