Target detection method, electronic equipment and storage medium

Abstract

The invention discloses a target detection method, electronic equipment and a storage medium, and the method comprises the steps: carrying out the transformation of a target image, and obtaining a plurality of to-be-extracted images; performing feature extraction on the plurality of to-be-extracted images to obtain a plurality of transformation feature maps, and obtaining image prompt information of the target image, the image prompt information being used for representing image content in the target image; fusing each transformation feature map and the image prompt information to obtain a plurality of fused feature maps; and performing target detection based on the plurality of fused feature maps to obtain a target detection result. According to the scheme, the accuracy of the target detection result can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a target detection method, electronic device, and storage medium. Background Art

[0002] Traditional object detection often relies on a single feature map, which limits detection effectiveness. Furthermore, the results of a single feature map can contain many false detections and missed detections, making it difficult to meet the accuracy requirements for object detection in complex scenarios. Summary of the Invention

[0003] This application at least provides a target detection method, electronic device, and storage medium, which can improve the accuracy of target detection results.

[0004] The first aspect of the present application provides a target detection method, which includes: transforming a target image to obtain several images to be extracted; performing feature extraction on the several images to be extracted respectively to obtain several transformed feature maps, and obtaining image prompt information of the target image, wherein the image prompt information is used to characterize the image content in the target image; fusing each transformed feature map and the image prompt information to obtain several fused feature maps; performing target detection based on the several fused feature maps to obtain a target detection result.

[0005] Among them, the resolutions of several images to be extracted are different, and feature extraction is performed on several images to be extracted respectively to obtain several transformation feature maps, including: feature extraction is performed on several images to be extracted respectively to obtain multi-resolution feature maps, and the multi-resolution feature maps include different resolution feature maps corresponding to each image to be extracted; feature enhancement is performed on the multi-resolution feature map to obtain several transformation feature maps.

[0006] The multi-resolution feature map includes four feature maps with different resolutions, and the plurality of transformation feature maps include three transformation feature maps with different resolutions; and / or, the multi-resolution feature map is feature enhanced to obtain a plurality of transformation feature maps, including: taking each non-minimum resolution feature map in the multi-resolution feature map as a target feature map; taking each target feature map as a current target feature map in order of resolution from small to large, and performing the following processing: upsampling the associated resolution feature map to obtain a first upsampled feature map, wherein, when the current target feature map is not the target feature map with the minimum resolution, the associated resolution feature map is an enhanced feature map corresponding to the previous target feature map; fusing the first upsampled feature map with the current target feature map to obtain a first fusion feature map. feature map; performing feature extraction on the first fused feature map to obtain a first enhanced feature map of the current target feature map, wherein, in the case where the current target feature map is the target feature map with the smallest resolution, the minimum resolution feature map is the first enhanced feature map of the minimum resolution feature map; in response to the current target feature map being the target feature map with the largest resolution, using the first enhanced feature corresponding to the current target feature map as the transformed feature corresponding to the current target feature map; in response to the current target feature map not being the target feature map with the largest resolution, fusing the first enhanced feature map corresponding to the current target feature map with the transformed feature map corresponding to the next target feature map to obtain a second fused feature map; and performing feature extraction on the second fused feature map to obtain a transformed feature map of the current target feature map.

[0007] Among them, the image prompt information is a prompt feature map obtained by extracting features of the target image; and / or the image prompt information of the target image includes several image prompt information whose sizes correspond to several transformation feature maps respectively; each transformation feature map and the image prompt information are fused to obtain several fused feature maps, including: each transformation feature map is fused with the image prompt information corresponding to the size to obtain several fused feature maps.

[0008] Among them, obtaining the image prompt information of the target image includes: respectively using several feature extraction modules connected in sequence to extract features of the target image, and using the prompt feature map output by each feature extraction module as the image prompt information of the target image, wherein the input of the first feature extraction module is the target image, and the input of the non-first feature extraction module is the prompt feature map output by the previous feature extraction module.

[0009] Among them, target detection is performed based on several fused feature maps to obtain target detection results, including: combining several fused feature maps to obtain a combined feature map; performing feature extraction on the combined feature map to obtain a multi-scale feature map; performing feature enhancement on the multi-scale feature map to obtain several second enhanced feature maps; performing target detection on each second enhanced feature map to obtain a target detection sub-result corresponding to each second enhanced feature map as the target detection result.

[0010] The multi-scale feature map includes three feature maps of different scales; and / or, feature enhancement is performed on the multi-scale feature maps respectively to obtain several second enhanced feature maps, including: taking each scale feature map as the current scale feature map in order from small to large scale, and performing the following processing: in response to the current scale feature map being the minimum scale feature map, performing convolution processing and feature extraction on the current scale feature map in sequence to obtain an intermediate feature map of the current scale feature map; in response to the current scale feature map not being the minimum scale feature map, upsampling the associated scale feature map to obtain a second upsampled feature map, wherein the associated scale feature map is the intermediate feature map corresponding to the previous scale feature map. feature map; fusing the second up-sampled feature map with the current scale feature map to obtain a third fused feature map; performing feature extraction on the third fused feature map to obtain an intermediate feature map of the current scale feature map; in response to the current scale feature map being the maximum scale feature map, using the intermediate feature map of the current scale feature map as the second enhanced feature map of the current scale feature map; in response to the current scale feature map not being the maximum scale feature map, fusing the intermediate feature map corresponding to the current scale feature map with the second enhanced feature map corresponding to the next scale feature map to obtain a fourth fused feature map; and performing feature extraction on the fourth fused feature map to obtain a second enhanced feature map of the current scale feature map.

[0011] Among them, the target detection sub-result corresponding to each second enhanced feature map includes the candidate category and category confidence of the target, as well as the candidate position information of the target; after performing target detection on each second enhanced feature map respectively and obtaining the target detection sub-result corresponding to each second enhanced feature map as the target detection result, it also includes: in each target detection sub-result, selecting the candidate category whose category confidence meets the confidence requirements as the category to which the target in the target image belongs; in each target detection sub-result, selecting the candidate position information that meets the position requirements as the position information of the target in the target image.

[0012] The step of transforming the target image to obtain a plurality of images to be extracted includes: performing a Laplace pyramid transform on the target image to obtain a plurality of images to be extracted with different resolutions.

[0013] The target detection model is used to extract features from a plurality of images to be extracted, obtain a plurality of transformed feature maps, detect targets based on a plurality of fused feature maps, and determine the target detection results. The target detection model includes a prompt feature generation module and a target detection module. The prompt feature generation module is used to extract the transformed feature maps and image prompt information, and fuse the transformed feature maps and image prompt information to obtain a plurality of fused feature maps. The target detection module is used to perform target detection based on a plurality of fused feature maps to obtain target detection results. And / or, the training method of the target detection model includes: transforming the sample image to obtain a plurality of sample images to be extracted, wherein the sample map There is at least one sample target in the image, and the sample image is annotated with the actual category information of the sample target and the actual position information corresponding to the sample target; a target detection model is used to perform feature extraction on several sample images to obtain several sample transformation feature maps, and sample image information corresponding to the sample image is obtained; each sample transformation feature map and the sample image information are fused to obtain several sample fusion feature maps; target detection is performed based on the several sample fusion feature maps to obtain predicted category information and predicted position information corresponding to the sample target; based on the difference between the predicted category information and the actual category information, and the difference between the predicted position information and the actual position information, the network parameters of the target detection model are optimized.

[0014] A second aspect of the present application provides an electronic device, comprising a memory and a processor coupled to each other, wherein the processor is configured to execute program instructions stored in the memory to implement the target detection method in the first aspect.

[0015] A third aspect of the present application provides a computer-readable storage medium having program instructions stored thereon, which implement the target detection method in the first aspect when the program instructions are executed by a processor.

[0016] In the above scheme, a transformation operation is first performed on the target image to generate multiple images to be extracted. Next, feature extraction is performed on each of these images to be extracted to obtain a number of transformed feature maps. Simultaneously, image cue information of the target image is obtained. Subsequently, each transformed feature map is fused with the image cue information to obtain a number of fused feature maps. Finally, target detection is performed using these fused feature maps to obtain target detection results. Since each fused feature map incorporates the image cue information of the target image, better detection results can be achieved during the target detection process. Furthermore, comprehensive analysis of the detection results of multiple fused feature maps can further improve the accuracy of target detection results.

[0017] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings herein are incorporated into and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and, together with the specification, are used to illustrate the technical solutions of the present application.

[0019] Figure 1 This is a flow chart of an embodiment of the target detection method of the present application;

[0020] Figure 2 This is a schematic diagram of the framework of an embodiment of the target detection model of the present application;

[0021] Figure 3 This is a schematic diagram of the network structure of an embodiment of the prompt feature generation module of the present application;

[0022] Figure 4 This is a schematic diagram of the network structure of an embodiment of the target detection module of the present application;

[0023] Figure 5 This is a schematic diagram of the network structure of an embodiment of the MobileOneBlock module of the present application;

[0024] Figure 6 This is a flow chart of an embodiment of the target detection model training method of the present application;

[0025] Figure 7 This is a flow chart of another embodiment of the target detection method of the present application;

[0026] Figure 8 This is a schematic diagram of the framework of an embodiment of the target detection device of the present application;

[0027] Figure 9 This is a schematic diagram of the framework of an embodiment of the electronic device of the present application;

[0028] Figure 10 It is a schematic diagram of a framework of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0029] The following describes the embodiments of the present application in detail with reference to the accompanying drawings.

[0030] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0031] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the previous and next associated objects are in an "or" relationship. In addition, "many" in this article means two or more than two. In addition, the term "at least one" in this article means any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C, can mean including any one or more elements selected from the set consisting of A, B, and C.

[0032] See also Figure 1 , Figure 1 It is a flowchart of an embodiment of the target detection method of the present application.

[0033] Specifically, the following steps may be included:

[0034] Step S110: transforming the target image to obtain a number of images to be extracted.

[0035] This application is mainly used in the fields of artificial intelligence and computer vision, and is committed to achieving accurate recognition and detection of various targets in images. Current technologies often encounter many difficulties and challenges when facing blurred target detection tasks in complex scenes, especially when implemented in resource-constrained end-side environments. Unlike existing technologies, this solution innovatively introduces image transformation and feature fusion mechanisms. Through image transformation, images can be processed from different angles and scales, effectively enhancing the recognizability of blurred targets; and feature fusion can integrate multi-layer and multi-dimensional feature information to fully explore useful details in the image. This technical solution that combines image transformation and feature fusion can capture and understand target information in images more comprehensively and accurately, thereby significantly improving the accuracy of blurred target detection and providing a more reliable solution for target detection in complex scenes.

[0036] In some embodiments, to highlight edges and details in an image and enhance image contrast, a Laplacian pyramid transform can be performed on the target image to obtain several images to be extracted at different resolutions. Specifically, the target image is first decomposed using a Gaussian pyramid to obtain a set of images of different resolutions. This process is achieved by continuously applying Gaussian filters and downsampling operations, with each downsampling operation halving the image size. Next, based on the Gaussian pyramid, a Laplacian pyramid is constructed by performing a difference operation on images at adjacent levels. Specifically, the Laplacian image of each level is equal to the Gaussian image of that level minus an upsampled version of the Gaussian image of the next level. After processing the Laplacian images of each level, image reconstruction is achieved through the inverse operation. That is, the Laplacian images of each level are first upsampled and Gaussian filtered, and then they are added together to obtain the final reconstructed image, i.e., the image to be extracted.

[0037] In other embodiments, it is also common to transform the target image through geometric transformation, thereby obtaining images to be extracted from different perspectives. This process generally involves operations such as translation, rotation, scaling, affine transformation, and perspective transformation. These transformations can not only adjust the geometric shape and position of the image, but also simulate the image effects under different perspectives. For example, in image registration, multiple images are aligned to the same coordinate system through geometric transformation for fusion or comparison; in 3D reconstruction, image transformations from different perspectives are used to obtain three-dimensional information of the object. These transformation operations can better understand and process image data, improving the accuracy and efficiency of image processing and analysis.

[0038] Step S120: performing feature extraction on a plurality of images to be extracted respectively to obtain a plurality of transformed feature maps, and obtaining image prompt information of the target image, wherein the image prompt information is used to represent the image content in the target image.

[0039] In the fields of computer vision and image processing, feature extraction is a key step in extracting useful information from images. For a number of images to be extracted, various feature extraction methods can be used to process them separately. First, a convolutional neural network (CNN) is used to extract features from the image. CNNs automatically learn features from images through convolutional layers, effectively capturing the image's local characteristics and hierarchical structure. Second, traditional feature extraction methods such as the scale-invariant feature transform (SIFT) and histogram of oriented gradients (HOG) are used. Image transformation is an essential step in the feature extraction process. For example, grayscale and geometric transformations can enhance certain image characteristics, making them more suitable for feature extraction. After these transformations, several transformed feature maps are obtained. These feature maps not only retain the key information of the original image but also highlight features of different levels and types, providing a solid foundation for subsequent tasks such as object detection and image classification.

[0040] In some embodiments, to obtain multiple transformed feature maps, feature extraction can be performed on multiple images to be extracted, resulting in multi-resolution feature maps. The multi-resolution feature maps include feature maps of different resolutions corresponding to each image to be extracted. Feature enhancement is then performed on the multi-resolution feature maps to obtain multiple transformed feature maps.

[0041] Please refer to Figure 2 and Figure 3 The target detection model 200 includes a prompt feature generation module 210 and a target detection module 220. The target detection model 200 is configured to execute steps S120 to S140. The prompt feature generation module 210 is configured to extract a transformation feature map and image prompt information, and fuse the transformation feature maps and image prompt information to obtain a plurality of fused feature maps. The target detection module 220 is configured to perform target detection based on the plurality of fused feature maps to obtain a target detection result.

[0042] Please continue reading Figure 3 After the target image is transformed, four resolution images are obtained: a first resolution image, a second resolution image, a third resolution image, and a fourth resolution image. The first resolution image has a length of H and a width of W, the second resolution image has a length of H / 2 and a width of W / 2, the third resolution image has a length of H / 4 and a width of W / 4, and the fourth resolution image has a length of H / 8 and a width of W / 8. The prompt feature generation module 210 includes a feature extraction network 211, a first feature enhancement network 212, a fusion network 213, and a prompt module 214. After the four resolution images are input into the feature extraction network 211, the feature extraction modules in the feature extraction network 211 perform feature extraction to obtain multi-resolution feature maps. Specifically, the multi-resolution feature map includes four feature maps with different resolutions. After the first resolution image is input into the feature extraction network 211, feature extraction is performed through the MobileOneBlock module 2111 in the feature extraction network 211 to obtain a first resolution transformation feature map M1; after the second resolution image is input into the feature extraction network 211, feature extraction is performed through the MobileOneBlock module 2112 in the feature extraction network 211 to obtain a second resolution transformation feature map M2; after the third resolution image is input into the feature extraction network 211, feature extraction is performed through the MobileOneBlock module 2113 in the feature extraction network 211 to obtain a third resolution transformation feature map M3; after the fourth resolution image is input into the feature extraction network 211, feature extraction is performed through the MobileOneBlock module 2114 in the feature extraction network 211 to obtain a fourth resolution transformation feature map M4.

[0043] After obtaining the multi-resolution feature map, feature enhancement is performed on the multi-resolution feature map to obtain several transformed feature maps.

[0044] Specifically, each non-minimum resolution feature map in the multi-resolution feature map can be used as a target feature map, and each target feature map can be used as the current target feature map in order of resolution from small to large, and steps S121 to S126 can be executed.

[0045] Step S121: upsample the associated resolution feature map to obtain a first upsampled feature map, wherein, when the current target feature map is not the target feature map with the minimum resolution, the associated resolution feature map is the enhanced feature map corresponding to the previous target feature map.

[0046] Step S122: Fusing the first upsampled feature map with the current target feature map to obtain a first fused feature map.

[0047] Step S123: performing feature extraction on the first fused feature map to obtain a first enhanced feature map of the current target feature map, wherein, when the current target feature map is the target feature map with the minimum resolution, the minimum resolution feature map is the first enhanced feature map of the minimum resolution feature map.

[0048] Step S124: In response to the current target feature map being the target feature map with the highest resolution, the first enhanced feature corresponding to the current target feature map is used as the transformed feature corresponding to the current target feature map.

[0049] Step S125: In response to the current target feature map not being the target feature map with the highest resolution, the first enhanced feature map corresponding to the current target feature map is fused with the transformed feature map corresponding to the next target feature map to obtain a second fused feature map.

[0050] Step S126: performing feature extraction on the second fused feature map to obtain a transformed feature map of the current target feature map.

[0051] Please continue reading Figure 3 The first resolution transformation feature map M1, the second resolution transformation feature map M2, the third resolution transformation feature map M3, and the fourth resolution transformation feature map M4 obtained by the feature extraction network 211 can be input into the first feature enhancement network 212 for feature enhancement processing.

[0052] Specifically, the resolutions of the first resolution transformation feature map M1, the second resolution transformation feature map M2, the third resolution transformation feature map M3, and the fourth resolution transformation feature map M4 are H*W, H / 2*W / 2, H / 4*W / 4, and H / 8*W / 8 respectively. Therefore, in ascending order of their resolutions, it is M4 < M3 < M2 < M1. Among them, several transformation feature maps include three different resolution transformation feature maps.

[0053] In the first feature enhancement network 212, first, the fourth resolution transformation feature map M4 is processed. At this time, the fourth resolution transformation feature map M4 is the current target feature map, and the enhanced feature map of the fourth resolution transformation feature map M4 is itself. Then, the first feature enhancement network 212 starts to process the third resolution transformation feature map M3. At this time, the third resolution transformation feature map M3 is the current target feature map, and the associated resolution feature map of the third resolution transformation feature map M3 is the fourth resolution transformation feature map M4 itself. The fourth resolution transformation feature map M4 is input into the UpSample module 21211 for upsampling to obtain the first upsampled feature map M4' (not shown in the figure). Then, the Concat module 21221 is used to fuse the first upsampled feature map M4' with the current target feature map M3 to obtain the first fused feature map. Then, the MobileOneBlock module 21231 is used to extract features from the first fused feature map to obtain the first enhanced feature map M3' (not shown in the figure) of the current target feature map M3.

[0054] The first feature enhancement network 212 starts to process the second resolution transformation feature map M2. At this time, the second resolution transformation feature map M2 is the current target feature map, and the associated resolution feature map of the second resolution transformation feature map M2 is the enhanced feature map M3'. The enhanced feature map M3' is input into the UpSample module 21212 for upsampling to obtain the first upsampled feature map M3'' (not shown in the figure). Then, the Concat module 21222 is used to fuse the first upsampled feature map M3'' with the current target feature map M2 to obtain the first fused feature map. The MobileOneBlock module 21232 is used to extract features from the first fused feature map to obtain the first enhanced feature map M2' (not shown in the figure) of the current target feature map M2.

[0055] The first feature enhancement network 212 begins to process the first resolution transformation feature map M1. At this time, the first resolution transformation feature map M1 is the current target feature map, and the associated resolution feature map of M1 is the enhanced feature map M2′. The enhanced feature map M2′ is input into the UpSample module 21213 for upsampling to obtain the first upsampled feature map M2″ (not shown in the figure), and then the first upsampled feature map M2″ is fused with the current target feature map M1 using the Concat module 21223 to obtain the first fused feature map. The MobileOneBlock module 21233 is used to extract features from the first fused feature map to obtain the first enhanced feature map M1′ (not shown in the figure) of the current target feature map M1. Because the current target feature map M1 is the target feature map with the highest resolution at this time, the first enhanced feature M1′ corresponding to the current target feature map M1 is used as the transformation feature map corresponding to the current target feature map.

[0056] After enhancing the target feature map M1 with the highest resolution, reverse fusion is performed. M2 is used as the current target feature map in the reverse fusion process. The enhanced feature map M2′ corresponding to M2 is fused with the transformed feature map corresponding to M1 using the Concat module 21224 to obtain a second fused feature map. The MobileOneBlock module 21234 is then used to extract features from the second fused feature map corresponding to M2 to obtain a transformed feature map corresponding to M2.

[0057] When M3 is used as the current target feature map in the reverse fusion process, the enhanced feature map M3′ corresponding to M3 is fused with the transformed feature map corresponding to M2 using the Concat module 21225 to obtain a second fused feature map. The MobileOneBlock module 21235 then performs feature extraction on the second fused feature map corresponding to M3 to obtain a transformed feature map corresponding to M3.

[0058] In some embodiments, in order to obtain image prompt information of the target image, several feature extraction modules connected in sequence are used to extract features of the target image, and the prompt feature map output by each feature extraction module is used as the image prompt information of the target image, wherein the input of the first feature extraction module is the target image, and the input of the non-first feature extraction module is the prompt feature map output by the previous feature extraction module.

[0059] Please continue reading Figure 3 The prompt module 214 is composed of three layers of feature extraction modules, each layer of modules is progressively used to gradually extract the deep features of the image. Specifically:

[0060] The first-layer feature extraction module includes a MobileOne Block, which performs preliminary feature extraction on the input target image to obtain the first-layer image prompt information. This layer is responsible for capturing the basic features of the image, laying the foundation for subsequent processing.

[0061] The second-layer feature extraction module, consisting of two MobileOne blocks, further extracts features from the image hint information extracted in the first layer to generate the second-layer image hint information. This layer increases the depth of feature extraction, enhancing the ability to capture complex features in the image.

[0062] The third-layer feature extraction module, also consisting of two MobileOne blocks, extracts features from the second-layer image prompt information, ultimately generating the third-layer image prompt information. This layer further deepens the feature extraction process to obtain more abstract and advanced feature representations.

[0063] Each layer of feature extraction is implemented using the MobileOne Block. This architectural design aims to balance computational efficiency and feature extraction capabilities. Through this layer-by-layer approach, the prompt module 214 can effectively extract feature information at different levels from the target image, providing rich feature support for subsequent image analysis and processing tasks.

[0064] Step S130: Fusing each transformed feature map with the image prompt information to obtain a plurality of fused feature maps.

[0065] Please continue reading Figure 3 , the image prompt information extracted by the prompt module 214 is a prompt feature map generated by extracting features from the target image. These prompt feature maps can reflect the key feature information of the target image and provide basic support for subsequent image processing and analysis. Specifically, the image prompt information of the target image contains multiple levels of prompt feature maps, and the size of each prompt feature map corresponds to a number of transformation feature maps. This correspondence ensures that the prompt feature map can be effectively matched and fused with the transformation feature map at different levels, thereby realizing multi-scale feature extraction and analysis of the target image. In this way, the prompt module 214 can not only capture the local details of the target image, but also retain its global structural information, providing a more comprehensive and accurate feature representation for image processing tasks in complex scenes.

[0066] Furthermore, the fusion network 213 can be used to fuse each transformed feature map with the image prompt information corresponding to the size to generate a fused feature map. The fusion network 213 includes a CPB Block module 2131, a CPB Block module 2132, and a CPBBlock module 2133. Specifically:

[0067] First-layer feature fusion: The first-layer extracted image cue information output by the cue module 214 and the transformed feature map corresponding to M1 are input into the CPB Block module 2131 for fusion, generating a fused feature map F1. Here, the first-layer extracted image cue information and the transformed feature map corresponding to M1 are both of size H × W. This fusion aims to combine low-level image cue information and the transformed feature map to capture the essential features of the image.

[0068] Second-layer feature fusion: The second-layer extracted image cue information output by the cue module 214 and the transformed feature map corresponding to M2 are input into the CPB Block module 2132 for fusion, similarly generating a fused feature map F1. The second-layer extracted image cue information and the transformed feature map corresponding to M2 both have dimensions of H / 2 × W / 2. This level of fusion further integrates mid-level features, enhancing the ability to capture image details.

[0069] Third-layer feature fusion: The third-layer extracted image cue information output by the cue module 214 and the transformed feature map corresponding to M3 are input into the CPB Block module 2133 for fusion, generating a fused feature map F1. The third-layer extracted image cue information and the transformed feature map corresponding to M3 are both of size H / 4 × W / 4. This level of fusion focuses on integrating high-level features, providing a more abstract and global feature representation.

[0070] Through this multi-layer feature fusion mechanism, the system captures image feature information from different layers, providing more comprehensive and accurate feature support for image analysis and object detection in complex scenarios. Each layer's fusion process is implemented through the CPB Block module, ensuring efficient and accurate feature extraction and fusion. This design not only improves the richness of feature representation but also enhances the system's robustness and adaptability when processing complex image data.

[0071] Step S140: Perform target detection based on several fused feature maps to obtain target detection results.

[0072] In some embodiments, based on the fused feature map, an appropriate object detection algorithm can be selected for object detection. Common object detection algorithms include YOLO, Faster R-CNN, and SSD. These algorithms can predict the location and category of an object based on the information in the feature map.

[0073] In other embodiments, to further improve the accuracy, robustness, and adaptability of target detection and meet the needs of target detection in complex scenarios, feature enhancement processing can be performed on the fused feature map. For details, please refer to steps S141 to S144.

[0074] Step S141: Combine several fused feature maps to obtain a combined feature map.

[0075] In some embodiments, the fused feature maps F1, F2, and F3 output by the prompt feature generation module 210 can be spliced to obtain a combined feature map. By integrating the information in different feature maps, the combined feature map is made more comprehensive and rich, and contains more target feature information, thereby providing a more accurate basis for subsequent detection. Among them, the prompt feature generation module 210 can be used to construct an improved YOLOv9 model. YOLOv9 is a real-time target detection model with high efficiency and high precision. It solves the problem of information loss in deep neural networks and improves parameter utilization and computational efficiency by introducing innovative technologies such as Programmable Gradient Information (PGI) and Generalized Efficient Layer Aggregation Network (GELAN). Its core structure includes GELAN as the backbone network, which can flexibly adapt to different computing requirements while maintaining efficient performance. YOLOv9 is suitable for real-time target detection tasks, especially in resource-limited environments.

[0076] Step S142: performing feature extraction on the combined feature map to obtain a multi-scale feature map.

[0077] Please refer to Figure 4 The target detection module 220 includes a backbone network 221, a second feature enhancement network 222, and a detection module 223. The backbone network 221 is used to extract features from the combined feature map, and the multi-scale feature map includes three feature maps of different scales, wherein the backbone network 221 can be GELAN. Specifically, the backbone network 221 includes five stages, wherein Stage 1 includes one MobileOneBlock, Stage 2 includes two MobileOneBlocks, Stage 3 includes eight MobileOneBlocks, Stage 4 includes four MobileOneBlocks, and Stage 5 includes two MobileOneBlocks. The combined feature map is sequentially subjected to feature extraction through five stages, and the different scale feature maps output by Stage 3, Stage 4, and Stage 5 are selected as the multi-scale feature map. For example, the scale feature map output by Stage 3 is 80*80, the scale feature map output by Stage 4 is 40*40, and the scale feature map output by Stage 5 is 20*20. The extraction of multi-scale feature maps can capture the characteristics of targets at different scales, avoid missed detection or false detection due to differences in target size, and improve the detection capability of targets of different sizes.

[0078] Step S143: performing feature enhancement on the multi-scale feature maps respectively to obtain a plurality of second enhanced feature maps.

[0079] In some embodiments, after obtaining the multi-scale feature maps, the second feature enhancement network 222 is used to perform feature enhancement on the multi-scale feature maps to obtain several second enhanced feature maps. Feature enhancement on the multi-scale feature maps can further highlight target features, suppress interference information such as background noise, make the target more visible in the feature maps, and thus improve detection accuracy.

[0080] Specifically, each scale feature map is used as the current scale feature map in order of scale from small to large, and reference may be made to steps S1431 to S1437 .

[0081] Step S1431: In response to the current scale feature map being the minimum scale feature map, convolution processing and feature extraction are sequentially performed on the current scale feature map to obtain an intermediate feature map of the current scale feature map.

[0082] Step S1432: In response to the current scale feature map not being the minimum scale feature map, upsampling the associated scale feature map to obtain a second upsampled feature map, wherein the associated scale feature map is an intermediate feature map corresponding to the previous scale feature map.

[0083] Step S1433: Fusing the second up-sampled feature map with the current scale feature map to obtain a third fused feature map.

[0084] Step S1434: performing feature extraction on the third fused feature map to obtain an intermediate feature map of the current scale feature map.

[0085] Step S1435: In response to the current scale feature map being the maximum scale feature map, the middle feature map of the current scale feature map is used as the second enhanced feature map of the current scale feature map.

[0086] Step S1436: In response to the current scale feature map not being the maximum scale feature map, the intermediate feature map corresponding to the current scale feature map is fused with the second enhanced feature map corresponding to the next scale feature map to obtain a fourth fused feature map.

[0087] Step S1437: performing feature extraction on the fourth fused feature map to obtain a second enhanced feature map of the current scale feature map.

[0088] Specifically, the scale feature map output by Stage 5 in the backbone network 221 has the smallest scale, while the scale feature map output by Stage 3 has the largest scale. When the scale feature map output by Stage 5 is used as the current scale feature map, the scale feature map output by Stage 5 is convolved using the Conv module 2221, and then feature extracted using the MobileOne Block module 22221 to obtain an intermediate feature map corresponding to the scale feature map output by Stage 5.

[0089] Next, when the scale feature map output by Stage 4 is used as the current scale feature map, its associated scale feature map is the intermediate feature map corresponding to the scale feature map output by Stage 5. The UpSample module 22231 is used to upsample the intermediate feature map corresponding to the scale feature map output by Stage 5 to obtain a second upsampled feature map corresponding to the scale feature map output by Stage 5. Then, the Concat module 22241 is used to fuse the second upsampled feature map with the scale feature map output by Stage 4 to obtain a third fused feature map corresponding to the scale feature map output by Stage 4. The MobileOne Block module 22222 is then used to perform feature extraction on the third fused feature map to obtain the intermediate feature map corresponding to the scale feature map output by Stage 4.

[0090] Finally, when the scale feature map output by Stage 3 is used as the current scale feature map, its associated scale feature map is the intermediate feature map corresponding to the scale feature map output by Stage 4. The intermediate feature map corresponding to the scale feature map output by Stage 4 is upsampled using the UpSample module 22232 to obtain a second upsampled feature map corresponding to the scale feature map output by Stage 4. This second upsampled feature map is then fused with the scale feature map output by Stage 3 using the Concat module 22242 to obtain a third fused feature map corresponding to the scale feature map output by Stage 3. Feature extraction is then performed on this third fused feature map using the MobileOne Block module 22223 to obtain an intermediate feature map corresponding to the scale feature map output by Stage 3. Since the scale feature map output by Stage 3 is the maximum scale feature map, the intermediate feature map corresponding to the scale feature map output by Stage 3 is used as the second enhanced feature map P1 of the current scale feature map. The scale of the second enhanced feature map P1 is 80×80×c, where c is the depth and c can be any value.

[0091] After completing the above processing, reverse fusion is performed. When the scale feature map output by Stage 4 is used as the current scale feature map, the intermediate feature map corresponding to the scale feature map output by Stage 4 is fused with the intermediate feature map corresponding to the scale feature map output by Stage 3 using the Concat module 22243 to obtain a fourth fused feature map corresponding to the scale feature map output by Stage 4. The MobileOne Block module 22224 is then used to perform feature extraction on this fourth fused feature map to obtain a second enhanced feature map P2 corresponding to the scale feature map output by Stage 4. The scale of the second enhanced feature map P2 is 40×40×c.

[0092] When the scale feature map output by Stage 5 is used as the current scale feature map, the intermediate feature map corresponding to the scale feature map output by Stage 5 is fused with the second enhanced feature map corresponding to the scale feature map output by Stage 4 using the Concat module 22244 to obtain a fourth fused feature map corresponding to the scale feature map output by Stage 5. The RepGhostBottleneck module 2225 is then used to perform feature extraction on the fourth fused feature map to obtain a second enhanced feature map P3 corresponding to the scale feature map output by Stage 5. The scale of the second enhanced feature map P3 is 20×20×c.

[0093] Step S144: performing target detection on each second enhanced feature map respectively, and obtaining a target detection sub-result corresponding to each second enhanced feature map as a target detection result.

[0094] In some embodiments, the second enhanced feature maps P1, P2, and P3 can be input into the detection module 223 for target detection. Specifically, in the detection module 223, the second enhanced feature maps P1, P2, and P3 are convolved using 1*1 Conv, and the depth c is converted to 16. The convolved enhanced feature maps are then input into the classification branch and the regression branch to obtain target detection results.

[0095] Please continue reading Figure 4 After the second enhanced feature maps P1, P2, and P3 are input into the detection module 223, each second enhanced feature map will obtain a target detector in the target detection result, and the target detection sub-result corresponding to each second enhanced feature map includes the candidate category and category confidence of the target, as well as the candidate position information of the target.

[0096] In addition, within each object detection sub-result, candidate categories whose category confidences meet the confidence requirement are selected as the category to which the object in the target image belongs. The confidence requirement can be that the category confidence is greater than a confidence threshold. Within each object detection sub-result, candidate location information that meets the location requirement is selected as the location information of the object in the target image. The location requirement is that the intersection-over-union ratio of the target bounding box and the surrounding bounding boxes is maximized.

[0097] In this way, this scheme can improve the adaptability of the detection algorithm in complex environments through multi-scale feature extraction and feature enhancement, so that it can better cope with situations such as target occlusion, lighting changes, and complex backgrounds.

[0098] Please refer to Figure 3-Figure 5 In the prompt feature generation module 210 and the target detection module 220, multiple MobileOne Block modules are included. The MobileOne Block is designed by integrating the depthwise separable convolution and the structural reparameterization idea. Its specific structure is as follows Figure 5 Each MobileOne-Block consists of two stages: the first stage includes a 1×1 channel-by-channel convolution (DWConv) branch, a 3×3 DWConv branch controlled by a hyperparameter k, and a batch normalization (BN) layer branch. In the MobileOneLite network, the hyperparameter k = 4, and the activation function used is the ReLU function with low computational latency.

[0099] Furthermore, the second stage includes a 1×1 ordinary convolution branch and a batch normalization (BN) layer branch, in which the activation function used is also the ReLU function with lower computational latency.

[0100] See also Figure 6 , Figure 6 This is a flowchart of an embodiment of the target detection model training method of the present application. Specifically, it can include the following steps:

[0101] Step S610: transforming the sample image to obtain a number of sample images to be extracted, wherein there is at least one sample target in the sample image, and the sample image is annotated with actual category information of the sample target and actual position information corresponding to the sample target.

[0102] In some embodiments, the fuzzy scene dataset can be divided into a training set, a validation set, and a test set. The dataset used in this embodiment is the public target detection dataset ExDark. The target detection model 200 is trained using sample images in the training set. First, a Laplace pyramid transform is performed on the sample image to obtain several sample images to be extracted with different resolutions. Among them, the length of the first sample image to be extracted is H and the width is W, the length of the second sample image to be extracted is H / 2 and the width is W / 2, the length of the third sample image to be extracted is H / 4 and the width is W / 4, and the length of the fourth sample image to be extracted is H / 8 and the width is W / 8.

[0103] Step S620: performing feature extraction on a number of sample images to be extracted using the target detection model to obtain a number of sample transformation feature maps, and obtaining sample image information corresponding to the sample images.

[0104] In some embodiments, the feature extraction network 211 in the prompt feature generation module 210 performs feature extraction on the first sample image to be extracted, the second sample image to be extracted, the third sample image to be extracted, and the fourth sample image to be extracted, thereby obtaining four sample-resolution feature maps. Subsequently, the first feature enhancement network 212 performs feature enhancement on the multi-sample-resolution feature maps, thereby obtaining three sample-transformed feature maps. Simultaneously, the three sequentially connected layers of feature extraction modules in the prompt module 214 perform feature extraction on the sample image, and the prompt feature maps output by each layer of feature extraction module serve as sample image prompt information for the sample image.

[0105] Step S630: Fusing each sample transformation feature map with the sample image information to obtain a plurality of sample fusion feature maps.

[0106] In some embodiments, the fusion network 213 is used to fuse each sample transformation feature map and the sample image information to obtain three sample fusion feature maps.

[0107] Step S640: Target detection is performed based on several sample fusion feature maps to obtain predicted category information and predicted position information corresponding to the sample target.

[0108] In some embodiments, the three sample fusion feature maps are input into the backbone network 221. After five stages of feature extraction, the different sample scale feature maps output by Stage 3, Stage 4, and Stage 5 are selected as multi-sample scale feature maps. The second feature enhancement network 222 then performs feature enhancement on the different sample scale feature maps output by Stage 3, Stage 4, and Stage 5. This feature enhancement process is the same as step S143 above and will not be further described here. After obtaining the three second enhanced feature maps, they are then input into the detection module 223 for target detection, obtaining predicted category information and predicted position information corresponding to the sample target.

[0109] Step S650: Optimizing the network parameters of the target detection model based on the difference between the predicted category information and the actual category information, and the difference between the predicted position information and the actual position information.

[0110] In some embodiments, QFL LOSS can be used to calculate the difference between the predicted category information and the actual category information, and DFL LOSS can be used to calculate the difference between the predicted position information and the actual position information. Specifically, the loss function used in the training phase is:

[0111] Loss=Loss cls +Loss reg (1)

[0112] QFL(σ)=-|m-σ| β ((1-m)log(1-σ)+mlogσ)· 2)

[0113] DFL(S i , S i+1 )=-((y i+1 -y)log(S i )+(yy i-1 )log(S i+1 ) (3)

[0114] Among them, Loss cls Represents classification loss, Loss reg represents regression loss, m represents actual category information, y represents actual position information, and y i-1 and y i+1 Indicates the left and right position of the actual position information. S i+1 and S i They represent the probabilities of the left and right positions of the actual position information, σ represents the predicted category information, and β represents the experience value.

[0115] The loss function LOSS is used to optimize the parameters of each MobileOneBlock in the target detection model 200, so that the network structure of the training phase in MobileOneBlock is transformed into a network structure including a 1×1 ordinary convolution branch and a batch normalization layer branch after structural reparameterization, thereby reducing the network complexity of the target detection model 200 and making the model more lightweight.

[0116] After training is completed, the images in the test set are input into the object detection model 200 for final evaluation of the model's performance.

[0117] See also Figure 7 , Figure 7 It is a flowchart of another embodiment of the target detection method of the present application.

[0118] Specifically, the following steps may be included:

[0119] Step S710: pre-process the target image to obtain a pre-processed image.

[0120] In some embodiments, the target image is preprocessed, including normalization, denoising, contrast enhancement and other operations, to improve image quality and reduce noise interference.

[0121] Step S720: transform the pre-processed image to obtain a number of images to be extracted.

[0122] This step is the same as the above step S110 and will not be described again here.

[0123] Step S730: performing feature extraction on a plurality of images to be extracted, obtaining a plurality of transformed feature maps, and acquiring image prompt information of the pre-processed images.

[0124] This step is the same as the above step S120 and will not be described again here.

[0125] Step S740: Fusing each transformed feature map with the image prompt information to obtain a plurality of fused feature maps.

[0126] This step is the same as the above step S130 and will not be described again here.

[0127] Step S750: Optimize the plurality of fused feature maps respectively to obtain a plurality of optimized feature maps.

[0128] In some embodiments, the fused feature map is further optimized, for example, by using methods such as dimensionality reduction, normalization, and feature selection to remove redundant information and enhance the distinguishability of features.

[0129] Step S760: Perform target detection based on several optimized feature maps to obtain target detection results.

[0130] This step is the same as the above step S140 and will not be described again here.

[0131] This application generates image-related semantic information through a prompt module, so that the model can recognize and adapt to the image after Laplace transformation, thereby using the target image to guide the feature extraction of the transformed image, thereby improving the detection accuracy and robustness of blurred objects.

[0132] This application adopts a lightweight structure, and both the feature generation module and the target detection module use MobileOne as the basic structure to improve the running speed and facilitate deployment on edge devices.

[0133] Those skilled in the art will understand that in the above-mentioned method of the specific implementation method, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0134] See also Figure 8 , Figure 8 It is a schematic diagram of the framework of an embodiment of the target detection device of the present application. The target detection device 800 includes a transformation module 810, a feature extraction module 820, a fusion module 830 and a fusion detection module 840. The transformation module 810 performs a transformation on the target image to obtain a number of images to be extracted. The feature extraction 820 performs feature extraction on a number of images to be extracted respectively to obtain a number of transformation feature maps, and obtains image prompt information of the target image, wherein the image prompt information is used to characterize the image content in the target image. The fusion module 830 performs a fusion of each transformation feature map and the image prompt information to obtain a number of fused feature maps. The fusion detection module 840 performs target detection based on a number of fused feature maps to obtain a target detection result.

[0135] In some embodiments, the feature extraction 820 performs feature extraction on several images to be extracted with different resolutions, and performs feature extraction on several images to be extracted respectively to obtain several transformed feature maps, including: performing feature extraction on several images to be extracted respectively to obtain multi-resolution feature maps, and the multi-resolution feature maps include feature maps of different resolutions corresponding to each image to be extracted; and performing feature enhancement on the multi-resolution feature maps to obtain several transformed feature maps.

[0136] In some embodiments, the feature extraction 820 performs a multi-resolution feature map including four different resolution feature maps, and a plurality of transformation feature maps including three different resolution transformation feature maps; and / or, performs feature enhancement on the multi-resolution feature map to obtain a plurality of transformation feature maps, including: taking each non-minimum resolution feature map in the multi-resolution feature map as a target feature map; taking each target feature map as a current target feature map in order of resolution from small to large, and performing the following processing: upsampling the associated resolution feature map to obtain a first upsampled feature map, wherein, when the current target feature map is not the target feature map with the minimum resolution, the associated resolution feature map is an enhanced feature map corresponding to the previous target feature map; fusing the first upsampled feature map with the current target feature map , obtaining a first fused feature map; performing feature extraction on the first fused feature map to obtain a first enhanced feature map of the current target feature map, wherein, in the case where the current target feature map is the target feature map with the smallest resolution, the minimum resolution feature map is the first enhanced feature map of the minimum resolution feature map; in response to the current target feature map being the target feature map with the largest resolution, using the first enhanced feature corresponding to the current target feature map as the transformed feature corresponding to the current target feature map; in response to the current target feature map not being the target feature map with the largest resolution, fusing the first enhanced feature map corresponding to the current target feature map with the transformed feature map corresponding to the next target feature map to obtain a second fused feature map; and performing feature extraction on the second fused feature map to obtain a transformed feature map of the current target feature map.

[0137] In some embodiments, the fusion module 830 executes image prompt information to extract features of the target image to obtain a prompt feature map; and / or, the image prompt information of the target image includes several image prompt information whose sizes correspond to several transformation feature maps; each transformation feature map and the image prompt information are fused to obtain several fused feature maps, including: fusing each transformation feature map with the image prompt information corresponding to the size to obtain several fused feature maps.

[0138] In some embodiments, feature extraction 820 is performed to obtain image prompt information of the target image, including: respectively using several feature extraction modules connected in sequence to extract features of the target image, and using the prompt feature map output by each feature extraction module as the image prompt information of the target image, wherein the input of the first feature extraction module is the target image, and the input of the non-first feature extraction module is the prompt feature map output by the previous feature extraction module.

[0139] In some embodiments, the fusion detection module 840 performs target detection based on several fused feature maps to obtain target detection results, including: combining several fused feature maps to obtain a combined feature map; performing feature extraction on the combined feature map to obtain a multi-scale feature map; performing feature enhancement on the multi-scale feature map to obtain several second enhanced feature maps; performing target detection on each second enhanced feature map to obtain a target detection sub-result corresponding to each second enhanced feature map as the target detection result.

[0140] In some embodiments, the fusion detection module 840 performs a multi-scale feature map including three different scale feature maps; and / or, performs feature enhancement on the multi-scale feature map respectively to obtain a plurality of second enhanced feature maps, including: taking each scale feature map as the current scale feature map in order from small to large scale, and performing the following processing: in response to the current scale feature map being the minimum scale feature map, performing convolution processing and feature extraction on the current scale feature map in sequence to obtain an intermediate feature map of the current scale feature map; in response to the current scale feature map not being the minimum scale feature map, upsampling the associated scale feature map to obtain a second upsampled feature map, wherein the associated scale feature map is the previous scale feature map. the intermediate feature map corresponding to the feature map of the current scale; fusing the second up-sampled feature map with the current scale feature map to obtain a third fused feature map; performing feature extraction on the third fused feature map to obtain an intermediate feature map of the current scale feature map; in response to the current scale feature map being the maximum scale feature map, using the intermediate feature map of the current scale feature map as the second enhanced feature map of the current scale feature map; in response to the current scale feature map not being the maximum scale feature map, fusing the intermediate feature map corresponding to the current scale feature map with the second enhanced feature map corresponding to the next scale feature map to obtain a fourth fused feature map; and performing feature extraction on the fourth fused feature map to obtain a second enhanced feature map of the current scale feature map.

[0141] In some embodiments, the fusion detection module 840 performs target detection sub-results corresponding to each second enhanced feature map, including candidate categories and category confidences of the target, as well as candidate position information of the target; after performing target detection on each second enhanced feature map respectively and obtaining target detection sub-results corresponding to each second enhanced feature map as target detection results, it also includes: in each target detection sub-result, selecting a candidate category whose category confidence meets the confidence requirements as the category to which the target in the target image belongs; in each target detection sub-result, selecting candidate position information that meets the position requirements as the position information of the target in the target image.

[0142] In some embodiments, the transformation module 810 performs a transformation on the target image to obtain a plurality of images to be extracted, including: performing a Laplace pyramid transformation on the target image to obtain a plurality of images to be extracted with different resolutions.

[0143] In some embodiments, the target detection device 800 performs the steps of extracting features from a plurality of images to be extracted, obtaining a plurality of transformed feature maps, performing target detection based on a plurality of fused feature maps, and determining the target detection result by the target detection model; wherein the target detection model includes a prompt feature generation module and a target detection module, the prompt feature generation module is used to extract the transformed feature map and the image prompt information, and fuse each transformed feature map and the image prompt information to obtain a plurality of fused feature maps; the target detection module is used to perform target detection based on the plurality of fused feature maps to obtain the target detection result; and / or, the training method of the target detection model includes: transforming the sample image to obtain a plurality of sample images to be extracted, Image, wherein there is at least one sample target in the sample image, and the sample image is annotated with the actual category information of the sample target and the actual position information corresponding to the sample target; a target detection model is used to perform feature extraction on several sample images to be extracted, and several sample transformation feature maps are obtained, as well as sample image information corresponding to the sample image; each sample transformation feature map and the sample image information are fused to obtain several sample fusion feature maps; target detection is performed based on the several sample fusion feature maps to obtain predicted category information and predicted position information corresponding to the sample target; based on the difference between the predicted category information and the actual category information, and the difference between the predicted position information and the actual position information, the network parameters of the target detection model are optimized.

[0144] See also Figure 9 , Figure 9 1 is a schematic diagram of an embodiment of an electronic device 90 of the present application. Electronic device 90 includes a memory 91 and a processor 92 coupled to each other. Processor 92 is configured to execute program instructions stored in memory 91 to implement the steps of any of the above-described target detection method embodiments. In a specific implementation scenario, electronic device 90 may include, but is not limited to, a microcomputer and a server. Furthermore, electronic device 90 may also include mobile devices such as laptops and tablet computers, without limitation herein.

[0145] Specifically, the processor 92 is used to control itself and the memory 91 to implement the steps in any of the above-mentioned target detection method embodiments. The processor 92 can also be called a CPU (Central Processing Unit). The processor 92 may be an integrated circuit chip with signal processing capabilities. The processor 92 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 92 can be implemented by an integrated circuit chip.

[0146] See also Figure 10 , Figure 10 1 is a schematic diagram of a framework of an embodiment of a computer-readable storage medium 100 of the present application. The computer-readable storage medium 100 stores program instructions 1001 that can be executed by a processor, and the program instructions 1001 are used to implement the steps of any of the above-mentioned target detection method embodiments.

[0147] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0148] The above description of the various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced with each other and will not be repeated herein for the sake of brevity.

[0149] In the several embodiments provided in this application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation methods described above are merely illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods, such as units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0150] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0151] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

Claims

1. A target detection method, characterized in that: include: Transform the target image to obtain several images to be extracted; Performing feature extraction on the plurality of images to be extracted respectively to obtain a plurality of transformation feature maps, and obtaining image prompt information of the target image, wherein the image prompt information is used to represent image content in the target image; Fusing the transformed feature maps with the image prompt information to obtain a plurality of fused feature maps; Target detection is performed based on the several fused feature maps to obtain a target detection result.

2. The method according to claim 1, characterized in that The resolutions of the plurality of images to be extracted are different, and the feature extraction is performed on the plurality of images to be extracted respectively to obtain a plurality of transformed feature maps, including: Performing feature extraction on the plurality of images to be extracted respectively to obtain a multi-resolution feature map, wherein the multi-resolution feature map includes feature maps of different resolutions corresponding to the respective images to be extracted; Feature enhancement is performed on the multi-resolution feature map to obtain the plurality of transformed feature maps.

3. The method according to claim 2, characterized in that The multi-resolution feature map includes four feature maps with different resolutions, and the plurality of transformed feature maps include three transformed feature maps with different resolutions; And / or, performing feature enhancement on the multi-resolution feature map to obtain the plurality of transformed feature maps includes: Using each non-minimum resolution feature map in the multi-resolution feature map as a target feature map; In order of resolution from small to large, each target feature map is used as the current target feature map, and the following processing is performed: Upsampling the associated resolution feature map to obtain a first upsampled feature map, wherein, when the current target feature map is not the target feature map with the minimum resolution, the associated resolution feature map is an enhanced feature map corresponding to the previous target feature map; Fusing the first upsampled feature map with the current target feature map to obtain a first fused feature map; Performing feature extraction on the first fused feature map to obtain a first enhanced feature map of the current target feature map, wherein, when the current target feature map is the target feature map with the minimum resolution, the minimum resolution feature map is the first enhanced feature map of the minimum resolution feature map; In response to the current target feature map being the target feature map with the largest resolution, using the first enhanced feature corresponding to the current target feature map as the transformed feature corresponding to the current target feature map; In response to the current target feature map not being the target feature map with the highest resolution, a first enhanced feature map corresponding to the current target feature map is fused with a transformed feature map corresponding to the next target feature map to obtain a second fused feature map; and feature extraction is performed on the second fused feature map to obtain a transformed feature map of the current target feature map.

4. The method according to claim 1, wherein The image prompt information is a prompt feature map obtained by extracting features from the target image; And / or, the image prompt information of the target image includes a plurality of image prompt information whose sizes respectively correspond to the plurality of transformation feature maps; The transformation feature maps and the image prompt information are fused to obtain a plurality of fused feature maps, including: Each of the transformed feature maps is fused with the image prompt information corresponding to the size to obtain a plurality of fused feature maps.

5. The method according to claim 4, characterized in that The acquiring of the image prompt information of the target image includes: The target image is subjected to feature extraction using several feature extraction modules connected in sequence, and the prompt feature map output by each feature extraction module is used as the image prompt information of the target image, wherein the input of the first feature extraction module is the target image, and the input of the non-first feature extraction module is the prompt feature map output by the previous feature extraction module.

6. The method according to claim 1, characterized in that The target detection is performed based on the plurality of fused feature maps to obtain a target detection result, including: Combining the plurality of fused feature maps to obtain a combined feature map; Performing feature extraction on the combined feature map to obtain a multi-scale feature map; Performing feature enhancement on the multi-scale feature maps respectively to obtain a plurality of second enhanced feature maps; Target detection is performed on each of the second enhanced feature maps respectively, and a target detection sub-result corresponding to each of the second enhanced feature maps is obtained as the target detection result.

7. The method according to claim 6, characterized in that The multi-scale feature map includes three feature maps of different scales; And / or, performing feature enhancement on the multi-scale feature maps respectively to obtain a plurality of second enhanced feature maps, including: In order from small to large scale, each scale feature map is used as the current scale feature map, and the following processing is performed: In response to the current scale feature map being a minimum scale feature map, sequentially performing convolution processing and feature extraction on the current scale feature map to obtain an intermediate feature map of the current scale feature map; In response to the current scale feature map not being the minimum scale feature map, upsampling the associated scale feature map to obtain a second upsampled feature map, wherein the associated scale feature map is an intermediate feature map corresponding to the previous scale feature map; Fusing the second up-sampled feature map with the current scale feature map to obtain a third fused feature map; Performing feature extraction on the third fused feature map to obtain an intermediate feature map of the current scale feature map; In response to the current scale feature map being the maximum scale feature map, using an intermediate feature map of the current scale feature map as a second enhanced feature map of the current scale feature map; In response to the current scale feature map not being the maximum scale feature map, the intermediate feature map corresponding to the current scale feature map is fused with the second enhanced feature map corresponding to the next scale feature map to obtain a fourth fused feature map; and feature extraction is performed on the fourth fused feature map to obtain a second enhanced feature map of the current scale feature map.

8. The method according to claim 6, characterized in that The target detection sub-result corresponding to each of the second enhanced feature maps includes candidate categories and category confidences of the target, and candidate location information of the target; After performing target detection on each of the second enhanced feature maps and obtaining target detection sub-results corresponding to each of the second enhanced feature maps as the target detection results, the method further includes: In each of the target detection sub-results, a candidate category whose category confidence satisfies the confidence requirement is selected as the category to which the target in the target image belongs; In each of the target detection sub-results, the candidate position information that meets the position requirement is selected as the position information of the target in the target image.

9. The method according to claim 1, characterized in that The target image is transformed to obtain a plurality of images to be extracted, including: Performing Laplace pyramid transformation on the target image to obtain a plurality of images to be extracted with different resolutions.

10. The method according to claim 1, characterized in that The steps of respectively extracting features from the plurality of images to be extracted to obtain a plurality of transformed feature maps to performing target detection based on the plurality of fused feature maps and determining the target detection results are performed by a target detection model; wherein the target detection model includes a prompt feature generation module and a target detection module, the prompt feature generation module is used to extract the transformed feature map and the image prompt information, and fuse each of the transformed feature maps and the image prompt information to obtain a plurality of fused feature maps; the target detection module is used to perform target detection based on the plurality of fused feature maps to obtain a target detection result; and / or, The training method of the target detection model includes: Transforming the sample image to obtain a plurality of sample images to be extracted, wherein the sample image contains at least one sample target and the sample image is annotated with actual category information of the sample target and actual position information corresponding to the sample target; Using the target detection model to perform feature extraction on the plurality of sample images to be extracted, obtaining a plurality of sample transformation feature maps, and obtaining sample image information corresponding to the sample images; Fusing each of the sample transformation feature maps with the sample image information to obtain a plurality of sample fusion feature maps; Performing target detection based on the sample fusion feature maps to obtain predicted category information and predicted position information corresponding to the sample target; Based on the difference between the predicted category information and the actual category information, and the difference between the predicted position information and the actual position information, the network parameters of the target detection model are optimized.

11. An electronic device, characterized in that: The invention comprises a memory and a processor coupled to each other, wherein the processor is used to execute program instructions stored in the memory to implement the target detection method according to any one of claims 1 to 10.

12. A computer-readable storage medium having program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the target detection method according to any one of claims 1 to 10 is implemented.

Images