Target detection method and device, electronic device and storage medium

Abstract

The invention relates to a target detection method and device, an electronic device and a storage medium, and the method comprises the steps: obtaining original feature maps of a to-be-detected image through extracting the image features of the to-be-detected image, and enabling the original feature maps to comprise feature maps of different sizes; based on a trained discrete number estimation model, performing discrete number estimation on the original feature map, and outputting a discrete number estimation feature map; fusing the original feature map and the discrete number estimation feature map to obtain a mixed feature map; and inputting the mixed feature map into a trained target detection model for target object detection to obtain a target object detection result. According to the invention, the accuracy of target detection is improved.

Description

Technical Field

[0001] The present application relates to the field of target detection, and in particular to a target detection method, device, electronic device and storage medium. Background Art

[0002] Dense target detection is a detection task performed when there are a large number of closely arranged target objects (such as pedestrians, vehicles, etc.) in an image. These targets may block or overlap each other, increasing the difficulty of detection. In practical application scenarios such as urban traffic monitoring and shopping mall passenger flow statistics, accurate detection and analysis of dense targets is crucial for traffic management and abnormal behavior identification, and has important practical significance.

[0003] At present, target detection for dense scenes mainly estimates the number of targets by generating a density map of target distribution and indirectly locates the target position. For example, models such as multi-column convolutional neural networks are used to extract multi-scale features and generate density maps, or the image is divided into grids through the anchor method. Each grid predicts a fixed number of anchor points and slides on the feature map to detect the target. However, due to the large number of targets in dense scenes, the close distance between targets and their small size, it is often difficult for the model to focus on a single target, resulting in limited detection performance. Overlap between anchor points may also lead to inaccurate detection results.

[0004] There is currently no effective solution to the problem of inaccurate target detection in related technologies. Summary of the invention

[0005] In this embodiment, a target detection method, device, electronic device and storage medium are provided to solve the problem of inaccurate target detection in the related art.

[0006] In a first aspect, a target detection method is provided in this embodiment, including: Extracting image features of the image to be detected to obtain an original feature map of the image to be detected, wherein the original feature map includes feature maps of different sizes; Based on the trained discrete quantity estimation model, performing discrete quantity estimation on the original feature map, and outputting a discrete quantity estimation feature map; Fusion of the original feature map and the discrete quantity estimation feature map to obtain a mixed feature map; The mixed feature map is input into the trained target detection model to perform target object detection to obtain a target object detection result.

[0007] In some embodiments, extracting image features of the image to be detected to obtain an original feature map of the image to be detected includes: Extracting an initial feature map from the image to be detected by using a preset convolutional network model; Downsampling the initial feature map and then upsampling it; The feature maps having the same size and the same number of channels as those in the downsampling process and the upsampling process are added together to obtain the original feature map of the image to be detected.

[0008] In some of the embodiments, during downsampling of the initial feature map, the method further comprises: Inputting the initial feature map into the first convolution module and the bottleneck module of the preset convolutional network model, and outputting the first feature map of the image to be detected; Inputting the initial feature map into the second convolution module of the preset convolutional network model, and outputting the second feature map of the image to be detected; The first feature map and the second feature map are concatenated to obtain an initial feature map after downsampling.

[0009] In some embodiments, the training process of the discrete quantity estimation model includes: According to the depth-separable convolution and the classifier, the initial discrete quantity estimation model is trained to obtain a trained discrete quantity estimation model; Wherein, in the classifier, training the initial discrete quantity estimation model includes: After partitioning the discrete quantity estimation feature map, the quantity of target objects in the discrete quantity estimation feature map is estimated to obtain a quantity estimation result of each target object, and an initial discrete quantity estimation model is trained according to the quantity estimation result of each target object.

[0010] In some embodiments, the fusing the original feature map and the discrete quantity estimation feature map to obtain a mixed feature map includes: Performing depth-separable convolution processing on the original feature map to obtain two third feature maps and a fourth feature map of the same size; Resize the discrete quantity estimation feature map, the third feature map, and the fourth feature map into feature maps of the same size to obtain a reshaped discrete quantity estimation feature map, a reshaped third feature map, and a reshaped fourth feature map; The reshaped third feature map, the reshaped fourth feature map and the reshaped discrete quantity estimation feature map are fused to obtain a mixed feature map.

[0011] In some embodiments, the step of fusing the reshaped third feature map, the reshaped fourth feature map and the reshaped discrete quantity estimation feature map to obtain a mixed feature map comprises: Transposing the reshaped discrete quantity estimation feature map to obtain a transposed discrete quantity estimation feature map, Multiplying the transposed discrete quantity estimation feature map and the reshaped third feature map and inputting the resultant into a classifier, and calculating the distribution probability of the target object through the classifier to obtain a fifth feature map; Performing a matrix operation on the fifth feature map and the reshaped fourth feature map to obtain a sixth feature map; The size of the sixth feature map is reshaped to be the same as the size of the original feature map to obtain the mixed feature map.

[0012] In some embodiments, the target detection model training process includes: Obtain a mixed feature map of the training image; Pre-labeling the target object in the training image to obtain an initial target object labeling result; Based on the initial target object labeling result and the mixed feature map of the training image, the initial target detection model is trained for a target object detection task to obtain a trained target detection model.

[0013] In a second aspect, a target detection device is provided in this embodiment, including: an original feature extraction module, a discrete quantity estimation model, a hybrid feature module and a target detection module, wherein: The original feature extraction module is used to extract image features of the image to be detected to obtain an original feature map of the image to be detected, wherein the original feature map includes feature maps of different sizes; The discrete quantity estimation model is used to perform discrete quantity estimation on the original feature map and output a discrete quantity estimation feature map; The mixed feature module is used to fuse the original feature map and the discrete quantity estimation feature map to obtain a mixed feature map; The target detection module is used to input the mixed feature map into the trained target detection model to perform target object detection and obtain a target object detection result.

[0014] In a third aspect, an electronic device is provided in this embodiment, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the target detection method described in the first aspect when executing the computer program.

[0015] In a fourth aspect, a storage medium is provided in this embodiment, on which a computer program is stored, and when the program is executed by a processor, the target detection method described in the first aspect is implemented.

[0016] Compared with the related art, the target detection method provided in this embodiment obtains the original feature map of the image to be detected by extracting the image features of the image to be detected, wherein the original feature map includes feature maps of different sizes; based on the trained discrete quantity estimation model, the original feature map is discretely estimated, and a discrete quantity estimation feature map is output; the original feature map and the discrete quantity estimation feature map are fused to obtain a mixed feature map; the mixed feature map is input into the trained target detection model to perform target object detection to obtain a target object detection result. It performs discrete quantity estimation on the original feature map through the discrete quantity estimation model to obtain a discrete quantity estimation feature map, then fuses the discrete quantity estimation feature map with the original feature map, enhances the image features, obtains a mixed feature map, and performs target detection based on the mixed feature map after image feature enhancement, thereby improving the accuracy of target detection.

[0017] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 It is a hardware structure block diagram of the terminal of the target detection method of this embodiment.

[0019] Figure 2 is a flow chart of the target detection method of this embodiment.

[0020] Figure 3 Schematic diagram of the original feature map extraction network structure of the target detection method of this embodiment.

[0021] Figure 4 3 is a diagram of the C3 convolution module processing structure of the target detection method of this embodiment.

[0022] Figure 5 Schematic diagram of the discrete quantity estimation model structure of the target detection method of this embodiment.

[0023] Figure 6 Schematic diagram of the partitioning of the discrete quantity estimation model of the target detection method of this embodiment.

[0024] Figure 7 Schematic diagram of the feature mixing module structure of the target detection method of this embodiment.

[0025] Figure 8 is a flow chart of another target detection method of this embodiment.

[0026] Fig. 9 It is a structural block diagram of the target detection device of this embodiment. DETAILED DESCRIPTION

[0027] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0028] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with general skills in the technical field to which this application belongs. The words "one", "a", "the", "these" and the like in this application do not indicate a quantitative limitation, and they may be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" may mean: A exists alone, A and B exist at the same time, and B exists alone. Generally, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0029] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 FIG. 1 is a hardware structure diagram of a terminal of the target detection method of this embodiment. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 and a memory 104 for storing data, wherein the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above terminal. Figure 1More or fewer components as shown, or with Figure 1 Different configurations shown.

[0030] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the target detection method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0031] The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by a communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0032] In this embodiment, a target detection method is provided. Figure 2 is a flow chart of the target detection method of this embodiment. Figure 2 As shown, the process includes the following steps: Step S201, extracting image features of the image to be detected to obtain an original feature map of the image to be detected, wherein the original feature map includes feature maps of different sizes.

[0033] Specifically, the image features of the image to be detected are extracted by a convolutional neural network (CNN) to obtain the original feature map of the image to be detected. Specifically, the input image to be detected is first processed by multiple convolutional layers and pooling layers to gradually extract local features and global features in the image, which constitute the original feature map of the image to be detected. Each layer of convolution operation generates a set of feature maps, which reflect the abstract representation of the image at different levels. As the number of network layers increases, the size of the feature map gradually decreases, but the number of channels increases, which means that the feature map contains more abstract and advanced semantic information. For example, a shallow feature map may contain low-level features such as edges and textures, while a deep feature map may contain semantic information of object parts or overall objects. Smaller feature maps can capture global information in the image, such as the overall structure, layout, and large-scale texture features of the image. These global information is crucial to understanding the overall semantics and contextual relationships of the image. Larger feature maps pay more attention to local details in the image and can extract microscopic features such as edges, textures, and shapes. These detailed features are of great significance for identifying specific targets or texture patterns in the image. By constructing an original feature representation containing feature maps of different sizes, the multi-scale information of the image can be more comprehensively retained, thereby providing a richer and more accurate feature basis for subsequent image analysis tasks. For example, in target detection tasks, this multi-scale feature map can simultaneously help the algorithm better locate the position of the target and identify the detailed features of the target; in image classification tasks, it can more effectively extract features related to the category, thereby improving the accuracy and robustness of classification.

[0034] Step S202: Based on the trained discrete quantity estimation model, the original feature map is estimated in discrete quantity, and a discrete quantity estimation feature map is output.

[0035] Specifically, in this embodiment, in order to enhance the accuracy of target detection, the original image features are estimated by discrete quantity estimation model, the continuous feature data is divided into several discrete intervals, the continuous quantity classification task is converted into a discrete classification task, each feature value in the original feature map is evaluated, and the evaluation result of each feature value is mapped to a preset discrete interval or category, a discrete quantity estimation feature map is generated, and the feature representation related to quantity is learned from the original feature map, and the discrete quantity of the target in the image is predicted to obtain a discrete quantity estimation feature map. Among them, the discrete quantity estimation model is obtained by pre-training, and no quantity estimation task training is required in the target detection task. The discrete quantity estimation model can reduce the complexity of the model and improve the accuracy of classification. Improve the accuracy of target recognition.

[0036] Step S203, fusing the original feature map and the discrete quantity estimation feature map to obtain a mixed feature map.

[0037] Specifically, in order to enhance the object detection model's ability to understand image features, the original feature map and the discrete quantity estimation feature map are fused to obtain a more effective feature representation. Combining quantity information and semantic information with the original features enables the model to better understand the distribution and quantity of the target. By combining quantity information, the object detection model can also be helped to locate the target more accurately. Among them, the fusion method can be to directly splice the original feature map and the discrete quantity estimation feature map together to form a higher-dimensional feature map by splicing. The direct splicing method will increase the number of channels of the feature, but will not change the information inside each feature map. For example, if the number of channels of the original feature map is C1 and the number of channels of the discrete feature map is C2, the number of channels of the spliced ​​mixed feature map is C1+C2. Alternatively, the two feature maps are added pixel by pixel to enhance the amount of information of each feature map. This method does not increase the number of channels of the feature, but will enhance the amount of information of each feature map. Alternatively, the feature maps can be fused by other element-by-element operations (such as multiplication, maximum value, etc.). For example, element-by-element multiplication can highlight the common significant areas in the two feature maps. By fusing the original feature map and the discrete quantity estimation feature map, we can integrate the detail information of the original feature map and the semantic information of the discrete quantity estimation feature map, and integrate the target quantity information in the discrete quantity estimation feature map to improve the detection accuracy.

[0038] Step S204: input the mixed feature map into the trained target detection model to perform target object detection to obtain a target object detection result.

[0039] The target detection model is a specially designed neural network used to extract the location and category information of the target object from the mixed feature map. Among them, the target detection model can be a two-stage detection model such as Faster R-CNN and Mask R-CNN, that is, it can be a single-stage detection model such as YOLO (You Only Look Once) and SSD (Single Shot MultiBoxDetector). This embodiment does not impose specific restrictions on the target detection model and can be selected according to actual conditions. The target detection model in this embodiment is obtained by pre-training a large number of mixed feature maps. The mixed feature map is input into the trained target detection model for target object detection to obtain the target object detection result, wherein the target object detection result includes a bounding box, label category, and confidence score.

[0040] Through the above steps S201 to S204, the image features of the image to be detected are extracted to obtain the original feature map of the image to be detected, wherein the original feature map includes feature maps of different sizes; based on the trained discrete quantity estimation model, discrete quantity estimation is performed on the original feature map, and a discrete quantity estimation feature map is output; the original feature map and the discrete quantity estimation feature map are fused to obtain a mixed feature map; the mixed feature map is input into the trained target detection model to perform target object detection, and a target object detection result is obtained. Compared with the density map or anchor point method predicted by the model in the prior art, this embodiment extracts original feature maps of different sizes, enhances the feature representation of the image through the original feature maps of different sizes, and inputs the original feature map into the trained discrete quantity estimation model for discrete quantity estimation to obtain a discrete quantity estimation feature map, and the discrete quantity estimation model is used to perform discrete quantity estimation on the original feature map, converts the continuous quantity classification task into a discrete classification task, evaluates each eigenvalue in the original feature map, and maps the evaluation result of each eigenvalue to a preset discrete interval or category, generates a discrete quantity estimation feature map, further enhances the feature representation of the image to be detected, and the combined discrete quantity estimation model auxiliary training can also further improve the target detection model The ability and accuracy of identifying targets, and then the original feature map and the discrete quantity estimation feature map are feature fused, and the feature representation of the image to be detected is further enhanced to obtain a mixed feature map, and finally, target detection is performed based on the processed mixed feature map to obtain a target detection result. The accuracy of target detection is improved. In addition, the target detection method in this embodiment is not only applicable to the detection of a single target object. Even if it is applied to a scene with a large number of targets and closely arranged targets, it can achieve higher target detection accuracy through auxiliary training and feature fusion of discrete quantity estimation models.

[0041] In some of the embodiments, image features of an image to be detected are extracted to obtain an original feature map of the image to be detected, including: extracting a preset number of image features of different sizes from the image to be detected through a preset convolutional network model to obtain initial feature maps; downsampling each initial feature map and then upsampling it; adding feature maps of the same size and the same number of channels in the downsampling process and the upsampling process to obtain the original feature map of the image to be detected.

[0042] Specifically, in the above step S201, in order to obtain an original feature map with better feature performance, this embodiment extracts the feature map by combining upsampling and downsampling. Figure 3 : is a schematic diagram of the original feature map extraction network structure of the target detection method of this embodiment, such as Figure 3 As shown, first, an initial feature map is extracted from the image to be detected through the existing convolutional network model. The size of the initial feature map is H×W and the number of channels is 3.

[0043] The initial feature map is downsampled for the first time using the convolution module Conv(3, 64, 3, 2, 2), where the dimensions of the convolution module Conv(3, 64, 3, 2, 2) include: Number of input channels: 3; Number of output channels: 64; Convolution kernel size: 3x3; Step length: 2; Number of groups: 2; The feature map output by the convolution module Conv(3, 64, 3, 2, 2) is input to the convolution module C3(64) for further processing, and the output is: the feature map size is H / 2×W / 2, and the number of channels is 64.

[0044] Based on the image output by the convolution module C3 (64), a convolution module Conv (64, 128, 3, 2) is used for a second downsampling. The dimensions of the convolution module Conv (64, 128, 3, 2) include: Number of input channels: 64; Number of output channels: 128; Convolution kernel size: 3x3; Step length: 2; The feature map output by the convolution module Conv(64, 128, 3, 2) is input to the convolution module C3(128) for further processing, and the output is: the feature map size is H / 4×W / 4; the number of channels is 128.

[0045] Based on the image output by the convolution module C3 (128), the convolution module Conv (128, 256, 3, 2) is used to continue the third downsampling. The dimensions of the convolution module Conv (128, 256, 3, 2) include: Number of input channels: 128; Number of output channels: 256; Convolution kernel size: 3x3; Step length: 2; The feature map output by the convolution module Conv(128, 256, 3, 2) is input to the convolution module C3(256) for further processing. Output: The feature map size is H / 8×W / 8; the number of channels is 256.

[0046] Based on the image output by the convolution module C3 (256), the convolution module Conv (256, 512, 3, 2) is used to continue the fourth downsampling. The dimensions of the convolution module Conv (256, 512, 3, 2) include: Number of input channels: 256; Number of output channels: 512; Convolution kernel size: 3x3; Step length: 2; The feature map output by the convolution module Conv(256, 512, 3, 2) is input to the convolution module C3(512) for further processing, and the output is: the feature map size is H / 16×W / 16, and the number of channels is 512.

[0047] Based on the image output by the convolution module C3 (512), the convolution module Conv (512, 1024, 3, 2) is used to continue the fifth downsampling. The dimensions of the convolution module Conv (512, 1024, 3, 2) include: Number of input channels: 512; Number of output channels: 1024; Convolution kernel size: 3x3; Step length: 2; The feature map output by the convolution module Conv(512, 1024, 3, 2) is input to the convolution module C3(1024) for further processing, and the output is: the feature map size is H / 32×W / 32, and the number of channels is 1024.

[0048] Based on the image output by the convolution module C3 (1024), the spatial resolution of the feature map is increased by deconvolution (transposed convolution) or interpolation methods (such as bilinear interpolation), while the number of channels is reduced for upsampling. The convolution module Conv (1024, 512, 1, 1) is used for the first upsampling to reduce the number of channels. The dimensions of the convolution module Conv (1024, 512, 1, 1) include: Number of input channels: 1024; Number of output channels: 512; Convolution kernel size: 1x1; Step length: 1.

[0049] An upsampling operation is used to restore the size of the feature map output by the convolution module C3 (1024) from H / 32×W / 32 to H / 16×W / 16.

[0050] Output: The feature map size is H / 16×W / 16 and the number of channels is 512.

[0051] Based on the image output by the convolution module Conv(1024, 512, 1, 1), the convolution module Conv(512, 256, 1, 1) is used for a second upsampling to reduce the number of channels. The dimensions of the convolution module Conv(512, 256, 1, 1) include: Number of input channels: 512; Number of output channels: 256; Convolution kernel size: 1x1; Step length: 1; Use upsampling operations (such as bilinear interpolation or deconvolution) to restore the feature map size of the convolution module Conv(1024, 512, 1, 1) from H / 16×W / 16 to H / 8×W / 8. Output: The feature map size is H / 8×W / 8 and the number of channels is 256.

[0052] Among them, the Conv module is a convolution module, and its parameters are the number of input channels, output dimension, convolution kernel size, step size, and number of groups of the standard convolution. A convolution module contains a standard convolution layer, a BN layer, and an activation function SiLU. Upsample is responsible for upsampling, with a multiple of 2, and the sampling method is bilinear sampling (Bilinear). The specific number of upsampling and downsampling is not limited and can be set according to actual conditions. The C3 module is another convolution module in the feature extraction network structure. The C3 convolution module (CSP Bottleneck with 3 convolutions) is an efficient feature extraction module introduced in YOLOv5 and YOLOv8. As an improved version of CSP (Cross Stage Partial), C3 further improves the feature expression ability by introducing multiple Bottleneck layers while maintaining a lightweight calculation amount. The main function of the C3 module is to extract and fuse features at a deeper level based on the branch structure and combine multiple Bottleneck modules. It plays an important role in the backbone network and detection head of the YOLO series model. The original feature map of the image to be detected is extracted through the convolution module Conv and the convolution module C3.

[0053] After upsampling and downsampling the initial feature map, add the feature maps with the same size and the same number of channels in the upsampling process and the downsampling process. For example, add the feature map with a size of H / 8×W / 8 and a channel number of 256 output by the convolution module Conv(512, 256, 1, 1) in the upsampling process and the feature map with a size of H / 8×W / 8 and a channel number of 256 output by the convolution module C3(256) in the downsampling process to obtain the original feature map P1; add the feature map with the same size of H / 8×W / 8 and a channel number of 256 output by the convolution module Conv(1024, 512, 1, 1) in the upsampling process. The feature map with a size of H / 16×W16 and a channel number of 512 outputted by the convolution module C3(512) during the downsampling process is added together to obtain the original feature map P2; the feature map with a size of H / 32×W / 32 and a channel number of 1024 outputted by the convolution module C3(1024) during the upsampling process is added together with the feature map with a size of H / 32×W / 32 and a channel number of 1024 outputted by the convolution module C3(1024) during the downsampling process to obtain the original feature map P3.

[0054] pass Figure 3 The convolutional network performs multiple downsampling and upsampling operations to gradually extract and fuse multi-scale features. The downsampling process reduces the size of the feature map through convolution and operations with a step size greater than 1, while the upsampling process restores the size of the feature map through deconvolution or interpolation methods, which can fuse shallow and deep feature maps and improve the feature performance of the original feature map.

[0055] In another embodiment, during the downsampling of the initial feature map, the method further includes: The initial feature map is input into the first convolution module and the bottleneck module of the preset convolution network model, and the first feature map of the image to be detected is output; the initial feature map is input into the second convolution module of the preset convolution network model, and the second feature map of the image to be detected is output; the first feature map and the second feature map are spliced ​​to obtain the initial feature map after downsampling.

[0056] Specifically, in the above-mentioned process of downsampling the initial feature map, this embodiment also uses the C3 convolution module to process the initial feature map. The C3 convolution module (CSP Bottleneck with 3 convolutions) is an efficient feature extraction module introduced in YOLOv5 and YOLOv8. As an improved version of CSP (Cross Stage Partial), C3 further enhances the expressiveness of features by introducing multiple Bottleneck layers while maintaining lightweight computation. The main function of the C3 module is to extract and fuse features at a deeper level based on the branch structure and in combination with multiple Bottleneck modules. It plays an important role in the backbone network and detection head of the YOLO series models. Figure 4 : is a C3 convolution module processing structure diagram of the target detection method of this embodiment, such as Figure 4 As shown in the figure, the basic structure of the C3 convolution module contains two branches: one branch includes the first convolution module and the bottleneck module (residual Bottleneck module). The Bottleneck module usually contains multiple convolution layers and residual connections to enhance feature representation. The other branch includes the second convolution module. Input feature map, the feature map size is C×H×W, where C is the number of channels, H is the height, and W is the width. The initial feature map is convolved by the first convolution module Conv(C, 0.5C, 1, 1): Number of input channels: C; Number of output channels: 0.5C; Convolution kernel size: 1x1; Step length: 1; After the convolution operation, the output feature map size is 0.5C×H×W.

[0057] Based on the image output by the first convolution module Conv(C, 0.5C, 1, 1), a Bottleneck operation is performed to further extract features.

[0058] The output feature map size is still 0.5C×H×W, and the first feature map is obtained.

[0059] The initial feature map is convolved through the second convolution module Conv(C, 0.5C, 1, 1): Number of input channels: C; Number of output channels: 0.5C; Convolution kernel size: 1x1; Step length: 1; After the convolution operation of the second convolution module Conv(C, 0.5C, 1, 1), the output feature map size is 0.5C×H×W, and the second feature map is obtained.

[0060] Concatenate the output feature maps of the two branches in the channel dimension: The size of the first feature map output by branch 1 is 0.5C×H×W; The size of the second feature map output by branch 2 is 0.5C×H×W; After concatenating the first feature map and the second feature map, the size of the initial feature map is C×H×W.

[0061] Perform the final convolution operation Conv(C, C, 1, 1) on the concatenated feature map: Number of input channels: C; Number of output channels: C; Convolution kernel size: 1x1; Step length: 1; After the convolution operation, the size of the initial feature map after output downsampling is still C×H×W.

[0062] The initial feature map after output downsampling has the same size as the input feature map, but has undergone feature enhancement and multi-branch fusion processing. This structure can enhance the feature representation capability and further improve the accuracy of target detection through multi-branch processing and feature fusion.

[0063] In some embodiments, the training process of the discrete quantity estimation model includes: According to the deep separable convolution and the classifier, the initial discrete quantity estimation model is trained to obtain the trained discrete quantity estimation model; wherein, in the classifier, the training of the initial discrete quantity estimation model includes: after partitioning the discrete quantity estimation feature map, the quantity of the target objects in the discrete quantity estimation feature map is estimated to obtain the quantity estimation results of each target object, and the initial discrete quantity estimation model is trained according to the quantity estimation results of each target object.

[0064] Specifically, the discrete quantity estimation model in this embodiment is obtained by pre-training the sample feature map. Figure 5 is a schematic diagram of the discrete quantity estimation model structure of the target detection method of this embodiment; the discrete quantity estimation model in this embodiment includes a depth-separable convolution and a classifier. Figure 5 As shown, the training process includes: a. The original feature maps P1, P2, and P3 are processed by the DWConv convolution module to obtain new feature maps D1, D2, and D3; wherein, the DWConv convolution module is a depth-separable convolution, which replaces the standard convolution in the Conv convolution module with a depth-separable convolution. The DWConv convolution module includes depth-wise convolution and point-wise convolution. Through depth-wise convolution, each channel of the input feature map is convolved to extract spatial features; through point-by-point convolution, the output of the depth-wise convolution is mapped to the target number of channels using a 1x1 convolution kernel, and the information between channels is fused to obtain new feature maps D1, D2, and D3; then, global average pooling (Global AvgPooling) is performed on each new feature map D1, D2, and D3 to compress the spatial dimensions (height and width) to 1x1 to obtain G1, G2, and G3; b. Map the feature vectors G1, G2, and G3 after global average pooling to a fixed dimension (e.g., 64 dimensions, linear (512, 64)) through a linear layer to obtain F1, F2, and F3, mapping features of different scales to the same dimensional space for subsequent fusion; c. Concatenate F1, F2, and F3 in the channel dimension to obtain the joint feature F. By fusing multi-scale features, the expressiveness of the model is enhanced.

[0065] Finally, the feature map is partitioned by the classifier, and the number of target objects in each partition is estimated. The loss is calculated based on the difference between the number estimation result of each partition and the true label, and the initial discrete quantity estimation model is optimized to obtain the trained discrete quantity estimation model. The classifier in this embodiment includes a linear layer (linear (192, 102)) and a Softmax layer. In the training of the classifier, the discrete quantity estimation feature map is first divided into multiple regions, and each partition corresponds to a local area in the image. The number of partitions can be adjusted according to the task requirements. Figure 6 is a schematic diagram of the partitioning of the discrete quantity estimation model of the target detection method of this embodiment, such as Figure 6 As shown in the figure, for example, the rule for dividing 102 interval categories is to divide them into intervals of 10, and use regression methods to predict the number of target objects in each partition. Or use classification methods to predict the number category of target objects in each partition (such as 0, 1, 2, etc.), 0 means no target, and 1-10 means the number of target objects is between 1 and 10. The loss is calculated based on the difference between the number estimation result of each partition and the true label: if the regression method is used, the mean square error (MSE) loss is usually used. If the classification method is used, the cross-entropy loss is usually used, and the model parameters are updated through the back propagation algorithm to minimize the loss function. Through feature map partitioning and quantity estimation, the model can learn the distribution and quantity information of the target object, thereby improving the accuracy of target detection.

[0066] In another embodiment, the original feature map and the discrete quantity estimation feature map are fused to obtain a mixed feature map, including: The original feature map is subjected to a depthwise separable convolution process to obtain two third feature maps and a fourth feature map of the same size; the discrete quantity estimation feature map, the third feature map and the fourth feature map are resized into feature maps of the same size to obtain a reshaped discrete quantity estimation feature map, a reshaped third feature map and a reshaped fourth feature map; the reshaped third feature map, the reshaped fourth feature map and the reshaped discrete quantity estimation feature map are fused to obtain a mixed feature map.

[0067] Specifically, when fusing the original feature map with the discrete quantity estimation feature map, the original feature map is first passed through two depth-separable convolution modules to generate a third feature map and a fourth feature map, respectively, where the third feature map and the fourth feature map have the same size, and then the discrete quantity estimation feature map is reshaped to be the same size as the third feature map and the fourth feature map. Figure 1The reshaped third feature map, the reshaped fourth feature map, and the reshaped discrete quantity estimation feature map are fused. The fusion process can combine the information of these feature maps through matrix multiplication, attention mechanism, or other operations to obtain a mixed feature map. The mixed feature map contains the information of the original feature map and the discrete quantity estimation feature map.

[0068] In some of the embodiments, the reshaped third feature map, the reshaped fourth feature map and the reshaped discrete quantity estimation feature map are fused to obtain a mixed feature map, including: transposing the reshaped discrete quantity estimation feature map to obtain a transposed discrete quantity estimation feature map, multiplying the transposed discrete quantity estimation feature map with the reshaped third feature map and inputting the resultant inputs into a classifier, calculating the distribution probability of the target object through the classifier to obtain a fifth feature map; performing a matrix operation on the fifth feature map and the reshaped fourth feature map to obtain a sixth feature map; and reshaping the size of the sixth feature map to the same size as the original feature map to obtain a mixed feature map.

[0069] Specifically, Figure 7 : is a schematic diagram of the structure of the feature mixing module of the target detection method of this embodiment. Figure 7 As shown in the figure, taking the original feature map P1 and the discrete quantity estimation feature map D1 as examples, the size of the original feature map P1 is 256×H / 8×W / 8, where H is the height of the feature map and W is the width of the feature map. The feature map P1 is subjected to two depthwise separable convolution (DWConv (256, 1, 1)) modules to obtain two feature maps of the same size. The two feature maps are reshaped (Reshape) to merge the width and height into one dimension to obtain the reshaped third feature map K and the reshaped fourth feature map V, whose size is 256×H / 8×W / 8. The discrete quantity estimation feature map D1 is reshaped (Reshape) to generate the reshaped discrete quantity estimation feature map Q, whose size is 256×H / 8×W / 8. The reshaped discrete quantity estimation feature map Q is transposed to obtain a transposed discrete quantity estimation feature map, and the transposed discrete quantity estimation feature map is multiplied with the reshaped third feature map K to obtain an intermediate result. The Softmax operation is applied to the intermediate result to obtain the fifth feature map A, whose size is (H / 8×W / 8)×(H / 8×W / 8). The fifth feature map A is matrix multiplied with the reshaped fourth feature map V to obtain a weighted feature map sixth feature map E. The sixth feature map is reshaped (reshape) to restore the sixth feature map to the size of the original feature map P1, that is, 256×(H / 8)×(W / 8), to obtain a mixed feature map E1, whose size is 256×H / 8×W / 8.

[0070] In another embodiment, the target detection model training process includes: obtaining a mixed feature map of a training image; pre-labeling the target objects in the training image to obtain an initial target object labeling result; based on the initial target object labeling result and the mixed feature map of the training image, performing target object detection task training on the initial target detection model to obtain a trained target detection model.

[0071] Specifically, according to the method for extracting a mixed feature map in the above embodiment, a mixed feature map is extracted from a training image, and the target object in the training image is pre-labeled first. The labeling includes the category and bounding box of the target object. The pre-standard can be completed manually or using a pre-trained model or an automated tool. Based on the requirements of the target detection task, an initial target detection model is constructed, and the initial target detection model is trained based on the initial target object labeling results and the mixed feature map of the training image. During the training process, the model learns how to identify and locate the target object from the mixed feature map. Each point of each mixed feature map predicts the position and category of a target box. The position is a regression task, and the category is a classification task. According to the pre-labeled results and the prediction results of the initial target detection model, a loss function (such as cross entropy loss, IoU loss, etc.) is calculated to measure the difference between the prediction results of the initial target detection model and the true labeling, and the initial target detection model parameters are optimized to minimize the loss. The performance of the initial target detection model is evaluated, and according to the evaluation results, the initial target detection model is adjusted and optimized to obtain a trained target detection model.

[0072] This embodiment also provides a target detection method. Figure 8 is a flow chart of another target detection method of this embodiment. Figure 8 As shown, the process includes the following steps: Step S801, extracting an initial feature map from the image to be detected by a preset convolutional network model; downsampling the initial feature map and then upsampling it; adding the feature maps with the same size and the same number of channels in the downsampling process and the upsampling process to obtain the original feature map of the image to be detected; Step S802, based on the trained discrete quantity estimation model, performing discrete quantity estimation on the original feature map, and outputting a discrete quantity estimation feature map; Step S803, performing depth-separable convolution processing on the original feature map to obtain two third feature maps and a fourth feature map of the same size; Step S804, resizing the discrete quantity estimation feature map, the third feature map, and the fourth feature map into feature maps of the same size, to obtain a reshaped discrete quantity estimation feature map, a reshaped third feature map, and a reshaped fourth feature map; Step S805, transposing the reshaped discrete quantity estimation feature map to obtain a transposed discrete quantity estimation feature map, multiplying the transposed discrete quantity estimation feature map with the reshaped third feature map and inputting the resultant multiplication into a classifier, and calculating the distribution probability of the target object through the classifier to obtain a fifth feature map; Step S806, performing a matrix operation on the fifth feature map and the reshaped fourth feature map to obtain a sixth feature map; Step S807, reshape the size of the sixth feature map to the same size as the original feature map to obtain a mixed feature map; Step S808, inputting the mixed feature map into the trained target detection model to perform target object detection to obtain a target object detection result.

[0073] Through the above steps S801 to S808, compared with the density map or anchor point method predicted by the model in the prior art, this embodiment extracts original image features of different sizes of the target object in the image to be detected; combines the original image features of different sizes to train a discrete quantity estimation model, and inputs the original image features into the trained discrete quantity estimation model to generate a discrete quantity estimation feature map; through a mixed feature module, the information of the discrete quantity estimation feature map and the original feature map is fused to generate a mixed feature map; the target detection task is performed based on the mixed feature map; wherein, by adding a new discrete quantity estimation model, the discrete quantity estimation feature map and the original feature map are combined to obtain a mixed feature map, and the target object is detected based on the mixed feature map, thereby improving the accuracy of target detection.

[0074] In this embodiment, a target detection device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. The terms "module", "unit", "subunit", etc. used below can implement a combination of software and / or hardware of predetermined functions. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0075] Fig. 9 is a structural block diagram of the target detection device of this embodiment, such as Fig. 9 As shown, the device 90 includes: an original feature extraction module 91, a discrete quantity estimation model 92, a mixed feature module 93 and a target detection module 94, wherein: The original feature extraction module 91 is used to extract the image features of the image to be detected and obtain the original feature map of the image to be detected, wherein the original feature map includes feature maps of different sizes; A discrete quantity estimation model 92, used for performing discrete quantity estimation on the original feature map and outputting a discrete quantity estimation feature map; The mixed feature module 93 is used to fuse the original feature map and the discrete quantity estimation feature map to obtain a mixed feature map; The target detection module 94 is used to input the mixed feature map into the trained target detection model to perform target object detection and obtain the target object detection result.

[0076] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0077] In this embodiment, an electronic device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0078] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0079] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program: S1, extracting image features of the image to be detected to obtain an original feature map of the image to be detected, wherein the original feature map includes feature maps of different sizes; S2, based on the trained discrete quantity estimation model, performs discrete quantity estimation on the original feature map and outputs a discrete quantity estimation feature map; S3, fuses the original feature map and the discrete quantity estimation feature map to obtain a mixed feature map; S4, inputting the mixed feature map into the trained target detection model to perform target object detection and obtain the target object detection result.

[0080] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.

[0081] In addition, in combination with the target detection method provided in the above embodiments, a storage medium may be provided in this embodiment to implement the target detection method. The storage medium stores a computer program; when the computer program is executed by a processor, any target detection method in the above embodiments is implemented.

[0082] It should be understood that the specific embodiments described herein are only used to explain the application, rather than to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of this application.

[0083] Obviously, the drawings are only some examples or embodiments of the present application. For ordinary technicians in the field, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it is understandable that although the work done in this development process may be complicated and lengthy, for ordinary technicians in the field, certain changes in design, manufacturing or production based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient content disclosed in this application.

[0084] The term "embodiment" in this application refers to a specific feature, structure or characteristic described in conjunction with the embodiment that can be included in at least one embodiment of the present application. The appearance of this phrase in various locations in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is clearly or implicitly understood by those of ordinary skill in the art that the embodiments described in this application can be combined with other embodiments without conflict.

[0085] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0086] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of patent protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.

Claims

1. A target detection method, characterized in that: include: Extracting image features of the image to be detected to obtain an original feature map of the image to be detected, wherein the original feature map includes feature maps of different sizes; Based on the trained discrete quantity estimation model, performing discrete quantity estimation on the original feature map, and outputting a discrete quantity estimation feature map; Fusion of the original feature map and the discrete quantity estimation feature map to obtain a mixed feature map; The mixed feature map is input into the trained target detection model to perform target object detection to obtain a target object detection result.

2. The target detection method according to claim 1, characterized in that: The step of extracting image features of the image to be detected to obtain an original feature map of the image to be detected includes: Extracting an initial feature map from the image to be detected by using a preset convolutional network model; Downsampling the initial feature map and then upsampling it; The feature maps having the same size and the same number of channels as those in the downsampling process and the upsampling process are added together to obtain the original feature map of the image to be detected.

3. The target detection method according to claim 2, characterized in that: In the process of downsampling the initial feature map, the method further includes: Inputting the initial feature map into the first convolution module and the bottleneck module of the preset convolutional network model, and outputting the first feature map of the image to be detected; Inputting the initial feature map into the second convolution module of the preset convolutional network model, and outputting the second feature map of the image to be detected; The first feature map and the second feature map are concatenated to obtain an initial feature map after downsampling.

4. The target detection method according to claim 1, characterized in that: The training process of the discrete quantity estimation model includes: According to the depth-separable convolution and the classifier, the initial discrete quantity estimation model is trained to obtain a trained discrete quantity estimation model; Wherein, in the classifier, training the initial discrete quantity estimation model includes: After partitioning the discrete quantity estimation feature map, the quantity of target objects in the discrete quantity estimation feature map is estimated to obtain a quantity estimation result of each target object, and an initial discrete quantity estimation model is trained according to the quantity estimation result of each target object.

5. The target detection method according to claim 1, characterized in that: The fusing the original feature map and the discrete quantity estimation feature map to obtain a mixed feature map includes: Performing depth-separable convolution processing on the original feature map to obtain two third feature maps and a fourth feature map of the same size; Resize the discrete quantity estimation feature map, the third feature map, and the fourth feature map into feature maps of the same size to obtain a reshaped discrete quantity estimation feature map, a reshaped third feature map, and a reshaped fourth feature map; The reshaped third feature map, the reshaped fourth feature map and the reshaped discrete quantity estimation feature map are fused to obtain a mixed feature map.

6. The target detection method according to claim 5, characterized in that: The step of fusing the reshaped third feature map, the reshaped fourth feature map and the reshaped discrete quantity estimation feature map to obtain a mixed feature map comprises: Transposing the reshaped discrete quantity estimation feature map to obtain a transposed discrete quantity estimation feature map, Multiplying the transposed discrete quantity estimation feature map and the reshaped third feature map and inputting the resultant into a classifier, and calculating the distribution probability of the target object through the classifier to obtain a fifth feature map; Performing a matrix operation on the fifth feature map and the reshaped fourth feature map to obtain a sixth feature map; The size of the sixth feature map is reshaped to be the same as the size of the original feature map to obtain the mixed feature map.

7. The target detection method according to claim 1, characterized in that: The target detection model training process includes: Obtain a mixed feature map of the training image; Pre-labeling the target object in the training image to obtain an initial target object labeling result; Based on the initial target object labeling result and the mixed feature map of the training image, the initial target detection model is trained for a target object detection task to obtain a trained target detection model.

8. A target detection device, characterized in that: include: Original feature extraction module, discrete quantity estimation model, mixed feature module and target detection module, among which, The original feature extraction module is used to extract image features of the image to be detected to obtain an original feature map of the image to be detected, wherein the original feature map includes feature maps of different sizes; The discrete quantity estimation model is used to perform discrete quantity estimation on the original feature map and output a discrete quantity estimation feature map; The mixed feature module is used to fuse the original feature map and the discrete quantity estimation feature map to obtain a mixed feature map; The target detection module is used to input the mixed feature map into the trained target detection model to perform target object detection and obtain a target object detection result.

9. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the target detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the target detection method according to any one of claims 1 to 7 are implemented.

Images