Target detection method and system, computer equipment and storage medium

Abstract

The invention discloses a target detection method and system, computer equipment and a storage medium, and relates to the technical field of target detection, and the method comprises the steps: obtaining a marked image sample, carrying out the feature extraction of the image sample, and obtaining a plurality of groups of first feature maps of different depths; determining an excitation weight corresponding to the first feature map, and strengthening the first feature map through the excitation weight to obtain a second feature map corresponding to the first feature map; frame prediction is carried out based on the second feature map, a detection frame is obtained, frame loss and category loss are calculated, weight parameters are updated according to the frame loss and the category loss, and a target detection model used for target detection is obtained; and inputting a to-be-detected image into the target detection model for target detection, and labeling a target object in the to-be-detected image. The technical problem of inaccurate target identification in small target detection and multi-target shielding scenes in the prior art is solved.

Description

Technical Field

[0001] The present application relates to the technical field of target detection, and in particular to a target detection method, system, computer device and storage medium. Background Art

[0002] Object detection is an important research direction in the field of computer vision and is widely used in scenarios such as regional intrusion, passenger flow counting, and crowd gathering. The core goal of object detection technology is to accurately and quickly identify objects in images or video streams and locate them. With the rapid development of technologies such as deep learning and convolutional neural networks (CNNs), the research and application of object detection has made significant progress.

[0003] However, the application scenarios of target detection are becoming more complex and the requirements are becoming more diverse. Take pedestrian detection as an example. In long-distance surveillance videos, pedestrians often appear as smaller targets, resulting in fewer pixels in the image. At this time, existing detection algorithms often fail to accurately detect small targets due to scale issues in feature extraction. In addition, in dense crowds or complex environments, multiple pedestrians may occlude each other, making it difficult for detection algorithms to distinguish different targets. Although traditional target detection technology performs well in detecting a single target, it is often difficult to accurately separate occluded pedestrians in the case of severe occlusion.

[0004] In summary, although target detection technology has made great improvements in accuracy and speed, the problem of inaccurate recognition in some complex environments, such as small target detection and multi-target occlusion, has not yet been effectively solved. Summary of the invention

[0005] The embodiments of the present application provide a target detection method, system, computer device and storage medium to solve the technical problems of inaccurate small target detection and target recognition in multi-target occlusion scenarios in the prior art.

[0006] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0007] In a first aspect, an embodiment of the present application provides a target detection method, the method comprising:

[0008] Acquire a pre-annotated image sample, wherein the annotation includes annotating a category of a target object and a first position of the target object in the image sample;

[0009] Performing feature extraction on the acquired image samples to obtain multiple groups of first feature maps at different depths;

[0010] Determine an excitation weight corresponding to the first feature map according to a distance between a second position of the first feature map on the image sample and the first position, and strengthen the first feature map by using the excitation weight to obtain a second feature map corresponding to the first feature map;

[0011] Performing border prediction based on the second feature map to obtain a detection border, calculating a border loss and a category loss, and updating a weight parameter according to the border loss and the category loss to obtain a target detection model;

[0012] The image to be detected is input into the target detection model for target detection, and the target object in the image to be detected is marked based on the target detection result.

[0013] In combination with the first aspect, in a possible design manner, the first position is indicated by a marked border, and determining the excitation weight corresponding to the first feature map according to a distance between a second position of the first feature map on the image sample and the first position includes:

[0014] If the second position of the first feature map on the image sample is outside the marked border, determining that the excitation weight corresponding to the first feature map is 0;

[0015] If the second position of the first feature map on the image sample is within the marked border, it is determined that the excitation weight corresponding to the first feature map is greater than 0.

[0016] In combination with the first aspect, in a possible design manner, determining that the excitation weight corresponding to the first feature map is greater than 0 includes:

[0017] Determining an excitation weight corresponding to the first feature map is positively correlated with a distance between the second position and the first position.

[0018] In combination with the first aspect, in a possible design manner, the step of strengthening the first feature map by using the incentive weight to obtain a second feature map corresponding to the first feature map includes:

[0019] The feature of each spatial position in the first feature map is multiplied by the excitation weight corresponding to the spatial position, and then superimposed with the feature of the spatial position to obtain a second feature map corresponding to the first feature map.

[0020] In combination with the first aspect, in a possible design manner, after updating the weight parameter according to the border loss and the category loss, the method further includes:

[0021] While keeping the weight parameters of the backbone network fixed, a mean square error loss is constructed to measure the error between the excitation weight output by the foreground excitation network and the true weight label.

[0022] With the goal of minimizing the mean square error loss, the foreground excitation network is trained to obtain the optimized target detection model.

[0023] In combination with the first aspect, in a possible design manner, the backbone network includes a CSPDarknet53 network, the CSPDarknet53 network includes a first convolution module, a second convolution module, a third convolution module, a fourth convolution module, a fifth convolution module, a first C2f module, a second C2f module, a third C2f module, a fourth C2f module and an SPPF module, and the acquired image samples are subjected to feature extraction to obtain multiple groups of first feature maps of different depths, including:

[0024] Input the image sample into the first convolution module, the second convolution module, the first C2f module, the third convolution module, and the second C2f module, and perform layer-by-layer feature extraction to obtain a first group of first feature maps;

[0025] The first group of first feature maps are subjected to a fourth convolution module and a third C2f module for layer-by-layer feature extraction to obtain a second group of first feature maps;

[0026] After performing layer-by-layer feature extraction on the second group of first feature maps through a fifth convolution module, a fourth C2f module, and an SPPF module, a third group of first feature maps is obtained;

[0027] The depth order of the first group of first feature maps, the second group of first feature maps, and the third group of first feature maps is from low to high.

[0028] In combination with the first aspect, in a possible design manner, performing border prediction based on the second feature map to obtain a detection border includes:

[0029] Fusing the second feature maps to obtain fused features;

[0030] Based on the fusion features, bounding box prediction is performed to obtain bounding box information and category confidence of multiple predicted bounding boxes, and a detection bounding box is screened out from the multiple predicted bounding boxes.

[0031] In a second aspect, an embodiment of the present application provides a target detection system, including:

[0032] A sample acquisition module, used to acquire a pre-annotated image sample, wherein the annotation includes annotating a category of a target object and a first position of the target object in the image sample;

[0033] A feature extraction module, used to extract features from the acquired image samples to obtain multiple groups of first feature maps of different depths;

[0034] a foreground excitation module, configured to determine an excitation weight corresponding to the first feature map according to a distance between a second position of the first feature map on the image sample and the first position, and to strengthen the first feature map by using the excitation weight to obtain a second feature map corresponding to the first feature map;

[0035] A parameter updating module, configured to perform border prediction based on the second feature map to obtain a detection border, calculate a border loss and a category loss, and update a weight parameter according to the border loss and the category loss to obtain a target detection model;

[0036] The model application module is used to input the image to be detected into the target detection model for target detection, and to mark the target object in the image to be detected based on the target detection result.

[0037] In a third aspect, an embodiment of the present application provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the method of the first aspect and possible design methods thereof.

[0038] In a fourth aspect, an embodiment of the present application provides a storage medium storing a computer program, wherein the computer program is configured to execute the method of the first aspect and its possible design method when running, and the display screen is configured to display the second video and audio processed by the processor.

[0039] Compared with the prior art, the target detection method, system, computer device and storage medium provided in the embodiments of the present application first obtain a pre-annotated image sample, wherein the annotation includes annotating the category of the target object and the first position of the target object in the image sample; then, feature extraction is performed on the acquired image sample to obtain multiple groups of first feature maps of different depths; then, based on the second position of the first feature map on the image sample and the distance between the first position, an excitation weight corresponding to the first feature map is determined, and the first feature map is strengthened by the excitation weight to obtain a second feature map corresponding to the first feature map; a border prediction is performed based on the second feature map to obtain a detection border, the border loss and the category loss are calculated, and the weight parameters are updated according to the border loss and the category loss to obtain a target detection model; the image to be detected is input into the target detection model for target detection, and the target object in the image to be detected is annotated based on the target detection result. This application strengthens the foreground target by excitation weights to highlight the foreground features. In this way, the model's ability to recognize targets in multi-target occlusion scenarios is improved. In addition, features of different scales are extracted and fused so that the fused features have richer semantic information and fine-grained spatial information, thereby improving the model's detection effect on small targets.

[0040] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0042] Figure 1 A hardware structure block diagram of a computer device provided in an embodiment of the present application is shown;

[0043] Figure 2 A flow chart of a target detection method provided by an embodiment of the present application is shown;

[0044] Figure 3A A schematic diagram of a backbone network provided in an embodiment of the present application is shown;

[0045] Figure 3B A schematic diagram of a residual module in a backbone network provided in an embodiment of the present application is shown;

[0046] Figure 3C A schematic diagram of a C2f module in a backbone network provided in an embodiment of the present application is shown;

[0047] Figure 3DA schematic diagram of a bottleneck layer in a C2f module provided in an embodiment of the present application is shown;

[0048] Figure 3E A schematic diagram of an SPPF in a backbone network provided by an embodiment of the present application is shown;

[0049] Figure 4 A schematic diagram of a prospect incentive module provided in an embodiment of the present application is shown;

[0050] Figure 5 A schematic diagram of a multi-scale feature fusion module provided in an embodiment of the present application is shown;

[0051] Figure 6 A schematic diagram of a fusion module provided in an embodiment of the present application is shown;

[0052] Figure 7 A structural block diagram of a target detection system provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0053] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0054] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the", "these" and the like in this application do not represent quantitative restrictions, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether directly or indirectly. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. Usually, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0055] The embodiment of the present application provides a target detection method, which increases the weight of foreground target features through a foreground excitation network, so that the target detection model pays more attention to the information of the foreground target, and fuses multiple groups of second feature maps of different depths to enhance the algorithm's recognition ability for targets of different scales, thereby solving the problem of inaccurate small target detection and target recognition in multi-target occlusion scenarios in the prior art.

[0056] The method can be executed in a terminal, a computer or a similar test system. For example, running on a computer device, Figure 1 FIG. 1 shows a hardware structure block diagram of a computer device provided in an embodiment of the present application. Figure 1 As shown, the computer device may include one or more ( Figure 1 The computer device may further include a transmission device 106 and an input / output device 108 for communication functions. It will be appreciated by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above-mentioned computer device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0057] The memory 104 can be used to store computer programs, for example, software programs and modules of application software. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, that is, to implement the above method. The memory 104 can be used to store data, such as image samples, images to be detected, etc. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0058] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer device. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0059] The embodiments of the present application can be applied to identify the target object in the image to be detected, which can be a pedestrian, a vehicle, a traffic sign, a license plate, etc. The target object in the image to be detected is detected (or recognized) by training a target detection model, and the category and position of the target object are marked in the image to be detected, wherein the position of the target object can be marked by detecting a border, and the target object is located within the detection border. The target detection method provided by the embodiments of the present application is mainly described below by taking the target object as a pedestrian as an example.

[0060] Figure 2 A flow chart of a target detection method provided by an embodiment of the present application is shown as follows: Figure 2 As shown, the method includes steps S201 to S205.

[0061] Step S201: Acquire a pre-annotated image sample, where the annotation includes annotating the category of the target object and the first position of the target object in the image sample.

[0062] The annotation may be made in the image sample by software, and the annotation content includes annotation of the category and the first position, and the first position is represented by coordinates.

[0063] The category refers to the type of object, usually a label such as "pedestrian" or "person". The coordinates are the position of the pedestrian in the image sample, which is usually represented by a bounding box, which contains the pixel coordinates of the upper left corner and lower right corner of the rectangular box where the pedestrian is located. These two pixel coordinates determine the exact position and size of the pedestrian in the image sample.

[0064] Step S202: extract features from the acquired image samples to obtain multiple groups of first feature maps at different depths.

[0065] Among them, feature extraction is performed through a preset backbone network. The backbone network has a multi-scale structure and can extract feature information of different resolutions at different levels. Therefore, multi-level feature extraction is performed on image samples through the backbone network, and features at multiple levels are output. The features at each level are called a set of first feature maps.

[0066] The backbone network can be a ResNet network (Residual Networks), a VGG network (Visual Geometry Group Networks), a DenseNet network (Densely Connected Convolutional Networks), etc.

[0067] Among them, the low-level features extracted by the initial layers of the ResNet network (such as Conv1d, conv2d) have higher spatial resolution and can capture the details of small targets, while the high-level features have more semantic information and can help the model better understand the global structure in the image, thereby outputting the first feature maps of different depths at different layers.

[0068] The VGG network extracts information of different scales from low layers to high layers by gradually increasing the network depth. The low-level features have higher spatial resolution, while the high-level features have more semantic information.

[0069] Each layer of the DenseNet network is connected to the output of all previous layers, so that each layer can utilize the information of all previous layers. Through dense connections, multi-level features are effectively integrated to meet the needs of targets of different scales.

[0070] In some embodiments, the backbone network is CSPDarkNet53, Figure 3A A schematic diagram of a backbone network provided by an embodiment of the present application is shown, such as Figure 3A As shown, the backbone network includes the first two residual modules, the first C2f module, the third residual module, the second C2f module, the fourth residual module, the third C2f module, the fifth residual module, the fourth C2f module and an SPPF module from top to bottom. Each module is described in turn below.

[0071] Figure 3B A schematic diagram of a residual module in a backbone network provided by an embodiment of the present application is shown, Figure 3BAs shown in the figure, the residual module (ConvModule) combines convolution operation to extract features, batch normalization (BatchNorm2d) to improve training stability, and SiLU activation function to provide nonlinear mapping. Specifically, the local pattern of the image or input features is extracted through the convolution operation to generate a convolution feature map; the convolution output is batch normalized to accelerate training and enhance the generalization ability of the model, so that the feature map remains stable during training; finally, the output of the convolution layer is nonlinearly transformed through the SiLU activation function to improve the gradient flow and make the training of the deep network more stable. In the residual module, k = 3, s = 2, p = 1 means that the number of convolution kernels is 3, the stride is 2, and the padding is 1.

[0072] Figure 3C A schematic diagram of a C2f module in a backbone network provided by an embodiment of the present application is shown, Figure 3C As shown in the figure, the C2f module (Cross-Stage Partial Block) fuses some features in the network across multiple stages through skip connections, thereby improving computational efficiency and feature expression capabilities. Specifically, the C2f module divides the input feature map into two parts, one of which is extracted through the convolution layer, and the other is directly passed. The output features of the two parts are then concat and input into the residual module to reduce the number of channels to the number before the feature map is input. In the C2f module, n BottleNecks (bottleneck layers) are used as convolution layers, where n is a positive integer.

[0073] Figure 3D A schematic diagram of a bottleneck layer in a C2f module provided in an embodiment of the present application is shown. Figure 3D As shown in the figure, BottleNeck divides the input feature map into two parts, one of which is used to extract features through two residual modules, and the other part is directly passed, and then the output features of the two parts are concatenated and output.

[0074] Figure 3E A schematic diagram of SPPF in a backbone network provided by an embodiment of the present application is shown, Figure 3E As shown, the SPPF module receives the feature map output from the fourth C2f module and uses multiple small pooling layers, such as the maximum pooling layer (MaxPool2d) to replace the pooling operation of a single large core. The purpose of the pooling operation is to reduce the number of parameters and the amount of calculation.

[0075] After the image samples are subjected to multi-level feature extraction by the backbone network, the shallow modules (such as the second C2f module) output the first group of first feature maps (referred to as feature s3), the deeper modules (such as the third C2f module) output the second group of first feature maps (referred to as feature s4), and the deepest modules (such as the fourth C2f module) output the third group of first feature maps (referred to as feature s5). Among them, the depth order of the first group of first feature maps, the second group of first feature maps, and the third group of first feature maps is from low to high, which means that the spatial resolution of features s3, s4, and s5 is ordered from high to low, and the semantic information is ordered from low to high.

[0076] In target detection, the information of foreground targets is more important than that of background, but most existing detection models detect the entire image without specifically distinguishing between foreground and background. In the output features s3, s4, and s5, the embodiment of the present application increases the weight value of the foreground target feature through a foreground excitation module (equivalent to a foreground excitation network) to highlight the foreground target information. For details, please refer to the description of step S203 below.

[0077] Step S203, step S203, determine the excitation weight corresponding to the first feature map according to the distance between the second position of the first feature map on the image sample and the first position, and strengthen the first feature map by the excitation weight to obtain the second feature map corresponding to the first feature map.

[0078] The foreground excitation network executes step S203 to excite the foreground, wherein the foreground refers to the area that needs to be marked, and the background refers to the area that does not need to be marked.

[0079] The foreground excitation network achieves the purpose of exciting the foreground by assigning different weights to the features of the foreground and background. It can be understood that exciting the foreground means increasing the proportion of foreground features and highlighting the foreground target information.

[0080] As an example, a higher excitation weight can be assigned to the foreground feature than the background feature to highlight the foreground information. Specifically, taking a set of first feature maps with a size of 2048×20×20 as an example, 20×20 spatial positions of a set of first feature maps are mapped to the original image (i.e., image samples), and the excitation weight corresponding to the spatial position whose position in the original image is the foreground is greater than the excitation weight corresponding to the spatial position whose position in the original image is the background.

[0081] As another example, the background may not be excited, that is, the excitation weight corresponding to the spatial position that is the background in the original image is 0, and the excitation weight corresponding to the spatial position that is the foreground in the original image is greater than 0.

[0082] In some embodiments, the above example can be expressed as: if the foreground excitation network determines that the second position of the first feature map on the image sample is outside the annotated border, then the excitation weight corresponding to the first feature map is determined to be 0; if the foreground excitation network determines that the second position of the first feature map on the image sample is within the annotated border, then the excitation weight corresponding to the first feature map is determined to be greater than 0.

[0083] Specifically, the second position of the first feature map on the image sample is compared with the pixel coordinates of the upper left corner and the lower right corner of the annotated border of the rectangle where the target object is located. If the second position is within the annotated border, it is determined that the first feature map is the foreground, so the excitation weight corresponding to the first feature map is determined to be greater than 0. If the second position is outside the annotated border, it is determined that the first feature map is the background, so the excitation weight corresponding to the first feature map is determined to be 0.

[0084] In some embodiments, the above-mentioned determination that the excitation weight corresponding to the first feature map is greater than 0 includes: determining that the excitation weight corresponding to the first feature map is positively correlated with the distance between the second position and the first position.

[0085] In this embodiment, the features belonging to the foreground can not only obtain a higher weight than the background features, but also the closer the features are to the center of the foreground target, the higher the weight, and the more prominent the foreground information is. Among them, the foreground target center is the center of the annotation frame, which can be understood as a rectangular frame, and the intersection of the two diagonal lines of the rectangular frame is the center of the annotation frame. Of course, when the annotation frame is represented by other graphics, the incentive weight to be allocated to the foreground can still be determined by calculating the distance between the center of the annotation frame and the second position.

[0086] Continuing with the example of a set of first feature maps with a size of 2048×20×20, the label of the foreground excitation module is a 20×20 matrix with a value between [0, 1] to match the size of the first feature map. Each value in the matrix represents the excitation weight corresponding to each feature block in the first feature map, and the value of the excitation weight is [0, 1]. Assume that the position (xl, yl) on the label is mapped to an image block on the original image, and its center point is (xc, yc). If (xc, yc) is not in any foreground target frame, the value score of the position (xl, yl) is 0; if (xc, yc) is inside any foreground target frame, and the center point of the foreground target frame is (xg, yg), the width is w, and the height is h, then the value score of the position (xl, yl) is calculated by the following formula:

[0087]

[0088] Among them, score corresponds to each element in the matrix.

[0089] Figure 4 A schematic diagram of a prospect incentive module provided in an embodiment of the present application is shown, such as Figure 4 As shown,

[0090] The size of feature s5 is 2048×20×20. It is reduced in dimension by a convolution module of conv2d, k=1. A part of it is sequentially passed through a reshape module, a Trasformer Encoder module, a reshape module, and a convolution module of conv2d, k=1 to obtain a hotmap. Hotmap is a matrix containing excitation weights that needs to be output by the Trasformer Encoder module. The shape of the tensor is adjusted in the reshape module. The size of the hotmap is 1×20×20, which is used to assign excitation weights to the 400 features of feature s5, and multiply the reduced feature s5 (256×20×20) and add it to the reduced feature s5 (256×20×20) to obtain feature F5. Specifically, the strengthening step includes: multiplying the feature of each spatial position in the first feature map (feature s5) by the excitation weight corresponding to the spatial position, and superimposing it with the feature of the spatial position to obtain a second feature map (feature F5) corresponding to the first feature map. Feature F5 highlights the foreground object compared to feature s5.

[0091] The size of feature s4 is 1024×40×40. Different from the above feature s5, since the hotmap size is designed according to the size of feature s5, the hotmap needs to be upsampled and then multiplied with the reduced feature s4 (256×40×40) and added to the reduced feature s4 (256×40×40) to obtain feature F4.

[0092] The size of feature s3 is 512×80×80. The hotmap is further upsampled and then multiplied with the reduced-dimensional feature s3 (256×80×80) and added to the reduced-dimensional feature s3 (256×80×80) to obtain feature F3.

[0093] During the training phase, the foreground excitation module is trained separately, the BackBone is fixed, and the loss between the label and the hotmap is calculated, so that only the Transformer branch is trained. Specifically, while the weight parameters of the backbone network are fixed, the mean square error loss is constructed to measure the error between the excitation weight output by the foreground excitation network and the true weight label; with the goal of minimizing the mean square error loss, the foreground excitation network is trained to obtain an optimized target detection model. Training only the Transformer branch means that only the parameters of the Transformer module are updated during the training process, while the backbone network remains unchanged. This training strategy is used to train the model step by step, that is, to optimize a certain part (such as the Transformer module) first to ensure that the model can better perform complex feature association modeling.

[0094] Step S204: perform border prediction based on the second feature map to obtain a detection border, calculate the border loss and the category loss, and update the weight parameters according to the border loss and the category loss to obtain a target detection model.

[0095] The border loss refers to the difference between the predicted border and the annotated border, and the category loss refers to the difference between the predicted category and the annotated category. The annotated border can be represented by the first position, which is the pixel coordinates of the upper left corner and the lower right corner of the annotated border. The target detection model is trained by updating the weight parameters of the network model with the goal of minimizing the border loss and category loss.

[0096] In some of the embodiments, the above-mentioned border prediction based on the second feature map to obtain the detection border further includes: fusing the second feature map by a multi-scale feature fusion module to obtain a fusion feature; performing border prediction based on the fusion feature to obtain border information and category confidence of multiple predicted borders, and screening out the detection border from the multiple predicted borders.

[0097] Figure 5 A schematic diagram of a multi-scale feature fusion module provided in an embodiment of the present application is shown, Figure 5 As shown, feature F3 is reduced in dimension by a convolution module with conv2d, k=1, and then downsampled. Feature F5 is reduced in dimension by a convolution module with conv2d, k=1, and the downsampled features are fused through a Fusion module (or fusion module) to obtain the fused features of F3 and F5.

[0098] Figure 6 A schematic diagram of a fusion module provided in an embodiment of the present application is shown, such as Figure 6As shown in the figure, the Fusion module concats the two inputs into two outputs, one of which is reduced in dimension by a conv2d, k=1 convolution module, and the other is reduced in dimension by a conv2d, k=1 convolution module and then passed through three RepBlock modules (RepetitionBlock modules) to reduce the number of parameters, and the two output results are added together as the output of the Fusion module.

[0099] Continue to refer Figure 5 Similarly, the features of F3 and F4 fusion and the features of F4 and F5 fusion can be obtained. Then, the features of F3 and F5 fusion are reduced in dimension by the conv2d, k=1 convolution module, and then upsampled and downsampled respectively. The upsampled results are fused with the features of F3 and F4 fusion and reduced in dimension by the conv2d, k=1 convolution module, and the results are input into the flatten+FC module, where Flatten flattens the multidimensional data into a one-dimensional vector, and FC (Fully Connected Layer) performs linear transformation and nonlinear activation on the one-dimensional vector to obtain features of 6400×256 size. Similarly, the features of F3 and F5 fusion can be processed to obtain features of 1600×256 size, and the features of F4 and F5 fusion can be processed to obtain features of 400×256 size, and the three large-size features are concat to output fusion features of 8400×256 size.

[0100] For example, the detection bounding box can be filtered out from the predicted bounding box by using the NMS (Non-Maximum Suppression) technology. Specifically, in the object detection task, an object may be detected by multiple predicted bounding boxes, which may have different positions and sizes but represent the same object. NMS can suppress these overlapping bounding boxes and only retain the bounding box with the highest confidence, thereby obtaining the final detection bounding box.

[0101] As another example, the top 300 features with the highest IOU (Intersection over Union) scores may be selected as candidate objects, and then the detection bounding box may be selected using the Hungarian algorithm.

[0102] In the detection head, it is composed of MLP. The 8400*256 features output by the feature fusion module are sorted into 8400*N after passing through MLP (multi-layer perceptron), where:

[0103] N = c (number of detection categories) + 1 (background category) + 4 (detection box coordinates)

[0104] In common object detection algorithms, NMS is performed in this step to select the k objects with the highest confidence as the final output. Since NMS is less effective for highly overlapping objects, such methods do not perform well in complex scenes.

[0105] This embodiment abandons NMS, selects 300 predicted bounding boxes with the highest IOU overlap with the real target box as candidate targets, and then uses the Hungarian algorithm for matching to obtain the final detection bounding box.

[0106] In some of the embodiments, the present application does not select the IOU loss, but uses a new scale and position sensitive loss to improve the accuracy of algorithm detection. The formula of the new scale and position sensitive loss is as follows:

[0107] Loss = L scale -L location

[0108] L scale is the scale-sensitive loss, and its formula is:

[0109]

[0110] L location It is a position-sensitive loss. To calculate this loss, the target coordinates need to be converted into polar coordinates. Assume that the center point of the predicted bounding box of the target is (x p ,y p ), the real target box of the target is (x gt ,y gt ), conversion polar coordinate formula:

[0111]

[0112] Then L location It can be expressed as:

[0113]

[0114] In the above steps S201 to S204, the network model is trained to obtain a target detection model, and then the model can be applied through step S205. For details, please refer to the following description.

[0115] Step S205: input the image to be detected into the target detection model to perform target detection, and annotate the target object in the image to be detected based on the target detection result.

[0116] Exemplarily, if the target object is a pedestrian, the target detection model detects the pedestrian in the image to be detected and outputs a labeling result for the pedestrian, where the labeling result includes labeling of the category and the first position, where the first position is represented by coordinates.

[0117] During the training process, the target detection model takes into account that the shape, color, texture, etc. of the foreground target are often quite different from the background, and highlighting the foreground features can improve the recognition ability of the model. In addition, considering that the fusion of multiple levels of features can obtain richer semantic information and fine-grained spatial information, the trained target detection model has a higher recognition accuracy in small targets and multi-target occlusion scenarios.

[0118] In the embodiment described in the above steps S201 to S205, a pre-annotated image sample is first obtained, and the annotation includes annotating the category of the target object and the first position of the target object in the image sample; the image sample is then input into a preset backbone network for feature extraction to obtain multiple sets of first feature maps of different depths; the foreground excitation network determines the excitation weight corresponding to the first feature map according to the distance between the second position of the first feature map on the image sample and the first position, and strengthens the first feature map by the excitation weight to obtain a second feature map corresponding to the first feature map; a border prediction is performed based on the second feature map to obtain a detection border, the border loss and the category loss are calculated, and the weight parameters are updated according to the border loss and the category loss to obtain a target detection model for target detection; the image to be detected is then input into the target detection model for target detection, and the target object is annotated in the image to be detected. This application strengthens the foreground target by excitation weights to highlight the foreground features. In this way, the model's ability to recognize targets in multi-target occlusion scenarios is improved. In addition, features of different scales are extracted and fused so that the fused features have richer semantic information and fine-grained spatial information, thereby improving the model's detection effect on small targets.

[0119] Figure 7 A structural block diagram of a target detection system provided in an embodiment of the present application is shown. Figure 7 As shown, the system includes:

[0120] A sample acquisition module 71 is used to acquire a pre-annotated image sample, wherein the annotation includes annotating the category of the target object and the first position of the target object in the image sample;

[0121] A feature extraction module 72 is used to extract features from the acquired image samples to obtain multiple groups of first feature maps at different depths;

[0122] A foreground excitation module 73 is used to determine an excitation weight corresponding to the first feature map according to the distance between the second position of the first feature map on the image sample and the first position, and to strengthen the first feature map by the excitation weight to obtain a second feature map corresponding to the first feature map;

[0123] A parameter updating module 74 is used to perform border prediction based on the second feature map to obtain a detection border, calculate a border loss and a category loss, and update a weight parameter according to the border loss and the category loss to obtain a target detection model;

[0124] The model application module 75 is used to input the image to be detected into the target detection model to perform target detection, and to mark the target object in the image to be detected based on the target detection result.

[0125] In some embodiments, the feature extraction module 72 is further used to input the image sample into the first convolution module, the second convolution module, the first C2f module, the third convolution module, and the second C2f module, and then perform layer-by-layer feature extraction to obtain a first set of first feature maps;

[0126] After the first group of first feature maps passes through the fourth convolution module and the third C2f module to perform layer-by-layer feature extraction, a second group of first feature maps is obtained;

[0127] After the second group of first feature maps are subjected to layer-by-layer feature extraction by the fifth convolution module, the fourth C2f module, and the SPPF module, a third group of first feature maps are obtained;

[0128] Among them, the depth sorting of the first group of first feature maps, the second group of first feature maps, and the third group of first feature maps is from low to high.

[0129] In some embodiments, the foreground excitation module 73 is further configured to determine that the excitation weight corresponding to the first feature map is 0 if the second position of the first feature map on the image sample is outside the marked border;

[0130] If the second position of the first feature map on the image sample is within the marked border, it is determined that the excitation weight corresponding to the first feature map is greater than 0.

[0131] In some embodiments, the foreground excitation module 73 is further configured to determine an excitation weight corresponding to the first feature map, which is positively correlated with a distance between the second position and the first position.

[0132] In some embodiments, the foreground excitation module 73 is further used to multiply the feature of each spatial position in the first feature map by the excitation weight corresponding to the spatial position, and then superimpose the product with the feature of the spatial position to obtain a second feature map corresponding to the first feature map.

[0133] In some of the embodiments, the foreground excitation module 73 is further used to construct a mean square error loss to measure the error between the excitation weight output by the foreground excitation network and the true weight label when the weight parameters of the backbone network are fixed;

[0134] With the goal of minimizing the mean square error loss, the foreground excitation network is trained to obtain the optimized object detection model.

[0135] In some embodiments, the parameter updating module 74 is further used to fuse the second feature map to obtain a fused feature;

[0136] Based on the fusion features, bounding box prediction is performed to obtain bounding box information and category confidence of multiple predicted bounding boxes, and the detection bounding box is screened out from multiple predicted bounding boxes.

[0137] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0138] In this embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0139] Optionally, the computer device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0140] Optionally, in this embodiment, the processor may be configured to execute the following steps S1 to S5 through a computer program.

[0141] S1. Obtain a pre-annotated image sample, where the annotation includes annotating the category of the target object and the first position of the target object in the image sample.

[0142] S2. Perform feature extraction on the acquired image samples to obtain multiple groups of first feature maps with different depths.

[0143] S3. Determine an excitation weight corresponding to the first feature map according to a distance between a second position of the first feature map on the image sample and the first position, and strengthen the first feature map by the excitation weight to obtain a second feature map corresponding to the first feature map.

[0144] S4. Based on the second feature map, a border prediction is performed to obtain a detection border, a border loss and a category loss are calculated, and a weight parameter is updated according to the border loss and the category loss to obtain a target detection model.

[0145] S5. Input the image to be detected into the target detection model for target detection, and mark the target object in the image to be detected based on the target detection result.

[0146] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.

[0147] In addition, in combination with the method provided in the above embodiments, a storage medium may be provided in this embodiment to implement the method. The storage medium stores a computer program; when the computer program is executed by a processor, any target detection method in the above embodiments is implemented.

[0148] The embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer executes each function or step executed by the processor in the above method embodiment.

[0149] It should be understood that the specific embodiments described herein are only used to explain the application, rather than to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of this application.

[0150] Obviously, the drawings are only some examples or embodiments of the present application. For ordinary technicians in the field, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it is understandable that although the work done in this development process may be complicated and lengthy, for ordinary technicians in the field, certain changes in design, manufacturing or production based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient content disclosed in this application.

[0151] The term "embodiment" in this application refers to a specific feature, structure or characteristic described in conjunction with the embodiment that can be included in at least one embodiment of the present application. The appearance of this phrase in various locations in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is clearly or implicitly understood by those of ordinary skill in the art that the embodiments described in this application can be combined with other embodiments without conflict.

[0152] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of patent protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.

Claims

1. A target detection method, characterized in that: The method comprises: Acquire a pre-annotated image sample, wherein the annotation includes annotating a category of a target object and a first position of the target object in the image sample; Performing feature extraction on the acquired image samples to obtain multiple groups of first feature maps at different depths; Determine an excitation weight corresponding to the first feature map according to a distance between a second position of the first feature map on the image sample and the first position, and strengthen the first feature map by using the excitation weight to obtain a second feature map corresponding to the first feature map; Performing border prediction based on the second feature map to obtain a detection border, calculating a border loss and a category loss, and updating a weight parameter according to the border loss and the category loss to obtain a target detection model; The image to be detected is input into the target detection model for target detection, and the target object in the image to be detected is marked based on the target detection result.

2. The target detection method according to claim 1, characterized in that: The first position is indicated by a marked border, and determining the excitation weight corresponding to the first feature map according to the distance between the second position of the first feature map on the image sample and the first position includes: If the second position of the first feature map on the image sample is outside the marked border, determining that the excitation weight corresponding to the first feature map is 0; If the second position of the first feature map on the image sample is within the marked border, it is determined that the excitation weight corresponding to the first feature map is greater than 0.

3. The target detection method according to claim 2, characterized in that: The determining that the excitation weight corresponding to the first feature map is greater than 0 includes: Determining an excitation weight corresponding to the first feature map is positively correlated with a distance between the second position and the first position.

4. The target detection method according to claim 1, characterized in that: The step of strengthening the first feature map by using the excitation weight to obtain a second feature map corresponding to the first feature map includes: The feature of each spatial position in the first feature map is multiplied by the excitation weight corresponding to the spatial position, and then superimposed with the feature of the spatial position to obtain a second feature map corresponding to the first feature map.

5. The target detection method according to any one of claims 1 to 4, characterized in that: After updating the weight parameters according to the border loss and the category loss, the method further includes: While keeping the weight parameters of the backbone network fixed, the mean square error loss is constructed to measure the error between the excitation weight output by the foreground excitation network and the true weight label. With the goal of minimizing the mean square error loss, the foreground excitation network is trained to obtain the optimized target detection model.

6. The target detection method according to claim 1, characterized in that: The method further comprises: The backbone network includes a CSPDarknet53 network, and the CSPDarknet53 network includes a first convolution module, a second convolution module, a third convolution module, a fourth convolution module, a fifth convolution module, a first C2f module, a second C2f module, a third C2f module, a fourth C2f module and an SPPF module. The acquired image samples are subjected to feature extraction to obtain multiple groups of first feature maps of different depths, including: Input the image sample into the first convolution module, the second convolution module, the first C2f module, the third convolution module, and the second C2f module, and perform layer-by-layer feature extraction to obtain a first group of first feature maps; The first group of first feature maps are subjected to a fourth convolution module and a third C2f module for layer-by-layer feature extraction to obtain a second group of first feature maps; After performing layer-by-layer feature extraction on the second group of first feature maps through a fifth convolution module, a fourth C2f module, and an SPPF module, a third group of first feature maps is obtained; The depth order of the first group of first feature maps, the second group of first feature maps, and the third group of first feature maps is from low to high.

7. The target detection method according to claim 1, characterized in that: The performing border prediction based on the second feature map to obtain a detection border includes: Fusing the second feature maps to obtain fused features; Based on the fusion features, bounding box prediction is performed to obtain bounding box information and category confidence of multiple predicted bounding boxes, and a detection bounding box is screened out from the multiple predicted bounding boxes.

8. A target detection system, characterized in that: include: A sample acquisition module, used to acquire a pre-annotated image sample, wherein the annotation includes annotating a category of a target object and a first position of the target object in the image sample; A feature extraction module, used to extract features from the acquired image samples to obtain multiple groups of first feature maps of different depths; a foreground excitation module, configured to determine an excitation weight corresponding to the first feature map according to a distance between a second position of the first feature map on the image sample and the first position, and to strengthen the first feature map by using the excitation weight to obtain a second feature map corresponding to the first feature map; A parameter updating module, configured to perform border prediction based on the second feature map to obtain a detection border, calculate a border loss and a category loss, and update a weight parameter according to the border loss and the category loss to obtain a target detection model; The model application module is used to input the image to be detected into the target detection model for target detection, and to mark the target object in the image to be detected based on the target detection result.

9. A computer device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the target detection method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to execute the target detection method according to any one of claims 1 to 7 when running.

Images