Target detection model training method and target detection method

Abstract

This application discloses a method for training an object detection model and an object detection method, belonging to the field of deep learning technology. The object detection model training method includes: acquiring multiple training samples; inputting each training sample into the model to be trained to obtain a predicted target heatmap; determining a first loss function heatmap based on the predicted target heatmap and a standard target heatmap; correcting the first loss function heatmap based on at least one weight heatmap to obtain a second loss function heatmap; adjusting the model parameters of the model to be trained based on the second loss function heatmap, and iteratively training the parameter-adjusted model using training samples until the training stopping condition is met, thus obtaining a trained object detection model. Using the object detection model training method and object detection method provided in this application, object detection can be performed according to the object detection requirements in actual application scenarios, improving the targeting accuracy of object detection.

Description

Technical Field

[0001] This application belongs to the field of deep learning technology, and in particular relates to a method for training an object detection model and an object detection method. Background Technology

[0002] In recent years, deep learning-based object detection models have made great strides. Existing object detection models are designed and trained primarily with the ability to detect all objects in mind.

[0003] However, in practical applications, different scenarios have different requirements for target detection. It is not necessary to detect all targets. In some scenarios, some targets do not need to be detected, while others must be detected accurately. For example, in autonomous driving applications, small targets far away from the vehicle do not need to be detected, and when multiple pedestrians overlap, only one needs to be detected. However, targets close to the front of the vehicle must be detected accurately.

[0004] This means that existing target detection models cannot meet the different target detection needs in real-world application scenarios. Summary of the Invention

[0005] This application provides a target detection model training method and a target detection method, which can at least solve the problem that the target detection models in the prior art cannot meet the different target detection needs in actual application scenarios.

[0006] In a first aspect, embodiments of this application provide a method for training an object detection model, the method comprising:

[0007] Acquire multiple training samples, which include sample images and standard target heatmaps corresponding to the sample images;

[0008] For each training sample, perform the following steps:

[0009] The training samples are input into the model to be trained to obtain the predicted target heatmap output by the model to be trained;

[0010] Based on the predicted target heatmap and the standard target heatmap, determine the first loss function heatmap of the model to be trained;

[0011] The first loss function heatmap is corrected based on at least one weighted heatmap to obtain the second loss function heatmap. The weighted heatmap is obtained by setting different weights for different targets in the heatmap, and the weights are determined according to the target detection requirements.

[0012] Based on the heatmap of the second loss function, the model parameters of the model to be trained are adjusted, and the model to be trained with adjusted parameters is iteratively trained using training samples until the training stopping condition is met, thus obtaining a trained object detection model.

[0013] Secondly, embodiments of this application provide a target detection method, the method comprising:

[0014] Acquire the image to be detected;

[0015] The image to be detected is input into the target detection model to obtain the target heatmap corresponding to the image to be detected. The target detection model is the model trained by any one of the methods in claims 1-15.

[0016] Thirdly, embodiments of this application provide a target detection model training apparatus, the apparatus comprising:

[0017] The sample acquisition module is used to acquire multiple training samples, which include sample images and standard target heatmaps corresponding to the sample images.

[0018] The first sample input module is used to perform the following for each training sample: input the training sample into the model to be trained, and obtain the predicted target heatmap output by the model to be trained;

[0019] The first determination module is used to perform the following for each training sample: determine the first loss function heatmap of the model to be trained based on the predicted target heatmap and the standard target heatmap;

[0020] The correction module is used to perform the following for each training sample: correct the first loss function heatmap according to at least one weight heatmap to obtain the second loss function heatmap. The weight heatmap is a graph obtained by setting different weights for different targets in the heatmap. The weights are determined according to the target detection requirements.

[0021] The first adjustment module is used to perform the following for each training sample: adjust the model parameters of the model to be trained according to the second loss function heatmap, and use the training samples to iteratively train the model to be trained with adjusted parameters until the training stopping condition is met, so as to obtain a trained object detection model.

[0022] Fourthly, embodiments of this application provide a target detection device, which includes:

[0023] The image acquisition module is used to acquire the image to be detected;

[0024] The first image input module is used to input the image to be detected into the target detection model to obtain the target heatmap corresponding to the image to be detected. The target detection model is a model trained by any one of the methods in claims 1-15.

[0025] Fifthly, embodiments of this application provide an electronic device, the device including: a processor and a memory storing computer program instructions;

[0026] When the processor executes computer program instructions, it implements the object detection model training method as shown in any embodiment of the first aspect or the object detection method as shown in any embodiment of the second aspect.

[0027] In a sixth aspect, embodiments of this application provide a computer storage medium storing computer program instructions, which, when executed by a processor, implement the target detection model training method shown in any embodiment of the first aspect or the target detection method shown in any embodiment of the second aspect.

[0028] In a seventh aspect, embodiments of this application provide a computer program product in which instructions, when executed by a processor of an electronic device, cause the electronic device to perform the target detection model training method shown in any embodiment of the first aspect or the target detection method shown in any embodiment of the second aspect.

[0029] The target detection model training method and target detection method of this application can acquire multiple training samples. For each training sample, the training sample is input into the model to be trained to obtain a predicted target heatmap output by the model. Then, based on the predicted target heatmap and the standard target heatmap included in the training samples, a first loss function heatmap of the model to be trained is determined. The first loss function heatmap is then corrected based on at least one weight heatmap to obtain a second loss function heatmap. Based on the second loss function heatmap, the model parameters of the model to be trained are adjusted, and the model to be trained with adjusted parameters is iteratively trained using training samples until the training stopping condition is met, resulting in a trained target detection model. Since the weight heatmap is obtained by assigning different weights to different targets in the heatmap, and the weights are determined according to the target detection requirements, the target detection model trained in this way can perform target detection according to the target detection requirements in the actual application scenario, improving the targeting of target detection. Attached Figure Description

[0030] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 This is a flowchart of a target detection model training method provided in one embodiment of this application;

[0032] Figure 2 This is a schematic diagram of the internal processing of a model to be trained according to an embodiment of this application;

[0033] Figure 3a This is a schematic diagram of a standard target heat map provided in one embodiment of this application;

[0034] Figure 3b This is a schematic diagram of a weighted heatmap provided in one embodiment of this application;

[0035] Figure 3c This is a schematic diagram of another weighted heatmap provided in one embodiment of this application;

[0036] Figure 4 This is a schematic diagram illustrating the process of determining a fifth loss function heatmap according to an embodiment of this application;

[0037] Figure 5 This is a flowchart of a target detection method provided in one embodiment of this application;

[0038] Figure 6 This is a schematic diagram of the structure of a target detection model training device provided in one embodiment of this application;

[0039] Figure 7 This is a schematic diagram of the structure of a target detection device provided in one embodiment of this application;

[0040] Figure 8 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application. Detailed Implementation

[0041] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0042] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0043] Figure 1 The diagram illustrates a flowchart of a target detection model training method according to an embodiment of this application. It should be noted that this target detection model training method can be applied to target detection model training devices, such as… Figure 1 As shown, the training method for this object detection model may include the following steps:

[0044] S110, acquire multiple training samples;

[0045] S120, For each training sample, perform the following: Input the training sample into the model to be trained, and obtain the predicted target heatmap output by the model to be trained;

[0046] S130, for each training sample, perform the following: determine the first loss function heatmap of the model to be trained based on the predicted target heatmap and the standard target heatmap;

[0047] S140, For each training sample, perform the following: Correct the first loss function heatmap based on at least one weight heatmap to obtain the second loss function heatmap;

[0048] S150, for each training sample, perform the following: adjust the model parameters of the model to be trained according to the heatmap of the second loss function, and use the training samples to iteratively train the model to be trained after parameter adjustment until the training stopping condition is met, so as to obtain a trained object detection model.

[0049] This allows for the acquisition of multiple training samples. For each training sample, it is input into the model to be trained, resulting in a predicted target heatmap output by the model. Then, based on the predicted target heatmap and the standard target heatmaps included in the training samples, a first loss function heatmap is determined for the model. This first loss function heatmap is then corrected based on at least one weight heatmap to obtain a second loss function heatmap. The model parameters are adjusted according to this second loss function heatmap, and the adjusted model is iteratively trained using the training samples until the training stopping condition is met, resulting in a trained target detection model. Since the weight heatmap is obtained by assigning different weights to different targets in the heatmap, and these weights are determined based on the target detection requirements, the target detection model trained in this way can perform target detection according to the specific needs of the application scenario, improving the targeting accuracy of target detection.

[0050] Regarding S110, training samples may include sample images and corresponding standard target heatmaps. These training samples can be used to train the model to be trained. The model to be trained can be an untrained target detection model; training this model yields a trained target detection model. Sample images can be images acquired through a camera or other image sensor. The standard target heatmap can be a heatmap obtained by manually marking the locations of targets in the sample images; alternatively, it can be a heatmap obtained by marking the locations of targets in the sample images using other methods, without limitation.

[0051] Regarding S120, the training samples are input into the training model to obtain a predicted target heatmap. This training model can construct the predicted target heatmap by using a Gaussian distribution to build parameters at the locations where targets exist; other distributions, such as a uniform distribution, can also be used, without limitation. The predicted target heatmap is a heatmap output by the training model with the target locations marked. The predicted target heatmap of this sample image usually differs somewhat from the standard target heatmap. The target location can be the location of the target's center point, for example, the coordinates of the target's center point.

[0052] In some implementations, the predicted target heatmap corresponding to the sample image can be obtained by extracting image features from the sample image.

[0053] Here, the model to be trained can be controlled to extract image features from sample images, thereby obtaining the predicted target heatmap corresponding to the sample images.

[0054] Furthermore, by controlling the model under training to extract image features from sample images, we can also obtain the predicted bias heatmap and predicted width-height heatmap corresponding to the sample images. The bias heatmap can be a heatmap representing the target's position drift; due to the existence of lateral and vertical drift, the bias heatmap has 2 channels. The width-height heatmap can be a heatmap representing the target's width and height, also with 2 channels.

[0055] It should be noted that all heat maps involved in the embodiments of this application have the same size, that is, all heat maps have the same width and height.

[0056] Thus, by controlling the model to be trained to extract image features from sample images, the model can output a predicted target heatmap corresponding to the sample images.

[0057] In some implementations, to make the predicted target heatmap output by the model to be trained more accurate and improve processing efficiency, the above-mentioned S120 may specifically include:

[0058] Extract image features from the sample image to obtain the first feature image;

[0059] The first feature image is downsampled by different downsampling factors to obtain multiple second feature images;

[0060] Image features of multiple second feature images are extracted separately to obtain multiple third feature images; the multiple third feature images are then fused to obtain a heatmap of the predicted target.

[0061] Here, multiple second feature images can be feature images of different sizes. The image features of different second feature images can be further extracted by the deep interlayer aggregation module to obtain third feature images. Then, these multiple third feature images are fused to obtain the predicted target heatmap.

[0062] In addition to obtaining the predicted target heatmap, we can also obtain the predicted bias heatmap and the predicted width and height heatmap.

[0063] In some examples, feature extraction can be performed on a sample image of size 512×512 to obtain a first feature image of size 512×512. Then, the first feature image is downsampled by a factor of 8, 16, and 32 respectively to obtain three second feature images of sizes 64×64, 32×32, and 16×16 respectively. The image features of these three second feature images are then extracted to obtain three third feature images. Finally, these three third feature images are fused to obtain the predicted target heatmap.

[0064] Thus, by extracting image features from sample images to obtain a first feature image, and then downsampling the first feature image to obtain multiple second feature images of different sizes, and then extracting image features from the multiple second feature images to obtain multiple third feature images, and then fusing the multiple third feature images to obtain a predicted target heatmap, the accuracy of the predicted target heatmap can be improved. Downsampling the first feature image to obtain multiple second feature images can make the size of the second feature images smaller than the first feature image. Feature extraction from the smaller feature images can reduce the processing difficulty and improve the processing efficiency.

[0065] In some implementations, since the multiple second feature images have different sizes, the multiple third feature images obtained based on the multiple second feature images also have different sizes. In order to fuse the multiple third feature images of different sizes together, the above-mentioned fusion of multiple third feature images to obtain the predicted target heatmap may specifically include:

[0066] Multiple third feature images are adjusted to the same size to obtain multiple fourth feature images;

[0067] By fusing multiple fourth feature images, a heatmap of the predicted target is obtained.

[0068] Here, multiple third feature images can be adjusted to the same size by the upsampling module to obtain multiple fourth feature images. Then, the multiple fourth feature images are fused by the feature stitching module to obtain the predicted target heatmap.

[0069] In some examples, the upsampling module can be used to upsample three third feature images with sizes of 64×64, 32×32 and 16×16 to obtain three fourth feature images with a size of 128×128. Then, these three fourth feature images with a size of 128×128 are fused to obtain the predicted target heatmap.

[0070] Thus, through the above process, multiple third feature images of different sizes can be fused to obtain a heatmap of the predicted target.

[0071] In some examples, the internal processing of the model to be trained can be as follows: Figure 2As shown, the sample image 201 is input into the backbone network 202 to extract image features, resulting in a first feature image. Then, the first feature image is downsampled by different downsampling factors to obtain second feature images with downsampling factors of 32x, 16x, and 8x. The first deep inter-layer aggregation module 203 extracts features from the 32x downsampled second feature image to obtain a first sub-feature image. The second deep inter-layer aggregation module 204 extracts features from the 16x downsampled second feature image to obtain a second sub-feature image. The third deep inter-layer aggregation module 205 extracts features from the 8x downsampled second feature image to obtain a third sub-feature image. Then, the first upsampling module 206 upsamples the first sub-feature image by 2x, adjusting it to the same size as the second sub-feature image. Finally, the second feature image is processed by the first feature stitching module 207. The sub-feature image and the resized first sub-feature image are stitched together and image features are extracted again to obtain the fourth sub-feature image. Then, the second upsampling module 208 upsamples the fourth sub-feature image by 2 times and adjusts the fourth sub-feature image to a feature image of the same size as the third sub-feature image. Then, the second feature stitching module 209 stitches the third sub-feature image and the resized fourth sub-feature image together and extracts image features again to obtain the fifth sub-feature image. Finally, the third upsampling module 210 upsamples the fifth sub-feature image by 2 times. The bias detection head 214 outputs a predicted bias heatmap 211 that is downsampled by 4 times compared to the first feature image. The width and height detection head 215 outputs a predicted width and height heatmap 212 that is downsampled by 4 times compared to the first feature image. The positioning detection head 216 outputs a predicted target heatmap 213 that is downsampled by 4 times compared to the first feature image.

[0072] In some implementations, in order to adjust multiple third feature images of different sizes to the same size, the above-mentioned adjustment of multiple third feature images to the same size to obtain multiple fourth feature images may specifically include:

[0073] Multiple third feature images are adjusted to multiple fifth feature images of the same size by bilinear interpolation upsampling;

[0074] By performing convolution operations, batch normalization, and activation functions, the feature values ​​of multiple fifth feature images are adjusted to obtain multiple fourth feature images.

[0075] Here, the upsampling module can include a bilinear interpolation upsampling submodule, a convolution submodule, a batch normalization submodule, and an activation function submodule. Specifically, bilinear interpolation upsampling can be used to adjust multiple third feature images into multiple fifth feature images of the same size. Then, through convolution operations, batch normalization processing, and activation functions, the feature values ​​of the multiple fifth feature images can be adjusted to obtain multiple fourth feature images.

[0076] In this way, through the above process, multiple third feature images of different sizes can be adjusted to the same size to obtain multiple fourth feature images.

[0077] In some embodiments, in addition to the methods described above, to adjust multiple third feature images of different sizes to the same size, thereby obtaining multiple fourth feature images, the method may further include:

[0078] Multiple third feature images are resized to the same size by deconvolution, resulting in multiple fourth feature images.

[0079] Here, the upsampling module may include a deconvolution module, which adjusts multiple third feature images to the same size through deconvolution operations, thereby obtaining multiple fourth feature images.

[0080] In this way, through the above process, multiple third feature images of different sizes can be adjusted to the same size to obtain multiple fourth feature images.

[0081] In some implementations, to fuse multiple fourth feature images, the aforementioned fusion of multiple fourth feature images to obtain a predicted target heatmap may specifically include:

[0082] The sixth feature image is obtained by performing element-wise addition on multiple fourth feature images;

[0083] Image features of the sixth feature image are extracted by convolution operation to obtain the predicted target heatmap.

[0084] Here, the feature concatenation module can include an element-wise addition module and a convolution module. Element-wise addition can be performed on multiple fourth feature images, summing elements at corresponding positions in the multiple fourth feature images to obtain a sixth feature image. Then, convolution operations are used to further extract image features from the sixth feature image to obtain the predicted target heatmap.

[0085] In this way, through the above process, multiple fourth feature images can be fused together to obtain a heatmap of the predicted target.

[0086] Regarding S130, since the predicted target heatmap output by the model to be trained usually differs from the standard target heatmap, a first loss function heatmap for the model to be trained can be determined based on the predicted target heatmap and the standard target heatmap. This first loss function heatmap includes the loss function value corresponding to each target.

[0087] In some implementations, since the areas without targets in the sample image are usually more numerous than the areas with targets, an imbalance between positive and negative samples may occur. To balance the positive and negative samples, S130 may specifically include:

[0088] Based on the attention loss function, calculate the loss function value corresponding to each target;

[0089] The first loss function heatmap is determined based on the loss function value corresponding to each target.

[0090] Here, focal loss can be used to calculate the loss function value corresponding to each target, thereby determining the thermal value of the first loss function.

[0091] Thus, since focal loss can balance positive and negative samples, it can solve the problem of imbalance between positive and negative samples in the sample image.

[0092] Regarding S140, the weighted heatmap can be obtained by assigning different weights to different targets in the heatmap, and the weights can be determined according to the target detection requirements. The weighted heatmap can also be constructed using a Gaussian distribution, and its Gaussian radius can be the same as or slightly smaller than the Gaussian radius of the predicted target heatmap. Adjusting the first loss function heatmap based on this weighted heatmap can make the model under training better suited to the target detection requirements.

[0093] In some implementations, in order to correct the first loss function heatmap based on at least one weighted heatmap, S140 may specifically include:

[0094] For each target, the loss function value corresponding to the target in the first loss function heatmap is weighted according to the weight corresponding to the target in the weighted heatmap to obtain the second loss function heatmap.

[0095] Here, for each target, the loss function value corresponding to the target in the first loss function heatmap can be weighted according to the weight corresponding to the target in the weight heatmap. That is, the weight corresponding to the target in the weight heatmap is multiplied by the loss function value corresponding to the target in the first loss function heatmap to obtain the second loss function heatmap.

[0096] When there are multiple weighted heatmaps, a fused weighted heatmap can be obtained by fusing the multiple weighted heatmaps. Based on the weight of the target in the fused weighted heatmap, the loss function value corresponding to the target in the first loss function heatmap is weighted and calculated.

[0097] Thus, through the above process, the heatmap of the first loss function can be corrected based on at least one weighted heatmap.

[0098] In some implementations, there may be multiple weighted heatmaps. To correct the first loss function heatmap based on these multiple weighted heatmaps, step S140 may specifically include:

[0099] When there are multiple weighted heatmaps, the multiple weighted heatmaps are merged to obtain a merged weighted heatmap;

[0100] Based on the fused weighted heatmap, the first loss function heatmap is corrected to obtain the second loss function heatmap.

[0101] Here, multiple weighted heatmaps are used to address different target detection needs. Weights can be assigned to multiple weighted heatmaps, and they can be fused by weighted calculations to obtain a fused weighted heatmap. Then, the first loss function heatmap is modified based on the fused weighted heatmap to obtain the second loss function heatmap.

[0102] In some examples, there are two weighted heatmaps. The weights in the first weighted heatmap are determined based on the first target detection requirement, and the weights in the second weighted heatmap are determined based on the second target detection requirement. The first target detection requirement might be to focus on larger targets, while the second target detection requirement might be to focus on more clearly defined targets. If users are more concerned with larger targets than clearly defined targets, then the weights in the first weighted heatmap can be set to be greater than the weights in the second weighted heatmap.

[0103] In some examples, the fused weighted heatmap is calculated as follows:

[0104]

[0105] Where Z can be the weight value corresponding to each target in the fused weighted heatmap, and a i W can be the weight of the i-th weighted heatmap. i It can be the weight value corresponding to each target in the i-th weighted heatmap.

[0106] Thus, through the above process, multiple weighted heatmaps can be merged when there are multiple weighted heatmaps, thereby correcting the first loss function heatmap based on the multiple weighted heatmaps.

[0107] In some implementations, the weights in the weight heatmap can be determined according to the target detection requirements. To determine the weight heatmap, the target detection model training method may further include, before S140:

[0108] Based on the size characteristics of each target in the sample image, determine the weight value corresponding to each target; and / or, based on the sharpness characteristics of each target in the sample image, determine the weight value corresponding to each target;

[0109] A weighted heatmap is determined based on the weight value corresponding to each target.

[0110] Here, the target detection requirement can be related to target size, such as detecting targets that are large or small; or it can be related to target sharpness, such as detecting targets that are sharp or blurry. When the target detection requirement is related to target size, a weight value can be determined for each target based on its size characteristics, thus creating a weighted heatmap. When the target detection requirement is related to target sharpness, a weight value can be determined for each target based on its sharpness characteristics, thus creating a weighted heatmap. Furthermore, the target detection requirement can also be related to target brightness, and of course, other target detection requirements can be included, without limitation here.

[0111] In some examples, if more attention is paid to larger targets, larger targets can be assigned larger weight values, and smaller targets can be assigned smaller weight values. Conversely, if more attention is paid to more clearly defined targets, clearer targets can be assigned larger weight values, and less clearly defined targets can be assigned smaller weight values. For example, a standard target heatmap can be... Figure 3a As shown, a large and clear target exists at location 301, a large but blurry target at location 302, and a small but clear target at location 303. If more attention is paid to large targets, the weighted heatmap can be prepared as follows: Figure 3b As shown, a larger weight of 10 is set at position 304 (corresponding to 301) and position 305 (corresponding to 302), while a smaller weight of 5 is set at position 306 (corresponding to 303). If a clearer objective is desired, the weighted heatmap can be... Figure 3c As shown, a larger weight of 10 is set at position 307 corresponding to 301 and position 309 corresponding to 303, and a smaller weight of 5 is set at position 308 corresponding to 302.

[0112] In this way, the weight values ​​in the weighted heatmap can be determined according to the target detection requirements.

[0113] In some implementations, greater attention is paid to larger targets. To assign larger weight values ​​to larger targets and smaller weight values ​​to smaller targets, the weight value for each target is determined based on its size characteristics in the sample image. This can specifically include:

[0114] The weight value for each objective is calculated using the following formula:

[0115]

[0116] Where W is the weight value corresponding to the target, h is the height of the target, w is the width of the target, and s min s is the preset minimum area. max 'a' is the preset maximum area, 'b' is the preset minimum weight, and 'b' is the preset maximum weight.

[0117] Here, since the shape of the target is not necessarily regular, h can be the height of the detection box containing the target, and w can be the width of the detection box containing the target. The target detection box can be a rectangle.

[0118] Thus, the above formula allows us to assign larger weight values ​​to larger targets and smaller weight values ​​to smaller targets.

[0119] In some implementations, more attention is paid to sharper targets. To assign larger weights to sharper targets and smaller weights to blurrier targets, the weight value for each target is determined based on its sharpness characteristics in the sample image. This can specifically include:

[0120] If the sharpness of the target is not less than the first preset sharpness, the weight value corresponding to the target is determined to be the first preset weight value;

[0121] If the clarity of the target is less than the first preset clarity but greater than the second preset clarity, the weight value corresponding to the target is determined to be the second preset weight value.

[0122] If the clarity of the target is not greater than the second preset clarity, the weight value corresponding to the target is determined to be the third preset weight value.

[0123] Here, the first weight value can be greater than the second weight value, and the second weight value can be greater than the third weight value. The first sharpness can be greater than the second sharpness.

[0124] In some examples, the formula for determining the weight value W corresponding to each objective can be:

[0125]

[0126] Thus, the above formula allows us to assign larger weight values ​​to clearer targets and smaller weight values ​​to more ambiguous targets.

[0127] Regarding S150, the second loss function heatmap is obtained by adjusting the first loss function heatmap based on the weight heatmap. The weight values ​​in the weight heatmap are determined according to the object detection requirements. Therefore, adjusting the model parameters of the model to be trained based on this second loss function heatmap allows the predicted object heatmap output by the model to gradually approach the object detection requirements. The model with adjusted parameters is iteratively trained using training samples until the training stopping condition is met. The resulting trained object detection model outputs a predicted object heatmap that better meets the object detection requirements. This trained object detection model can then be used to perform object detection on the image to be detected according to the object detection requirements.

[0128] In some implementations, to improve the efficiency of model inference and ensure a sufficient number of parameters during training, the backbone network of the object detection model can be a structurally reparameterized deep convolutional network.

[0129] Therefore, using a structurally reparameterized deep convolutional network as the backbone network of the object detection model can improve the efficiency of model inference and ensure a sufficient number of parameters during training.

[0130] In addition, the backbone network of the object detection model can also be a basic network such as ResNet neural network, Res2Net neural network or VGG neural network.

[0131] In some implementations, the predicted target heatmap can only indicate the location of the target, not its size. Furthermore, if the size of the sample image is not divisible by the sampling factor, the target's location will shift. Therefore, to determine the target's size and accurate location, the training samples may also include a standard bias heatmap and a standard width-height heatmap corresponding to the sample image. After S110, the target detection model training method may further include:

[0132] For each training sample, perform the following steps:

[0133] The training samples are input into the model to be trained to obtain the predicted bias heatmap and predicted width and height heatmap output by the model to be trained.

[0134] Based on the predicted bias heatmap and the standard bias heatmap, the third loss function heatmap of the model to be trained is determined, and based on the predicted width and height heatmap and the standard width and height heatmap, the fourth loss function heatmap of the model to be trained is determined.

[0135] Based on the second, third, and fourth loss function heatmaps, determine the fifth loss function heatmap;

[0136] Based on the heatmap of the fifth loss function, the model parameters of the model to be trained are adjusted, and the model to be trained with adjusted parameters is iteratively trained using training samples until the training stopping condition is met, thus obtaining a trained object detection model.

[0137] Here, the ability of the model to output width-height heatmaps and bias heatmaps can be trained. Training samples, including sample images, standard width-height heatmaps, and standard bias heatmaps, are input into the model to be trained. This outputs predicted width-height heatmaps and predicted bias heatmaps. Then, a third loss function heatmap is determined based on the predicted and standard width-height heatmaps, and a fourth loss function heatmap is determined based on the predicted and standard bias heatmaps. The second, third, and fourth loss function heatmaps are then fused to obtain a fifth loss function heatmap. Based on this fifth loss function heatmap, the model parameters are adjusted, and the adjusted model is iteratively trained using training samples until the training stopping condition is met, resulting in a trained object detection model. This model can then perform object detection on the image to be detected, outputting width-height heatmaps, bias heatmaps, and object heatmaps. The bias heatmap can be used to correct the location of the object.

[0138] Specifically, the predicted target heatmap can be output by the localization detection head of the model to be trained, the predicted bias heatmap can be output by the bias detection head of the model to be trained, and the predicted width and height heatmap can be output by the width and height detection head of the model to be trained.

[0139] Correspondingly, the target heatmap can be output through the localization detection head of the target detection model, the offset heatmap can be output through the offset detection head of the target detection model, and the width and height heatmap can be output through the width and height detection head of the target detection model.

[0140] In some examples, the process of determining the fifth loss function can be as follows: Figure 4 As shown, the sample image 401 is input into the model 402 to be trained. The model 402 outputs a predicted bias heatmap 403, a predicted width and height heatmap 404, and a predicted target heatmap 405. Then, a third loss function heatmap 407 is determined based on the predicted bias heatmap 403 and the standard bias heatmap 406. A fourth loss function heatmap 409 is determined based on the predicted width and height heatmap 404 and the standard width and height heatmap 408. A first loss function heatmap 411 is determined based on the predicted target heatmap 405 and the standard target heatmap 410. A second loss function heatmap 413 is determined based on the first loss function heatmap 411 and the weight heatmap 412. Finally, a fifth loss function 414 is determined based on the second loss function heatmap 413, the third loss function heatmap 407, and the fourth loss function heatmap 409.

[0141] Thus, through the above training process, the target detection model can output width and height heatmaps, bias heatmaps, and target heatmaps, thereby allowing users to know the accurate location and width and height of the target.

[0142] The following is combined Figure 5 The target detection method provided in the embodiments of this application will be described in detail.

[0143] Figure 5 The diagram illustrates a flowchart of a target detection method according to an embodiment of this application. It should be noted that this target detection method can be applied to target detection devices, such as… Figure 5 As shown, the target detection method may include the following steps:

[0144] S510, acquire the image to be detected;

[0145] S520: Input the image to be detected into the target detection model to obtain the target heatmap corresponding to the image to be detected.

[0146] Therefore, by inputting the acquired image to be detected into the target detection model, a target heatmap corresponding to the image to be detected can be obtained. Since the target detection model used can perform target detection according to the target detection requirements in the actual application scenario, the target detection is more targeted. Therefore, the obtained target heatmap is more in line with the target detection requirements in the actual application scenario.

[0147] Regarding S510, the image to be detected can be an image captured by a camera. For example, if the object detection model is applied to the field of autonomous driving, the image to be detected can be an image captured by a camera installed on the vehicle. Of course, the image to be detected can also be an image captured by other image sensors, which is not limited here.

[0148] Regarding S520, the object detection model can be a model trained using the object detection model training method described above. The specific process of inputting the image to be detected into the object detection model to obtain the target heatmap corresponding to the image to be detected is the same as the object detection model training process described above, so it will not be repeated here.

[0149] In some implementations, to make the target's position in the target heatmap more accurate and to determine the target's width and height, the target detection method may further include, after S510:

[0150] The image to be detected is input into the target detection model to obtain the bias heatmap and width-height heatmap corresponding to the image to be detected.

[0151] Here, the offset heatmap can show the target's position offset in both the horizontal and vertical directions. The target's position in the target heatmap can be corrected based on the offset data in the offset heatmap. The width-height heatmap can include the target's width and height.

[0152] Thus, by obtaining the offset heatmap and width-height heatmap corresponding to the image to be detected through the above process, the position of the target in the target heatmap can be more accurate and the width and height of the target can be determined.

[0153] Based on the same inventive concept, embodiments of this application also provide an object detection model training device. The following, in conjunction with… Figure 6 The target detection model training device provided in the embodiments of this application will be described in detail.

[0154] Figure 6 This illustration shows a schematic diagram of the structure of a target detection model training device provided in one embodiment of this application.

[0155] like Figure 6 As shown, the target detection model training device may include:

[0156] The sample acquisition module 601 is used to acquire multiple training samples, including sample images and standard target heatmaps corresponding to the sample images.

[0157] The first sample input module 602 is used to perform the following for each training sample: inputting the training sample into the model to be trained, and obtaining the predicted target heatmap output by the model to be trained;

[0158] The first determining module 603 is used to perform the following for each training sample: determine the first loss function heatmap of the model to be trained based on the predicted target heatmap and the standard target heatmap;

[0159] The correction module 604 is used to perform the following for each training sample: correct the first loss function heatmap according to at least one weight heatmap to obtain the second loss function heatmap. The weight heatmap is a graph obtained by setting different weights for different targets in the heatmap. The weights are determined according to the target detection requirements.

[0160] The first adjustment module 605 is used to perform the following for each training sample: adjust the model parameters of the model to be trained according to the second loss function heatmap, and use the training samples to iteratively train the model to be trained after parameter adjustment until the training stopping condition is met, so as to obtain a trained target detection model.

[0161] This allows for the acquisition of multiple training samples. For each training sample, it is input into the model to be trained, resulting in a predicted target heatmap output by the model. Then, based on the predicted target heatmap and the standard target heatmaps included in the training samples, a first loss function heatmap is determined for the model. This first loss function heatmap is then corrected based on at least one weight heatmap to obtain a second loss function heatmap. The model parameters are adjusted according to this second loss function heatmap, and the adjusted model is iteratively trained using the training samples until the training stopping condition is met, resulting in a trained target detection model. Since the weight heatmap is obtained by assigning different weights to different targets in the heatmap, and these weights are determined based on the target detection requirements, the target detection model trained in this way can perform target detection according to the specific needs of the application scenario, improving the targeting accuracy of target detection.

[0162] In some implementations, to make the predicted target heatmap output by the model to be trained more accurate and improve processing efficiency, the first sample input module 602 may specifically include:

[0163] The first extraction submodule is used to extract image features from the sample image to obtain a first feature image;

[0164] The downsampling submodule is used to downsample the first feature image by different downsampling factors to obtain multiple second feature images;

[0165] The second extraction submodule is used to extract image features from multiple second feature images to obtain multiple third feature images;

[0166] The fusion submodule is used to fuse multiple third-feature images to obtain a heatmap of the predicted target.

[0167] In some implementations, there may be multiple weighted heatmaps. To correct the first loss function heatmap based on these multiple weighted heatmaps, correction module 604 may specifically include:

[0168] The fusion submodule is used to merge multiple weighted heatmaps to obtain a merged weighted heatmap when there are multiple weighted heatmaps.

[0169] The correction submodule is used to correct the first loss function heatmap based on the fused weight heatmap to obtain the second loss function heatmap.

[0170] In some implementations, the weights in the weighted heatmap can be determined according to the target detection requirements. To determine this weighted heatmap, the correction module 604 may specifically include:

[0171] The first determining submodule is used to determine the weight value corresponding to each target based on the size characteristics of each target in the sample image before correcting the first loss function heatmap according to at least one weighted heatmap to obtain the second loss function heatmap; and / or; determine the weight value corresponding to each target based on the sharpness characteristics of each target in the sample image.

[0172] The second determination submodule is used to determine the weight heatmap based on the weight value corresponding to each target.

[0173] In some implementations, greater attention is paid to larger targets. To assign larger weight values ​​to larger targets and smaller weight values ​​to smaller targets, the first determining submodule may include:

[0174] The calculation unit is used to calculate the weight value corresponding to each target using the following formula:

[0175]

[0176] Where W is the weight value corresponding to the target, h is the height of the target, w is the width of the target, and s min s is the preset minimum area. max 'a' is the preset maximum area, 'b' is the preset minimum weight, and 'b' is the preset maximum weight.

[0177] In some implementations, more attention is paid to clearer targets. To assign larger weight values ​​to clearer targets and smaller weight values ​​to less clear targets, the first determination submodule may include:

[0178] The first determining unit is used to determine the weight value corresponding to the target as the first preset weight value when the sharpness of the target is not less than the first preset sharpness.

[0179] The second determining unit is used to determine the weight value corresponding to the target as the second preset weight value when the target's clarity is less than the first preset clarity but greater than the second preset clarity.

[0180] The third determining unit is used to determine the weight value corresponding to the target as the third preset weight value when the sharpness of the target is not greater than the second preset sharpness.

[0181] The first weight value is greater than the second weight value, and the second weight value is greater than the third weight value.

[0182] In some implementations, the predicted target heatmap can only indicate the location of the target, not its size. Furthermore, if the size of the sample image is not divisible by the sampling factor, the target's location will shift. Therefore, to determine the target's size and accurate location, the training samples also include a standard bias heatmap and a standard width-height heatmap corresponding to the sample image. The target detection model training apparatus may further include:

[0183] The second sample input module is used to perform the following steps for each training sample after acquiring multiple training samples: inputting the training sample into the model to be trained, and obtaining the predicted bias heatmap and predicted width and height heatmap output by the model to be trained.

[0184] The second determination module is used to perform the following for each training sample: determine the third loss function heatmap of the model to be trained based on the predicted bias heatmap and the standard bias heatmap, and determine the fourth loss function heatmap of the model to be trained based on the predicted width and height heatmap and the standard width and height heatmap.

[0185] The third determination module is used to perform the following for each training sample: determine the fifth loss function heatmap based on the second, third, and fourth loss function heatmaps;

[0186] The second adjustment module is used to perform the following for each training sample: adjust the model parameters of the model to be trained according to the heatmap of the fifth loss function, and use the training samples to iteratively train the model to be trained with adjusted parameters until the training stopping condition is met, so as to obtain a trained object detection model.

[0187] Based on the same inventive concept, embodiments of this application also provide a target detection device. The following, in conjunction with… Figure 7 The target detection device provided in the embodiments of this application will be described in detail.

[0188] Figure 7 A schematic diagram of the structure of a target detection device provided in one embodiment of this application is shown.

[0189] like Figure 7 As shown, the target detection device may include:

[0190] Image acquisition module 701 is used to acquire the image to be detected;

[0191] The first image input module 702 is used to input the image to be detected into the target detection model to obtain the target heatmap corresponding to the image to be detected. The target detection model is a model trained using the above-mentioned target detection model training method.

[0192] In some embodiments, to make the target location in the target heatmap more accurate and to determine the target's width and height, the target detection device may further include:

[0193] The second image input module is used to input the image to be detected into the target detection model after acquiring the image to be detected, so as to obtain the bias heatmap and width-height heatmap corresponding to the image to be detected.

[0194] Therefore, by inputting the acquired image to be detected into the target detection model, a target heatmap corresponding to the image to be detected can be obtained. Since the target detection model used can perform target detection according to the target detection requirements in the actual application scenario, the target detection is more targeted. Therefore, the obtained target heatmap is more in line with the target detection requirements in the actual application scenario.

[0195] Figure 8 A schematic diagram of the structure of an electronic device provided in one embodiment of this application is shown.

[0196] like Figure 8 As shown, the electronic device 8 is a structural diagram of an exemplary hardware architecture of an electronic device capable of implementing the target detection model training method and target detection method according to the embodiments of this application, as well as the target detection model training device and the target detection device. This electronic device may refer to the electronic device in the embodiments of this application.

[0197] The electronic device 8 may include a processor 801 and a memory 802 storing computer program instructions.

[0198] Specifically, the processor 801 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0199] Memory 802 may include mass storage for data or instructions. For example, and not limitingly, memory 802 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 802 may include removable or non-removable (or fixed) media. Where appropriate, memory 802 may be internal or external to an integrated gateway disaster recovery device. In a particular embodiment, memory 802 is non-volatile solid-state memory. In a particular embodiment, memory 802 may include read-only memory (ROM), random access memory (RAM), disk storage media device, optical storage media device, flash memory device, electrical, optical, or other physical / tangible memory storage device. Thus, generally, memory 802 includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to one aspect of this application.

[0200] The processor 801 reads and executes computer program instructions stored in the memory 802 to implement any of the target detection model training methods or target detection methods in the above embodiments.

[0201] In one example, the electronic device may also include a communication interface 803 and a bus 804. For example, Figure 8 As shown, the processor 801, memory 802, and communication interface 803 are connected through bus 804 and complete communication with each other.

[0202] The communication interface 803 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0203] Bus 804 includes hardware, software, or both, that couples components of an electronic device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 804 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, any suitable bus or interconnect is contemplated herein.

[0204] The electronic device can execute the data storage method in the embodiments of this application, thereby achieving the combination Figures 1 to 7 The document describes a method for training an object detection model and an object detection method, as well as an object detection model training device and an object detection device.

[0205] Furthermore, in conjunction with the target detection model training method and target detection method in the above embodiments, this application embodiment can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any one of the target detection model training methods or target detection methods in the above embodiments.

[0206] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0207] The functional blocks shown in the above-described structural diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0208] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0209] The aspects of this application have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by dedicated hardware performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0210] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A method for training an object detection model, characterized in that, include: Multiple training samples are acquired, the training samples including sample images and standard target heatmaps corresponding to the sample images; For each training sample, perform the following steps: The training samples are input into the model to be trained to obtain the predicted target heatmap output by the model to be trained; Based on the predicted target heatmap and the standard target heatmap, the first loss function heatmap of the model to be trained is determined; The first loss function heatmap is corrected based on at least one weighted heatmap to obtain a second loss function heatmap. The weighted heatmap is obtained by assigning different weights to different targets in the heatmap, and the weights are determined according to the target detection requirements. The weighted heatmap is constructed using a Gaussian distribution, and the Gaussian radius of the weighted heatmap is determined based on the Gaussian radius of the predicted target heatmap. Based on the second loss function heatmap, the model parameters of the model to be trained are adjusted, and the training samples are used to iteratively train the model to be trained after parameter adjustment until the training stopping condition is met, so as to obtain a trained target detection model. Before correcting the first loss function heatmap based on at least one weighted heatmap to obtain the second loss function heatmap, the method further includes: Based on the size characteristics of each target in the sample image, determine the weight value corresponding to each target; and / or, based on the sharpness characteristics of each target in the sample image, determine the weight value corresponding to each target; The weight heatmap is determined based on the weight value corresponding to each target; The step of inputting the training samples into the model to be trained to obtain the predicted target heatmap output by the model to be trained includes: Extract the image features from the sample image to obtain the first feature image; The first feature image is downsampled by different downsampling factors to obtain multiple second feature images; the multiple second feature images are multiple feature images of different sizes; By using deep interlayer aggregation, image features of the multiple second feature images are extracted to obtain multiple third feature images; By fusing the multiple third feature images, the predicted target heatmap is obtained.

2. The method according to claim 1, characterized in that, The step of correcting the first loss function heatmap based on at least one weighted heatmap to obtain a second loss function heatmap includes: When there are multiple weighted heatmaps, the multiple weighted heatmaps are merged to obtain a merged weighted heatmap; Based on the fused weighted heatmap, the first loss function heatmap is corrected to obtain the second loss function heatmap.

3. The method according to claim 1, characterized in that, The step of determining the weight value corresponding to each target based on the size features of each target in the sample image includes: The weight value for each objective is calculated using the following formula: Where W is the weight value corresponding to the target, h is the height of the target, and w is the width of the target. This is the preset minimum area. 'a' is the preset maximum area, 'b' is the preset minimum weight, and 'b' is the preset maximum weight.

4. The method according to claim 1, characterized in that, The step of determining the weight value corresponding to each target based on the sharpness features of each target in the sample image includes: If the sharpness of the target is not less than the first preset sharpness, the weight value corresponding to the target is determined to be the first preset weight value; If the clarity of the target is less than the first preset clarity but greater than the second preset clarity, the weight value corresponding to the target is determined to be the second preset weight value. If the sharpness of the target is not greater than the second preset sharpness, the weight value corresponding to the target is determined to be the third preset weight value; The first weight value is greater than the second weight value, and the second weight value is greater than the third weight value.

5. The method according to claim 1, characterized in that, The training samples also include a standard bias heatmap and a standard width-height heatmap corresponding to the sample images. After acquiring multiple training samples, the method further includes: For each training sample, perform the following steps: The training samples are input into the model to be trained to obtain the predicted bias heatmap and predicted width and height heatmap output by the model to be trained. Based on the predicted bias heatmap and the standard bias heatmap, the third loss function heatmap of the model to be trained is determined, and based on the predicted width and height heatmap and the standard width and height heatmap, the fourth loss function heatmap of the model to be trained is determined. Based on the second loss function heatmap, the third loss function heatmap, and the fourth loss function heatmap, a fifth loss function heatmap is determined; Based on the heatmap of the fifth loss function, the model parameters of the model to be trained are adjusted, and the training samples are used to iteratively train the model with adjusted parameters until the training stopping condition is met, thus obtaining a trained target detection model.

6. A target detection method, characterized in that, include: Acquire the image to be detected; The image to be detected is input into the target detection model to obtain the target heatmap corresponding to the image to be detected. The target detection model is a model trained by any one of the methods in claims 1-5.

7. The method according to claim 6, characterized in that, After acquiring the image to be detected, the method further includes: The image to be detected is input into the target detection model to obtain the bias heatmap and width-height heatmap corresponding to the image to be detected.

8. A target detection model training device, characterized in that, The device includes: The sample acquisition module is used to acquire multiple training samples, the training samples including sample images and standard target heatmaps corresponding to the sample images; The first sample input module is used to perform the following for each training sample: inputting the training sample into the model to be trained to obtain the predicted target heatmap output by the model to be trained; The first determining module is used to perform the following for each training sample: determining the first loss function heatmap of the model to be trained based on the predicted target heatmap and the standard target heatmap; The correction module is used to perform the following for each training sample: correcting the first loss function heatmap according to at least one weight heatmap to obtain a second loss function heatmap, wherein the weight heatmap is a graph obtained by setting different weights for different targets in the heatmap, and the weights are determined according to the target detection requirements; the weight heatmap is constructed using a Gaussian distribution, and the Gaussian radius of the weight heatmap is determined based on the Gaussian radius of the predicted target heatmap; The first adjustment module is used to perform the following for each training sample: adjust the model parameters of the model to be trained according to the second loss function heatmap, and use the training sample to iteratively train the model to be trained after parameter adjustment until the training stopping condition is met, so as to obtain a trained target detection model. The correction module specifically includes: The first determining submodule is configured to, before correcting the first loss function heatmap according to at least one weighted heatmap to obtain the second loss function heatmap, determine the weight value corresponding to each target based on the size characteristics of each target in the sample image; and / or, determine the weight value corresponding to each target based on the sharpness characteristics of each target in the sample image; The second determining submodule is used to determine the weight heatmap based on the weight value corresponding to each target; The first sample input module is further configured to extract image features from the sample image to obtain a first feature image; downsample the first feature image by different downsampling factors to obtain multiple second feature images; the multiple second feature images are multiple feature images of different sizes; extract image features from the multiple second feature images respectively through deep interlayer aggregation to obtain multiple third feature images; and fuse the multiple third feature images to obtain the predicted target heatmap.

9. A target detection device, characterized in that, The device includes: The image acquisition module is used to acquire the image to be detected; The first image input module is used to input the image to be detected into the target detection model to obtain the target heatmap corresponding to the image to be detected, wherein the target detection model is a model trained by any one of the methods of claims 1-5.

10. An electronic device, characterized in that, The device includes: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the target detection model training method as described in any one of claims 1-5 or the target detection method as described in any one of claims 6-7.

11. A computer storage medium, characterized in that, The computer storage medium stores computer program instructions, which, when executed by a processor, implement the target detection model training method as described in any one of claims 1-5 or the target detection method as described in any one of claims 6-7.

12. A computer program product, characterized in that, When the instructions in the computer program product are executed by the processor of the electronic device, the electronic device performs the target detection model training method as described in any one of claims 1-5 or the target detection method as described in any one of claims 6-7.

Images