Target detection method, electronic device, storage medium and computer program product

Abstract

The present invention provides a target detection method, electronic device, storage medium and computer program product, the method comprising obtaining an image to be detected containing a target to be detected; inputting the image to be detected into a target detection model, performing target detection, and obtaining a target detection result output by the target detection model, wherein the target detection model is obtained by training based on a candidate frame and its corresponding positive and negative labels, and the positive and negative labels are determined based on the intersection-and-union ratio of the candidate frame and the real frame corresponding to the candidate frame, and a dynamically changing intersection-and-union ratio threshold. The present invention dynamically changes the positive and negative labels of the candidate frame through a dynamically changing intersection-and-union ratio threshold, so that the positive and negative labels finally assigned to the candidate frame are accurate labels, thereby improving the label assignment accuracy of the candidate frame, thereby improving the recall rate of the target detection model, and finally achieving high-performance target detection.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a target detection method, electronic equipment, storage medium and computer program product. Background Art

[0002] Object detection is a major task in the field of computer vision. It can identify multiple objects in an image and locate the bounding boxes of different objects. This object detection technology is widely used in various fields, such as autonomous driving, drone recognition, security monitoring, and medical diagnosis.

[0003] At present, target detection is to learn the real object box through deep learning, so as to build a target detection model for target detection, and then obtain the classification and regression results of the candidate box. Among them, the training strategy of the target detection model is fixed, and most algorithms have different learning capabilities for objects of different sizes, especially for small target objects. Therefore, the current fixed training strategy for model training will lead to a decrease in the accuracy of label assignment of candidate boxes, and then a decrease in the recall rate of the target detection model. Summary of the invention

[0004] The present invention provides a target detection method, an electronic device, a storage medium and a computer program product, which are used to solve the defects of low label assignment accuracy of candidate boxes and low recall rate of target detection models in the prior art, and realize high-performance target detection.

[0005] The present invention provides a target detection method, comprising:

[0006] Acquire an image to be detected containing an object to be detected;

[0007] The image to be detected is input into a target detection model to perform target detection, and a target detection result output by the target detection model is obtained. The target detection model is trained based on candidate boxes and their corresponding positive and negative labels, and the positive and negative labels are determined based on the intersection-and-union ratio of the candidate box and the real box corresponding to the candidate box, as well as a dynamically changing intersection-and-union ratio threshold.

[0008] A target detection method provided by the present invention also includes a training method for the target detection model:

[0009] Obtain a training sample image, input the training sample image into the model to be trained, perform feature extraction, and obtain a feature map corresponding to the training sample image;

[0010] Based on the feature map, construct candidate boxes of different scales and different aspect ratios;

[0011] Determine the positive and negative labels of the candidate frame based on the intersection-and-union ratio of the candidate frame and the real frame corresponding to the candidate frame, and a dynamically changing intersection-and-union ratio threshold;

[0012] Based on the candidate boxes and the positive and negative labels, the model to be trained is trained to obtain a target detection model.

[0013] According to a target detection method provided by the present invention, constructing candidate boxes of different scales and different aspect ratios based on the feature map includes:

[0014] Based on the feature map and preset size categories, candidate boxes of different scales and different aspect ratios are constructed, wherein the preset size categories are obtained based on statistical analysis of sizes of each target in a general target dataset, and the general target dataset includes targets of various sizes.

[0015] According to a target detection method provided by the present invention, the positive and negative labels of the candidate box are determined based on the intersection-and-union ratio of the candidate box and the real box corresponding to the candidate box, and a dynamically changing intersection-and-union ratio threshold, including:

[0016] Determine the positive and negative labels of the candidate box based on the intersection-over-union ratio of the candidate box and the real box corresponding to the candidate box, and the size relationship of the intersection-over-union ratio threshold;

[0017] Counting the number of positive candidate boxes of the corresponding size of the candidate box;

[0018] Based on the number of positive candidate boxes, the intersection-over-union threshold is adjusted, and the step of determining the positive and negative labels of the candidate box based on the intersection-over-union ratio of the candidate box and the real box corresponding to the candidate box and the size relationship of the intersection-over-union ratio threshold is returned until the number of positive candidate boxes meets the requirement.

[0019] According to a target detection method provided by the present invention, the step of adjusting the intersection-over-union threshold based on the number of positive candidate boxes, and returning the intersection-over-union ratio based on the intersection-over-union ratio of the candidate box and the real box corresponding to the candidate box, and the size relationship of the intersection-over-union ratio threshold, to determine the positive and negative labels of the candidate box until the number of positive candidate boxes meets the requirement includes:

[0020] Counting the total number of candidate boxes of the corresponding size of the candidate box, and calculating the number of positive candidate boxes and the positive ratio of the total number of candidate boxes;

[0021] If the proportion of positive examples is greater than the candidate hyperparameter of the size corresponding to the candidate box, the intersection-over-union threshold is increased, and the step of determining the positive and negative labels of the candidate box based on the intersection-over-union ratio of the candidate box and the real box corresponding to the candidate box and the size relationship of the intersection-over-union ratio threshold is returned until the proportion of positive examples is less than or equal to the candidate hyperparameter of the size corresponding to the candidate box.

[0022] According to a target detection method provided by the present invention, the model to be trained is trained based on the candidate box and the positive and negative labels to obtain a target detection model, including:

[0023] The candidate box, the positive and negative labels and the preset loss function are used to train the model to be trained to obtain a target detection model, and the gradient value of the preset loss function is dynamically adjusted based on the training duration or the number of training times.

[0024] According to a target detection method provided by the present invention, the method further includes a method for adjusting the gradient value of the preset loss function:

[0025] Calculate the relative normalized regression loss of the positive candidate box in the candidate box and the target real box corresponding to the positive candidate box;

[0026] If the relative normalized regression loss is less than the regression threshold, the gradient value of the preset loss function is reduced, and the regression threshold is dynamically adjusted based on the training duration or the number of training times.

[0027] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any of the target detection methods described above are implemented.

[0028] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of any of the target detection methods described above are implemented.

[0029] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the steps of any of the target detection methods described above are implemented.

[0030] The target detection method, electronic device, storage medium and computer program product provided by the present invention acquire an image to be detected containing a target to be detected, then input the image to be detected into a target detection model, perform target detection, and obtain a target detection result output by the target detection model, thereby realizing detection of the target to be detected in the image to be detected. The target detection model is obtained by training based on a candidate frame and its corresponding positive and negative labels, and the positive and negative labels are determined based on the intersection-and-union ratio of the candidate frame and the real frame corresponding to the candidate frame, and a dynamically changing intersection-and-union ratio threshold. Based on this, the present invention dynamically changes the positive and negative labels of the candidate frame through a dynamically changing intersection-and-union ratio threshold, so that the positive and negative labels finally assigned to the candidate frame are accurate labels, thereby improving the label assignment accuracy of the candidate frame, thereby improving the recall rate of the target detection model, and finally achieving high-performance target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0032] Figure 1 One of the flow charts of the target detection method provided by the present invention;

[0033] Figure 2 The second flowchart of the target detection method provided by the present invention;

[0034] Figure 3 The third flowchart of the target detection method provided by the present invention;

[0035] Figure 4 An example physical structure diagram of an electronic device is provided. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] Figure 1 One of the flow charts of the target detection method provided by the present invention is as follows: Figure 1 As shown, the target detection method provided by the present invention includes:

[0038] Step 110, obtaining an image to be detected containing the target to be detected;

[0039] In this embodiment, the target detection method can be applied to various fields, such as unmanned driving of automobiles, drone identification, security monitoring, and medical diagnosis, etc. Correspondingly, the images to be detected can be images around the car, images taken by drones, monitoring images, color ultrasound images, and CT images, etc.

[0040] The image to be detected may contain one or more targets to be detected. For example, in the field of autonomous driving, the targets to be detected may be people, vehicles, and obstacles; in the field of security monitoring, the targets to be detected may be people, animals, and objects; in the field of medical diagnosis, the targets to be detected may be lesion areas, joint areas, cell areas, and organ areas.

[0041] Step 120, input the image to be detected into the target detection model, perform target detection, and obtain the target detection result output by the target detection model. The target detection model is trained based on the candidate box and its corresponding positive and negative labels, and the positive and negative labels are determined based on the intersection-and-union ratio of the candidate box and the real box corresponding to the candidate box, and a dynamically changing intersection-and-union ratio threshold.

[0042] In this embodiment, the target detection result includes the classification result and regression result of the target to be detected. The classification result is the classification result obtained by classifying the candidate box (suggestion box, detection box or prediction box), that is, the target category of the candidate box. The regression result is the result obtained by performing bounding box regression on the candidate box, that is, transforming the candidate box as much as possible to the position of the real box corresponding to the candidate box. The transformation may include up and down translation, left and right translation and scaling.

[0043] Among them, the target detection model is a machine learning model, specifically, the target detection model is a deep learning network model built based on the target detection algorithm. The target detection algorithm may include Faster R-CNN, Fast R-CNN, R-CNN, R-FCN and SSD, etc. In this embodiment and the following embodiments, the target detection model built based on the Faster R-CNN algorithm is used as an example for description.

[0044] Among them, the candidate box is the suggested area corresponding to the target to be detected. One target to be detected corresponds to multiple candidate boxes, and one image to be detected may include multiple targets to be detected. Therefore, one image to be detected corresponds to multiple candidate boxes. The multiple targets to be detected in one image to be detected may be targets of different sizes. Based on this, targets to be detected of different sizes correspond to candidate boxes of different sizes, that is, candidate boxes of different scales and different aspect ratios.

[0045] Among them, the positive and negative labels corresponding to the candidate boxes include positive candidate boxes (positive samples) and negative candidate boxes (negative samples). The positive and negative labels of each candidate box are dynamically changed, that is, after the candidate boxes are assigned labels, the labels can be dynamically redistributed according to the training process. The specific dynamic change process of the positive and negative labels can be referred to the second embodiment below, which will not be repeated here.

[0046] The initial value of the intersection-over-union ratio threshold is a preset value, which can be set according to actual needs, for example, 0.4, 0.6, 0.7. The intersection-over-union ratio threshold is used to determine whether a candidate box is a positive candidate box, that is, to determine whether it is a positive sample, so as to achieve label assignment or label marking of the candidate box.

[0047] In a specific embodiment, the target detection model includes a regression detection sub-model and a classification sub-model. Specifically, based on the feature extractor in the trained target detection model, a feature map of the image to be detected is extracted; a positive candidate box is generated in the feature map; a border regression process is performed on the positive candidate box based on the regression detection sub-model; and a classification process is performed on the positive candidate box based on the classification sub-model.

[0048] In one embodiment, the feature extractor includes a convolutional network layer, the convolutional network layer includes a convolutional layer, a pooling layer and an activation function, and the step of extracting a feature map of the image to be detected based on the feature extractor in the trained target detection model includes:

[0049] Based on the convolution layer of the convolution network layer in the target detection model, the image to be detected is convolved to obtain a convolution result; based on the pooling layer of the convolution network layer in the target detection model, the convolution result is pooled to obtain a pooling result; and the pooling result is nonlinearly processed using the activation function to obtain a feature map of the image to be detected. The activation function can be set according to actual needs, such as a RELU activation function.

[0050] According to the target detection method of the embodiment of the present invention, by acquiring an image to be detected containing the target to be detected, and then inputting the image to be detected into the target detection model, performing target detection, and obtaining the target detection result output by the target detection model, the target to be detected in the image to be detected is detected. The target detection model is trained based on the candidate box and its corresponding positive and negative labels, and the positive and negative labels are determined based on the intersection-and-union ratio of the candidate box and the real box corresponding to the candidate box, and the dynamically changing intersection-and-union ratio threshold. Based on this, the embodiment of the present invention dynamically changes the positive and negative labels of the candidate box through the dynamically changing intersection-and-union ratio threshold, so that the positive and negative labels finally assigned to the candidate box are accurate labels, thereby improving the label assignment accuracy of the candidate box, and then improving the recall rate of the target detection model, and finally achieving high-performance target detection.

[0051] Furthermore, based on the above first embodiment, a second embodiment of the target detection method of the present invention is proposed. Figure 2 The second flowchart of the target detection method provided by the present invention is as follows: Figure 2 As shown, in this embodiment, a training method for the target detection model is also included:

[0052] Step 210, obtaining a training sample image, inputting the training sample image into the model to be trained, performing feature extraction, and obtaining a feature map corresponding to the training sample image;

[0053] In this embodiment, the training sample images are object images captured by a camera, that is, the training sample images are images captured by capturing real target objects.

[0054] Among them, the training sample images for training the training model may include multiple ones. The training sample images may include one or more targets to be detected. The targets to be detected may be various objects. For example, the target detection model obtained by training the training model is applied to the field of unmanned driving of automobiles, and the targets to be detected may be people, vehicles, obstacles, etc.; the target detection model obtained by training the training model is applied to the field of security monitoring, and the targets to be detected may be people, animals, objects, etc.; the target detection model obtained by training the training model is applied to the field of medical diagnosis, and the targets to be detected may be lesion areas, joint areas, cell areas, organ areas, etc.

[0055] It should be noted that, in order to improve the learning ability of objects to be detected of different sizes, the training sample images include various general objects, that is, objects to be detected of different sizes. Specifically, the objects to be detected of different sizes can be divided into N types of sizes, and N can be set according to actual needs, such as 3, that is, the objects to be detected can be divided into 3 types of sizes. For example, the objects to be detected are divided into three types of sizes: large, medium and small, so that the corresponding three types of candidate box sizes can be set later.

[0056] In a specific embodiment, the step of inputting the training sample image into the model to be trained, performing feature extraction, and obtaining a feature map corresponding to the training sample image includes: extracting the feature map of the training sample image based on a feature extractor of the model to be trained.

[0057] In one embodiment, the feature extractor includes a convolutional network layer, the convolutional network layer includes a convolutional layer, a pooling layer and an activation function, and the feature extractor based on the model to be trained, the step of extracting the feature map of the training sample image includes:

[0058] Based on the convolution layer of the convolution network layer in the model to be trained, the training sample image is convolved to obtain a convolution result; based on the pooling layer of the convolution network layer in the model to be trained, the convolution result is pooled to obtain a pooling result; and the pooling result is nonlinearly processed using the activation function to obtain a feature map of the training sample image. The activation function can be set according to actual needs, such as a RELU activation function.

[0059] Step 220, constructing candidate boxes of different scales and different aspect ratios based on the feature map;

[0060] Specifically, candidate boxes of different scales and different aspect ratios are obtained by dividing the feature map. The target sizes in the general target database can be summarized and statistically analyzed to divide the target to be detected into sizes, and the corresponding candidate box sizes are set, so as to determine the scale of the candidate box based on the candidate box size.

[0061] Among them, different aspect ratios can be 1:1, 1:2, 1:3.

[0062] In a specific embodiment, the model to be trained includes an RPN (Region Proposal Networks) layer, and the above step 220 includes:

[0063] Based on the RPN layer of the model to be trained, candidate boxes of different scales and different aspect ratios are constructed on the feature map. Specifically, the RPN layer uses a sliding window to traverse the entire feature map, and during the traversal process, each window center generates a candidate box according to the corresponding scale and aspect ratio. The size of the sliding window can be 3x3.

[0064] Furthermore, the above step 220 includes:

[0065] Step 221, constructing candidate boxes of different scales and different aspect ratios based on the feature map and preset size categories, wherein the preset size categories are obtained based on statistical analysis of each target size in a general target dataset, and the general target dataset includes targets of various sizes.

[0066] In this embodiment, the general target data set includes general targets, and each general target can be a target of multiple sizes, that is, the general target data set includes targets of multiple sizes, so that a preset size category can be obtained based on statistical analysis of each target size in the general target data set.

[0067] Among them, the preset size categories may include N size categories, and N can be set according to actual needs. For example, if N is 3, the preset size categories may include three sizes: large, medium, and small (L, M, S), so that candidate boxes of three sizes can be set.

[0068] In a specific embodiment, the step of constructing candidate boxes of different scales and different aspect ratios based on the feature map and the preset size category includes:

[0069] Based on the preset size category, a scale size set is determined; based on the feature map and the scale size set, candidate boxes of different scale sizes and different aspect ratios are constructed. The scale size set includes multiple scale sizes, that is, the preset size category is used to determine the scale size of the candidate box.

[0070] In this embodiment, by analyzing the size of general targets, candidate boxes of different scales and aspect ratios are constructed on the feature map, so that the model can learn targets of different sizes, especially the learning ability for small targets, thereby improving the detection accuracy of the target detection model, and ultimately further improving the detection effect of the target detection model.

[0071] Step 230, determining the positive and negative labels of the candidate box based on the intersection-over-union ratio of the candidate box and the real box corresponding to the candidate box, and a dynamically changing intersection-over-union ratio threshold;

[0072] In this embodiment, after the candidate box is generated, the candidate box can be classified into two categories through the fully connected layer in the model to be trained, that is, whether the candidate box is a positive candidate box (positive sample) or a negative candidate box (negative sample), that is, whether the candidate box is a foreground or a background. Among them, the positive and negative labels include positive candidate boxes (positive samples) and negative candidate boxes (negative samples).

[0073] Specifically, if the intersection-over-union ratio is greater than or equal to the intersection-over-union ratio threshold, the candidate box is determined to be a positive candidate box; if the intersection-over-union ratio is less than the intersection-over-union ratio threshold, the candidate box is determined to be a negative candidate box.

[0074] The intersection-over-union ratio of the candidate box is used to measure the overlap between the candidate box and the corresponding real box. That is, the ratio of the intersection of the candidate box and the real box to the union of the candidate box and the real box is calculated. The intersection is the overlapping area of ​​the candidate box and the real box, and the union is the combined area of ​​the candidate box and the real box.

[0075] Specifically, the calculation formula for calculating the intersection-over-union ratio is:

[0076]

[0077] Among them, C represents the candidate box, G represents the real box, area() is the area of ​​the corresponding box, and IoU is the intersection over union ratio.

[0078] The dynamically changing IoU threshold is adjusted based on the number of positive candidate frames in the current candidate frame. It should be noted that the IoU threshold is adjusted based on the number of positive candidate frames in the current candidate frame to obtain an accurate IoU threshold, and the IoU threshold is used to determine whether a candidate frame is a positive candidate frame, and then an appropriate number of positive candidate frames can be selected.

[0079] Furthermore, the above step 230 includes:

[0080] Step 231, based on the IoU ratio of the candidate box and the real box corresponding to the candidate box, and the magnitude relationship of the IoU ratio threshold, determine the positive and negative labels of the candidate box;

[0081] The initial value of the intersection-over-union ratio threshold is a preset value, which can be set according to actual needs, for example, 0.4, 0.6, 0.7. The intersection-over-union ratio threshold is used to determine whether a candidate box is a positive candidate box, that is, to determine whether it is a positive sample, so as to achieve label assignment or label marking of the candidate box.

[0082] Specifically, if the intersection-over-union ratio of the candidate box and the real box corresponding to the candidate box is greater than or equal to the intersection-over-union ratio threshold, the candidate box is determined to be a positive candidate box, and the candidate box is labeled based on the positive label corresponding to the positive candidate box; if the intersection-over-union ratio of the candidate box and the real box corresponding to the candidate box is less than the intersection-over-union ratio threshold, the candidate box is determined to be a negative candidate box, and the candidate box is labeled based on the negative label corresponding to the negative candidate box.

[0083] In some embodiments, candidate boxes of various sizes are included, and each candidate box of each size has its corresponding IoU threshold, so the IoU threshold is determined based on the corresponding size of the candidate box. For example, if candidate boxes of three sizes are included, the corresponding IoU thresholds are also three thresholds.

[0084] Specifically, the above step 231 includes: determining the positive and negative labels of the candidate box based on the intersection-and-union ratio of the candidate box and the real box corresponding to the candidate box, and the size relationship of the intersection-and-union ratio threshold corresponding to the candidate box.

[0085] Step 232, counting the number of positive candidate boxes of the corresponding size of the candidate box;

[0086] Specifically, the number of positive candidate boxes of a size corresponding to the current candidate box in all candidate boxes in the feature map is counted.

[0087] Among them, the feature map includes candidate boxes of various sizes, and the number of positive candidate boxes corresponding to each size can be counted separately.

[0088] Step 233, based on the number of positive candidate boxes, adjust the intersection-over-union ratio threshold, and return to the step of determining the positive and negative labels of the candidate box based on the relationship between the intersection-over-union ratio of the candidate box and the real box corresponding to the candidate box, and the intersection-over-union ratio threshold, until the number of positive candidate boxes meets the requirements.

[0089] In this embodiment, based on the number of positive candidate boxes, it is determined whether the number of current positive candidate boxes is appropriate. That is, if the number of positive candidate boxes is too large, the intersection-over-union ratio threshold is increased to reduce the number of positive candidate boxes; if the number of positive candidate boxes is too small, the intersection-over-union ratio threshold is reduced to increase the number of positive candidate boxes.

[0090] It should be noted that returning to step 231 and re-executing steps 231, 232, and 233 can redetermine the positive and negative labels of the candidate boxes until the number of positive candidate boxes meets the requirement. In other words, the positive and negative labels of the candidate boxes are dynamically changed, and the labels of the candidate boxes are dynamically redistributed according to the training process.

[0091] In one embodiment, whether the number of positive candidate boxes meets the requirement can be determined based on the ratio of the number of positive candidate boxes to the number of all candidate boxes. If the ratio is greater than a preset threshold, the number of positive candidate boxes is too large and does not meet the requirement; if the ratio is less than or equal to the preset threshold, the number of positive candidate boxes meets the requirement. The preset threshold can be set according to actual needs and is not limited here.

[0092] In another embodiment, the feature map includes candidate boxes of various sizes, and the number of positive example candidate boxes corresponding to each size can be counted separately, and then determined based on the ratio of the number of positive example candidate boxes to the number of all candidate boxes of the size corresponding to the candidate boxes. If the ratio is greater than a preset threshold, the number of positive example candidate boxes is too large and does not meet the requirements; if the ratio is less than or equal to the preset threshold, the number of positive example candidate boxes meets the requirements.

[0093] Specifically, the above step 233 includes:

[0094] Step 2331, counting the total number of candidate boxes of the size corresponding to the candidate box, and calculating the number of positive candidate boxes and the positive ratio of the total number of candidate boxes;

[0095] Specifically, the total number of candidate boxes of a size corresponding to the current candidate box in all candidate boxes in the feature map is counted.

[0096] Among them, the feature map includes candidate boxes of various sizes, and the total number of candidate boxes corresponding to the candidate boxes of each size can be counted separately.

[0097] The positive case ratio is obtained by dividing the number of positive candidate boxes by the total number of candidate boxes.

[0098] Step 2332, if the proportion of positive examples is greater than the candidate hyperparameter of the size corresponding to the candidate box, increase the intersection-over-union threshold, and return to the step of determining the positive and negative labels of the candidate box based on the intersection-over-union ratio of the candidate box and the real box corresponding to the candidate box, and the size relationship of the intersection-over-union ratio threshold, until the proportion of positive examples is less than or equal to the candidate hyperparameter of the size corresponding to the candidate box.

[0099] The candidate hyperparameter is used to select an appropriate number of positive candidate boxes. The candidate hyperparameter can be set according to actual needs and is not limited here.

[0100] In addition, the feature map includes candidate boxes of various sizes. Based on this, the candidate hyperparameters corresponding to candidate boxes of different sizes are different. For example, the candidate boxes are divided into three types of sizes: large, medium and small (L, M, S), and the candidate hyperparameters are divided into large candidate hyperparameters C L , the candidate hyperparameter C M and a small candidate hyperparameter C S .

[0101] It should be noted that the value by which the intersection-over-union ratio threshold is increased each time can be set according to actual needs and is not limited here.

[0102] It can be understood that when the positive example ratio of the number of positive candidate boxes to the total number of candidate boxes of the corresponding size is greater than the candidate hyperparameter of the corresponding size, the intersection-over-union ratio threshold is increased to redistribute the positive and negative labels of the candidate boxes, that is, based on the dynamically changing positive and negative labels, so that the positive and negative labels finally assigned to the candidate boxes are accurate labels, thereby improving the accuracy of the label assignment of the candidate boxes, and then improving the recall rate of the target detection model, and finally achieving high-performance target detection.

[0103] In another embodiment, after the above step 2331, the target detection method further includes:

[0104] If the positive example ratio is less than or equal to the candidate hyperparameter of the size corresponding to the candidate box, the candidate box is labeled based on the positive and negative labels. Specifically, if the positive example ratio is less than or equal to the candidate hyperparameter of the size corresponding to the candidate box, there is no need to adjust the intersection-over-union ratio threshold, that is, the previously determined positive and negative labels meet the requirements, and there is no need to reallocate the labels of the candidate boxes.

[0105] In this embodiment, the intersection-over-union ratio threshold is adjusted based on the number of positive candidate boxes, so as to reallocate the positive and negative labels of the candidate boxes again, that is, based on the dynamically changing positive and negative labels, so that the positive and negative labels finally assigned to the candidate boxes are accurate labels, thereby improving the accuracy of label assignment of the candidate boxes, and then improving the recall rate of the target detection model, and finally achieving high-performance target detection.

[0106] Step 240: training the model to be trained based on the candidate boxes and the positive and negative labels to obtain a target detection model.

[0107] Specifically, based on the candidate box and the positive and negative labels, a positive candidate box is determined; based on the positive candidate box, the model to be trained is trained to obtain a target detection model.

[0108] It should be noted that the positive example candidate box is input into the model to be trained, the model prediction is performed, the model output label is obtained, and then the model loss is calculated based on the preset loss function, and then the model to be trained is updated based on the model loss, that is, the model parameters in the model to be trained are updated until the number of iterations of the model to be trained reaches the preset number of iterations or the corresponding preset loss function (objective function) reaches the preset value. Among them, the preset number of iterations and the preset value can be set according to actual needs.

[0109] In a specific embodiment, the model to be trained includes a regression detection sub-model to be trained and a classification sub-model to be trained. In this embodiment, the regression detection sub-model and the classification sub-model can be jointly trained.

[0110] In this embodiment, the target detection model is trained based on the candidate box and its corresponding positive and negative labels. The positive and negative labels are determined based on the intersection-over-union ratio of the candidate box and the real box corresponding to the candidate box, and a dynamically changing intersection-over-union ratio threshold. Based on this, the embodiment of the present invention dynamically changes the positive and negative labels of the candidate box through a dynamically changing intersection-over-union ratio threshold, so that the positive and negative labels finally assigned to the candidate box are accurate labels, thereby improving the label assignment accuracy of the candidate box, and then improving the recall rate of the target detection model, and ultimately achieving high-performance target detection.

[0111] Furthermore, based on the above second embodiment, a third embodiment of the target detection method of the present invention is proposed. Figure 3 The third flowchart of the target detection method provided by the present invention is as follows: Figure 3 As shown, in this embodiment, the above step 240 includes:

[0112] Step 241, using the candidate box, the positive and negative labels and the preset loss function, the model to be trained is trained to obtain a target detection model, and the gradient value of the preset loss function is dynamically adjusted based on the training duration or the number of training times.

[0113] In this embodiment, the preset loss function is the loss function (objective function) of the model to be trained, which is used to optimize the model parameters in the model to be trained by using gradient descent.

[0114] Specifically, based on the candidate box and the positive and negative labels, a positive candidate box is determined; the positive candidate box is input into the model to be trained, model prediction is performed, and a model output label is obtained, and then based on a preset loss function, the model loss is calculated, and then based on the model loss, the model to be trained is updated, that is, the model parameters in the model to be trained are updated, until the number of iterations of the model to be trained reaches a preset number of iterations or the corresponding preset loss function (objective function) reaches a preset value. Among them, the preset number of iterations and the preset value can be set according to actual needs.

[0115] The gradient value of the preset loss function is dynamically adjusted based on the training duration or the number of training times. In other words, the gradient of the preset loss function is dynamically adjusted according to the regression situation.

[0116] Specifically, the gradient value of the preset loss function is gradually reduced based on the training time or the number of training times, so that in the early stage of training, a larger gradient value can quickly approach the local optimal point, and in the later stage of training, the gradient value is appropriately reduced to better approach the local optimal point.

[0117] The preset loss function includes a classification loss function and a bounding box regression loss function. The classification loss function is the loss function of the classification sub-model in the model to be trained, and the bounding box regression loss function is the loss function of the bounding box regression sub-model in the model to be trained.

[0118] The border regression loss function may be a SmoothL1 loss function, and the SmoothL1 loss function may be:

[0119]

[0120] Among them, x is the distance between the predicted box and the true box, and the initial value of β(time) is 1.

[0121] Furthermore, it also includes a method for adjusting the gradient value of the preset loss function:

[0122] Step 300, calculating the relative normalized regression loss of the positive candidate box in the candidate box and the target real box corresponding to the positive candidate box;

[0123] Among them, the relative normalized regression loss is the distance between the positive candidate box and the corresponding target real box. The calculation formula of the relative normalized regression loss is:

[0124]

[0125] Among them, the predicted value of the positive candidate box is t i , the true frame value is

[0126]

[0127]

[0128] x a ,y a 、w a 、h a represents the center coordinates, width, and height of the Anchor; x, y, w, and h represent the center coordinates, width, and height of the prediction box; x * ,y * 、w* 、h * Indicates the center coordinates, width, and height of the target real box.

[0129] Step 400: If the relative normalized regression loss is less than a regression threshold, the gradient value of the preset loss function is reduced, and the regression threshold is dynamically adjusted based on the training duration or the number of training times.

[0130] The regression threshold is dynamically adjusted based on the training duration or the number of training times. In other words, the gradient of the preset loss function is dynamically adjusted according to the regression situation.

[0131] The initial value of the regression threshold is set in advance and changes with the training duration or number of training times. It is used to determine whether the loss function is close to the local optimal point.

[0132] The preset loss function includes a classification loss function and a regression loss function. The classification loss function is the loss function of the classification sub-model in the model to be trained, and the bounding box regression loss function is the loss function of the bounding box regression sub-model in the model to be trained.

[0133] The border regression loss function may be a SmoothL1 loss function, and the SmoothL1 loss function may be:

[0134]

[0135] Among them, x is the distance between the predicted box and the true box, and the initial value of β(time) is 1.

[0136] It should be noted that as the training progresses, the gradient of the loss function gradually decreases, that is, β(time) is gradually increased, so that the gradient of the regression loss function gradually decreases, allowing the loss function to cross the local optimal saddle and fit better.

[0137] It can be understood that when the relative normalized regression loss is less than the regression threshold, the gradient value of the regression loss function is reduced so that the regression loss function can cross the local optimal saddle, thereby improving the regression effect.

[0138] In another embodiment, after the above step 300, the target detection method further includes:

[0139] If the relative normalized regression loss is greater than or equal to the regression threshold, the gradient value of the preset loss function is not changed, and the regression threshold is dynamically adjusted based on the training duration or the number of training times.

[0140] In this embodiment, the gradient value of the loss function of the model to be trained changes dynamically based on the training duration or the number of training times, so that the loss function quickly approaches the local optimal point in the early stage of training and better approaches the local optimal point in the later stage of training, thereby improving the regression fitting effect of the candidate box, and then improving the detection accuracy of the target detection model, and finally further realizing high-performance target detection.

[0141] The target detection device provided by the present invention is described below. The target detection device described below and the target detection method described above can be referenced to each other.

[0142] The target detection device provided by the present invention comprises:

[0143] An acquisition module is used to acquire an image to be detected containing an object to be detected;

[0144] The detection module is used to input the image to be detected into the target detection model, perform target detection, and obtain the target detection result output by the target detection model. The target detection model is trained based on the candidate box and its corresponding positive and negative labels. The positive and negative labels are determined based on the intersection-and-union ratio of the candidate box and the real box corresponding to the candidate box, as well as a dynamically changing intersection-and-union ratio threshold.

[0145] The target detection device also includes a training module, which is used to obtain a training sample image, input the training sample image into a model to be trained, perform feature extraction, and obtain a feature map corresponding to the training sample image; based on the feature map, construct candidate boxes of different scales and different aspect ratios; based on the intersection-and-union ratio of the candidate box and the real box corresponding to the candidate box, and a dynamically changing intersection-and-union ratio threshold, determine the positive and negative labels of the candidate box; based on the candidate box and the positive and negative labels, train the model to be trained to obtain a target detection model.

[0146] The training module is also used to construct candidate boxes of different scales and different aspect ratios based on the feature map and preset size categories, wherein the preset size categories are obtained based on statistical analysis of each target size in a general target dataset, and the general target dataset includes targets of multiple sizes.

[0147] The training module is also used to determine the positive and negative labels of the candidate box based on the intersection-and-union ratio of the candidate box and the real box corresponding to the candidate box, and the size relationship of the intersection-and-union ratio threshold; count the number of positive candidate boxes of the corresponding size of the candidate box; adjust the intersection-and-union ratio threshold based on the number of positive candidate boxes, and return to the step of determining the positive and negative labels of the candidate box based on the intersection-and-union ratio of the candidate box and the real box corresponding to the candidate box, and the size relationship of the intersection-and-union ratio threshold, until the number of positive candidate boxes meets the requirements.

[0148] The training module is also used to count the total number of candidate boxes of the size corresponding to the candidate box, and calculate the number of positive candidate boxes and the positive example ratio of the total number of candidate boxes; if the positive example ratio is greater than the candidate hyperparameter of the size corresponding to the candidate box, the intersection-and-union ratio threshold is increased, and the step of determining the positive and negative labels of the candidate box based on the intersection-and-union ratio of the candidate box and the real box corresponding to the candidate box and the size relationship of the intersection-and-union ratio threshold is returned until the positive example ratio is less than or equal to the candidate hyperparameter of the size corresponding to the candidate box.

[0149] The training module is also used to train the model to be trained using the candidate box, the positive and negative labels and a preset loss function to obtain a target detection model, and the gradient value of the preset loss function is dynamically adjusted based on the training duration or the number of training times.

[0150] The training module is also used to calculate the relative normalized regression loss of the positive candidate frame in the candidate frame and the target real frame corresponding to the positive candidate frame;

[0151] If the relative normalized regression loss is less than the regression threshold, the gradient value of the preset loss function is reduced, and the regression threshold is dynamically adjusted based on the training duration or the number of training times.

[0152] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the target detection method, which includes: obtaining an image to be detected containing a target to be detected; inputting the image to be detected into a target detection model, performing target detection, and obtaining a target detection result output by the target detection model, wherein the target detection model is obtained by training based on a candidate box and its corresponding positive and negative labels, and the positive and negative labels are determined based on the intersection-and-union ratio of the candidate box and the real box corresponding to the candidate box, and a dynamically changing intersection-and-union ratio threshold.

[0153] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0154] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the target detection method provided by the above methods, which method includes: obtaining an image to be detected containing a target to be detected; inputting the image to be detected into a target detection model to perform target detection, and obtaining a target detection result output by the target detection model, wherein the target detection model is trained based on candidate boxes and their corresponding positive and negative labels, and the positive and negative labels are determined based on the intersection-and-union ratio of the candidate box and the real box corresponding to the candidate box, as well as a dynamically changing intersection-and-union ratio threshold.

[0155] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the target detection method provided by the above-mentioned methods, the method comprising: obtaining an image to be detected containing a target to be detected; inputting the image to be detected into a target detection model, performing target detection, and obtaining a target detection result output by the target detection model, wherein the target detection model is trained based on a candidate box and its corresponding positive and negative labels, and the positive and negative labels are determined based on the intersection-and-union ratio of the candidate box and the real box corresponding to the candidate box, as well as a dynamically changing intersection-and-union ratio threshold.

[0156] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0157] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0158] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A target detection method, characterized in that: include: Acquire an image to be detected containing an object to be detected; Input the image to be detected into a target detection model to perform target detection, and obtain a target detection result output by the target detection model, wherein the target detection model is trained based on candidate boxes and their corresponding positive and negative labels, and the positive and negative labels are determined based on an intersection-and-union ratio between the candidate box and a real box corresponding to the candidate box, and a dynamically changing intersection-and-union ratio threshold; Also included is a method for training the target detection model: Obtain a training sample image, input the training sample image into the model to be trained, perform feature extraction, and obtain a feature map corresponding to the training sample image; Based on the feature map, construct candidate boxes of different scales and different aspect ratios; Determine the positive and negative labels of the candidate frame based on the intersection-and-union ratio of the candidate frame and the real frame corresponding to the candidate frame, and a dynamically changing intersection-and-union ratio threshold; Based on the candidate boxes and the positive and negative labels, the model to be trained is trained to obtain a target detection model; The determining of the positive and negative labels of the candidate frame based on the intersection-and-union ratio of the candidate frame and the real frame corresponding to the candidate frame, and the dynamically changing intersection-and-union ratio threshold, includes: Based on the IoU ratio between the candidate frame and the real frame corresponding to the candidate frame, and the IoU ratio threshold, the positive and negative labels of the candidate frame are determined; each size of the candidate frame corresponds to a different IoU ratio threshold; Counting the number of positive candidate boxes of the corresponding size of the candidate box; Based on the number of positive candidate boxes, the intersection-over-union ratio threshold is adjusted, and the positive and negative labels of the candidate boxes are determined based on the relationship between the intersection-over-union ratio of the candidate box and the real box corresponding to the candidate box and the intersection-over-union ratio threshold, until the number of positive candidate boxes meets the requirement; The step of adjusting the intersection-and-union ratio threshold based on the number of positive candidate boxes, and returning the relationship between the intersection-and-union ratio of the candidate box and the real box corresponding to the candidate box and the intersection-and-union ratio threshold, and determining the positive and negative labels of the candidate box until the number of positive candidate boxes meets the requirement includes: Counting the total number of candidate boxes of the corresponding size of the candidate box, and calculating the number of positive candidate boxes and the positive ratio of the total number of candidate boxes; If the proportion of positive examples is greater than the candidate hyperparameter of the size corresponding to the candidate box, the intersection-over-union ratio threshold is increased, and the step of determining the positive and negative labels of the candidate box based on the intersection-over-union ratio of the candidate box and the real box corresponding to the candidate box and the size relationship of the intersection-over-union ratio threshold is returned until the proportion of positive examples is less than or equal to the candidate hyperparameter of the size corresponding to the candidate box; the candidate hyperparameters corresponding to candidate boxes of different sizes are different.

2. The target detection method according to claim 1, characterized in that: The step of constructing candidate boxes of different scales and different aspect ratios based on the feature map includes: Based on the feature map and preset size categories, candidate boxes of different scales and different aspect ratios are constructed, wherein the preset size categories are obtained based on statistical analysis of sizes of each target in a general target dataset, and the general target dataset includes targets of various sizes.

3. The target detection method according to any one of claims 1 to 2, characterized in that: The training of the model to be trained based on the candidate box and the positive and negative labels to obtain the target detection model includes: The candidate box, the positive and negative labels and the preset loss function are used to train the model to be trained to obtain a target detection model, and the gradient value of the preset loss function is dynamically adjusted based on the training duration or the number of training times.

4. The target detection method according to claim 3, characterized in that: It also includes a method for adjusting the gradient value of the preset loss function: Calculate the relative normalized regression loss of the positive candidate box in the candidate box and the target real box corresponding to the positive candidate box; If the relative normalized regression loss is less than the regression threshold, the gradient value of the preset loss function is reduced, and the regression threshold is dynamically adjusted based on the training duration or the number of training times.

5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the target detection method according to any one of claims 1 to 4 are implemented.

6. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the target detection method according to any one of claims 1 to 4 are implemented.

7. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the target detection method according to any one of claims 1 to 4 are implemented.

Images