Sample distribution and target detection model training method and system, medium and equipment
By calculating the normalized distance between the prediction box and the actual box, filtering and reducing the prediction box to obtain more accurate positive samples, the problem of traditional object detection technology being poor in small object detection is solved, and the performance of small object detection is significantly improved.
Patent Information
- Application Number
- CN202510023890.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-07
- Publication Date
- 2025-04-25
AI Technical Summary
Traditional object detection technology has challenges in small object detection tasks, especially due to poor model training results due to low IoU.
By obtaining the two-dimensional Gaussian distribution of the prediction box and the actual box, and calculating the normalized distance between them, filtering out the candidate prediction box, narrowing the candidate prediction box to obtain more accurate positive samples, and then training the object detection model.
The performance of small object detection is significantly improved, and the training effect of the model is improved through more accurate positive sample matching results.
Smart Images
Figure CN119888189A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of target detection technology, and in particular to a sample allocation and target detection model training method, system, medium and equipment. Background Art
[0002] In recent years, object detection technology has flourished, but it still faces challenges when facing small object detection tasks. Traditional label assignment strategies mostly rely on the IoU (Intersection over Union) between the predicted box and the true box for judgment, but this method has large differences in sensitivity to objects of different scales, especially for small objects. Even if the target category is accurately predicted, it cannot be effectively labeled due to the low IoU, resulting in poor model training results.
[0003] Existing improved methods for small object detection, such as OTA (Optimized Transport Algorithm for Instance Segmentation) and TAL (Tasks-Aligned Labeling Assignment), try to optimize label assignment through more complex strategies, but still have limitations. Although methods such as OTA take into account the IoU cost, they still cannot completely solve the problem of lack of overlap between small objects and anchor boxes or center points. Although TAL attempts to integrate the indicators of classification and localization tasks, it does not fully consider the characteristics of objects of different scales. Summary of the invention
[0004] Based on this, it is necessary to propose a sample allocation and target detection model training method, system, medium and equipment to address the above problems.
[0005] A method for sample allocation and target detection model training, the method comprising:
[0006] An image sample is obtained, and the image sample is input into a target detection model to obtain a prediction box predicted by the target detection model.
[0007] The two-dimensional Gaussian distributions corresponding to the predicted frame and the actual frame of the image sample are obtained, and a first normalized distance between the two-dimensional Gaussian distributions of the predicted frame and the actual frame is determined.
[0008] According to the first normalization-distance, the first k prediction boxes are selected as candidate prediction boxes.
[0009] The candidate prediction box is reduced to obtain a reduced prediction box, a two-dimensional Gaussian distribution of the reduced prediction box is obtained, and a second normalized distance between the two-dimensional Gaussian distribution of the reduced prediction box and the two-dimensional Gaussian distribution of the actual box is determined.
[0010] A reduced prediction box closest to the actual box is determined according to the first normalized distance and the second normalized distance as a positive sample.
[0011] The object detection model is trained based on the positive samples of the actual frame to obtain a trained object detection model.
[0012] The obtaining of the two-dimensional Gaussian distributions corresponding to the predicted frame and the actual frame of the image sample, respectively, and determining a first normalized distance between the two-dimensional Gaussian distributions of the predicted frame and the actual frame specifically includes:
[0013] Get the probability density function of the predicted box and the probability density function of the actual box.
[0014] When the probability density function of the predicted frame and the probability density function of the actual frame meet preset conditions, the parameters of the inscribed ellipse equation of the predicted frame are determined, and the parameters of the inscribed ellipse equation of the actual frame are determined.
[0015] The two-dimensional Gaussian distribution of the predicted box is determined by the parameters of the inscribed ellipse equation of the predicted box, and the two-dimensional Gaussian distribution of the actual box is determined by the parameters of the inscribed ellipse equation of the actual box.
[0016] The determining of a first normalized distance between the two-dimensional Gaussian distribution of the predicted box and the actual box specifically includes:
[0017] A Wasserstein distance between the two-dimensional Gaussian distribution of the predicted box and the two-dimensional Gaussian distribution of the actual box is determined according to the two-dimensional Gaussian distribution of the predicted box and the two-dimensional Gaussian distribution of the actual box.
[0018] The Wasserstein distance is normalized to determine the first normalized distance.
[0019] Among them, it also includes:
[0020] If the probability density function satisfies (x-μ) T Σ -1 (x-μ)=1, it is determined that the probability density function meets the preset condition.
[0021] The two-dimensional Gaussian distribution is:
[0022]
[0023] Among them, cx is the horizontal coordinate of the predicted box or the actual box, c y is the ordinate of the predicted box or the actual box, w is the width of the predicted box or the actual box, and h is the height of the predicted box or the actual box.
[0024] The step of selecting the first k prediction boxes as candidate prediction boxes according to the first normalized distance specifically includes:
[0025] Obtain the classification score and positioning score output by the target detection model for the sample image.
[0026] The classification score, the positioning score and the first normalized distance are combined to obtain a first metric.
[0027] The first k prediction boxes are selected according to the first metric as candidate prediction boxes.
[0028] The step of training the target detection model based on the positive sample of the actual frame to obtain the trained target detection model specifically includes:
[0029] According to the binary cross entropy loss function and the positioning loss function, the total loss function between the actual frame and the positive sample is determined by the auxiliary positioning loss function. The total loss function is:
[0030] L loss =λ 1 L BCE +λ 2 L CIoU +λ 3 L DFL ;
[0031] Where, L BCE is the binary cross entropy loss function, L CIoU is the positioning loss function, L DFL is the loss function to assist in localization, λ 1 is the weight of the binary cross entropy loss function, λ 2 is the weight of the positioning loss function, λ 3 To help locate the weight of the loss function.
[0032] The model parameters of the target detection model are optimized by the total loss function to obtain a trained target detection model.
[0033] Among them, it also includes:
[0034] Select a large model as the teacher model and a lightweight model as the student model.
[0035] The prediction results are obtained through the teacher model as soft labels, and the student model uses the soft labels to perform knowledge distillation and performs target alignment during the knowledge distillation process.
[0036] The student model after knowledge distillation is trained to obtain the final object detection model.
[0037] Among them, it also includes:
[0038] The parameters of the target detection model are optimized by the position distillation loss function, which is:
[0039]
[0040] in, is the position distillation loss function, is the KL divergence loss function, z S is the predicted output of the student model, z t is the predicted output of the teacher model, and τ is the distillation temperature parameter.
[0041] A sample allocation and target detection model training system, the system comprising:
[0042] The prediction box acquisition module is used to acquire image samples, input the image samples into the target detection model, and obtain the prediction box predicted by the target detection model.
[0043] The first normalized distance determination module is used to obtain the two-dimensional Gaussian distributions corresponding to the predicted frame and the actual frame of the image sample, and determine the first normalized distance between the two-dimensional Gaussian distributions of the predicted frame and the actual frame.
[0044] The candidate prediction box screening module is used to screen out the first k prediction boxes as candidate prediction boxes according to the first normalized distance.
[0045] The second normalized distance determination module is used to reduce the candidate prediction box to obtain a reduced prediction box, obtain the two-dimensional Gaussian distribution of the reduced prediction box, and determine the second normalized distance between the two-dimensional Gaussian distribution of the reduced prediction box and the two-dimensional Gaussian distribution of the actual box.
[0046] A positive sample determination module is used to determine a reduced prediction frame closest to the actual frame according to the first normalized distance and the second normalized distance, and use the reduced prediction frame as a positive sample.
[0047] The target detection model training module is used to train the target detection model based on the positive samples of the actual frame to obtain a trained target detection model.
[0048] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor executes the steps of the method described above.
[0049] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the above method.
[0050] The embodiments of the present invention have the following beneficial effects:
[0051] The present invention significantly improves the performance of small target detection through a sample allocation and target detection model training method. The method first uses normalized distance to quantify the similarity between the prediction box and the actual box. In view of the problem of insufficient small target samples, the first normalized distance between the two-dimensional Gaussian distribution of the prediction box and the actual box is first determined. The first k prediction boxes are selected according to the first normalized distance as candidate prediction boxes. Further, the candidate prediction box is reduced to obtain a reduced prediction box, and the two-dimensional Gaussian distribution of the reduced prediction box is obtained. The second normalized distance between the two-dimensional Gaussian distribution of the reduced prediction box and the two-dimensional Gaussian distribution of the actual box is determined. Finally, the reduced prediction box closest to the actual box is determined according to the first normalized distance and the second normalized distance to obtain a more accurate positive sample matching result. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0053] in:
[0054] Figure 1 A schematic diagram of a flow chart of an embodiment of a method for sample allocation and target detection model training provided by the present invention;
[0055] Figure 2 A flow chart of another embodiment of a sample allocation and target detection model training method provided by the present invention;
[0056] Figure 3 A schematic flow chart of another embodiment of a sample allocation and target detection model training method provided by the present invention;
[0057] Figure 4 A schematic diagram of the structure of an embodiment of a sample allocation and target detection model training system provided by the present invention;
[0058] Figure 5 A schematic diagram of the structure of an embodiment of the device provided by the present invention;
[0059] Figure 6 A schematic structural diagram of an embodiment of a storage medium provided by the present invention. DETAILED DESCRIPTION
[0060] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0061] like Figure 1 As shown, Figure 1 A flow chart of an embodiment of a method for sample allocation and target detection model training provided by the present invention. A method for sample allocation and target detection model training, the method comprising:
[0062] S101: Obtain an image sample, input the image sample into a target detection model, and obtain a prediction box predicted by the target detection model.
[0063] S102: Obtain two-dimensional Gaussian distributions corresponding to the predicted frame and the actual frame of the image sample, respectively, and determine a first normalized distance between the two-dimensional Gaussian distributions of the predicted frame and the actual frame.
[0064] Exemplarily, an image sample is obtained, and the image sample is input into the target detection model to obtain the predicted box predicted by the target detection model. The probability density function of the predicted box and the probability density function of the actual box are obtained, and the probability density function is:
[0065] The probability density function of the prediction box is:
[0066]
[0067] Among them, μ 1 is the two-dimensional Gaussian distribution of the prediction box, Σ 1 is the covariance matrix of the two-dimensional Gaussian distribution of the prediction box, x is the center coordinate, including the horizontal coordinate and the vertical coordinate.
[0068] The probability density function of the actual box is:
[0069]
[0070] Among them, μ 2 is the two-dimensional Gaussian distribution of the actual box, Σ 2 is the covariance matrix of the two-dimensional Gaussian distribution of the actual box.
[0071] When the probability density function of the predicted box and the probability density function of the actual box meet the preset conditions, the parameters of the inscribed ellipse equation of the predicted box and the parameters of the inscribed ellipse equation of the actual box are determined;
[0072] Furthermore, the two-dimensional Gaussian distribution of the predicted box is determined by the parameters of the inscribed ellipse equation, and the two-dimensional Gaussian distribution of the actual box is determined by the parameters of the inscribed ellipse equation of the actual box. Specifically, the two-dimensional Gaussian distribution of the predicted box is:
[0073]
[0074] Among them, c x1 is the horizontal coordinate of the prediction box, c y1 is the ordinate of the prediction box, w 1 is the width of the prediction box, h 1 is the height of the prediction box.
[0075] The two-dimensional Gaussian distribution of the actual box is:
[0076]
[0077] Among them, c x2 is the horizontal coordinate of the actual box, c y2 is the ordinate of the actual box, w 2 is the actual box width, h 2 The actual frame height.
[0078] Furthermore, the Wasserstein distance of the predicted box is determined according to the two-dimensional Gaussian distribution of the predicted box, and the Wasserstein distance of the actual box is determined according to the two-dimensional Gaussian distribution of the actual box. The Wasserstein distance is normalized to determine the first normalized distance, which is:
[0079]
[0080] in, is the Wasserstein distance between the two-dimensional Gaussian distribution of the predicted box and the actual box, and c is an adjustment factor between the classification score and the positioning IoU score.
[0081] S103: Filter out the first k prediction boxes as candidate prediction boxes according to the first normalized distance.
[0082] Exemplarily, the classification score and the positioning score output by the target detection model for the sample image are obtained; the classification score, the positioning score and the first normalized distance are combined to obtain a first metric; and the top k prediction boxes are selected according to the first metric as candidate prediction boxes.
[0083] S104: Shrinking the candidate prediction box to obtain a shrunken prediction box, obtaining a two-dimensional Gaussian distribution of the shrunken prediction box, and determining a second normalized distance between the two-dimensional Gaussian distribution of the shrunken prediction box and the two-dimensional Gaussian distribution of the actual box.
[0084] Exemplarily, by reducing the area of the candidate prediction box to focus on a smaller target, a reduced prediction box is obtained, a two-dimensional Gaussian distribution of the reduced prediction box is obtained, and a second normalized distance between the two-dimensional Gaussian distribution of the reduced prediction box and the two-dimensional Gaussian distribution of the actual box is determined. The second normalized distance is:
[0085]
[0086] in, is the Wasserstein distance between the two-dimensional Gaussian distribution of the reduced prediction box and the actual box, and c is an adjustment factor between the classification score and the positioning IoU score.
[0087] It should be noted that when reducing, 80% of the area of the previous pre-allocated prediction box is used as the prediction box size of the current stage.
[0088] S105: Determine a reduced prediction frame that is closest to the actual frame according to the first normalized distance and the second normalized distance, and use the reduced prediction frame as a positive sample.
[0089] Exemplarily, the reduced prediction frame closest to the actual frame is determined according to the first normalized distance and the second normalized distance, and the distance of the reduced prediction frame closest to the actual frame is:
[0090] d 3 =d 1 ×r+d 2 ×(1-r);
[0091] Among them, d 3 is the distance between the shrunken prediction box and the actual box, d 1 is the first normalized distance, d 2 is the second normalized distance, and r is the mask.
[0092] S106: Train the target detection model based on the positive samples of the actual frame to obtain a trained target detection model.
[0093] Exemplarily, according to the binary cross entropy loss function, the positioning loss function, and the auxiliary positioning loss function, the total loss function between the actual box and the positive sample is determined, and the total loss function is:
[0094] L loss =λ 1 L BCE+λ 2 L CIoU +λ 3 L DFL ;
[0095] Where, L BCE is the binary cross entropy loss function, L CIoU is the positioning loss function, L DFL is the loss function to assist in localization, λ 1 is the weight of the binary cross entropy loss function, λ 2 is the weight of the positioning loss function, λ 3 To help locate the weight of the loss function.
[0096] The model parameters of the target detection model are optimized through the total loss function to obtain the trained target detection model.
[0097] The present invention significantly improves the performance of small target detection through a sample allocation and target detection model training method. The method first uses normalized distance to quantify the similarity between the prediction box and the actual box. In view of the problem of insufficient small target samples, the first normalized distance between the two-dimensional Gaussian distribution of the prediction box and the actual box is first determined. The first k prediction boxes are selected according to the first normalized distance as candidate prediction boxes. Further, the candidate prediction box is reduced to obtain a reduced prediction box, and the two-dimensional Gaussian distribution of the reduced prediction box is obtained. The second normalized distance between the two-dimensional Gaussian distribution of the reduced prediction box and the two-dimensional Gaussian distribution of the actual box is determined. Finally, the reduced prediction box closest to the actual box is determined according to the first normalized distance and the second normalized distance to obtain a more accurate positive sample matching result.
[0098] like Figure 2 As shown, Figure 2 A flow chart of another embodiment of a method for sample allocation and target detection model training provided by the present invention. A method for sample allocation and target detection model training, the method comprising:
[0099] S201: Obtain an image sample, input the image sample into a target detection model, and obtain a prediction box predicted by the target detection model.
[0100] S202: Obtain a probability density function of the predicted box and a probability density function of the actual box.
[0101] Exemplarily, the probability density function of the prediction box is:
[0102]
[0103] Among them, μ 1 is the two-dimensional Gaussian distribution of the prediction box, Σ 1is the covariance matrix of the two-dimensional Gaussian distribution of the prediction box, x is the center coordinate, including the horizontal coordinate and the vertical coordinate.
[0104] The probability density function of the actual box is:
[0105]
[0106] Among them, μ 2 is the two-dimensional Gaussian distribution of the actual box, Σ 2 is the covariance matrix of the two-dimensional Gaussian distribution of the actual box.
[0107] S203: When the probability density function satisfies a preset condition, the parameters of the equation of the inscribed ellipse of the predicted frame are determined, and the parameters of the equation of the inscribed ellipse of the actual frame are determined.
[0108] Exemplarily, if the probability density function satisfies (x-μ) T Σ -1 (x-μ)=1, it is determined that the probability density function meets the preset condition;
[0109] The two-dimensional Gaussian distribution is:
[0110]
[0111] Among them, c x is the horizontal coordinate of the predicted box or the actual box, c y is the ordinate of the predicted box or the actual box, w is the width of the predicted box or the actual box, and h is the height of the predicted box or the actual box.
[0112] Specifically, if the probability density function of the predicted box satisfies (x-μ 1 ) T Σ -1 (x-μ 1 )=1, it is determined that the prediction box meets the preset conditions, and the parameters of the inscribed ellipse equation of the prediction box are determined. The inscribed ellipse equation of the prediction box is:
[0113]
[0114] Among them, μ x1 is the horizontal coordinate of the center coordinate of the inscribed ellipse of the prediction box, μ y1 is the ordinate of the center coordinate of the inscribed ellipse of the prediction box, σ x1 is the semi-axis length of the horizontal axis of the inscribed ellipse of the prediction box, σ y1 It is the length of the longitudinal semi-axis of the inscribed ellipse of the prediction box.
[0115] At the same time, if the probability density function of the actual box satisfies (x-μ 2 ) T Σ -1(x-μ 2 )=1, it is determined that the actual frame meets the preset conditions, and the parameters of the inscribed ellipse equation of the actual frame are determined. The inscribed ellipse equation of the actual frame is:
[0116]
[0117] Among them, μ x2 is the horizontal coordinate of the center coordinate of the inscribed ellipse of the actual frame, μ y2 is the ordinate of the center coordinate of the inscribed ellipse of the actual frame, σ x2 is the semi-axis length of the horizontal axis of the inscribed ellipse of the actual frame, σ y2 It is the length of the longitudinal semi-axis of the inscribed ellipse of the actual frame.
[0118] S204: Determine the two-dimensional Gaussian distribution of the predicted box through the parameters of the inscribed ellipse equation of the predicted box, and determine the two-dimensional Gaussian distribution of the actual box through the parameters of the inscribed ellipse equation of the actual box.
[0119] Exemplarily, the two-dimensional Gaussian distribution of the prediction box is:
[0120]
[0121] Among them, c x1 is the horizontal coordinate of the prediction box, c y1 is the ordinate of the prediction box, w 1 is the width of the prediction box, h 1 is the height of the prediction box.
[0122] The two-dimensional Gaussian distribution of the actual box is:
[0123]
[0124] Among them, c x2 is the horizontal coordinate of the actual box, c y2 is the ordinate of the actual box, w 2 is the actual box width, h 2 The actual frame height.
[0125] S205: Determine a first normalized distance between the two-dimensional Gaussian distributions of the predicted box and the actual box.
[0126] Exemplarily, the Wasserstein distance between the two-dimensional Gaussian distribution of the predicted box and the two-dimensional Gaussian distribution of the actual box is determined according to the two-dimensional Gaussian distribution of the predicted box and the two-dimensional Gaussian distribution of the actual box. The Wasserstein distance is:
[0127]
[0128] Among them, μ 1is the two-dimensional Gaussian distribution of the prediction box, Σ 1 is the covariance matrix of the two-dimensional Gaussian distribution of the prediction box, μ 2 is the two-dimensional Gaussian distribution of the actual box, Σ 2 is the covariance matrix of the two-dimensional Gaussian distribution of the actual box, m 1 is the mean vector of the two-dimensional Gaussian distribution of the prediction box, m 2 is the mean vector of the two-dimensional Gaussian distribution of the actual box;
[0129] Furthermore, the Wasserstein distance is normalized to determine the first normalized distance, the first normalized distance d 1 for:
[0130]
[0131] in, is the Wasserstein distance between the two-dimensional Gaussian distribution of the predicted box and the actual box, and c is an adjustment factor between the classification score and the positioning IoU score.
[0132] S206: Obtain the classification score and positioning score output by the target detection model for the sample image.
[0133] S207: Combining the classification score, the positioning score and the first normalized distance to obtain a first metric.
[0134] Exemplarily, the classification score and positioning score output by the target detection model for the sample image are obtained. In order to alleviate the inconsistency between the classification task and the positioning task, the first metric is obtained by combining the classification score, the positioning score and the first normalized distance. The first metric is:
[0135] T=s α ×(γ 1 ×μ+γ 2 ×d 1 ) β ;
[0136] Among them, s is the classification score of the predicted box, μ is the positioning IoU score, and d 1 is the first normalized distance between the predicted box and the true box, α is the hyperparameter for adjusting the classification, β is the hyperparameter for the positioning task, and γ 1 and γ 2 To tune the hyperparameters of the positioning indicators in the metric.
[0137] After obtaining the normalized distance, this metric improves the overall performance of the model by integrating the scores of these two different tasks.
[0138] S208: Filter out the first k prediction boxes as candidate prediction boxes according to the first metric.
[0139] Exemplarily, the prediction boxes are sorted in descending order according to the first metric, and the top k prediction boxes with the highest scores are used as candidate prediction boxes.
[0140] S209: Shrinking the candidate prediction box to obtain a reduced prediction box, obtaining a two-dimensional Gaussian distribution of the reduced prediction box, and determining a second normalized distance between the two-dimensional Gaussian distribution of the reduced prediction box and the two-dimensional Gaussian distribution of the actual box.
[0141] Exemplarily, the candidate prediction box is reduced to obtain a reduced prediction box, and a two-dimensional Gaussian distribution of the reduced prediction box is obtained. The two-dimensional Gaussian distribution of the reduced prediction box is:
[0142]
[0143] Among them, μ 3 is the two-dimensional Gaussian distribution of the reduced prediction box, Σ 3 is the covariance matrix of the two-dimensional Gaussian distribution of the reduced prediction box, c x3 is the horizontal coordinate of the reduced prediction box, c y3 is the ordinate of the reduced prediction box, w 3 is the width of the reduced prediction box, h 3 is the height of the reduced prediction box.
[0144] Further, a second normalized distance between the two-dimensional Gaussian distribution of the reduced prediction box and the two-dimensional Gaussian distribution of the actual box is determined.
[0145] According to the two-dimensional Gaussian distribution of the reduced prediction box and the two-dimensional Gaussian distribution of the actual box, the Wasserstein distance between the two-dimensional Gaussian distribution of the reduced prediction box and the actual box is determined. The Wasserstein distance is:
[0146]
[0147] Among them, μ 3 is the two-dimensional Gaussian distribution of the reduced prediction box, μ 2 is the two-dimensional Gaussian distribution of the actual box, m 3 is the mean vector of the two-dimensional Gaussian distribution of the prediction box, m 2 is the mean vector of the two-dimensional Gaussian distribution of the actual box, Σ 2 is the covariance matrix of the two-dimensional Gaussian distribution of the actual box, Σ 3is the covariance matrix of the two-dimensional Gaussian distribution of the reduced prediction box;
[0148] Furthermore, the Wasserstein distance between the two-dimensional Gaussian distribution of the reduced prediction box and the actual box is normalized to determine the second normalized distance, which is:
[0149]
[0150] in, is the Wasserstein distance between the two-dimensional Gaussian distribution of the reduced prediction box and the actual box, and c is an adjustment factor between the classification score and the positioning IoU score.
[0151] S210: Determine the reduced prediction box closest to the actual box according to the first normalized distance and the second normalized distance as a positive sample.
[0152] Exemplarily, the reduced prediction frame closest to the actual frame is determined according to the first normalized distance and the second normalized distance, and the distance of the reduced prediction frame closest to the actual frame is:
[0153] d 3 =d 1 ×r+d 2 ×(1-r);
[0154] Among them, d 3 is the distance between the shrunken prediction box and the actual box, d 1 is the first normalized distance, d 2 is the second normalized distance, and r is the mask.
[0155] According to the first normalized distance and the second normalized distance, the reduced prediction box closest to the actual box is determined as a positive sample.
[0156] S211: Train the target detection model based on the positive samples of the actual frame to obtain a trained target detection model.
[0157] Exemplarily, according to the binary cross entropy loss function, the positioning loss function, and the auxiliary positioning loss function, the total loss function between the actual box and the positive sample is determined, and the total loss function is:
[0158] L loss =λ 1 L BCE +λ 2 L CIoU +λ 3 L DFL ;
[0159] Where, L BCEis the binary cross entropy loss function, L CIoU is the positioning loss function, L DFL is the loss function to assist in localization, λ 1 is the weight of the binary cross entropy loss function, λ 2 is the weight of the positioning loss function, λ 3 To help locate the weight of the loss function;
[0160] The model parameters of the target detection model are optimized through the total loss function to obtain the trained target detection model.
[0161] The present invention is based on the cross entropy loss function as the classification loss and the localization loss function as the loss function of the localization branch, realizes the classification and localization tasks through independent branches, and further enhances the localization ability of the bounding box by introducing the auxiliary localization loss function, thereby effectively improving the performance of small target detection.
[0162] like Figure 3 As shown, Figure 3 A flow chart of another embodiment of a method for sample allocation and target detection model training provided by the present invention. A method for sample allocation and target detection model training, the method comprising:
[0163] S301: Select a large model as the teacher model and a lightweight model as the student model.
[0164] S302: The prediction results are obtained as soft labels through the teacher model, and the student model uses the soft labels to perform knowledge distillation and performs target alignment during the knowledge distillation process.
[0165] For example, the high-performance large-parameter model FCOS-ResNet101 is selected as the teacher model, and the lightweight model FCOSLite is selected as the student model. For the same input image, both the teacher model and the student model independently predict the target information. The teacher model only uses the inference mode to obtain the prediction results as soft labels and does not participate in gradient back propagation. The student model uses the soft labels for knowledge distillation and performs target alignment during the knowledge distillation process. Specifically, the prediction output of the teacher model is represented as z T , the predicted output of the student model is denoted as z S . To align the outputs of the two models into a probability distribution p, a SoftMax function S(·,τ)=SoftMax(·,τ) is used, where τ denotes the distillation temperature parameter. It is worth noting that when τ=1, the function reflects the original SoftMax function. When τ approaches 0, the probability distribution becomes a Dirac distribution, while for τ→∞, it converges to a uniform distribution. Empirical observations show that for τ>1, the probabilities exhibit smoother behavior, enhancing the information content carried by the probability distribution.
[0166] S303: Train the student model after knowledge distillation to obtain the final target detection model.
[0167] Exemplarily, the parameters of the target detection model are optimized by a position distillation loss function, where the position distillation loss function is:
[0168]
[0169] in, is the position distillation loss function, is the KL divergence loss function, z S is the predicted output of the student model, z T is the predicted output of the teacher model, and τ is the distillation temperature parameter.
[0170] From the above description, we can see that by applying knowledge distillation technology, the knowledge of the high-performance model is transferred to the lightweight model to improve the performance of the lightweight target detection model. In addition, the output of the teacher model and the student model are aligned through the SoftMax function and the KL divergence loss function, which effectively improves the accuracy of target detection.
[0171] like Figure 4 As shown, Figure 4 A schematic diagram of a sample allocation and target detection model training system according to an embodiment of the present invention. A sample allocation and target detection model training system, the system comprises:
[0172] The prediction box acquisition module 11 is used to acquire image samples, input the image samples into the target detection model, and obtain the prediction box predicted by the target detection model.
[0173] The first normalized distance determination module 12 is used to obtain the two-dimensional Gaussian distributions corresponding to the predicted frame and the actual frame of the image sample, and determine the first normalized distance between the two-dimensional Gaussian distributions of the predicted frame and the actual frame.
[0174] The candidate prediction box screening module 13 is used to screen out the first k prediction boxes as candidate prediction boxes according to the first normalized distance.
[0175] The second normalized distance determination module 14 is used to reduce the candidate prediction box to obtain a reduced prediction box, obtain the two-dimensional Gaussian distribution of the reduced prediction box, and determine the second normalized distance between the two-dimensional Gaussian distribution of the reduced prediction box and the two-dimensional Gaussian distribution of the actual box.
[0176] The positive sample determination module 15 is used to determine the reduced prediction frame closest to the actual frame according to the first normalized distance and the second normalized distance, and use the reduced prediction frame as the positive sample.
[0177] The target detection model training module 16 is used to train the target detection model based on the positive samples of the actual frame to obtain a trained target detection model.
[0178] Exemplarily, in the prediction frame acquisition module 11, an image sample is obtained, and the image sample is input into the target detection model to obtain the prediction frame predicted by the target detection model. In the first normalized distance determination module 12, the probability density function of the prediction frame and the probability density function of the actual frame are obtained; when the probability density function of the prediction frame and the probability density function of the actual frame meet the preset conditions, the parameters of the inscribed ellipse equation of the prediction frame and the parameters of the inscribed ellipse equation of the actual frame are determined; the two-dimensional Gaussian distribution of the prediction frame is determined by the parameters of the inscribed ellipse equation of the prediction frame, and the two-dimensional Gaussian distribution of the actual frame is determined by the parameters of the inscribed ellipse equation of the actual frame. In the candidate prediction frame screening module 13, the classification score and positioning score output by the target detection model for the sample image are obtained; the classification score, positioning score and the first normalized distance are combined to obtain the first metric; the top k prediction frames are screened out according to the first metric as candidate prediction frames. In the second normalized distance determination module 14, the candidate prediction box is reduced to obtain a reduced prediction box, the two-dimensional Gaussian distribution of the reduced prediction box is obtained, and the second normalized distance between the two-dimensional Gaussian distribution of the reduced prediction box and the two-dimensional Gaussian distribution of the actual box is determined. In the positive sample determination module 15, the reduced prediction box closest to the actual box is determined according to the first normalized distance and the second normalized distance, and the reduced prediction box is used as a positive sample. In the target detection model training module 16, the target detection model is trained based on the positive sample of the actual box to obtain a trained target detection model.
[0179] like Figure 5 As shown, Figure 5 The device 20 includes a memory 21 and a processor 22. The memory 21 stores a computer program, and the processor 22 executes the computer program when working to implement the following. Figure 1 , Figure 2 and Figure 3 The method shown.
[0180] The specific technical details of a sample allocation and target detection model training method implemented when the above-mentioned device 20 executes a computer program have been discussed in detail in the aforementioned method steps, so they will not be repeated here.
[0181] like Figure 6 As shown, Figure 6 The storage medium 30 stores at least one computer program 31, which is executed by the processor 22 to implement the following. Figure 1 , Figure 2 and Figure 3 In one embodiment, the medium 30 may be a storage chip, a hard disk, a mobile hard disk, a USB flash drive, an optical disk, or other readable and writable storage tools, or a server, etc.
[0182] The above describes specific embodiments of the present specification, and other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily have to be performed in the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0183] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, equipment, and non-volatile computer-readable storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.
[0184] The apparatus, device, non-volatile computer-readable storage medium and method provided in the embodiments of this specification correspond to each other, and therefore, the apparatus, device, and non-volatile computer storage medium also have similar beneficial technical effects as the corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the corresponding apparatus, device, and non-volatile computer storage medium will not be repeated here.
[0185] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
[0186] For the convenience of description, the above device is described by being divided into various units according to their functions and described separately. Of course, when implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware. It should be understood by those skilled in the art that this specification embodiment can be provided as a method, system, or computer program product. Therefore, this specification embodiment can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, this specification embodiment can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0187] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0188] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0189] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0190] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0191] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0192] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0193] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0194] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.
[0195] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0196] The above disclosure is only the preferred embodiment of the present invention, which certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.
Claims
1. A sample allocation and target detection model training method, characterized in that: The method comprises: Obtain an image sample, input the image sample into a target detection model, and obtain a prediction box predicted by the target detection model; Obtaining two-dimensional Gaussian distributions corresponding to the predicted frame and the actual frame of the image sample, respectively, and determining a first normalized distance between the two-dimensional Gaussian distributions of the predicted frame and the actual frame; Filter out the first k prediction boxes as candidate prediction boxes according to the first normalized distance; Shrinking the candidate prediction box to obtain a reduced prediction box, obtaining a two-dimensional Gaussian distribution of the reduced prediction box, and determining a second normalized distance between the two-dimensional Gaussian distribution of the reduced prediction box and the two-dimensional Gaussian distribution of the actual box; Determine, according to the first normalized distance and the second normalized distance, a reduced prediction frame closest to the actual frame as a positive sample; The object detection model is trained based on the positive samples of the actual frame to obtain a trained object detection model.
2. A sample allocation and target detection model training method according to claim 1, characterized in that: The acquiring the two-dimensional Gaussian distributions corresponding to the predicted frame and the actual frame of the image sample, respectively, and determining a first normalized distance between the two-dimensional Gaussian distributions of the predicted frame and the actual frame, specifically includes: Get the probability density function of the predicted box and the probability density function of the actual box; When the probability density function of the predicted frame and the probability density function of the actual frame meet a preset condition, the parameters of the inscribed ellipse equation of the predicted frame and the parameters of the inscribed ellipse equation of the actual frame are determined; The two-dimensional Gaussian distribution of the predicted box is determined by the parameters of the inscribed ellipse equation of the predicted box, and the two-dimensional Gaussian distribution of the actual box is determined by the parameters of the inscribed ellipse equation of the actual box.
3. A sample allocation and target detection model training method according to claim 2, characterized in that: The determining a first normalized distance between the two-dimensional Gaussian distribution of the predicted frame and the actual frame specifically includes: Determine a Wasserstein distance between the two-dimensional Gaussian distribution of the predicted box and the two-dimensional Gaussian distribution of the actual box according to the two-dimensional Gaussian distribution of the predicted box and the two-dimensional Gaussian distribution of the actual box; The Wasserstein distance is normalized to determine the first normalized distance.
4. A sample allocation and target detection model training method according to claim 3, characterized in that: Also includes: If the probability density function satisfies (x-μ) T Σ -1 (x-μ)=1, it is determined that the probability density function meets the preset condition; The two-dimensional Gaussian distribution is: Among them, c x is the horizontal coordinate of the predicted box or the actual box, c y is the ordinate of the predicted box or the actual box, w is the width of the predicted box or the actual box, and h is the height of the predicted box or the actual box.
5. The method for sample allocation and target detection model training according to claim 1, characterized in that: The step of selecting the first k prediction boxes as candidate prediction boxes according to the first normalized distance specifically includes: Obtaining a classification score and a positioning score output by the target detection model for the sample image; Combining the classification score, the positioning score and the first normalized distance to obtain a first metric; The first k prediction boxes are selected according to the first metric as candidate prediction boxes.
6. A sample allocation and target detection model training method according to claim 1, characterized in that: The training of the target detection model based on the positive sample of the actual frame to obtain the trained target detection model specifically includes: According to the binary cross entropy loss function and the positioning loss function, the total loss function between the actual frame and the positive sample is determined by the auxiliary positioning loss function. The total loss function is: L loss =λ1L BCE +λ2L CIoU +λ3L DFL ; Where, L BCE is the binary cross entropy loss function, L CIoU is the positioning loss function, L DFL is the auxiliary positioning loss function, λ1 is the weight of the binary cross entropy loss function, λ2 is the weight of the positioning loss function, and λ3 is the weight of the auxiliary positioning loss function; The model parameters of the target detection model are optimized by the total loss function to obtain a trained target detection model.
7. A sample allocation and target detection model training method according to claim 1, characterized in that: Also includes: Choose a large model as the teacher model and a lightweight model as the student model; The prediction results are obtained through the teacher model as soft labels, and the student model uses the soft labels to perform knowledge distillation and performs target alignment during the knowledge distillation process; The student model after knowledge distillation is trained to obtain the final object detection model.
8. The method for extracting entities in the power field based on a scenario learning model according to claim 7 is characterized in that: Also includes: The parameters of the target detection model are optimized by the position distillation loss function, which is: in, is the position distillation loss function, is the KL divergence loss function, z S is the predicted output of the student model, z T is the predicted output of the teacher model, and τ is the distillation temperature parameter.
9. A sample allocation and target detection model training system, characterized in that: The system comprises: A prediction box acquisition module is used to acquire image samples, input the image samples into the target detection model, and obtain the prediction box predicted by the target detection model; a first normalized distance determination module, configured to obtain two-dimensional Gaussian distributions corresponding to the predicted frame and the actual frame of the image sample, respectively, and determine a first normalized distance between the two-dimensional Gaussian distributions of the predicted frame and the actual frame; A candidate prediction box screening module is used to screen out the first k prediction boxes as candidate prediction boxes according to the first normalized distance; A second normalized distance determination module, used to reduce the candidate prediction box to obtain a reduced prediction box, obtain a two-dimensional Gaussian distribution of the reduced prediction box, and determine a second normalized distance between the two-dimensional Gaussian distribution of the reduced prediction box and the two-dimensional Gaussian distribution of the actual box; a positive sample determination module, configured to determine a reduced prediction frame closest to the actual frame according to the first normalized distance and the second normalized distance, and use the reduced prediction frame as a positive sample; The target detection model training module is used to train the target detection model based on the positive samples of the actual frame to obtain a trained target detection model.
10. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 8.
11. A computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 8.