Sample selection method and device, classifier training method and device

Abstract

The present invention provides a sample selection method and device, and a classifier training method and device, which are applied to a single-stage target detector, wherein the sample selection method includes: constructing a cost function that simultaneously considers classification loss and regression loss; using the cost function to calculate the cost of all candidate samples according to the prediction information output by the detection network and the target true value; using prior information to screen candidate samples; sorting the screened candidate samples according to the cost, and selecting the candidate samples with the highest priority. N The samples with the lowest cost are positive samples, and the rest are negative samples. The classifier training method and device distinguish all positive samples according to the IoU value between the regression box and the target box, improve the contribution of high-quality candidate samples to the classification loss, and suppress the influence of low-quality samples.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, in particular to target detection, and is specifically related to a sample selection method and device, and a classifier training method and device. Background Art

[0002] At present, the target detection network based on deep learning is mainly divided into two-stage detectors and single-stage detectors. Two-stage detectors, such as Faster-RCNN, first extract candidate regions through the region generation network, and then send the candidate regions to the detection network for target classification and position regression; the detection accuracy of the two-stage detector is high, but the speed is slow and it is difficult to meet the real-time requirements. The single-stage detector first divides the input image into multiple grids, uses the grid information to select appropriate samples for training, and directly predicts the classification and regression information on the grid. It has a fast detection speed and can detect targets in real time, so it is widely used in various fields.

[0003] At present, the single-stage detector mainly includes four core modules: data preprocessing, network model construction, sample selection and model training. Sample selection is mainly to select suitable positive samples for the target and suitable negative samples for the background. It is one of the important research contents of the single-stage detector. There are two main sample selection methods for single-stage detectors: sample selection method based on IoU loss and sample selection method based on center point distance. Among them, the sample selection method based on IoU loss first calculates the Intersection-over-Union (IoU) value between the candidate sample and the target according to the manually set anchor, and selects the sample with a value greater than the threshold. T IOU The samples are positive samples, and the rest are negative samples. The sample selection method based on the center point distance selects samples with a distance less than the threshold according to the L1 distance between the grid coordinate point where the candidate sample is located and the center point of the target. T c The samples are positive samples, and the rest are negative samples. Both sample selection methods adopt a many-to-one approach, that is, each target corresponds to multiple positive samples, which can provide rich supervision information for model training.

[0004] However, the above two sample selection methods have the following problems:

[0005] (1) Both of these sample selection methods are based on prior information and calculate the loss according to the pre-set anchor or grid coordinate points of the image. They do not take into account the prediction information of the model, and the selected samples are not optimal samples.

[0006] (2) These two sample selection methods only consider regression loss. Object detection combines classification and positioning tasks. Only considering regression loss for sample selection is not conducive to classifier training, such asFigure 1 As shown;

[0007] (3) Currently, most single-stage detectors use FocalLoss to train classifiers, considering samples with low classification scores as difficult samples and increasing their contribution to classification loss. Using a many-to-one sample selection method, each target corresponds to multiple positive samples, but these positive samples are not all meaningful samples, such as Figure 2 As shown, all positive samples need to be treated differently. Summary of the invention

[0008] In view of the above situation, the present invention provides a sample selection method and device, constructs a cost function that considers both classification loss and regression loss, and selects each sample before N The candidate samples with the smallest cost are selected as positive samples, and a classifier training method and device are provided at the same time, which distinguish all positive samples according to the IoU value between the regression box and the target box, thereby improving the contribution of high-quality candidate samples to the classification loss and suppressing the influence of low-quality samples.

[0009] In order to achieve the purpose of the present invention, the following scheme is proposed:

[0010] A sample selection method based on prediction information, applied to a single-stage target detector, comprises the steps of:

[0011] Construct cost function;

[0012] Calculating the costs of all candidate samples using the cost function according to the prediction information output by the detection network of the single-stage target detector and the target true value;

[0013] Use prior information to screen candidate samples;

[0014] The selected candidate samples are sorted according to the cost, and the first N samples with the smallest cost are selected as positive samples, and the rest are negative samples.

[0015] Furthermore, the cost function is constructed by a weighted combination of three loss functions: the cross entropy function for classification loss, the IoU loss function for regression loss, and the L1 loss function of the center point distance between the regression box and the target box.

[0016] The cost function is:

[0017] C = l loc C loc + l cls C cls

[0018] in, l loc and l cls Represent the weight coefficients of regression loss and classification loss respectively, C loc represents the regression error, C cls Represents the classification error between the candidate sample and the target box;

[0019] C loc Including IoU loss, center point L1 distance:

[0020] C loc = l IoU C IoU + l L1 C L1

[0021] in, l IoU and l L1 Represent the weight coefficients of IoU loss and center point L1 distance respectively, C IoU Represents the IoU loss, which is obtained by calculating the IoU value between the regression box of the sample and the target box. C L1 Represents the center point L1 distance, which is obtained by calculating the distance between the regression box of the sample and the center point of the target box;

[0022] C cls The formula is:

[0023]

[0024] Among them, C cls In the formula, the weight coefficient α cls Used to balance the weights of the target category and the remaining categories, c cls is the adjustment factor used to adjust the importance of the sample. p Represents the classification information predicted by the detection network, i、j Represents the target category index.

[0025] Furthermore, we use the prior information to filter candidate samples, retain the candidate samples whose center points are within the target box, and delete the candidate samples outside the target box, using the formula:

[0026] C 1 =Ω*( l loc C loc + l cls C cls )

[0027] Among them, Ω represents the prior information, and the grid coordinate point where the candidate sample is located is within the target box, then Ω i ∈Ω is 1, otherwise it is infinite.

[0028] A sample selection device based on prediction information, applied to a single-stage target detector, comprising:

[0029] Construction module, used to construct the cost function;

[0030] A calculation module, configured to calculate the cost of all candidate samples using the cost function according to the prediction information output by the detection network of the single-stage target detector and the target true value;

[0031] A screening module is used to screen candidate samples using prior information;

[0032] The selection module is used to sort the screened candidate samples according to the cost, select the first N samples with the smallest cost as positive samples, and the rest as negative samples.

[0033] A classifier training method based on joint representation of classification and positioning information, applied to a single-stage target detector, comprises the steps of:

[0034] According to the regression information of the candidate sample, the IoU value between the regression box of the candidate sample and the target box is calculated; the candidate sample is a positive sample and a negative sample selected by the sample selection method based on the prediction information;

[0035] For each target, the normalized weight of the sample is calculated based on the IoU value between the regression box and the target box of all the candidate samples:

[0036]

[0037]

[0038] in, oh k represents the weight coefficient of the kth sample, q k Represents the IoU value between the regression box and the target box of the kth sample, q l Indicates lThe IoU value between the candidate box and the target box of the sample, N Indicates the total number of samples corresponding to the target;

[0039] When the positive sample is used to train the classifier, the classification loss of the positive sample is weighted by the weight coefficient calculated by the following formula:

[0040]

[0041] In this formula, α s Used to adjust the contribution of positive and negative samples to classification loss. p Represents the classification information predicted by the detection network, y is the positive and negative sample label, if y =1, it indicates a positive sample; if y =0, it indicates a negative sample. c s It is a regulation factor that adjusts the contribution of different negative samples to the classification loss.

[0042] A classifier training device based on joint representation of classification and positioning information, applied to a single-stage target detector, comprising:

[0043] An IoU value module is used to calculate the IoU value between the regression box of the candidate sample and the target box according to the regression information of the candidate sample; the candidate sample is a positive sample and a negative sample selected by the sample selection method based on the prediction information;

[0044] The normalization module is used to calculate the normalized weight of each target according to the IoU value between the regression box and the target box of all candidate samples:

[0045]

[0046]

[0047] in, oh k Indicates k The weight coefficient of the samples, q k Indicates k The IoU value between the regression box and the target box of the sample, q l Indicates l The IoU value between the candidate box and the target box of the sample, N Indicates the total number of samples corresponding to the target;

[0048] A weighting module is used to weight the classification loss of the positive sample by a weight coefficient calculated by the following formula when training the classifier using the positive sample:

[0049]

[0050] In this formula, α s Used to adjust the contribution of positive and negative samples to classification loss. p Represents the classification information predicted by the detection network, y is the positive and negative sample label, if y =1, it indicates a positive sample; if y =0, it indicates a negative sample. c s It is a regulation factor that adjusts the contribution of different negative samples to the classification loss.

[0051] An electronic device comprises: at least one processor and a memory; wherein the memory stores computer-executable instructions; the computer-executable instructions stored in the memory are executed on the at least one processor, so that the at least one processor executes the sample selection method based on prediction information, or executes the classifier training method based on the joint representation of classification and positioning information.

[0052] A computer-readable storage medium stores a computer program, which, when executed by a processor, controls the device where the storage medium is located to execute the sample selection method based on prediction information or the classifier training method based on the joint representation of classification and positioning information.

[0053] The beneficial effects of the present invention are:

[0054] 1. Using the sample selection method based on prediction information, according to the prediction information output by the detection network, the positive samples are selected while considering the classification loss and regression loss, which improves the stability of sample selection, not only speeds up the convergence of the model, but also improves the detection performance of the model.

[0055] 2. Use the joint representation based on classification and positioning information to train the classifier. In the loss function of the training classifier, both the classification error and the regression error are considered. The normalized weight coefficient is calculated through the IoU value between the regression box and the target box of the sample, and the normalized weight coefficient is used to distinguish the contribution of different samples to the classification loss, thereby increasing the contribution of high-quality samples to the loss function and suppressing the impact of low-quality samples. In addition, the use of the joint representation based on classification and regression information to train the classifier further strengthens the correlation between classification and positioning and improves the performance of the detector. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The drawings described herein are for illustrative purposes only of selected embodiments and not all possible implementations, and are not intended to limit the scope of the present invention.

[0057] Figure 1 Positive samples selected by the existing sample selection strategy.

[0058] Figure 2 Positive samples selected by the existing one-to-one sample selection strategy.

[0059] Figure 3 Polysemy for sample selection based on IoU loss and center point L1 distance.

[0060] Figure 4 This is a flow chart of a sample selection method based on prediction information according to an embodiment of the present application.

[0061] Figure 5 This is a structural block diagram of a sample selection device based on prediction information according to an embodiment of the present application.

[0062] Figure 6 This is a flow chart of a classifier training method based on joint representation of classification and positioning information according to an embodiment of the present application.

[0063] Figure 7 This is a structural block diagram of a classifier training device based on joint representation of classification and positioning information according to an embodiment of the present application. DETAILED DESCRIPTION

[0064] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings.

[0065] In one aspect of an embodiment of the present application, a sample selection method based on prediction information is provided, which is applied to a single-stage target detector, such as Figure 4 As shown, the following steps are included:

[0066] First, the cost function is constructed, which is implemented by a weighted combination of three loss functions: the cross entropy function for classification loss, the IoU loss function for regression loss, and the L1 loss function of the center point distance between the regression box and the target box.

[0067] The constructed cost function is:

[0068] C = l loc C loc + l cls C cls

[0069] in, l loc and lcls Represent the weight coefficients of regression loss and classification loss respectively, C loc represents the regression error, C cls Represents the classification error between the candidate sample and the target box;

[0070] C loc Including IoU loss, center point L1 distance:

[0071] C loc = l IoU C IoU + l L1 C L1

[0072] in, l IoU and l L1 Represent the weight coefficients of IoU loss and center point L1 distance respectively, C IoU Represents the IoU loss, which is obtained by calculating the IoU value between the regression box of the sample and the target box. C L1 It represents the L1 distance of the center point, which is obtained by calculating the distance between the regression box of the sample and the center point of the target box; since both the IoU loss and the L1 distance of the center point are ambiguous, such as Figure 3 As shown in , the weighted combination of these two losses can effectively reduce the ambiguity of the regression loss. Figure 3 When using IoU values ​​to select samples, there are many candidate samples with equal IoU, such as Figure 3 As shown in (a), the IoU values ​​of grids with equal anchors in the target box are equal; and the samples are selected by using the center point distance, such as Figure 3 As shown in the image (b), samples with the same center point have the same distance loss. When the two are combined to select samples, the ambiguity will be greatly reduced. Figure 3 As shown in the image (c), C loc The three candidate samples are not equal.

[0073] C cls Calculated by Fcoal Loss, the formula is:

[0074]

[0075] Among them, C clsIn the formula, the weight coefficient α cls Used to balance the weights of the target category and the remaining categories, c cls is the adjustment factor used to adjust the importance of the sample. p Represents the classification information predicted by the detection network, i、j Represents the target category index. The classification loss not only considers the loss of the target category, but also the influence of the remaining categories. The selected samples should not only have a small target category cost, but also have good discrimination from other categories.

[0076] Then, the prediction information and the target true value are obtained according to the detection network, and the cost function is used to calculate the cost of all candidate samples.

[0077] Then, we use the prior information to filter candidate samples. Since the network's prediction information is inaccurate in the initial stage of training, the samples selected based on the prediction information are very unstable. In order to further improve the stability of sample selection, we filter candidate samples based on the prior information, retain all candidate samples whose center points are within the target box, and delete candidate samples outside the target box, using the formula:

[0078] C 1 =Ω*( l loc C loc + l cls C cls )

[0079] Among them, Ω represents the prior information, which refers to whether the grid coordinate point where the candidate sample is located is within the target frame. If the grid coordinate point where the candidate sample is located is within the target frame, then Ω m ∈Ω is 1, otherwise it is infinite. The cost of candidate samples within the target box is finite, while the cost of candidate samples outside the target box is infinite.

[0080] Finally, the selected candidate samples are sorted according to the cost, and the cost values ​​are sorted from small to large, and the first N samples with the smallest costs are selected as positive samples, and the rest are negative samples.

[0081] Another aspect of the present application embodiment provides a sample selection device based on prediction information, which is applied to a single-stage target detector, such as Figure 5 As shown, it includes a construction module, a calculation module, a screening module, and a selection module.

[0082] The construction module is used to construct the cost function; it is specifically implemented by a weighted combination of three losses: the cross entropy function for classification loss, the IoU loss function for regression loss, and the L1 loss function of the center point distance between the regression box and the target box.

[0083] The cost function constructed by the construction module is:

[0084] C = l loc C loc + l cls C cls

[0085] in, l loc and l cls Represent the weight coefficients of regression loss and classification loss respectively, C loc represents the regression error, C cls Represents the classification error between the candidate sample and the target box;

[0086] C loc Including IoU loss, center point L1 distance:

[0087] C loc = l IoU C IoU + l L1 C L1

[0088] in, l IoU and l L1 Represent the weight coefficients of IoU loss and center point L1 distance respectively, C IoU Represents the IoU loss, which is obtained by calculating the IoU value between the regression box of the sample and the target box. C L1 It represents the L1 distance of the center point, which is obtained by calculating the distance between the regression box of the sample and the center point of the target box; since both the IoU loss and the L1 distance of the center point are ambiguous, such as Figure 3 As shown in , the weighted combination of these two losses can effectively reduce the ambiguity of the regression loss. Figure 3When using IoU values ​​to select samples, there are many candidate samples with equal IoU, such as Figure 3 As shown in (a), the IoU values ​​of grids with equal anchors in the target box are equal; and the samples are selected by using the center point distance, such as Figure 3 As shown in the image (b), samples with the same center point have the same distance loss. When the two are combined to select samples, the ambiguity will be greatly reduced. Figure 3 As shown in the image (c), C loc The three candidate samples are not equal.

[0089] C cls Calculated by Fcoal Loss, the formula is:

[0090]

[0091] Among them, C cls In the formula, the weight coefficient α cls Used to balance the weights of the target category and the remaining categories, c cls is the adjustment factor used to adjust the importance of the sample. p Represents the classification information predicted by the detection network, i、j Represents the target category index. The classification loss not only considers the loss of the target category, but also the influence of the remaining categories. The selected samples should not only have a small target category cost, but also have good discrimination from other categories.

[0092] The calculation module is used to calculate the cost of all candidate samples using the cost function according to the prediction information and the target true value output by the detection network of the single-stage target detector.

[0093] The screening module is used to screen candidate samples using prior information, retaining all candidate samples whose center points are within the target box and deleting candidate samples outside the target box, using the formula:

[0094] C 1 =Ω*( l loc C loc + l cls C cls )

[0095] Among them, Ω represents the prior information, which refers to whether the grid coordinate point where the candidate sample is located is within the target frame. If the grid coordinate point where the candidate sample is located is within the target frame, then Ω m∈Ω is 1, otherwise it is infinite. The cost of candidate samples within the target box is finite, while the cost of candidate samples outside the target box is infinite.

[0096] The selection module is used to sort the selected candidate samples according to the cost, sort them from small to large according to the cost value, select the first N samples with the smallest cost as positive samples, and the rest as negative samples.

[0097] Another aspect of an embodiment of the present application provides a classifier training method based on joint representation of classification and positioning information, which is applied to a single-stage target detector.

[0098] Most target detectors use FocalLoss to train classifiers. The specific formula is as follows:

[0099]

[0100] In this formula, y is the sample category label. If it is a positive sample, then y =1; if it is a negative sample, then y =0, weight coefficient α f Used to balance the importance of positive and negative samples. c f is an adjustment factor used to adjust the importance between simple samples and difficult samples. p Represents the predicted classification score. Using a many-to-one sample selection method, each target corresponds to multiple samples. FcoalLoss treats each positive sample equally and considers samples with low classification scores as difficult samples, and the greater their contribution to classification loss. However, for each target, samples with low scores are not necessarily meaningful samples, such as Figure 2 shown.

[0101] To address this problem, this example proposes a classifier training method based on the joint representation of classification and positioning information, such as Figure 7 As shown, the following steps are included:

[0102] First, according to the regression information of the candidate sample, the IoU value between the regression box of the candidate sample and the target box is calculated; the candidate sample is a positive sample and a negative sample selected by the sample selection method based on prediction information in the above embodiment;

[0103] Then, for each target, the normalized weight of the sample is calculated based on the IoU value between the regression box and the target box of all the candidate samples:

[0104]

[0105]

[0106] in, oh k Indicates k The weight coefficient of the samples, q k Represents the IoU value between the regression box and the target box of the kth sample, q l Indicates l The IoU value between the candidate box and the target box of the sample, N Indicates the total number of samples corresponding to the target;

[0107] Then, when the positive sample is used to train the classifier, the classification loss of the positive sample is weighted by the weight coefficient calculated by the following formula:

[0108]

[0109] In this formula, α s Used to adjust the contribution of positive and negative samples to classification loss. p Represents the classification information predicted by the detection network, y is the positive and negative sample label, if y =1, it indicates a positive sample; if y =0, it indicates a negative sample. c s It is a regulation factor that adjusts the contribution of different negative samples to the classification loss.

[0110] Another aspect of the present application embodiment provides a classifier training device based on the joint representation of classification and positioning information, which is applied to a single-stage target detector, such as Figure 7 As shown, it includes: IoU value module, normalization module and weighting module.

[0111] The IoU value module is used to calculate the IoU value between the regression box of the candidate sample and the target box according to the regression information of the candidate sample; the candidate sample is a positive sample and a negative sample selected by the sample selection method based on prediction information.

[0112] The normalization module is used to calculate the normalized weight of each target according to the IoU value between the candidate box and the target box of all candidate samples:

[0113]

[0114]

[0115] in, oh k Indicates k The weight coefficient of the samples, qk Indicates k The IoU value between the regression box and the target box of the sample, q l Indicates l The IoU value between the candidate box and the target box of the sample, N Indicates the total number of samples corresponding to the target.

[0116] The weighting module is used to weight the classification loss of the positive sample by the weight coefficient calculated by the following formula when using the positive sample to train the classifier:

[0117]

[0118] In this formula, α s Used to adjust the contribution of positive and negative samples to classification loss. p Represents the classification information predicted by the detection network, y is the positive and negative sample label, if y =1, it indicates a positive sample; if y =0, it indicates a negative sample. c s It is a regulation factor that adjusts the contribution of different negative samples to the classification loss.

[0119] Another aspect of an embodiment of the present application provides an electronic device, comprising: at least one processor and a memory; wherein the memory stores computer-executable instructions; the computer-executable instructions stored in the memory are executed by the at least one processor, so that the at least one processor executes the sample selection method based on prediction information described in the previous embodiment, or executes the classifier training method based on the joint representation of classification and positioning information described in the previous embodiment.

[0120] In another aspect of an embodiment of the present application, a computer-readable storage medium stores a computer program thereon, which, when the computer program is executed by a processor, controls the device where the storage medium is located to execute the sample selection method based on prediction information described in the previous embodiment, or to execute the classifier training method based on the joint representation of classification and positioning information described in the previous embodiment.

[0121] The above are only preferred embodiments of the present invention, and are not intended to be the only or limiting embodiments of the present invention. Those skilled in the art should understand that various changes or equivalent substitutions made to the present invention without departing from the scope of the present invention are within the scope of protection of the present invention.

Claims

1. A sample selection method based on prediction information, applied to a single-stage target detector, characterized in that: Includes steps: Input image data; Construct cost function; Calculating the costs of all candidate samples using the cost function according to the prediction information output by the detection network of the single-stage target detector and the target true value; Use prior information to screen candidate samples; Sort the selected candidate samples according to the cost, select the first N samples with the smallest cost as positive samples, and the rest as negative samples; The cost function is constructed by a weighted combination of three losses: the cross entropy function for classification loss, the IoU loss function for regression loss, and the L1 loss function of the center point distance between the regression box and the target box. The cost function is: C = λ loc C loc + λ cls C cls in, λ loc and λ cls Represent the weight coefficients of regression loss and classification loss respectively, C loc represents the regression error, C cls Represents the classification error between the candidate sample and the target box; C loc Including IoU loss, center point L1 distance: C loc = λ IoU C IoU + λ L1 C L1 in, λ IoU and λ L1 Represent the weight coefficients of IoU loss and center point L1 distance respectively, C IoU Represents the IoU loss, which is obtained by calculating the IoU value between the regression box of the sample and the target box. C L1 Represents the center point L1 distance, which is obtained by calculating the distance between the regression box of the sample and the center point of the target box; C cls The formula is: Among them, C cls In the formula, the weight coefficient α cls Used to balance the weights of the target category and the remaining categories, γ cls is the adjustment factor used to adjust the importance of the sample. p Represents the classification information predicted by the detection network, i, j Represents the target category index; Using prior information to filter candidate samples is to retain candidate samples whose center points are within the target box and delete candidate samples outside the target box. The formula is: C 1 =Oh*( λ loc C loc + λ cls C cls ) Among them, Ω represents the prior information, which refers to whether the grid point where the candidate sample is located is within the target frame. If the grid coordinate point where the candidate sample is located is within the target frame, then Ω i ∈Ω is 1, otherwise it is infinity.

2. A sample selection device based on prediction information, applied to a single-stage target detector, characterized in that: include: An input module, used for inputting image data; Construction module, used to construct the cost function; A calculation module, configured to calculate the cost of all candidate samples using the cost function according to the prediction information output by the detection network of the single-stage target detector and the target true value; A screening module is used to screen candidate samples using prior information; A selection module is used to sort the screened candidate samples according to the cost, select the first N samples with the smallest cost as positive samples, and the rest as negative samples; The cost function is implemented by a weighted combination of three losses: a cross entropy function for classification loss, an IoU loss function for regression loss, and an L1 loss function for the distance between the center point of the regression box and the target box. The cost function constructed by the construction module is: C = λ loc C loc + λ cls C cls in, λ loc and λ cls Represent the weight coefficients of regression loss and classification loss respectively, C loc represents the regression error, C cls Represents the classification error between the candidate sample and the target box; C loc Including IoU loss, center point L1 distance: C loc = λ IoU C IoU + λ L1 C L1 in, λ IoU and λ L1 Represent the weight coefficients of IoU loss and center point L1 distance respectively, C IoU Represents the IoU loss, which is obtained by calculating the IoU value between the regression box of the sample and the target box. C L1 Represents the center point L1 distance, which is obtained by calculating the distance between the regression box of the sample and the center point of the target box; C cls The formula is: Among them, C cls In the formula, the weight coefficient α cls Used to balance the weights of the target category and the remaining categories, γ cls is the adjustment factor used to adjust the importance of the sample. p Represents the classification information predicted by the detection network, i, j Represents the target category index; Using prior information to filter candidate samples is to retain candidate samples whose center points are within the target box and delete candidate samples outside the target box. The formula is: C 1 =Oh*( λ loc C loc + λ cls C cls ) Among them, Ω represents the prior information, which refers to whether the grid point where the candidate sample is located is within the target frame. If the grid coordinate point where the candidate sample is located is within the target frame, then Ω i ∈Ω is 1, otherwise it is infinity.

3. A classifier training method based on joint representation of classification and positioning information, applied to a single-stage target detector, characterized in that: Includes steps: According to the regression information of the candidate sample, the IoU value between the regression box of the candidate sample and the target box is calculated; the candidate sample is a positive sample and a negative sample selected by the sample selection method based on prediction information according to claim 1; For each target, the normalized weight of the sample is calculated based on the IoU values ​​of all the candidate samples: in, ω i represents the weight coefficient of the kth sample, q k Represents the IoU value between the regression box and the target box of the kth sample, q l Indicates l The IoU value between the candidate box and the target box of the sample, N Indicates the total number of samples corresponding to the target; When the positive sample is used to train the classifier, the classification loss of the positive sample is weighted by the weight coefficient calculated by the following formula: In this formula, α s Used to adjust the contribution of positive and negative samples to classification loss. p Represents the classification information predicted by the detection network, y is a positive or negative sample label, if y =1, it indicates a positive sample; if y =0, it indicates a negative sample. γ s It is a regulation factor that adjusts the contribution of different negative samples to the classification loss.

4. A classifier training device based on joint representation of classification and positioning information, applied to a single-stage target detector, characterized in that: include: The IoU value module is used to calculate the IoU value between the regression box of the candidate sample and the target box based on the regression information of the candidate sample; The candidate samples are positive samples and negative samples selected by the sample selection method based on prediction information according to claim 1; The normalization module is used to calculate the normalized weight of each target according to the IoU value between the regression box and the target box of all candidate samples: in, ω k represents the weight coefficient of the kth sample, q k Represents the IoU value between the regression box and the target box of the kth sample, q l Indicates l The IoU value between the candidate box and the target box of the sample, N Indicates the total number of samples corresponding to the target; A weighting module is used to weight the classification loss of the positive sample by a weight coefficient calculated by the following formula when training the classifier using the positive sample: In this formula, α s Used to adjust the contribution of positive and negative samples to classification loss. p Represents the classification information predicted by the detection network, y is a positive or negative sample label, if y =1, it indicates a positive sample; if y =0, it indicates a negative sample. γ s It is a regulation factor that adjusts the contribution of different negative samples to the classification loss.

5. An electronic device comprising: At least one processor and a memory; wherein the memory stores computer-executable instructions; characterized in that the computer-executable instructions stored in the memory are executed by the at least one processor, so that the at least one processor executes the sample selection method based on prediction information as described in claim 1, or executes the classifier training method based on the joint representation of classification and positioning information as described in claim 3.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, the device where the storage medium is located is controlled to execute the sample selection method based on prediction information as claimed in claim 1, or to execute the classifier training method based on the joint representation of classification and positioning information as claimed in claim 3.

Images