Confidence threshold determination method and apparatus, electronic device, and storage medium

Abstract

The application provides a confidence threshold determination method and device, electronic equipment and a storage medium, and belongs to the technical field of deep learning. The method comprises the following steps: obtaining unlabeled samples, and obtaining classification prediction results of the unlabeled samples in each training batch; obtaining sample difficulty scores of sample levels of each training batch and preliminary classification categories corresponding to each unlabeled sample according to the classification prediction results; and determining a confidence threshold corresponding to each preliminary classification category according to the sample difficulty scores and the preliminary classification categories. The application considers the mutual differences between samples, reflects the characteristics of the sample levels through the sample difficulty scores of the sample levels, and obtains the confidence threshold corresponding to each preliminary classification category according to the sample difficulty scores of the sample levels, which is more in line with the training process of the model, can make more unlabeled samples participate in the model training, ensures the correctness of the pseudo labels of the unlabeled samples participating in the model training, and improves the final effect of the model.

Description

Technical Field

[0001] This invention relates to the field of deep learning technology, and in particular to a method, apparatus, electronic device, and storage medium for determining a confidence threshold. Background Technology

[0002] Current deep learning methods heavily rely on massive amounts of labeled data, but obtaining such data requires significant human and material resources, especially in fields like medical and remote sensing images. To reduce sample labeling and fully utilize large amounts of unlabeled data, semi-supervised learning (SSL) has emerged and gained widespread attention in recent years. Researchers have proposed many algorithms; however, how to further improve the utilization rate of unlabeled samples and thus enhance model training performance remains a crucial problem to be solved.

[0003] To address the issue of improving the utilization rate of unlabeled samples in SSL (Supported Learning) methods, recent SSL approaches have achieved effective utilization of unlabeled samples through two core methods: consistency regularization and pseudo-labeling techniques. The key to pseudo-labeling is finding a suitable confidence threshold to determine which unlabeled samples can participate in model training. Most existing methods, such as FixMatch, Dash, FlexMatch, and FreeMatch, employ methods such as fixed thresholds, fixed threshold variation strategies, and consideration of class difficulty for the confidence threshold. Combining pseudo-labeling and consistency regularization techniques mainly involves algorithmic innovation in setting the pseudo-label confidence threshold. However, none of the above methods for obtaining the pseudo-label confidence threshold consider the sample-level issue, i.e., the differences between samples. Therefore, the obtained confidence threshold does not match the model's learning process, resulting in low utilization of unlabeled samples and consequently affecting the model's final performance. Summary of the Invention

[0004] This invention provides a method, apparatus, electronic device, and storage medium for determining a confidence threshold, in order to solve the defect in the prior art where the confidence threshold does not match the model's learning process, resulting in a low utilization rate of unlabeled samples, which in turn affects the final performance of the model.

[0005] This invention provides a method for determining a confidence threshold, comprising:

[0006] Obtain unlabeled samples and get the classification prediction results of the unlabeled samples in each training batch;

[0007] Based on the classification prediction results, the sample difficulty score at the sample level for each training batch and the preliminary classification category corresponding to each unlabeled sample are obtained.

[0008] Based on the sample difficulty score and the preliminary classification category, a confidence threshold is determined for each preliminary classification category.

[0009] According to a method for determining a confidence threshold provided by the present invention, the step of determining a confidence threshold corresponding to each preliminary classification category based on the sample difficulty score and the preliminary classification category includes:

[0010] Based on the sample difficulty score, the initial update amount of the confidence threshold corresponding to each preliminary classification category is obtained;

[0011] The initial update amount is smoothed based on a preset momentum update constant to obtain a smoothed update amount of the confidence threshold between training batches.

[0012] Based on the momentum update constant and the smooth update amount, determine the confidence threshold corresponding to each of the preliminary classification categories.

[0013] According to a method for determining a confidence threshold provided by the present invention, the step of obtaining the initial update amount of the confidence threshold corresponding to each preliminary classification category based on the sample difficulty score includes:

[0014] Determine the number of unlabeled samples corresponding to each of the preliminary classification categories;

[0015] If the number of unlabeled samples is 1, the maximum value of the classification prediction result corresponding to the unlabeled sample is used as the initial update amount of the confidence threshold corresponding to the preliminary classification category;

[0016] If the number of unlabeled samples is greater than 1, the initial update amount of the confidence threshold corresponding to each preliminary classification category is obtained based on the proportion of the sample difficulty score corresponding to each unlabeled sample and the maximum value of the classification prediction result.

[0017] The confidence threshold determination method provided by the present invention further includes:

[0018] The number of unlabeled samples whose maximum value of the classification prediction result is higher than a preset threshold is normalized to obtain the normalized sample number.

[0019] The normalized sample size is scaled based on preset boundary parameters to obtain the pseudo-label data class distribution.

[0020] Correspondingly, determining the confidence threshold for each preliminary classification category based on the momentum update constant and the smoothing update amount includes:

[0021] Based on the momentum update constant, the smooth update amount, and the pseudo-label data class distribution, determine the confidence threshold corresponding to each of the preliminary classification categories.

[0022] According to a confidence threshold determination method provided by the present invention, the step of obtaining a sample difficulty score at the sample level for each training batch and a preliminary classification category corresponding to each unlabeled sample based on the classification prediction result includes:

[0023] The difference between the maximum and the second maximum value of the classification prediction results is taken as the sample difficulty score of the unlabeled sample.

[0024] The maximum value of the classification prediction result is used as the preliminary classification category for each unlabeled sample.

[0025] The confidence threshold determination method provided by the present invention further includes:

[0026] Based on the confidence threshold corresponding to each of the initial classification categories, a loss function is determined for backpropagation.

[0027] The confidence threshold determination method provided by the present invention further includes:

[0028] Obtain labeled samples and determine the first loss function corresponding to the labeled samples;

[0029] Correspondingly, determining the loss function for backpropagation based on the confidence threshold corresponding to each preliminary classification category includes:

[0030] Based on the confidence threshold corresponding to each of the preliminary classification categories, determine the second loss function corresponding to the unlabeled sample;

[0031] The loss function is determined based on the first loss function and the second loss function for backpropagation.

[0032] The present invention also provides a confidence threshold determination device, comprising:

[0033] The classification module is used to acquire unlabeled samples and obtain the classification prediction results of the unlabeled samples in each training batch.

[0034] The first determining module is used to obtain, based on the classification prediction results, a sample difficulty score at the sample level for each training batch and a preliminary classification category corresponding to each unlabeled sample.

[0035] The second determining module is used to determine the confidence threshold corresponding to each of the preliminary classification categories based on the sample difficulty score and the preliminary classification category.

[0036] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the confidence threshold determination method as described above.

[0037] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the confidence threshold determination method as described above.

[0038] The confidence threshold determination method, apparatus, electronic device, and storage medium provided by this invention consider the heterogeneity between samples, reflect the characteristics of the sample level through the sample difficulty score of unlabeled samples, and obtain the confidence threshold corresponding to each preliminary classification category based on the sample difficulty score of the sample level. This is more in line with the model training process, which not only allows more unlabeled samples to participate in model training, but also ensures the accuracy of the pseudo-labels among the unlabeled samples participating in model training, thereby improving the final effect of the model. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0040] Figure 1 This is one of the flowcharts illustrating the confidence threshold determination method provided by the present invention;

[0041] Figure 2 This is a flowchart illustrating step S200 provided by the present invention;

[0042] Figure 3 This is a flowchart illustrating step S300 provided by the present invention;

[0043] Figure 4 This is the second flowchart illustrating the confidence threshold determination method provided by the present invention;

[0044] Figure 5 This is a schematic diagram of the confidence threshold determination device provided by the present invention;

[0045] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0047] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the embodiments of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0048] Figure 1 This is one of the flowcharts illustrating the confidence threshold determination method provided by the present invention, such as... Figure 1 As shown, the present invention provides a method for determining a confidence threshold, comprising:

[0049] Step S100: Obtain unlabeled samples and obtain the classification prediction results of the unlabeled samples in each training batch; unlabeled samples are image samples without labels. Input the unlabeled samples into the classification model. The classification model is used to output the classification prediction results of the unlabeled samples. The classification prediction results are used to characterize the results of classifying the unlabeled samples into different categories or labels, including the probability distribution of each category.

[0050] A training batch refers to the number of datasets input into a machine learning or deep learning model at once for training. Typically, a training batch contains multiple samples. The model updates and optimizes its parameters based on the features and labels of these samples to improve its performance and accuracy. The size of the training batch can be adjusted according to the specific problem and dataset. Larger batch sizes may speed up training but may also increase memory consumption and computational costs. Common batch sizes include 32, 64, and 128.

[0051] Step S200: Based on the classification prediction results, obtain the sample difficulty score at the sample level for each training batch and the preliminary classification category corresponding to each unlabeled sample. The sample difficulty score can be the probability difference between probability distributions. The larger the probability difference, the simpler the unlabeled sample, and the easier it is for the classification model to identify its category. Conversely, the smaller the probability difference, the more difficult the unlabeled sample, and the harder it is to identify its category. The category corresponding to the maximum probability in the classification prediction results is taken as the preliminary classification category of the unlabeled sample.

[0052] Step S300: Based on the sample difficulty score and the preliminary classification category, determine the confidence threshold corresponding to each preliminary classification category. Classify the sample difficulty scores based on the preliminary classification categories, weight the unlabeled samples according to the sample difficulty scores to obtain the initial update amount of the confidence threshold. To balance information between batches, an exponential moving average update can be used to obtain the update amount of the confidence threshold. Finally, update the confidence threshold. The confidence threshold can be used for adaptive learning of semi-supervised or unsupervised learning models.

[0053] It is understandable that this invention takes into account the dissimilarity between samples, reflects the characteristics of the sample level through the sample difficulty score of unlabeled samples, and obtains the confidence threshold corresponding to each preliminary classification category based on the sample difficulty score of the sample level. This is more in line with the model training process, which not only allows more unlabeled samples to participate in model training, but also ensures the accuracy of the pseudo-labels among the unlabeled samples participating in model training, thereby improving the final effect of the model.

[0054] Figure 2 This is a flowchart illustrating step S200 provided by the present invention, as shown below. Figure 2 As shown, based on the above embodiments, as an optional embodiment, obtaining the sample difficulty score at the sample level for each training batch and the preliminary classification category corresponding to each unlabeled sample based on the classification prediction results includes:

[0055] Step S210: The difference between the maximum value and the second maximum value of the classification prediction results is taken as the sample difficulty score of the unlabeled sample.

[0056] For assessing sample difficulty, the difference between the two largest classification prediction results is used as the difficulty of the unlabeled sample. The larger the difference, the simpler the unlabeled sample; conversely, the smaller the difference, the more difficult the unlabeled sample. The sample difficulty score is calculated as follows:

[0057] ξ = max(q) - max(q′);

[0058] Where ξ represents the sample difficulty score of the unlabeled sample, q refers to the classification prediction result of the classification model for the unlabeled sample, and q′ is the prediction result after removing the maximum value in q.

[0059] Step S220: The maximum value of the classification prediction result is used as the preliminary classification category for each unlabeled sample. Specifically, max(q) is used as the preliminary classification category for the unlabeled sample.

[0060] Understandably, this invention provides a scheme for determining sample difficulty scores and preliminary classification categories, taking into account sample-level factors to update the confidence threshold. This solves the problem in the prior art where the confidence threshold does not match the model's learning process, resulting in low utilization of unlabeled samples and thus affecting the final performance of the model.

[0061] Figure 3 This is a flowchart illustrating step S300 provided by the present invention, as shown below. Figure 3 As shown, based on the above embodiments, as an optional embodiment, determining the confidence threshold corresponding to each preliminary classification category according to the sample difficulty score and the preliminary classification category includes:

[0062] Step S310: Based on the sample difficulty score, obtain the initial update amount of the confidence threshold corresponding to each preliminary classification category;

[0063] Optionally, the step of obtaining the initial update amount of the confidence threshold corresponding to each preliminary classification category based on the sample difficulty score includes:

[0064] Step S311: Determine the number of unlabeled samples corresponding to each of the preliminary classification categories;

[0065] Step S312: If the number of unlabeled samples is 1, the maximum value of the classification prediction result corresponding to the unlabeled sample is used as the initial update amount of the confidence threshold corresponding to the preliminary classification category.

[0066] Step S313: If the number of unlabeled samples is greater than 1, the initial update amount of the confidence threshold corresponding to each preliminary classification category is obtained based on the proportion of the sample difficulty score corresponding to each unlabeled sample and the maximum value of the classification prediction result.

[0067] Specifically, based on the initial classification results of each unlabeled sample using the classification model, the preliminary classification category is determined. Then, the initial update amount of the confidence threshold for each category is calculated using the following formula:

[0068]

[0069] Where, η update (c) represents the initial update of the confidence threshold for category c, and m refers to the number of unlabeled samples in this batch that are classified as category c.

[0070] Step S320: Smooth the initial update amount based on the preset momentum update constant to obtain the smoothed update amount of the confidence threshold between training batches.

[0071] After obtaining the initial update values ​​for each category in this training batch, in order to smooth out the update values... Exponential moving updates can be used to balance updates between batches. The formula for calculating the smooth update amount is as follows:

[0072]

[0073] Where λ is the momentum renewal constant.

[0074] Step S330: Determine the confidence threshold corresponding to each of the preliminary classification categories based on the momentum update constant and the smooth update amount.

[0075] The confidence threshold τ is adjusted using the update amount of each class. t (c) Perform the update as shown in the following formula:

[0076]

[0077] in, t represents the current training batch.

[0078] It is understood that the present invention provides a confidence threshold update scheme for each category, taking into account the dissimilarity between samples, and obtaining the confidence threshold corresponding to the category based on the characteristics at the sample level. The obtained confidence threshold is more in line with the model training process, which can both enable more unlabeled samples to participate in model training and ensure the accuracy of the pseudo-labels among the unlabeled samples participating in model training.

[0079] Based on the above embodiments, as an optional embodiment, the confidence threshold determination method provided by the present invention further includes:

[0080] The number of unlabeled samples whose maximum value of the classification prediction result is higher than a preset threshold is normalized to obtain the normalized sample number.

[0081] Specifically, in classification tasks where difficulty varies depending on the category, the model will first master the categories that are easy to classify during the learning process. To encourage the model to make different predictions for each class, this invention proposes a class balancing penalty scheme based on the number of pseudo-label samples for different classes at the sample level. The normalization formula is shown below:

[0082]

[0083] Where, β t To normalize the sample size, βt =[β t (1),β t (2),β t (3),...β t (C)], where C is the total number of categories, τ′ is a constant, and Count can count the classification of all unlabeled data.

[0084] The normalized sample size is scaled based on preset boundary parameters to obtain the pseudo-label data class distribution.

[0085] Normalizing the number of unlabeled samples that meet the threshold requirements for each category directly affects the update of the confidence threshold, significantly impacting its updating. To ensure the confidence threshold updates correctly and the model can distinguish between different categories, the β threshold is adjusted accordingly. t The following scaling was performed:

[0086] β t (c)=β t (c)·(u b -l b )+l b +1-u b ;

[0087] Among them, u b and l b They represent β respectively t In this embodiment, the upper boundary is set to 0.1 and the lower boundary to 0 after scaling.

[0088] Correspondingly, determining the confidence threshold for each preliminary classification category based on the momentum update constant and the smoothing update amount includes:

[0089] Based on the momentum update constant, the smooth update amount, and the pseudo-label data class distribution, determine the confidence threshold corresponding to each of the preliminary classification categories.

[0090] After obtaining the pseudo-label data class distribution that meets the conditions, it is added to the confidence threshold update process. The confidence threshold update formula is as follows:

[0091]

[0092] Where, τ t (c) represents the confidence threshold, C represents the total number of categories, λ represents the momentum update constant, and τ represents the inequality. t-1 (c) represents the confidence threshold of the previous training batch.

[0093] Understandably, this invention incorporates a category-specific balancing penalty during the confidence threshold update process. For categories with fewer occurrences, the confidence threshold update is suppressed, ultimately resulting in a confidence threshold that better matches the model's learning process. This leads to superior model training performance and more diverse prediction results.

[0094] Based on the above embodiments, as an optional embodiment, the confidence threshold determination method provided by the present invention further includes:

[0095] Based on the confidence threshold corresponding to each of the initial classification categories, a loss function is determined for backpropagation.

[0096] The formula for calculating the loss function corresponding to the unlabeled samples is as follows:

[0097]

[0098] in, Here, μB represents the loss function for unlabeled samples, and q represents the number of unlabeled samples. i This indicates the prediction result after weak enhancement of the sample. It is q i After one-hot encoding, Ω(·) indicates strong enhancement of the sample. It is an indicator function; if the condition is met, it is assigned the value 1, τ. t It is the confidence threshold. This refers to calculating the cross-entropy, where p(·) represents the model's prediction result for the sample.

[0099] Optionally, the loss function corresponding to the unlabeled samples can be used to back-train the unsupervised model, thereby improving the performance of the unsupervised model.

[0100] It is understood that this invention introduces sample difficulty into the update process of the confidence threshold corresponding to the category, and obtains a confidence threshold that is more in line with the model training process by considering the sample-level difficulty, and / or introduces the balance penalty corresponding to the category into the update process of the confidence threshold, taking into account the prediction balance of each category in the confidence threshold update process.

[0101] Based on the above embodiments, as an optional embodiment, the confidence threshold determination method provided by the present invention further includes:

[0102] Obtain labeled samples and determine the first loss function corresponding to the labeled samples;

[0103] Specifically, in the SSL classification task, the samples are used in two ways: label samples and other samples. and unlabeled samples Represented as Where x i Represents the sample, y i N represents the corresponding label. L This indicates the total number of labeled samples. Where u i N represents unlabeled samples. U This represents the total number of unlabeled samples. The formula for calculating the first loss function is as follows:

[0104]

[0105] in, Let B be the first loss function, and let B be the number of labeled samples in this training batch. This refers to calculating the cross-entropy, where ω(·) indicates weak enhancement of the model, and p(·) represents the model's prediction result for the sample.

[0106] Correspondingly, determining the loss function for backpropagation based on the confidence threshold corresponding to each preliminary classification category includes:

[0107] Based on the confidence threshold corresponding to each of the preliminary classification categories, determine the second loss function corresponding to the unlabeled sample;

[0108] The formula for calculating the second loss function is as follows:

[0109]

[0110] in, Let q be the loss function for unlabeled samples, μ represent the ratio of the number of unlabeled samples to the number of labeled samples in a training batch, and q be the loss function for unlabeled samples. i This indicates the prediction result after weak enhancement of the sample. It is q i After one-hot encoding, Ω(·) indicates strong enhancement of the sample. It is an indicator function; if the condition is met, it is assigned the value 1, τ. t It is the confidence threshold. This refers to calculating the cross-entropy, where p(·) represents the model's prediction result for the sample.

[0111] If the initial confidence threshold is τ0, then the initial second loss function is calculated as follows:

[0112]

[0113] The loss function is determined based on the first loss function and the second loss function for backpropagation.

[0114] The loss function for gradient update during backpropagation in a semi-supervised model is:

[0115] It is understood that this invention introduces sample difficulty into the update process of the confidence threshold corresponding to the category, and obtains a confidence threshold that is more in line with the model training process by considering the sample-level difficulty, and / or introduces the balance penalty corresponding to the category into the update process of the confidence threshold, taking into account the prediction balance of each category in the confidence threshold update process.

[0116] Figure 4 This is the second flowchart illustrating the confidence threshold determination method provided by the present invention, as shown below. Figure 4 As shown, this invention proposes a sample-level difficulty-penalized confidence thresholding method (SampMatch). The figure illustrates the operation on unlabeled samples in a batch. Pre-classification is performed using the highest probability prediction result for each sample. Then, samples in each category are weighted according to their sample difficulty to obtain the initial update amount of the confidence threshold, which is finally applied to the class confidence threshold. The class confidence threshold is the confidence threshold for a specific category.

[0117] This invention is the first to consider sample-level factors in updating the confidence threshold. In a training batch, unlabeled samples are first initially classified based on the model's predictions. Then, unlabeled samples classified into the same class are weighted according to their difficulty scores to obtain the initial update amount of the confidence threshold for that class. To balance information across batches, an exponential moving average update is used to obtain the update amount of the confidence threshold. Finally, the confidence threshold is updated. To diversify the prediction results of the classification model, a class balance penalty (a balance penalty corresponding to a specific class) is added during the confidence threshold update process. Classes with fewer occurrences are suppressed from updating their confidence thresholds. Ultimately, a confidence threshold that better matches the learning process of the classification model is obtained, resulting in superior training performance.

[0118] Compared with current pseudo-label confidence thresholding methods, this invention considers the heterogeneity between samples and obtains class confidence thresholds based on sample-level characteristics. The obtained confidence thresholds are more consistent with the model training process, enabling more unlabeled samples to participate in model training while ensuring the accuracy of pseudo-labels among the unlabeled samples participating in model training.

[0119] The confidence threshold determination device provided by the present invention is described below. The confidence threshold determination device described below and the confidence threshold determination method described above can be referred to in correspondence.

[0120] Figure 5 This is a schematic diagram of the confidence threshold determination device provided by the present invention, as shown below. Figure 5 As shown, the present invention also provides a confidence threshold determination device, comprising:

[0121] The classification module 510 is used to acquire unlabeled samples and obtain the classification prediction results of the unlabeled samples in each training batch.

[0122] The first determining module 520 is used to obtain, based on the classification prediction results, a sample difficulty score at the sample level for each training batch and a preliminary classification category corresponding to each unlabeled sample.

[0123] The second determining module 530 is used to determine the confidence threshold corresponding to each of the preliminary classification categories based on the sample difficulty score and the preliminary classification category.

[0124] As one embodiment, the second determining module 530 is further configured to:

[0125] Based on the sample difficulty score, the initial update amount of the confidence threshold corresponding to each preliminary classification category is obtained;

[0126] The initial update amount is smoothed based on a preset momentum update constant to obtain a smoothed update amount of the confidence threshold between training batches.

[0127] Based on the momentum update constant and the smooth update amount, determine the confidence threshold corresponding to each of the preliminary classification categories.

[0128] As one embodiment, the second determining module 530 is further configured to:

[0129] Determine the number of unlabeled samples corresponding to each of the preliminary classification categories;

[0130] If the number of unlabeled samples is 1, the maximum value of the classification prediction result corresponding to the unlabeled sample is used as the initial update amount of the confidence threshold corresponding to the preliminary classification category;

[0131] If the number of unlabeled samples is greater than 1, the initial update amount of the confidence threshold corresponding to each preliminary classification category is obtained based on the proportion of the sample difficulty score corresponding to each unlabeled sample and the maximum value of the classification prediction result.

[0132] As one embodiment, the second determining module 530 is further configured to:

[0133] The number of unlabeled samples whose maximum value of the classification prediction result is higher than a preset threshold is normalized to obtain the normalized sample number.

[0134] The normalized sample size is scaled based on preset boundary parameters to obtain the pseudo-label data class distribution.

[0135] Correspondingly, determining the confidence threshold for each preliminary classification category based on the momentum update constant and the smoothing update amount includes:

[0136] Based on the momentum update constant, the smooth update amount, and the pseudo-label data class distribution, determine the confidence threshold corresponding to each of the preliminary classification categories.

[0137] As one embodiment, the second determining module 530 is further configured to:

[0138] The difference between the maximum and the second maximum value of the classification prediction results is taken as the sample difficulty score of the unlabeled sample.

[0139] The maximum value of the classification prediction result is used as the preliminary classification category for each unlabeled sample.

[0140] As one embodiment, it also includes:

[0141] The model training module is used to determine the loss function for backpropagation based on the confidence threshold corresponding to each of the initial classification categories.

[0142] As an example, the model training module is also used for:

[0143] Obtain labeled samples and determine the first loss function corresponding to the labeled samples;

[0144] Correspondingly, determining the loss function for backpropagation based on the confidence threshold corresponding to each preliminary classification category includes:

[0145] Based on the confidence threshold corresponding to each of the preliminary classification categories, determine the second loss function corresponding to the unlabeled sample;

[0146] The loss function is determined based on the first loss function and the second loss function for backpropagation.

[0147] Figure 6 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 6 As shown, the electronic device may include: a processor 610, a communications interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communications interface 620, and the memory 630 communicate with each other via the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute a confidence threshold determination method, which includes:

[0148] Obtain unlabeled samples and get the classification prediction results of the unlabeled samples in each training batch;

[0149] Based on the classification prediction results, the sample difficulty score at the sample level for each training batch and the preliminary classification category corresponding to each unlabeled sample are obtained.

[0150] Based on the sample difficulty score and the preliminary classification category, a confidence threshold is determined for each preliminary classification category.

[0151] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0152] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program that can be stored on a non-transitory computer-readable storage medium, wherein when the computer program is executed by a processor, the computer is capable of executing a confidence threshold determination method, the method comprising:

[0153] Obtain unlabeled samples and get the classification prediction results of the unlabeled samples in each training batch;

[0154] Based on the classification prediction results, the sample difficulty score at the sample level for each training batch and the preliminary classification category corresponding to each unlabeled sample are obtained.

[0155] Based on the sample difficulty score and the preliminary classification category, a confidence threshold is determined for each preliminary classification category.

[0156] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform a confidence threshold determination method, the method comprising:

[0157] Obtain unlabeled samples and get the classification prediction results of the unlabeled samples in each training batch;

[0158] Based on the classification prediction results, the sample difficulty score at the sample level for each training batch and the preliminary classification category corresponding to each unlabeled sample are obtained.

[0159] Based on the sample difficulty score and the preliminary classification category, a confidence threshold is determined for each preliminary classification category.

[0160] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0161] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0162] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for determining a confidence threshold, characterized in that, include: Obtain unlabeled samples and get the classification prediction results of the unlabeled samples in each training batch; Unlabeled samples are image samples without labels. The unlabeled samples are input into the classification model, and the classification model is used to output the classification prediction results of the unlabeled samples. Based on the classification prediction results, the sample difficulty score at the sample level for each training batch and the preliminary classification category corresponding to each unlabeled sample are obtained. Based on the sample difficulty score and the preliminary classification category, determine the confidence threshold corresponding to each preliminary classification category; The step of determining the confidence threshold corresponding to each preliminary classification category based on the sample difficulty score and the preliminary classification category includes: Based on the sample difficulty score, the initial update amount of the confidence threshold corresponding to each preliminary classification category is obtained; The formula for calculating the initial update amount is as follows: ； in, Representative category The initial update amount of the confidence threshold. m This refers to the batch that is classified as a category. The number of unlabeled samples; The initial update amount is smoothed based on a preset momentum update constant to obtain a smoothed update amount of the confidence threshold between training batches. After obtaining the initial update amounts for each category in the training batch, an exponential moving average update method is used to balance the updates between batches. The formula for calculating the smooth update amount is as follows: ； in, Let this be the momentum update constant; Based on the momentum update constant and the smooth update amount, determine the confidence threshold corresponding to each of the preliminary classification categories; Utilize the update amount of each class to adjust the confidence threshold. The update is performed as shown in the following formula: ； in, , This is the current training batch.

2. The confidence threshold determination method according to claim 1, characterized in that, The initial update amount for obtaining the confidence threshold corresponding to each preliminary classification category based on the sample difficulty score includes: Determine the number of unlabeled samples corresponding to each of the preliminary classification categories; If the number of unlabeled samples is 1, the maximum value of the classification prediction result corresponding to the unlabeled sample is used as the initial update amount of the confidence threshold corresponding to the preliminary classification category; If the number of unlabeled samples is greater than 1, the initial update amount of the confidence threshold corresponding to each preliminary classification category is obtained based on the proportion of the sample difficulty score corresponding to each unlabeled sample and the maximum value of the classification prediction result.

3. The confidence threshold determination method according to claim 1, characterized in that, Also includes: The number of unlabeled samples whose maximum value of the classification prediction result is higher than a preset threshold is normalized to obtain the normalized sample number. The normalized sample size is scaled based on preset boundary parameters to obtain the pseudo-label data class distribution. Correspondingly, determining the confidence threshold for each preliminary classification category based on the momentum update constant and the smoothing update amount includes: Based on the momentum update constant, the smooth update amount, and the pseudo-label data class distribution, determine the confidence threshold corresponding to each of the preliminary classification categories.

4. The confidence threshold determination method according to claim 1, characterized in that, The step of obtaining the sample difficulty score at the sample level for each training batch and the preliminary classification category corresponding to each unlabeled sample based on the classification prediction results includes: The difference between the maximum and the second maximum value of the classification prediction results is taken as the sample difficulty score of the unlabeled sample. The maximum value of the classification prediction result is used as the preliminary classification category for each unlabeled sample.

5. The confidence threshold determination method according to claim 1, characterized in that, Also includes: Based on the confidence threshold corresponding to each of the initial classification categories, a loss function is determined for backpropagation.

6. The confidence threshold determination method according to claim 5, characterized in that, Also includes: Obtain labeled samples and determine the first loss function corresponding to the labeled samples; Correspondingly, determining the loss function for backpropagation based on the confidence threshold corresponding to each preliminary classification category includes: Based on the confidence threshold corresponding to each of the preliminary classification categories, determine the second loss function corresponding to the unlabeled sample; The loss function is determined based on the first loss function and the second loss function for backpropagation.

7. A confidence threshold determination device, characterized in that, include: The classification module is used to acquire unlabeled samples and obtain the classification prediction results of the unlabeled samples in each training batch. Unlabeled samples are image samples without labels. The unlabeled samples are input into the classification model, and the classification model is used to output the classification prediction results of the unlabeled samples. The first determining module is used to obtain, based on the classification prediction results, a sample difficulty score at the sample level for each training batch and a preliminary classification category corresponding to each unlabeled sample. The second determining module is used to determine the confidence threshold corresponding to each of the preliminary classification categories based on the sample difficulty score and the preliminary classification category; The step of determining the confidence threshold corresponding to each preliminary classification category based on the sample difficulty score and the preliminary classification category includes: Based on the sample difficulty score, the initial update amount of the confidence threshold corresponding to each preliminary classification category is obtained; The formula for calculating the initial update amount is as follows: ； in, Representative category The initial update amount of the confidence threshold. m This refers to the batch that is classified as a category. The number of unlabeled samples; The initial update amount is smoothed based on a preset momentum update constant to obtain a smoothed update amount of the confidence threshold between training batches. After obtaining the initial update amounts for each category in the training batch, an exponential moving average update method is used to balance the updates between batches. The formula for calculating the smooth update amount is as follows: ； in, Let this be the momentum update constant; Based on the momentum update constant and the smooth update amount, determine the confidence threshold corresponding to each of the preliminary classification categories; Utilize the update amount of each class to adjust the confidence threshold. The update is performed as shown in the following formula: ； in, , This is the current training batch.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the confidence threshold determination method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the confidence threshold determination method as described in any one of claims 1 to 6.

Images