Method, device and equipment for determining probability threshold of classification model, and storage medium

Abstract

The application relates to a method and device for determining a probability threshold of a classification model, equipment and a storage medium, and belongs to the technical field of artificial intelligence. The method comprises the following steps: determining a plurality of probabilities based on a classification model and a plurality of test data sets, wherein the plurality of probabilities are used to represent probabilities that a plurality of test data in the test data sets belong to a first category of the classification model; for each probability, determining predicted labels of the plurality of test data by taking the probability as a probability threshold; determining an accuracy rate and a recall rate corresponding to the probability threshold based on the predicted labels of the plurality of test data, wherein the accuracy rate is used to reflect a false detection condition of the classification model, and the recall rate is used to reflect a missed detection condition of the classification model; and determining a target probability threshold meeting a condition based on the accuracy rate and the recall rate corresponding to the probability threshold, wherein the target probability threshold is taken as the probability threshold of the classification model. The application improves the reasoning capability of the classification model.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the field of artificial intelligence technology, and particularly relate to a method and apparatus for determining a probability threshold of a classification model, a device, and a storage medium. BACKGROUND

[0002] With the development of artificial intelligence technology, it has become very popular to classify images through an image classification model. When classifying images through an image classification model, an image is input into the image classification model, and the probability that the image belongs to multiple categories is output. The multiple categories are multiple categories pre-set by the image classification model. For each category, if the probability that the image belongs to the category is greater than the probability threshold corresponding to the category, the category will be output as the label of the image. Therefore, before classifying images through an image classification model, the probability threshold corresponding to each category needs to be determined.

[0003] In related technologies, 0.5 is used as the default probability threshold. However, the sample data used to train the image classification model may not be completely labeled, and therefore, when the image classification model is used to classify images, the probability may be too high or too low. At this time, using the default threshold will cause the problem of missed detection or false detection, that is, the inference ability of the image classification model will be poor. SUMMARY

[0004] Embodiments of the present application provide a method and apparatus for determining a probability threshold of a classification model, a device, and a storage medium, which can improve the inference ability of the classification model. The technical solution is as follows:

[0005] In one aspect, a method for determining a probability threshold of a classification model is provided, and the method comprises:

[0006] determining multiple probabilities based on a classification model and multiple test data sets, the multiple probabilities being used to represent the probability that multiple test data in the test data set belongs to a first category of the classification model;

[0007] for each probability, determining the predicted label of the multiple test data by taking the probability as a probability threshold;

[0008] determining the accuracy rate and the recall rate corresponding to the probability threshold based on the predicted label of the multiple test data, the accuracy rate being used to reflect the false detection situation of the classification model, and the recall rate being used to reflect the missed detection situation of the classification model;

[0009] determining a target probability threshold that meets a condition based on the accuracy rate and the recall rate corresponding to the probability threshold, the target probability threshold being taken as the probability threshold of the classification model.

[0010] In an aspect, a device for determining a probability threshold of a classification model is provided. The device includes:

[0011] a first determining module configured to determine a plurality of probabilities based on the classification model and a plurality of test data sets, the plurality of probabilities indicating probabilities that a plurality of test data in the test data sets belong to a first class of the classification model;

[0012] a second determining module configured to determine predicted labels of the plurality of test data sets by taking each probability as a probability threshold;

[0013] a third determining module configured to determine an accuracy rate and a recall rate corresponding to the probability threshold based on the predicted labels of the plurality of test data sets, the accuracy rate reflecting a false detection situation of the classification model, and the recall rate reflecting a missed detection situation of the classification model;

[0014] a fourth determining module configured to determine a target probability threshold satisfying a condition based on the accuracy rate and the recall rate corresponding to the probability threshold, the target probability threshold being taken as the probability threshold of the classification model.

[0015] In an aspect, an electronic device is provided. The electronic device includes a processor and a memory. The memory stores at least one program code for being executed by the processor to implement the method for determining a probability threshold of a classification model according to any one of the above aspects.

[0016] In an aspect, a computer-readable storage medium is provided. The computer-readable storage medium stores at least one program code for being executed by a processor to implement the method for determining a probability threshold of a classification model according to any one of the above aspects.

[0017] In an aspect, a computer program product is provided. When program codes in the computer program product are executed by a processor of an electronic device, the electronic device is enabled to perform the method for determining a probability threshold of a classification model according to any one of the above aspects.

[0018] In the embodiments of the present application, a plurality of candidate probability thresholds are determined based on a classification model by means of a plurality of test data sets, and then an accuracy rate and a recall rate corresponding to each probability threshold are determined based on the plurality of candidate probability thresholds. The accuracy rate can reflect a false detection situation of the classification model, and the recall rate can reflect a missed detection situation of the classification model. Therefore, the accuracy rate of a target probability threshold determined based on the accuracy rate and the recall rate is improved, and the inference ability of the classification model is improved. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1A schematic diagram showing an implementation environment according to an example embodiment of the present application is shown.

[0020] Figure 2 A structural block diagram of a terminal according to an example embodiment of the present application is shown.

[0021] Figure 3 A structural block diagram of a server according to an example embodiment of the present application is shown.

[0022] Figure 4 A flow chart of a method for determining a probability threshold of a classification model according to an example embodiment of the present application is shown.

[0023] Figure 5 A schematic diagram of a method for determining a probability threshold of a classification model according to an example embodiment of the present application is shown.

[0024] Figure 6 A flow chart of a method for determining a probability threshold of a classification model according to an example embodiment of the present application is shown.

[0025] Figure 7 A flow chart of a data classification method according to an example embodiment of the present application is shown.

[0026] Figure 8 A structural block diagram of a device for determining a probability threshold of a classification model according to an example embodiment of the present application is shown. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solutions and advantages of the present application clearer, the embodiments of the present application will be described in further detail below with reference to the drawings.

[0028] In the present document, "a plurality of" refers to two or more. The "and / or" describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the three cases of A alone, A and B together, and B alone. The character " / " generally represents that the associated objects before and after are in an "or" relationship.

[0029] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data involved in the present application are all information and data authorized by the user or authorized by all parties.

[0030] The method for determining the probability threshold of a classification model provided in this application is applied to an electronic device. In one possible implementation, the electronic device can be a terminal 100, that is, the terminal 100 determines the probability threshold of the classification model, and then classifies the target data to be classified based on the classification model and the probability threshold. The terminal 100 can be a mobile phone, tablet computer, computer, or Internet of Things (IoT) device, etc. In another possible implementation, the electronic device can be a server 200, that is, the server 200 determines the probability threshold of the classification model, and then classifies the target data to be classified based on the classification model and the probability threshold. The server 200 can be a single server, a server cluster consisting of several servers, or a cloud computing service center.

[0031] In another possible implementation, the electronic devices may be provided as terminal 100 and server 200; please refer to Figure 1 This diagram illustrates an implementation environment provided by an exemplary embodiment of this application. The implementation environment includes a terminal 100 and a server 200; the terminal 100 and the server 200 are connected via a wireless or wired network. Optionally, the server 200 is used to determine a probability threshold for a classification model using the method provided in the embodiments of this application. The terminal 100 sends the target data to be classified to the server 200, which classifies the target data based on the classification model and the probability threshold, and sends the classification result back to the terminal 100.

[0032] It should be noted that, Figure 1 The example described here is that the terminal 100 classifies the target data to be classified with the help of the server 200. In another embodiment, the server 200 can also send the classification model and probability threshold to the terminal 100, and the terminal 100 can classify the target data based on the classification model and probability threshold.

[0033] The method for determining the probability threshold of the classification model provided in this application embodiment can be applied to any scenario.

[0034] In the first scenario, the target data is an image, and the corresponding classification model is an image classification model. Therefore, the method provided in this application can be applied to scenarios involving image classification. For example, an image is input into the image classification model, which outputs the probability that the image belongs to multiple first categories. Based on the probability threshold of each first category and the probability that the image belongs to multiple first categories, the category of the image is determined. This category can be a label for the image. For example, the category could be "cat," "dog," etc., meaning the image classification model is used to identify animals in the image. Similarly, the category could be "text," "people," etc., meaning the image classification model is used to identify the category of the image.

[0035] In the second scenario, the target data is a speech signal, and the corresponding classification model is a speech signal classification model. The method provided in this application can then be applied to scenarios involving the classification of speech signals. For example, the speech signal is input into the speech signal classification model, which outputs the probability that the speech signal belongs to multiple first categories. Based on the probability threshold of each first category and the probability that the speech signal belongs to multiple first categories, the category of the speech signal is determined. This category can be a label for the speech signal. For example, the category could be the age of the object corresponding to the speech signal, meaning the speech signal classification model is used to identify the user's age. Alternatively, the category could be that the speech signal is a control command or a wake-up command, meaning the speech signal classification model is used to identify the category of the speech signal.

[0036] In this application embodiment, an electronic device is provided, which includes a processor and a memory; the memory stores at least one piece of program code, which is executed by the processor to implement the method for determining the probability threshold of the classification model provided in this application embodiment.

[0037] In one possible implementation, the electronic device can be provided as a terminal; please refer to [reference needed]. Figure 2 This illustration shows a schematic diagram of the structure of a terminal 100 provided in an exemplary embodiment of this application. The terminal 100 can be a mobile phone, tablet computer, computer, or IoT device, etc., with functions such as determining the probability threshold of a classification model. The terminal 100 in this application may include one or more of the following components: a processor 110, a memory 120, and a display screen 130.

[0038] Processor 110 may include one or more processing cores. Processor 110 connects to various parts within terminal 100 using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory 120, and by calling data stored in memory 120. Optionally, processor 110 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). Processor 110 may integrate one or more of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), Neural-network Processing Unit (NPU), and modem. Specifically, the CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required to be displayed on display screen 130; the NPU is used to implement Artificial Intelligence (AI) functions; and the modem is used to handle wireless communication. It is understandable that the aforementioned modem may not be integrated into the processor 110, but may be implemented using a separate chip.

[0039] The memory 120 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 120 may include a non-transitory computer-readable storage medium. The memory 120 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 120 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the various method embodiments described below, etc.; the data storage area may store data created according to the use of the terminal 100 (such as audio data, phone book, etc.).

[0040] Display screen 130 is a display component used to display a user interface. Optionally, the display screen 130 is a touch-enabled display screen, through which users can use their fingers, styluses, or any suitable object to perform touch operations on the display screen 130.

[0041] The display screen 130 is typically located on the front panel of the terminal 100. The display screen 130 can be designed as a full-screen, curved screen, irregularly shaped screen, dual-sided screen, or foldable screen. The display screen 130 can also be designed as a combination of a full-screen and a curved screen, or a combination of an irregularly shaped screen and a curved screen, etc., but this embodiment does not limit it in this way.

[0042] In addition, those skilled in the art will understand that the structure of the terminal 100 shown in the above figures does not constitute a limitation on the terminal 100. The terminal 100 may include more or fewer components than shown, or combine certain components, or have different component arrangements. For example, the terminal 100 may also include a microphone, speaker, radio frequency circuit, input unit, sensor, audio circuit, Wireless Fidelity (Wi-Fi) module, power supply, Bluetooth module, etc., which will not be described in detail here.

[0043] In another possible implementation, the electronic device can be provided as a server; please refer to [reference needed]. Figure 3 This illustration shows a schematic diagram of the structure of a server 200 provided in an exemplary embodiment of this application. The server 200 can vary considerably due to differences in configuration or performance, and may include one or more Central Processing Units (CPUs) 201 and one or more memories 202. The memories 202 are used to store executable instructions, and the processors 201 are configured to execute these executable instructions to implement the method for determining the probability threshold of the classification model provided in the various method embodiments described above. Of course, the server 200 may also have wired or wireless network interfaces, a keyboard, and input / output interfaces for input and output. The server 200 may also include other components for implementing device functions, which will not be elaborated upon here.

[0044] Please refer to Figure 4 The diagram illustrates a flowchart of a method for determining the probability threshold of a classification model provided in an exemplary embodiment of this application, the method comprising:

[0045] Step 401: The electronic device acquires multiple test datasets.

[0046] Each test dataset includes at least one test data point. For example, if the classification model is an image classification model, then multiple test datasets constitute multiple image test sets, and each image test set includes at least one test image. Similarly, if the classification model is a speech signal classification model, then multiple test datasets constitute multiple speech signal test sets, and each speech signal test set includes at least one test speech signal.

[0047] Each test data point may or may not be labeled with its first category. Theoretically, every test data point should be labeled with its first category; however, the case where a test data point is not labeled with its first category only occurs when there are omissions. Furthermore, this classification model is a multi-label classification model, meaning it has multiple preset labels, each label corresponding to one first category, resulting in multiple first categories. The multiple test datasets are test datasets with multiple first categories, and each first category corresponds to one test dataset. Moreover, the multiple first categories can cover all categories of the classification model, or only some categories.

[0048] Step 402: The electronic device determines multiple probabilities based on the classification model and multiple test datasets. These multiple probabilities represent the probability that multiple test data in the test datasets belong to the first category of the classification model.

[0049] In this step, the electronic device inputs each test data point from multiple test datasets into the classification model and outputs the probability that each test data point belongs to that first category. This classification model is a multi-label classification model, meaning it corresponds to multiple first categories. Therefore, in this step, for each test data point, the electronic device outputs the probability that the test data point belongs to each first category. Here, the test data is equivalent to sample data, and the probability that the test data point belongs to each first category is equivalent to the score of the test data point; please refer to [the relevant documentation / reference]. Figure 5 The electronic device first calculates the score for all samples. Multiple probabilities are then used as candidate probability thresholds, essentially finding all possible probability thresholds based on the scores.

[0050] For example, if the classification model is an image classification model, and the multiple test datasets are multiple image test sets (taking 5 image data as an example), and the first category is cat; then the probabilities of the 5 image test data belonging to cat are 0.5, 0.4, 0.6, 0.7, and 0.2, respectively.

[0051] In this embodiment, the electronic device finds all possible thresholds based on the probabilities (scores) of multiple test data determined by a classification model, replacing manually determined candidate thresholds. This saves manual workload and improves the efficiency of determining probability thresholds. Furthermore, since these candidate thresholds are determined by the classification model, they are closer to the actual thresholds, thus improving the accuracy of the determined probability thresholds.

[0052] Since multiple probabilities may contain the same probability, if such probabilities exist, the electronic device first performs deduplication on the multiple probabilities before executing step 403; if no multiple probabilities contain the same probability, the electronic device directly executes step 403 after completing step 402. In this embodiment, deduplication of the same probabilities reduces workload and improves efficiency.

[0053] It should be noted that a probability threshold range can also be set, limiting the probability threshold to within this range. Before the electronic device executes step 403, it first filters multiple probabilities based on this probability threshold range, deletes probabilities within the probability threshold range, and then executes step 403, thereby avoiding excessively extreme probability thresholds, such as avoiding a probability threshold of 0.

[0054] Step 403: For each probability, the electronic device uses that probability as a probability threshold to determine the predicted label for multiple test data.

[0055] For each test data point, if the probability of the test data point is not less than the probability threshold, the electronic device determines that the predicted label of the test data point is the first category; if the probability of the test data point is less than the probability threshold, the electronic device determines that the predicted label of the test data point is not the first category.

[0056] For example, in step 402, the electronic device determines that the probabilities of the five image test data belonging to cats are 0.5, 0.4, 0.6, 0.7, and 0.2, respectively. In this step, the electronic device first uses a probability threshold of 0.5 to determine the predicted labels of the five image test data as: cat, not cat, cat, cat, and not cat. The electronic device then uses a probability threshold of 0.4 to determine the predicted labels of the five image test data as: cat, cat, cat, cat, and not cat. The electronic device then uses a probability threshold of 0.6 to determine the predicted labels of the five image test data as: not cat, not cat, cat, cat, and not cat. The electronic device then uses a probability threshold of 0.7 to determine the predicted labels of the five image test data as: not cat, not cat, not cat, cat, and not cat. Finally, the electronic device uses a probability threshold of 0.2 to determine the predicted labels of the five image test data as: cat, cat, cat, cat, and cat.

[0057] Step 404: The electronic device determines the accuracy and recall corresponding to the probability threshold based on the predicted labels of multiple test data.

[0058] Among them, accuracy reflects the false positive rate of the classification model, and recall reflects the false negative rate of the classification model. This step can be achieved through the following steps (1) to (2), including:

[0059] (1) The electronic device determines the first quantity, the second quantity and the third quantity based on the predicted labels of multiple test data.

[0060] The first quantity represents the number of test data points where the predicted label and the labeled label are the same, i.e., the number of true positive test data points. The second quantity represents the number of test data points where the predicted label and the labeled label are different, i.e., the number of false positive test data points. The third quantity represents the number of test data points that were labeled but did not output a test label, i.e., the number of false negative test data points.

[0061] (2) The electronic device determines the accuracy and recall corresponding to the probability threshold based on the first quantity, the second quantity and the third quantity.

[0062] Precision is positively correlated with the first quantity and negatively correlated with the second quantity; recall is positively correlated with the first quantity and negatively correlated with the third quantity. Accordingly, the step of the electronic device determining the precision and recall corresponding to the probability threshold based on the first, second, and third quantities includes: the electronic device determining the precision and recall corresponding to the probability threshold based on the first, second, and third quantities using the following formulas one and two:

[0063] Formula 1: Precision = TP / (TP + TP)

[0064] Formula 2: Recall = TP / (TP + FN)

[0065] In this context, Precision and Precision represent precision and recall, respectively; TP, TP, and FN represent the first, second, and third quantities, respectively.

[0066] In another implementation, the electronic device can incorporate grayscale labels, where a grayscale label refers to a label at the defined boundary. The classification model is considered correct when it outputs this label, and not incorrect when it misses it. Correspondingly, the steps for the electronic device to determine the accuracy and recall corresponding to the probability threshold based on the first, second, and third quantities include: determining a fourth and fifth quantity; the fourth quantity being the number of first test data points (data whose sample label is grayscale and whose probability of belonging to the sample label is greater than the first probability threshold); the fifth quantity being the number of second test data points (data whose sample label is grayscale and whose probability of belonging to the sample label is less than the first probability threshold); determining the sum of the first and fourth quantities to obtain a sixth quantity; determining the difference between the second and fifth quantities to obtain a seventh quantity; and determining the accuracy and recall corresponding to the probability threshold based on the sixth, seventh, and third quantities.

[0067] Among them, precision is positively correlated with the sixth quantity and negatively correlated with the seventh quantity; recall is positively correlated with the sixth quantity and negatively correlated with the third quantity. The steps for the electronic device to determine the precision and recall corresponding to the probability threshold based on the sixth, seventh, and third quantities are the same as the process for the electronic device to determine the precision and recall corresponding to the probability threshold based on the first quantity (equivalent to the sixth quantity), the second quantity (equivalent to the seventh quantity), and the third quantity, and will not be repeated here.

[0068] In the embodiments of this application, the electronic device also takes grayscale labels into account, thereby reducing the difference between the expected result and the actual result corresponding to the probability threshold, and thus further improving the reasoning ability of the classification model.

[0069] Step 405: The electronic device determines a first weight and a second weight, where the first weight corresponds to accuracy and the second weight corresponds to recall.

[0070] Electronic devices can assign different weights to accuracy and recall based on their relative importance. For example, for categories with low tolerance for false positives, a higher weight can be assigned to accuracy; for categories with low tolerance for false negatives, a higher weight can be assigned to recall. Accordingly, the steps for determining the first and second weights by the electronic device can be as follows: the electronic device determines the tolerance parameter of the first category, which represents the degree to which the classification model tolerates false positives; based on the tolerance parameter, the first and second weights are determined; wherein, the first weight is positively correlated with the tolerance parameter, and the second weight is negatively correlated with the tolerance parameter, that is, the larger the tolerance parameter, the larger the first weight, and the smaller the tolerance parameter, the larger the second weight.

[0071] In this embodiment, the sum of the first weight and the second weight is 1, or the sum of the first weight and the second weight is not 1. In this embodiment, the example where the sum of the first weight and the second weight is not 1 is used for illustration. Furthermore, in this embodiment, the electronic device fixes the second weight to 1 and determines the first weight based on a tolerance parameter. Accordingly, the steps for the electronic device to determine the first weight and the second weight can be: the electronic device determines the tolerance parameter for the first category, determines the first weight based on the tolerance parameter, and the second weight is 1. The first weight is positively correlated with the tolerance parameter; that is, the larger the tolerance parameter, the larger the first weight, and the smaller the tolerance parameter, the smaller the first weight.

[0072] Step 406: The electronic device performs a weighted sum of the precision and recall corresponding to the probability threshold based on the first weight and the second weight to obtain the quality parameter corresponding to the probability threshold.

[0073] Based on the first and second weights, the electronic device uses the following formula (Formula 3) to perform a weighted sum of the precision and recall corresponding to the probability threshold, thus obtaining the quality parameter corresponding to the probability threshold:

[0074] Formula 3: Score=w1*Precision+w2*Recall

[0075] Where Score is the quality parameter corresponding to the probability threshold, w1 and w2 are the first weight and the second weight, respectively, and Precision and Recall are the precision and recall corresponding to the probability threshold, respectively.

[0076] It should be noted that, for the case where the second weight is fixed at 1, step 406 can be: the electronic device, based on the first weight, uses the following formula four to weight the accuracy corresponding to the probability threshold, and then sums it with the recall rate to obtain the quality parameter corresponding to the probability threshold:

[0077] Formula 4: Score=w1*Precision+Recall

[0078] Where Score is the quality parameter corresponding to the probability threshold, w1 is the first weight, and Precision and Recall are the precision and recall corresponding to the probability threshold, respectively.

[0079] In this embodiment, by assigning weights to precision and recall respectively, the relative importance of precision and recall is measured through these weights, thereby improving the accuracy of the quality parameters determined by weighted summation of precision and recall based on these weights. Furthermore, the electronic device determines the two weights corresponding to precision and recall based on the tolerance of the first category for false detections; that is, the two weights are flexibly set based on the actual needs of the first category to meet the requirements of different categories.

[0080] It should be noted that the electronic device can also correct the recall rate; accordingly, step 406, where the electronic device performs a weighted sum of the precision and recall rates corresponding to the probability threshold based on the first and second weights to obtain the quality parameter corresponding to the probability threshold, includes:

[0081] The electronic device determines a correction coefficient, which is used to correct the recall rate; based on the correction coefficient, the recall rate corresponding to the probability threshold is corrected; based on the first weight and the second weight, the precision rate corresponding to the probability threshold and the corrected recall rate are weighted and summed to obtain the quality parameter corresponding to the probability threshold.

[0082] The correction factor can be a default value; or it can be determined based on multiple test datasets; if the correction factor is determined based on multiple test datasets; for example, see [link to relevant documentation]. Figure 5 The electronic device determines whether the first test dataset corresponding to the first category is included in the multiple test datasets; if the first test dataset is included in the multiple test datasets, step (1) is executed; if the first test dataset is not included in the multiple test datasets, step (2) is executed; accordingly, the steps for the electronic device to determine the correction coefficient include the following steps (1) and (2):

[0083] (1) If the first test dataset is included in multiple test datasets, the electronic device determines the correction coefficient as the first value.

[0084] The first test dataset is the test dataset corresponding to the first category; the first value can be set and changed as needed, and in this embodiment, the first value is not specifically limited; for example, the first value is 1 or 1.1, etc., see below. Figure 5 .

[0085] (2) If the test dataset does not include the first test dataset, the electronic device determines the second category to which the first category belongs, determines the second test dataset corresponding to the second category, determines the number of test data of multiple first categories included in the second category based on the second test dataset, and determines the correction coefficient based on the number of test data of multiple first categories.

[0086] The step of determining the correction coefficient based on the quantity of test data for multiple first categories by the electronic device includes: the electronic device determining the proportion of test data for a specific first category to the total test data for multiple first categories to obtain the correction coefficient. For example, if the first category is "Ragdoll cat" and there is no test set corresponding to "Ragdoll cat" in multiple test datasets, then the second category (coarse category) to which the first category belongs is found to be "cat"; and all subcategories mapped to "cat" and the quantity of test data are found: 10 for "Ragdoll cat", 5 for "Siamese cat", 5 for "Garfield cat", and 10 for "tabby cat". Therefore, the correction coefficient corresponding to "Ragdoll cat" is (10+5+5+10) / 10=3, and the test dataset corresponding to the coarse category "cat" is used as the test dataset for "Ragdoll cat". In the embodiments of this application, even if there is no corresponding test set for the first category, the threshold can be adjusted according to the coarse category, which has wide applicability; for example, see below. Figure 5 .

[0087] It should be noted that the electronic device can first determine the correction coefficients and then determine the first weight corresponding to the accuracy; or it can first determine the first weight corresponding to the accuracy and then determine the correction coefficients. In this embodiment, the order of these two steps is not specifically limited. Figure 5 The following explanation uses the determination of the correction coefficient as an example.

[0088] Another point to note is that once the correction coefficients are determined, the electronic device can be calibrated based on these coefficients during the recall determination process. In other words, calibration and recall determination can be combined into a single step; for example, see [link to relevant documentation]. Figure 5 .

[0089] After the electronic device determines the quality parameter corresponding to the probability threshold, it can determine the target probability threshold using only the quality parameter. This process includes: the electronic device determining whether the quality parameter corresponding to the probability threshold meets the conditions; if the quality parameter corresponding to the probability threshold meets the conditions, the probability threshold is determined as the target probability threshold; if the quality parameter corresponding to the probability threshold does not meet the conditions, the probability threshold is discarded, and the quality parameter corresponding to the next probability threshold is re-determined.

[0090] It should be noted that after the electronic device determines the quality parameters corresponding to multiple probability thresholds, it can also select the probability threshold with the largest quality parameter from among the multiple probability thresholds based on the quality parameters corresponding to each probability threshold, and determine that probability threshold as the target probability threshold. In the embodiments of this application, since the target probability threshold is determined only by means of this quality parameter, the amount of computation is small, thereby improving efficiency.

[0091] It should be noted that the electronic device can also be set to set other limiting conditions for accuracy, recall and probability thresholds; accordingly, after the electronic device determines the quality parameter corresponding to each probability threshold, it executes step 407 to determine the target probability threshold.

[0092] Step 407: The electronic device determines the target probability threshold that meets the conditions based on the quality parameters, accuracy, recall and probability threshold corresponding to the probability threshold.

[0093] The electronic device determines a first parameter threshold, a second parameter threshold, and a probability threshold range. The first parameter threshold corresponds to the accuracy, the second parameter threshold corresponds to the recall, and the probability threshold range corresponds to the threshold. From multiple probability thresholds, a target probability threshold is determined. The quality parameter corresponding to the target probability threshold is the highest, and the accuracy of the probability threshold is not lower than the first parameter threshold, the recall of the probability threshold is not lower than the second parameter threshold, and the probability threshold is within the probability threshold range.

[0094] In this embodiment, by setting a first parameter threshold, the accuracy is limited to a level not lower than the first parameter threshold, thereby ensuring that the false positive rate does not exceed 1 - the first parameter threshold, thus limiting the false positive rate and preventing a large false positive rate. Furthermore, by setting a second parameter threshold, the recall is limited to a level not lower than the second parameter threshold, thereby ensuring that the false negative rate does not exceed 1 - the second parameter threshold, thus preventing a large false negative rate. Moreover, by setting a probability threshold range, the probability threshold is limited to a certain range, thus avoiding excessively extreme probability thresholds, such as avoiding a probability threshold of 0.

[0095] It should be noted that if no target probability threshold satisfying the above three conditions exists among the multiple probability thresholds, the electronic device determines the target probability threshold as the default probability threshold. This avoids the situation where no probability threshold can be determined. By using the default probability threshold as a fallback, the normal use of the classification model is ensured. The default probability threshold can be set and changed as needed. In this embodiment, no specific limitation is made on the default probability threshold; for example, the default probability threshold can be 0.5.

[0096] Another point to note is that if no target probability threshold satisfies the above three conditions among the multiple probability thresholds, the electronic device selects a target probability threshold that satisfies any two of the above conditions from among the multiple probability thresholds. If no target probability threshold satisfies any two of the above conditions among the multiple probability thresholds, the electronic device selects a target probability threshold that satisfies any one of the above conditions from among the multiple probability thresholds; for example, the electronic device selects the target probability threshold with the largest quality parameter from among the multiple probability thresholds based on the quality parameters corresponding to the multiple probability thresholds. In this embodiment of the application, if no target probability threshold satisfies the above three conditions among the multiple probability thresholds, a downgrade process is performed to determine the target probability threshold. In this embodiment of the application, the limiting conditions can be flexibly set according to the actual needs of the first category to meet the needs of different categories.

[0097] It should be noted that the classification model is a multi-label classification model, that is, the classification model corresponds to multiple first categories. For each first category, the target probability threshold corresponding to the first category is determined according to steps 401-407, thereby achieving fine-tuning for each first category. Compared with the coarse-tuning method where the probability thresholds corresponding to all first categories are the same, the embodiments of this application can achieve better efficiency.

[0098] In this embodiment of the application, multiple test datasets are used to determine multiple candidate probability thresholds based on a classification model. Then, based on the multiple candidate probability thresholds, the accuracy and recall corresponding to each probability threshold are determined. Accuracy reflects the false detection rate of the classification model, and recall reflects the false negative rate of the classification model. This improves the accuracy of the target probability threshold determined based on accuracy and recall, thereby improving the reasoning ability of the classification model.

[0099] Please refer to Figure 6 The diagram illustrates a flowchart of a method for determining the probability threshold of a classification model provided in an exemplary embodiment of this application, the method comprising:

[0100] Step 601: The electronic device acquires multiple test datasets.

[0101] Step 602: The electronic device determines multiple probabilities based on the classification model and multiple test datasets. These multiple probabilities represent the probability that multiple test data in the test datasets belong to the first category of the classification model.

[0102] It should be noted that steps 601 and 602 are the same as steps 401 and 402, respectively, and will not be repeated here.

[0103] Step 603: The electronic device selects the highest probability from among multiple probabilities.

[0104] The electronic device sorts multiple probabilities in descending order and selects the probability that is first in the sort; or, the electronic device sorts multiple probabilities in ascending order and selects the probability that is last in the sort.

[0105] Step 604: The electronic device determines the predicted labels for multiple test data using the selected probability as the probability threshold.

[0106] Step 605: The electronic device determines the accuracy and recall corresponding to the probability threshold based on the predicted labels of multiple test data.

[0107] Step 606: The electronic device determines a first weight and a second weight, where the first weight corresponds to accuracy and the second weight corresponds to recall.

[0108] Step 607: The electronic device performs a weighted sum of the precision and recall corresponding to the probability threshold based on the first weight and the second weight to obtain the quality parameter corresponding to the probability threshold.

[0109] Steps 604-607 are the same as steps 403-406, and will not be repeated here.

[0110] Step 608: The electronic device determines whether the probability threshold meets the conditions based on the quality parameters, accuracy, recall, and probability threshold corresponding to the probability threshold.

[0111] If the quality parameter is not lower than the third parameter threshold, the accuracy is not lower than the first parameter threshold, the recall is not lower than the second parameter threshold, and the probability threshold is within the probability threshold range, the electronic device determines that the probability threshold meets the conditions and executes step 609; if the quality parameter is lower than the third parameter threshold, or the accuracy is lower than the first parameter threshold, or the recall is lower than the second parameter threshold, or the probability threshold is not within the probability threshold range, the electronic device determines that the probability threshold does not meet the conditions and executes step 510.

[0112] Step 609: If the probability threshold meets the condition, the electronic device determines the probability threshold as the target probability threshold.

[0113] Step 610: If the probability threshold does not meet the condition, the electronic device deletes the probability threshold from the multiple probabilities and repeats step 603 until the target probability threshold is determined.

[0114] When the electronic device re-executes step 603, it selects the probability with the highest probability from the multiple probabilities after deleting the probability threshold.

[0115] In this embodiment, the electronic device sorts multiple probabilities in descending order, first taking the highest probability as the probability threshold, and then determining the precision and recall corresponding to the probability threshold until the target probability threshold is determined. This eliminates the need to determine the precision and recall corresponding to all probability thresholds, thereby reducing the computational load and improving the efficiency of determining the target probability threshold.

[0116] Please refer to Figure 7 The diagram illustrates a flowchart of a data classification method provided in an exemplary embodiment of this application, the method comprising:

[0117] Step 701: The electronic device determines the target data to be classified.

[0118] The target data to be classified can be an image or a voice signal; in this embodiment of the application, the type of target data to be classified is not specifically limited.

[0119] Step 702: The electronic device inputs the target data into the classification model and outputs the probability that the target data belongs to the first category.

[0120] If the target data is an image, then the classification model is an image classification model; if the target data is a speech signal, then the classification model is a speech signal classification model. Furthermore, if the classification model is a multi-label classification model, then the electronic device outputs the probability that the target data belongs to each first label.

[0121] Step 703: If the probability is greater than the target probability threshold, the electronic device determines that the target data is labeled as the first category.

[0122] If the probability is not greater than the target probability threshold, the electronic device determines that the label of the target data is the first label; for example, the input label is empty. The target data can have one or more labels; and for each first category, if the probability of the target data belonging to that first category is greater than the target probability threshold corresponding to that first category, the electronic device determines that the label of the target data is that first category. For example, if the classification model is an image classification model, and the image classification model corresponds to multiple first labels, namely cat, dog, and rabbit; and the image includes a dog and a cat, then the probability of the image belonging to the cat is greater than the target probability threshold corresponding to the cat, and the probability of the image belonging to the dog is greater than the target probability threshold corresponding to the dog, then the electronic device determines that the label of the image is cat and dog.

[0123] In this embodiment, since the target probability threshold corresponding to each first category is determined, the target data is classified based on the target probability threshold corresponding to each first category, which enables refined classification of the target data and improves the accuracy of classification.

[0124] Please refer to Figure 8The diagram illustrates a structural block diagram of a probability threshold determination device for a classification model provided in an exemplary embodiment of this application. The device includes:

[0125] The first determining module 801 is used to determine multiple probabilities based on a classification model and multiple test datasets. The multiple probabilities represent the probability that multiple test data in the test datasets belong to the first category of the classification model.

[0126] The second determining module 802 is used to determine the predicted label of multiple test data for each probability, using the probability as a probability threshold;

[0127] The third determination module 803 is used to predict labels based on multiple test data, determine the accuracy and recall corresponding to the probability threshold, the accuracy is used to reflect the false detection of the classification model, and the recall is used to reflect the false negative of the classification model.

[0128] The fourth determination module 804 is used to determine the target probability threshold that meets the conditions based on the accuracy and recall corresponding to the probability threshold. The target probability threshold is used as the probability threshold of the classification model.

[0129] In one possible implementation, the fourth determining module 804 includes:

[0130] The first determining unit is used to determine the first weight and the second weight, where the first weight corresponds to the accuracy and the second weight corresponds to the recall.

[0131] The weighted summation unit is used to perform a weighted summation of the accuracy and recall corresponding to the probability threshold based on the first weight and the second weight, so as to obtain the quality parameter corresponding to the probability threshold.

[0132] The second determining unit is used to determine the target probability threshold that meets the conditions based on the quality parameters, accuracy, recall and probability threshold corresponding to the probability threshold.

[0133] In one possible implementation, the second determining unit is used to determine a first parameter threshold, a second parameter threshold, and a probability threshold range. The first parameter threshold corresponds to accuracy, the second parameter threshold corresponds to recall, and the probability threshold range corresponds to a threshold. From multiple probability thresholds, a target probability threshold is determined. The quality parameter corresponding to the target probability threshold is the highest, and the accuracy of the probability threshold is not lower than the first parameter threshold, the recall of the probability threshold is not lower than the second parameter threshold, and the probability threshold is within the probability threshold range.

[0134] In one possible implementation, a first determining unit is used to determine a tolerance parameter for a first category, the tolerance parameter representing the degree to which the classification model tolerates false detections; based on the tolerance parameter, a first weight and a second weight are determined; wherein the first weight is positively correlated with the tolerance parameter, and the second weight is negatively correlated with the tolerance parameter.

[0135] In one possible implementation, a weighted summation unit is used to determine correction coefficients, which are used to correct the recall rate; the recall rate corresponding to each probability threshold is corrected based on the correction coefficients; and the precision rate and the corrected recall rate corresponding to the probability threshold are weighted and summed based on a first weight and a second weight to obtain the quality parameter corresponding to the probability threshold.

[0136] In one possible implementation, the weighted summation unit is used to determine the correction coefficient as a first value if the first test dataset is included in the multiple test datasets, and the first test dataset is the test dataset corresponding to the first category; if the first test dataset is not included in the test datasets, it determines the second category to which the first category belongs, determines the second test dataset corresponding to the second category, determines the number of test data of the multiple first categories included in the second category based on the second test dataset, and determines the correction coefficient based on the number of test data of the multiple first categories.

[0137] In one possible implementation, the third determining module 803 includes:

[0138] The third determining unit is used to determine a first quantity, a second quantity, and a third quantity based on the predicted labels of multiple test data. The first quantity is the number of test data with the same predicted label and the same annotation label. The second quantity is the number of test data with different predicted labels and annotation labels. The third quantity is the number of test data with annotation labels but no test labels are output.

[0139] The fourth determining unit is used to determine the precision and recall based on the first quantity, the second quantity, and the third quantity.

[0140] In one possible implementation, a fourth determining unit is used to determine a fourth quantity and a fifth quantity. The fourth quantity is the quantity of first test data, which consists of data whose sample label is a grayscale label and whose probability of belonging to the sample label is greater than a first probability threshold. The fifth quantity is the quantity of second test data, which consists of data whose sample label is a grayscale label and whose probability of belonging to the sample label is less than a first probability threshold. The sum of the first and fourth quantities is determined to obtain a sixth quantity. The difference between the second and fifth quantities is determined to obtain a seventh quantity. Based on the sixth, seventh, and third quantities, the precision and recall are determined.

[0141] In one possible implementation, the second determining module 802 includes:

[0142] The selection unit is used to select the highest probability from multiple probabilities;

[0143] The fifth determining unit is used to determine the predicted labels for multiple test data using the selected probability as the probability threshold.

[0144] The device also includes:

[0145] The fifth determination module is used to determine the probability threshold as the target probability threshold if the predicted label based on multiple test data meets the conditions.

[0146] The deletion module is used to delete the probability threshold from multiple probabilities if the predicted label based on multiple test data does not meet the condition.

[0147] The selection unit is used to select the step with the highest probability from multiple probabilities until the target probability threshold is determined.

[0148] In one possible implementation, the device further includes:

[0149] The fifth determination module is used to determine the target data to be classified;

[0150] The classification module is used to input target data into the classification model and output the probability that the target data belongs to the first category;

[0151] The sixth determination module is used to determine the label of the target data as the first category if the probability is greater than the target probability threshold.

[0152] In this embodiment of the application, multiple test datasets are used to determine multiple candidate probability thresholds based on a classification model. Then, based on the multiple candidate probability thresholds, the accuracy and recall corresponding to each probability threshold are determined. Accuracy reflects the false detection rate of the classification model, and recall reflects the false negative rate of the classification model. This improves the accuracy of the target probability threshold determined based on accuracy and recall, thereby improving the reasoning ability of the classification model.

[0153] This application also provides a computer-readable storage medium storing at least one piece of program code, which is executed by a processor to implement the method for determining the probability threshold of the classification model shown in the above embodiments.

[0154] This application also provides a computer program product, which, when the program code in the computer program product is executed by an electronic device, enables the electronic device to execute the method for determining the probability threshold of the classification model as shown in the above embodiments.

[0155] In some embodiments, the computer program involved in the present application embodiments may be deployed on an electronic device for execution, or executed on multiple electronic devices located in one location, or executed on multiple electronic devices distributed in multiple locations and interconnected through a communication network. Multiple electronic devices distributed in multiple locations and interconnected through a communication network may constitute a blockchain system.

[0156] Those skilled in the art will recognize that the functions described in the embodiments of this application in one or more of the above examples can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more program codes or code on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transfer of computer programs from one place to another. Storage media can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0157] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for determining the probability threshold of a classification model, characterized in that, The method includes: Based on a classification model and multiple test datasets, multiple probabilities are determined. These multiple probabilities represent the probability that multiple test data in the test datasets belong to the first category of the classification model. The classification model is preset with multiple labels, each corresponding to a first category. The multiple test datasets are test datasets for the multiple first categories. For each probability, the predicted label of the plurality of test data is determined using the probability as a probability threshold; Based on the predicted labels of the multiple test data, a first quantity, a second quantity, and a third quantity are determined. The first quantity is the number of test data with the same predicted label and the same annotation label. The second quantity is the number of test data with different predicted labels and annotation labels. The third quantity is the number of test data with annotation labels but no test labels are output. The fourth quantity and the fifth quantity are determined. The fourth quantity is the quantity of the first test data, which is data whose sample label is a grayscale label and whose probability of belonging to the sample label is greater than a first probability threshold. The fifth quantity is the quantity of the second test data, which is data whose sample label is a grayscale label and whose probability of belonging to the sample label is less than a first probability threshold. The sum of the first quantity and the fourth quantity is determined to obtain the sixth quantity; the difference between the second quantity and the fifth quantity is determined to obtain the seventh quantity; based on the sixth quantity, the seventh quantity, and the third quantity, the accuracy and recall are determined, wherein the accuracy is used to reflect the false detection of the classification model, and the recall is used to reflect the false negative of the classification model. Determine a first weight and a second weight, wherein the first weight corresponds to the accuracy and the second weight corresponds to the recall. Based on the first weight and the second weight, the precision and recall corresponding to the probability threshold are weighted and summed to obtain the quality parameter corresponding to the probability threshold. Based on the quality parameters, accuracy, recall, and probability threshold corresponding to the probability threshold, a target probability threshold that meets the conditions is determined.

2. The method according to claim 1, characterized in that, The step of determining the target probability threshold that meets the conditions based on the quality parameters, precision, recall, and the probability threshold corresponding to the probability threshold includes: Determine a first parameter threshold, a second parameter threshold, and a probability threshold range, wherein the first parameter threshold corresponds to the accuracy, the second parameter threshold corresponds to the recall, and the probability threshold range corresponds to the threshold; From the plurality of probability thresholds, a target probability threshold is determined, wherein the quality parameter corresponding to the target probability threshold is the highest, and the accuracy of the probability threshold is not lower than the first parameter threshold, the recall of the probability threshold is not lower than the second parameter threshold, and the probability threshold is within the range of the probability thresholds.

3. The method according to claim 1, characterized in that, The determination of the first weight and the second weight includes: Determine the tolerance parameter for the first category, wherein the tolerance parameter represents the degree to which the classification model tolerates false detections; Based on the tolerance parameter, the first weight and the second weight are determined; The first weight is positively correlated with the tolerance parameter, and the second weight is negatively correlated with the tolerance parameter.

4. The method according to claim 1, characterized in that, The step of weighted summing of precision and recall corresponding to the probability threshold based on the first weight and the second weight to obtain the quality parameter corresponding to the probability threshold includes: Determine the correction factor, which is used to correct the recall rate; The recall rate corresponding to each probability threshold is corrected based on the correction coefficient; Based on the first weight and the second weight, the accuracy and corrected recall corresponding to the probability threshold are weighted and summed to obtain the quality parameter corresponding to the probability threshold.

5. The method according to claim 4, characterized in that, The determination of the correction coefficient includes: If the plurality of test datasets includes a first test dataset, the correction coefficient is determined to be a first value, and the first test dataset is the test dataset corresponding to the first category; If the test dataset does not include the first test dataset, determine the second category to which the first category belongs, determine the second test dataset corresponding to the second category, determine the number of test data for multiple first categories included in the second category based on the second test dataset, and determine the correction coefficient based on the number of test data for multiple first categories.

6. The method according to claim 1, characterized in that, For each probability, using that probability as a probability threshold to determine the predicted label for the plurality of test data includes: Select the largest probability from the plurality of probabilities; The predicted labels for the multiple test data are determined using the selected probability as a probability threshold. The method further includes: If the probability threshold is determined to meet the condition based on the predicted label of the multiple test data, the probability threshold is determined as the target probability threshold. If the predicted label based on the multiple test data determines that the probability threshold does not meet the condition, then the probability threshold is deleted from the multiple probabilities, and the step of selecting the largest probability from the multiple probabilities is performed until the target probability threshold is determined.

7. The method according to claim 1, characterized in that, The method further includes: Identify the target data to be classified; The target data is input into the classification model, and the probability that the target data belongs to the first category is output. If the probability is greater than the target probability threshold, the label of the target data is determined to be the first category.

8. A device for determining the probability threshold of a classification model, characterized in that, The device includes: The first determining module is used to determine multiple probabilities based on a classification model and multiple test datasets. The multiple probabilities are used to represent the probability that multiple test data in the test datasets belong to the first category of the classification model. The classification model is preset with multiple labels, the multiple labels correspond to multiple first categories, and the multiple test datasets are test datasets of the multiple first categories. The second determining module is used to determine the predicted label of the plurality of test data for each probability, using the probability as a probability threshold; The third determining module is used to determine the accuracy and recall corresponding to the probability threshold based on the predicted labels of the multiple test data. The accuracy is used to reflect the false detection of the classification model, and the recall is used to reflect the false negative of the classification model. The fourth determining module is used to determine a target probability threshold that meets the conditions based on the accuracy and recall corresponding to the probability threshold, and the target probability threshold is used as the probability threshold of the classification model. The fourth determining module includes: A first determining unit is configured to determine a first weight and a second weight, wherein the first weight corresponds to the accuracy and the second weight corresponds to the recall. The weighted summation unit is used to perform a weighted summation of the accuracy and recall corresponding to the probability threshold based on the first weight and the second weight, so as to obtain the quality parameter corresponding to the probability threshold. The second determining unit is used to determine the target probability threshold that meets the conditions based on the quality parameter, accuracy, recall and the probability threshold corresponding to the probability threshold. The third determining module includes: The third determining unit is used to determine a first quantity, a second quantity, and a third quantity based on the predicted labels of the multiple test data. The first quantity is the number of test data with the same predicted label and the same annotation label. The second quantity is the number of test data with different predicted labels and annotation labels. The third quantity is the number of test data with annotation labels but no test labels output. The fourth determining unit is used to determine a fourth quantity and a fifth quantity, wherein the fourth quantity is the quantity of first test data, the first test data being data whose sample label is a grayscale label and whose probability of belonging to the sample label is greater than a first probability threshold; the fifth quantity is the quantity of second test data, the second test data being data whose sample label is a grayscale label and whose probability of belonging to the sample label is less than a first probability threshold; the sum of the first quantity and the fourth quantity is determined to obtain a sixth quantity; the difference between the second quantity and the fifth quantity is determined to obtain a seventh quantity; and the accuracy and the recall are determined based on the sixth quantity, the seventh quantity, and the third quantity.

9. The apparatus according to claim 8, characterized in that, The second determining unit is configured to determine a first parameter threshold, a second parameter threshold, and a probability threshold range, wherein the first parameter threshold corresponds to the accuracy, the second parameter threshold corresponds to the recall, and the probability threshold range corresponds to the threshold; and from the plurality of probability thresholds, a target probability threshold is determined, wherein the quality parameter corresponding to the target probability threshold is the highest, and the accuracy of the probability threshold is not lower than the first parameter threshold, the recall of the probability threshold is not lower than the second parameter threshold, and the probability threshold is within the probability threshold range.

10. The apparatus according to claim 8, characterized in that, The first determining unit is configured to determine a tolerance parameter for the first category, the tolerance parameter representing the degree of tolerance of the classification model to false detections; and based on the tolerance parameter, determine a first weight and a second weight; wherein the first weight is positively correlated with the tolerance parameter, and the second weight is negatively correlated with the tolerance parameter.

11. The apparatus according to claim 8, characterized in that, The weighted summation unit is used to determine the correction coefficient, which is used to correct the recall rate; to correct the recall rate corresponding to each probability threshold based on the correction coefficient; and to perform a weighted summation of the precision rate and the corrected recall rate corresponding to the probability threshold based on the first weight and the second weight to obtain the quality parameter corresponding to the probability threshold.

12. The apparatus according to claim 11, characterized in that, The weighted summation unit is used to determine the correction coefficient as a first value if the plurality of test datasets include a first test dataset, wherein the first test dataset is the test dataset corresponding to the first category. If the test dataset does not include the first test dataset, determine the second category to which the first category belongs, determine the second test dataset corresponding to the second category, determine the number of test data for multiple first categories included in the second category based on the second test dataset, and determine the correction coefficient based on the number of test data for multiple first categories.

13. The apparatus according to claim 8, characterized in that, The second determining module includes: A selection unit is used to select the largest probability from the plurality of probabilities; The fifth determining unit is used to determine the predicted label of the plurality of test data using the selected probability as a probability threshold; The device further includes: The fifth determining module is used to determine the probability threshold as the target probability threshold if the predicted label based on the multiple test data satisfies the condition. The deletion module is used to delete the probability threshold from the plurality of probabilities if the predicted label based on the plurality of test data is determined to not meet the condition, and to perform the step of selecting the largest probability from the plurality of probabilities until the target probability threshold is determined.

14. The apparatus according to claim 8, characterized in that, The device further includes: The fifth determination module is used to determine the target data to be classified; The classification module is used to input the target data into the classification model and output the probability that the target data belongs to the first category; The sixth determining module is used to determine the label of the target data as the first category if the probability is greater than the target probability threshold.

15. An electronic device, characterized in that, The electronic device includes a processor and a memory; the memory stores at least one piece of program code, which is executed by the processor to implement the method for determining the probability threshold of the classification model as described in any one of claims 1 to 7.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one piece of program code, which is executed by a processor to implement the method for determining the probability threshold of the classification model as described in any one of claims 1 to 7.

17. A computer program product, characterized in that, When the program code in the computer program product is executed by the processor of the electronic device, the electronic device is able to perform the method for determining the probability threshold of the classification model as described in any one of claims 1 to 7.

Images