Method and system for training model and selecting model threshold
By training multiple intermediate models and calculating their boundary values, the threshold of the maximum boundary difference is automatically selected, which solves the problem of inconsistent model training results for detecting foreign matter adsorbed by rotor iron cord in industrial production and difficulty in selecting thresholds, and realizes high reliability and automation model training and threshold selection.
Patent Information
- Application Number
- CN202311097614.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-08-28
- Publication Date
- 2025-02-28
AI Technical Summary
The prior art is used in industrial production to detect foreign matter adsorption by rotor iron cord, which has problems such as inconsistent model training results, low reliability and difficult threshold selection, resulting in the inability to form an automated closed-loop training process.
By using the data set to train multiple intermediate models, and calculate the upper boundary value, lower boundary value and boundary difference of each intermediate model, select the upper and lower boundary value corresponding to the maximum boundary difference to determine the model threshold, thereby realizing automated model training and threshold selection.
The automated closed-loop training process is realized, which improves the reliability and robustness of the model, and solves the problems of inconsistent model training results and difficulty in selecting thresholds.
Smart Images
Figure CN119540674A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence technology, and more specifically, to a method and system for training models and selecting model thresholds. Background Art
[0002] Manufacturing is an important pillar of the national economy and has received great attention since the Industrial Revolution. In the process of industrial production, we often encounter problems on the production line, such as the problem of foreign matter adsorbed by the rotor core. On the production line, the rotor core may adsorb foreign matter such as nuts, screws, and bolts. If the rotor adsorbs foreign matter, it is likely to cause the production line to be paralyzed. In order to prevent the paralysis of the production line due to the adsorption of foreign matter by the rotor core, the prior art usually adopts the method of manual visual inspection to detect in real time whether the core has adsorbed foreign matter. However, this method will waste a lot of manpower. Summary of the invention
[0003] In order to improve the level of automation and avoid waste of manpower, this application adopts a big data model in industrial production to solve problems encountered on the production line, such as foreign matter adsorbed by the rotor core. For example, in the case of foreign matter adsorbed by the rotor core, we try to use an image classification model to make a judgment. First, let the industrial camera take a picture of the rotor at a fixed position, and then use the image classification model to judge whether the rotor adsorbs foreign matter based on the picture. However, in the training of the image classification model, it is difficult to ensure that the results of each model training are consistent, which easily leads to the following problems: the training results are sometimes good and sometimes bad, and a lot of manual intervention is required. There is no way to form an automated closed-loop training process; the trained model has low reliability and it is difficult to meet industrial needs. Moreover, after the trained model is online, how to select the optimal threshold is also a problem.
[0004] The embodiments of the present application provide a method and system for training a model and selecting a model threshold, which can realize an automated closed-loop training process, improve the reliability of the trained model, and ensure that the model has good robustness. On the other hand, the model threshold can be automatically selected based on the trained model, thereby realizing complete intelligent operation of model training and threshold selection. The "good robustness" referred to in this article means that the results of model training are very good and the online test results are also very good.
[0005] According to one aspect of the present application, a method for training models and selecting model thresholds is provided, the method comprising: using a data set to train a model to be trained to obtain multiple intermediate models; for the multiple intermediate models, using a test set of the data set to calculate an upper boundary value, a lower boundary value and a boundary difference value of each intermediate model, wherein the boundary difference value represents the difference between the upper boundary value and the lower boundary value, the upper boundary value represents the minimum value of the probability prediction value of the intermediate model for the positive samples in the test set, and the lower boundary value represents the maximum value of the probability prediction value of the intermediate model for the negative samples in the test set; corresponding to the multiple intermediate models, selecting the maximum boundary difference value among multiple boundary difference values; and determining the model threshold based on the upper boundary value and the lower boundary value corresponding to the maximum boundary difference value.
[0006] In some embodiments of the present application, optionally, the method further includes: determining an intermediate model corresponding to the maximum boundary difference as a final selection model for performing a probability prediction task.
[0007] In some embodiments of the present application, optionally, determining the model threshold based on the upper boundary value and the lower boundary value corresponding to the maximum boundary difference includes: calculating the model threshold by the following formula:
[0008] T=B low +λ*ΔB,
[0009] Where T represents the model threshold, ΔB represents the maximum boundary difference, and B low represents the lower boundary value corresponding to the maximum boundary difference, λ represents the threshold coefficient and 0<λ<1.
[0010] In some embodiments of the present application, optionally, the threshold coefficient λ is set to 1 / 3.
[0011] In some embodiments of the present application, optionally, the use of a data set to train a model to be trained to obtain multiple intermediate models includes: obtaining a hyperparameter for indicating the number of intermediate models to be obtained; and when the number of intermediate models currently obtained is less than the number of intermediate models indicated by the hyperparameter, using the data set to perform an intermediate model determination step.
[0012] In some embodiments of the present application, optionally, the intermediate model determination step includes: obtaining a model to be trained; performing iterative training on the model to be trained for a threshold number of times using the data set, wherein each time the iterative training is performed on the model to be trained, an iterative model can be obtained; for multiple iterative models obtained after the threshold number of iterative training, determining multiple loss values of the multiple iterative models on the test set of the data set; selecting the minimum loss value among the multiple loss values; and determining the iterative model corresponding to the minimum loss value as the intermediate model.
[0013] In some embodiments of the present application, optionally, the threshold number of times is any number selected from 10 to 20 times.
[0014] In some embodiments of the present application, optionally, the number of intermediate models indicated by the hyperparameter is any number selected from 10 to 100.
[0015] According to another aspect of the present application, a system for training a model and selecting a model threshold is provided, the system comprising: a memory configured to store instructions; and a processor configured to execute the instructions so that the system performs any one of the methods described above.
[0016] According to another aspect of the present application, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium. When the instructions are executed by a processor, the processor executes any one of the methods described above.
[0017] The training model and the method and system for selecting the model threshold provided by the present application are easy to use, and the processes of model training, model selection and threshold selection are all automated, without any manual operation, and the whole process is very simple. Compared with manually training the model, the training model and the method and system for selecting the model threshold provided by the present application can effectively improve the model accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] The above and other objects and advantages of the present application will become more fully apparent from the following detailed description in conjunction with the accompanying drawings, wherein the same or similar elements are represented by the same reference numerals.
[0019] Figure 1A to Figure 1B A flowchart of a method 100 for training a model and selecting a model threshold according to one or more embodiments of the present application is shown;
[0020] Figure 2 A schematic diagram of a system 200 for training a model and selecting a model threshold according to one or more embodiments of the present application is shown. DETAILED DESCRIPTION
[0021] For the purpose of brevity and illustration, the principles of the present application are described herein mainly with reference to its exemplary embodiments. However, those skilled in the art will readily recognize that the same principles are equally applicable to all types of training models and methods and systems for selecting model thresholds, and these same or similar principles may be implemented therein, and any such changes do not depart from the true spirit and scope of the present application.
[0022] The model to be trained in the embodiment of the present application may be an image classification model for classifying images. Accordingly, the image classification model may make a judgment based on the received picture to output a prediction value. In some embodiments, the prediction value may be a probability value between 0 and 1. Optionally, the prediction value "1" may represent that the current state is normal, and the prediction value "0" may represent that the current state is abnormal. Accordingly, the closer the prediction value is to "1", the greater the probability that the current state is normal, and the smaller the probability that it is abnormal; the closer the prediction value is to "0", the greater the probability that the current state is abnormal, and the smaller the probability that it is normal.
[0023] For example, in the case of foreign matter adsorbed by the rotor core, if the predicted value of the image classification model is closer to "1", it means that the probability that the rotor core shown in the current picture does not have the phenomenon of foreign matter adsorbed is greater; if the predicted value is closer to "0", it means that the probability that the rotor core shown in the current picture has the phenomenon of foreign matter adsorbed is greater.
[0024] The "model threshold" referred to in this article refers to the standard value used to compare with the predicted value to determine whether the final prediction result of the image classification model is normal or abnormal. For example, when the predicted value of the image classification model is greater than or equal to the model threshold, the current state can be determined to be normal; when the predicted value of the image classification model is less than the model threshold, the current state can be determined to be abnormal.
[0025] The method 100 and system 200 for training models and selecting model thresholds in the embodiments of the present application can first train a relatively good model as a final model, and then automatically select a model threshold based on the final model.
[0026] The following will be combined Figure 1A to Figure 1B The method 100 for training a model and selecting a model threshold value in one or more embodiments of the present application is described. In some embodiments, the method 100 for training a model and selecting a model threshold value can be provided by a system 200 for training a model and selecting a model threshold value (see Figure 2 ) to execute.
[0027] Figure 1A to Figure 1B FIG. 1 is a flow chart showing a method 100 for training a model and selecting a model threshold according to one or more embodiments of the present application. Figure 1A to Figure 1BAs shown, the method 100 for training a model and selecting a model threshold may include steps S110 to S194.
[0028] In step S110, a data set is obtained, wherein the data set may include multiple samples. In some embodiments, for example, for the example of predicting the adsorption of foreign matter on the rotor core, the multiple samples may be image samples. Table 1 shows the sample size of a data set of an embodiment of the present application.
[0029] Table 1 Sample size of a data set of an embodiment
[0030]
[0031] As shown in Table 1, in some embodiments, the data set can be divided into: a training set, a validation set, and a test set. The number of samples in the data set is the sum of the corresponding number of samples in the training set, the validation set, and the test set. As shown in Table 1, the total number of samples in the data set is 1072, of which the training set has 635 samples, the validation set has 114 samples, and the test set has 323 samples.
[0032] Each of the training set, validation set and test set includes positive sample data and negative sample data. Samples in a normal state (such as normal part samples) are referred to as "positive samples" herein, and samples in an abnormal state (such as faulty part samples) are referred to as "negative samples". The embodiment of the present application can set the label of the positive sample to "0" and the label of the negative sample to "1". Based on the category of the sample, the number of samples counted in Table 1 includes the number of positive samples, the number of negative samples and the total number of samples, where the total number of samples is the sum of the number of positive samples and the number of negative samples. As shown in Table 1, the number of positive samples in each set of the data set is roughly equivalent to the number of negative samples. Step S120 can be further performed after the data set is obtained via step S110.
[0033] In step S120, a hyperparameter for indicating the number of intermediate models to be obtained is obtained. Through step S120, it can be determined how many intermediate models to be trained in this process. Optionally, the hyperparameter indicating the number of intermediate models to be obtained can be a default by the system 200, or can be customized by the user. As an example, the number of intermediate models to be obtained indicated by the hyperparameter by default is 50, and the user can redefine the number of intermediate models to be obtained based on factors such as computing cost and application requirements. For example, based on the default number of intermediate models of 50 set by the hyperparameter, the user can customize the hyperparameter to indicate that the number of intermediate models to be obtained is 20 at the initial stage of the execution of the method 100 for training models and selecting model thresholds. In some embodiments, the number of intermediate models indicated by the hyperparameter is any number selected from 10 to 100. Step S122 can be further performed after step S120.
[0034] In step S122, the number of iterations in this process and the number of intermediate models accumulated are assigned with initial values 0. Through step S122, the value of the number of iterations in this process is 0, and the number of intermediate models accumulated is 0. In some embodiments, the value of the number of iterations and the number of intermediate models accumulated can be stored in a storage device of the system 200. After step S122, step S130 can be further performed.
[0035] In step S130, the model to be trained is obtained. In some embodiments, the model to be trained can be formed by construction. In some embodiments, for example, for the example of predicting the adsorption of foreign matter on the rotor core, the model to be trained can be an image classification model. As an example, the model to be trained formed by construction can perform data augmentation (Data Argumentation) on the input picture through the input part, and extract features through a backbone network (such as ResNet50). In the model to be trained, as an example, except for the last predicted layer, the parameters of other parts can be frozen and not participate in the training. In addition, in the model to be trained, as an example, dropout can be added to the prediction layer to avoid overfitting. Via step S130, the model to be trained formed by construction can be obtained. Step S140 can be further performed after step S130.
[0036] In step S140, the model to be trained is trained using the data set to obtain an iterative model. The data set may be obtained via step S110, and the model to be trained may be obtained via step S130. In some embodiments, the model to be trained may be trained and verified using the training set and the validation set in the data set to obtain an iterative model. Step S142 may be further performed after step S140.
[0037] In step S142, the number of iterations stored in this process is incremented by 1. In an embodiment of the present application, during the training of the model, multiple iterative models can be obtained by iterating the model multiple times, and an intermediate model can be obtained based on the multiple iterative models. Accordingly, in order to ensure that a sufficient number of iterative models are obtained during the training of the intermediate model, the number of iterations can be counted during the training of each intermediate model through step S142. Step S150 can be further performed after step S142.
[0038] In step S150, the loss value of the iterative model obtained in step S140 is determined. In some embodiments, step S150 may use a test set in the data set to determine the loss value of the iterative model. Step S160 may be further performed after step S150.
[0039] In step S160, based on the current number of iterations determined by step S142, it is determined whether the current number of iterations reaches a threshold number. The threshold number may be a number value pre-set in the system to determine the number of iterative models on which each intermediate model is to be trained. In some embodiments, the threshold number may be any number selected from 10 to 20 times, for example, the threshold number may be 15 times. In the case where it is determined that the current number of iterations does not reach the threshold number, the process may return to step S130, so that the model to be trained may be re-acquired by initializing the model for obtaining the next iterative model. In the case where it is determined that the current number of iterations reaches the threshold number, step S170 may be further performed.
[0040] In step S170, based on multiple loss values of multiple iterative models obtained through a threshold number of iterations, the smallest loss value is selected, and the iterative model corresponding to the minimum loss value is determined as the intermediate model.
[0041] As can be seen from the above, steps S130 to S170 can be used to train and determine the intermediate model, wherein each time an intermediate model is trained, the model to be trained must be iterated a threshold number of times (e.g., 15 times) to obtain multiple iterative models. For multiple iterative models, the iterative model with the smallest loss value on the test set can be saved as the intermediate model.
[0042] The method 100 for training a model and selecting a model threshold of the present application can train an intermediate model in a threshold number of iterations. After the intermediate model is trained in step S170, step S180 can be further performed.
[0043] In step S180, the number of intermediate models accumulated in this process is incremented by 1. In an embodiment of the present application, a final model can be obtained based on multiple intermediate models obtained through training. Accordingly, in order to ensure that a sufficient number of intermediate models are obtained during model training, the number of intermediate models accumulated can be counted through step S180 after each intermediate model is trained. Step S182 can be further executed after step S180.
[0044] In step S182, the upper boundary value, lower boundary value and boundary difference value of the intermediate model on the test set are calculated. The boundary difference value can represent the difference between the upper boundary value and the lower boundary value, and its calculation formula is shown in Formula 1:
[0045] ΔB=B up -B low (Formula 1)
[0046] Where ΔB represents the boundary difference, B up Indicates the upper boundary value, B low Indicates the lower boundary value. The upper boundary value B up It can represent the minimum value of the probability prediction value of the intermediate model for the positive samples in the test set, the lower boundary value B low It can represent the maximum value of the probability prediction value of the intermediate model for negative samples in the test set.
[0047] Through step S182, each time an intermediate model is trained, the boundary difference between the maximum value of the probability prediction value of the intermediate model for negative samples and the minimum value of the probability prediction value for positive samples on the test set can be calculated. The embodiment of the present application can characterize the ability of the model to distinguish between positive and negative samples by the boundary difference, wherein the larger the boundary difference of the model, the stronger the ability of the model to distinguish between positive and negative samples. Step S184 can be further performed after step S182.
[0048] In step S184, it is determined whether the number of intermediate models currently accumulated is 1. If it is determined that the number of intermediate models currently accumulated is 1, step S188 is further executed to save the current intermediate model (i.e., the intermediate model determined by step S170 this time) as the intermediate model to be compared. If it is determined that the number of intermediate models currently accumulated is not 1, there is a previously saved intermediate model to be compared, and step S186 can be further executed.
[0049] In step S186, it is compared whether the boundary difference of the current intermediate model is greater than the boundary difference of the intermediate model to be compared. Corresponding to the calculation of the boundary difference for each trained intermediate model in step S182, step 186 can compare the boundary difference of the currently obtained intermediate model and the boundary difference of the intermediate model to be compared. In the case of determining that the boundary difference of the current intermediate model is greater than the boundary difference of the intermediate model to be compared, step S188 can be further executed. In the case of determining that the boundary difference of the current intermediate model is not greater than the boundary difference of the intermediate model to be compared, step S190 can be further executed.
[0050] In step S188, the currently obtained intermediate model is saved as the intermediate model to be compared. That is to say, when the currently obtained intermediate model is the intermediate model obtained for the first time, the intermediate model obtained for the first time can be directly saved as the intermediate model to be compared; and when the currently obtained intermediate model is not the intermediate model obtained for the first time, by comparing the currently obtained intermediate model with the current intermediate model to be compared, the intermediate model with the larger boundary difference is stored as the intermediate model to be compared. The present application judges the quality of the training results based on the size of the boundary difference between the maximum value of the negative sample prediction value and the minimum value of the positive sample prediction value of the model on the test set, and each time an intermediate model is trained, the intermediate model with the best training result is saved through comparison. Step S190 can be further executed after step S188.
[0051] In step S190, it is determined whether the number of intermediate models obtained cumulatively reaches the number of intermediate models to be obtained indicated by the hyperparameter. In some embodiments, the hyperparameter indicating the number of intermediate models to be obtained can be determined via step S120. In the case where it is determined that the number of intermediate models obtained at this time does not reach the number of intermediate models indicated by the hyperparameter, it is possible to return to step S130, so that the model to be trained can be obtained by initializing the model to perform training on the next intermediate model. In the case where it is determined that the number of intermediate models obtained at this time reaches the number of intermediate models indicated by the hyperparameter, step S192 can be further performed.
[0052] In step S192, the currently determined intermediate model to be compared is saved as the final model, where the final model can be used to perform subsequent probability prediction tasks. The embodiment of the present application can train multiple intermediate models with the same model to be trained while keeping the data set unchanged, so as to select the model with the best training effect from the multiple intermediate models as the final model.
[0053] The method 100 for training models and selecting model thresholds in the embodiment of the present application can compare the boundary difference between the currently trained intermediate model and the boundary difference between the previously trained intermediate model through steps S182 to S192, select the intermediate model with the larger boundary difference to update the intermediate model to be compared, and save the final intermediate model to be compared as the final selected model for performing subsequent probability prediction tasks. That is, among multiple intermediate models (the number of which is determined by the hyperparameter obtained in step S120), the final selected model is the intermediate model with the largest boundary difference.
[0054] After the training is completed using the method 100 for training models and selecting model thresholds, the saved model weights and thresholds are the current optimal model and the corresponding optimal threshold. In some embodiments, the intermediate models can be sorted according to the size of the boundary difference of multiple intermediate models, and the lower boundary, upper boundary, recall rate, accuracy and boundary difference of each intermediate model are output. As an example, Table 2 shows a data table of multiple intermediate models accumulated by training models using method 100.
[0055] Table 2 Data table of multiple intermediate models
[0056]
[0057] In the embodiment shown in Table 1, the number of intermediate models indicated by the hyperparameter is 20. Accordingly, 20 different intermediate models can be obtained by using the training model and the method 100 for selecting the model threshold of the embodiment of the present application. As shown in Table 1, since the intermediate models 1 to 20 are each the iterative models with the smallest loss value on the test set of the corresponding multiple iterative models, the recall rate and accuracy of the intermediate models 1 to 20 are both 1. The "recall rate" described in this article is used to characterize the probability that the model recognizes abnormal situations. The higher the recall rate, the more abnormal situations the model can recognize. For the embodiment of predicting the situation of foreign matter adsorbed by the rotor core, the recall rate can be used to characterize the probability of identifying all situations of foreign matter adsorbed by the rotor core. The higher the recall rate, the more situations the model can recognize. In addition, according to the sorting, among the intermediate models 1 to 20, the intermediate model 1 has the largest boundary difference (ΔB), so the intermediate model 1 can be selected as the final model through step S192.
[0058] In the embodiment of the present application, when each intermediate model is trained, the boundary of the intermediate model for the positive sample prediction value and the negative sample prediction value is calculated, and by comparing the boundary difference of the currently trained intermediate model with the boundary difference of the intermediate model to be compared, the intermediate model with the largest boundary difference is saved as the final model through step S192. Step S194 can be further executed after step S192.
[0059] In step S194, the model threshold is determined based on the upper boundary value and the lower boundary value of the final selected model. In some embodiments, the model threshold can be calculated by the following formula 2:
[0060] T=B low +λ*ΔB (Equation 2)
[0061] Where T represents the model threshold, ΔB represents the boundary difference of the final selected model, and B low represents the lower boundary value of the final selection model, λ represents the threshold coefficient and 0<λ<1.
[0062] In some embodiments, the threshold coefficient λ can be taken as an intermediate value of 1 / 2, and in some other embodiments, the threshold coefficient λ can be taken as other values, for example, a value of 1 / 3, or a value of 2 / 3. In some embodiments, the threshold coefficient λ can be determined based on considerations of factors such as recall rate and accuracy. For embodiments that prioritize recall rate (i.e., hoping that the model can identify as many faulty parts as possible), in order to ensure that a sufficient number of faulty parts are recalled, the model threshold can be taken as a larger value. As an example, considering that the industry requires the recall rate to be as high as possible, the model threshold can be taken at a position 1 / 3 between the upper and lower boundaries. The model threshold is taken at 1 / 3 of the upper and lower boundaries, which ensures that the model has a certain robustness while also taking into account the recall rate.
[0063] The training model and the method 100 for selecting the model threshold of the present application can automatically extract the optimal threshold, set it as a macro parameter, and automatically use it in the running algorithm model. In this way, everything from solving the optimal model threshold to applying the optimal model threshold to the algorithm model can be automated, without the need for manual solution and manual intervention. In other words, the use of the training model and the method 100 for selecting the model threshold of the present application can, on the one hand, ensure the stability of the model training and ensure the selection of the optimal threshold of the model, thereby ensuring the robustness of the model. On the other hand, the model training does not require manual intervention, that is, it will automatically update and upload the model threshold, thereby realizing a completely intelligent operation.
[0064] The training model and the method 100 for selecting the model threshold of the present application can be applied to the image classification tasks of the Internet and industry, and have strong universality. The longitude of the model obtained by training can be greatly improved by using the training model and the method 100 for selecting the model threshold of the present application, wherein the accuracy of the obtained model can reach 100%.
[0065] It should be noted that although Figure 1A to Figure 1B The steps S110 to S194 are shown to be executed in a specific order, but the above diagram is only an example. In some other embodiments, the steps S110 to S194 may not be executed in a specific order according to the needs of a specific scenario. Figure 1A to Figure 1B The method 100 for training a model and selecting a model threshold is performed in the order shown. For example, in some other embodiments, steps S110 and S120 may be performed simultaneously, or step S120 may be performed first and then step S110.
[0066] The following will be combined Figure 2 The system 200 for training models and selecting model thresholds according to one or more embodiments of the present application is described.
[0067] Figure 2A schematic diagram of a system 200 for training a model and selecting a model threshold according to one or more embodiments of the present application is shown.
[0068] like Figure 2 As shown in , the system 200 for training models and selecting model thresholds may include a memory 210 and a processor 220. The memory 210 and the processor 220 may communicate with each other. In some embodiments, the memory 210 may be a non-volatile memory such as a flash memory, a ROM, a hard disk drive, a disk, or an optical disk. In other embodiments, the memory 210 may also be other types of memory. The memory 210 may be configured to store instructions. The processor 220 may be configured to execute the instructions so that the system 200 for training models and selecting model thresholds performs the method 100 for training models and selecting model thresholds according to one or more embodiments of the present application.
[0069] According to another aspect of the present application, a computer-readable storage medium is provided, in which instructions are stored. When the instructions are executed by a processor, the processor executes any one of the methods 100 for training a model and selecting a model threshold as described above. The computer-readable medium referred to in the present application includes various types of computer storage media, which can be any available medium that can be accessed by a general or special-purpose computer. For example, the computer-readable medium can include RAM, ROM, EPROM, EEPROM, etc. 2 PROM, register, hard disk, removable disk, CD-ROM or other optical disk storage, disk storage or other magnetic storage device, or can be used to carry or store the desired program code unit in the form of instruction or data structure and can be accessed by general or special computer, or general or special processor. As used herein, the disk usually copies data magnetically, while the dish uses laser to optically copy data. The above combination should also be included in the protection scope of computer-readable medium. Exemplary storage medium is coupled to the processor so that the processor can read and write information from / to the storage medium. In an alternative solution, the storage medium can be integrated into the processor. The processor and the storage medium can reside in the ASIC. The ASIC can reside in the user terminal. In an alternative solution, the processor and the storage medium can reside in the user terminal as discrete components.
[0070] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Those skilled in the art can think of other feasible changes or substitutions based on the technical scope disclosed in the present application, and such changes or substitutions are all included in the protection scope of the present application. In the absence of conflict, the implementation modes of the present application and the features in the implementation modes can also be combined with each other. The protection scope of the present application shall be subject to the description of the claims.
Claims
1. A method for training a model and selecting a model threshold, characterized in that: The method comprises: Using the data set to train the model to be trained to obtain multiple intermediate models; For the multiple intermediate models, use the test set of the data set to calculate the upper boundary value, the lower boundary value and the boundary difference value of each intermediate model, wherein the boundary difference value represents the difference between the upper boundary value and the lower boundary value, the upper boundary value represents the minimum value of the probability prediction value of the intermediate model for the positive samples in the test set, and the lower boundary value represents the maximum value of the probability prediction value of the intermediate model for the negative samples in the test set; Corresponding to the plurality of intermediate models, selecting a maximum boundary difference value among a plurality of boundary differences; and A model threshold is determined based on an upper boundary value and a lower boundary value corresponding to the maximum boundary difference value.
2. The method according to claim 1, characterized in that The method also includes: determining the intermediate model corresponding to the maximum boundary difference as a final selection model for performing a probability prediction task.
3. The method according to claim 1, characterized in that Determining the model threshold based on the upper boundary value and the lower boundary value corresponding to the maximum boundary difference includes: The model threshold is calculated by the following formula: T=B low +λ*ΔB, Where T represents the model threshold, ΔB represents the maximum boundary difference, and B low represents the lower boundary value corresponding to the maximum boundary difference, λ represents the threshold coefficient and 0<λ<1.
4. The method according to claim 3, characterized in that The threshold coefficient λ is set to 1 / 3.
5. The method according to claim 1, characterized in that: The step of training the model to be trained using the data set to obtain multiple intermediate models includes: obtaining a hyperparameter indicating the number of intermediate models to obtain; and When the number of currently obtained intermediate models is less than the number of intermediate models indicated by the hyperparameter, the intermediate model determination step is performed using the data set.
6. The method according to claim 5, characterized in that The intermediate model determination step comprises: Get the model to be trained; Performing iterative training on the model to be trained using the data set for a threshold number of times, wherein each time the iterative training is performed on the model to be trained, an iterative model can be obtained; For a plurality of iterative models obtained through a threshold number of iterative trainings, determining a plurality of loss values of the plurality of iterative models on a test set of the data set; Selecting the minimum loss value among the multiple loss values; and The iterative model corresponding to the minimum loss value is determined as the intermediate model.
7. The method according to claim 6, characterized in that The threshold number of times is any number selected from 10 to 20 times.
8. The method according to claim 6, characterized in that The number of intermediate models indicated by the hyperparameter is any number selected from 10 to 100.
9. A system for training a model and selecting a model threshold, characterized in that: The system comprises: a memory configured to store instructions; and A processor configured to execute the instructions so that the system performs the method according to any one of claims 1 to 8.
10. A computer-readable storage medium, wherein instructions are stored in the computer-readable storage medium, characterized in that: When the instructions are executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 8.