A method for information classification and related device and server

Abstract

The present application discloses a method for information classification and related devices and servers, which are used to improve the accuracy of category prediction with less sample data in the classification model. The method of the present application includes: obtaining information to be classified; obtaining a first probability corresponding to the information to be classified through a classification model; modifying the first probability based on an information classification rule to obtain a second probability, wherein the information classification rule represents the corresponding relationship between the original probability set and the modified probability set, the original probability set includes M original probabilities, the modified probability set includes M modified probabilities, M is an integer greater than 1, the original probability and the modified probability have a one-to-one corresponding relationship, and the modified probability in the modified probability set and the original probability in the original probability set satisfy the same distribution order; and generating an information classification result according to the second probability.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an information classification method and related devices and servers. Background Art

[0002] With the rapid development of computer technology, computers have become an indispensable part of people's lives, and people are gradually using computers to replace manual work to complete certain tasks. For example, in fields such as text classification or image classification, where information needs to be classified, people usually use classification models based on machine learning to classify information such as text or images.

[0003] In classification models based on machine learning, the input is usually information such as text or images to be classified, and the output is the category to which the text or image to be classified belongs. Generally speaking, these classification models first output the probability that the text or image to be classified belongs to each category, and then determine the category to which the text or image to be classified belongs based on these probabilities, that is, output the category with the largest probability value as the category to which the text or image to be classified belongs.

[0004] At present, most classification models based on machine learning have problems predicting imbalanced data. That is, when the classification model is trained using imbalanced data, the results predicted by the classification model will shift to the category with more sample data in the training data, resulting in low prediction accuracy for the category with less sample data. Summary of the invention

[0005] The embodiments of the present application provide a method for information classification and related devices and servers. After obtaining the predicted probability output by the classification model, the predicted probability corresponding to each category is re-corrected according to the information classification rules, so that the predicted probability corresponding to each category can be reconstructed and aligned according to its distribution in the original probability corresponding to the category to which it belongs, so that the distribution of each category is as similar as possible, and then classification is performed, which can effectively improve the accuracy of category prediction with less sample data.

[0006] A first aspect of an embodiment of the present application provides a method for information classification, including:

[0007] Get the information to be classified;

[0008] Obtaining a first probability corresponding to the information to be classified through a classification model;

[0009] The first probability is modified based on the information classification rule to obtain a second probability, wherein the information classification rule represents a corresponding relationship between an original probability set and a modified probability set, the original probability set includes M original probabilities, the modified probability set includes M modified probabilities, M is an integer greater than 1, the original probability and the modified probability have a one-to-one corresponding relationship, and the modified probability in the modified probability set and the original probability in the original probability set satisfy the same distribution order;

[0010] An information classification result is generated according to the second probability.

[0011] A second aspect of an embodiment of the present application provides an information classification device, including:

[0012] An acquisition unit, used for acquiring information to be classified;

[0013] The acquisition unit is further used to acquire the first probability corresponding to the information to be classified through the classification model;

[0014] a correction unit, configured to perform correction processing on the first probability based on the information classification rule to obtain a second probability, wherein the information classification rule represents a correspondence between an original probability set and a modified probability set, the original probability set includes M original probabilities, the modified probability set includes M modified probabilities, M is an integer greater than 1, the original probability and the modified probability have a one-to-one correspondence, and the modified probability in the modified probability set and the original probability in the original probability set satisfy the same distribution order;

[0015] A generating unit is used to generate an information classification result according to the second probability.

[0016] In a possible design, in an implementation manner of the second aspect of the embodiments of the present application, a determining unit is further included;

[0017] The acquisition unit is also used to obtain the classification threshold of the classification model, and the classification threshold belongs to the original probability set;

[0018] A determination unit, used to determine a target probability threshold according to a classification threshold;

[0019] The generating unit is further used to generate information classification rules according to the target probability threshold, wherein the modified probability set includes the target probability threshold, the target probability threshold belongs to the Nth modified probability among the M modified probabilities, the first modified probability to the N-1th modified probability are all less than the target probability threshold, the N+1th modified probability to the Mth modified probability are all greater than the target probability threshold, and N is an integer greater than 1 and less than M.

[0020] In a possible design, in an implementation of the second aspect of the embodiments of the present application,

[0021] The acquisition unit is also used to obtain a receiver operating characteristic ROC curve of the classification model;

[0022] The acquisition unit is also used to acquire the classification threshold of the classification model according to the ROC curve.

[0023] In a possible design, in an implementation of the second aspect of the embodiments of the present application,

[0024] The acquisition unit is also used to obtain the model accuracy value of the coordinate point on the ROC curve, and the model accuracy value is determined by the precision and recall rate corresponding to the coordinate point;

[0025] The acquisition unit is also used to acquire the classification threshold of the classification model according to the model accuracy value.

[0026] In a possible design, in an implementation of the second aspect of the embodiments of the present application,

[0027] The acquisition unit is further used to acquire the distance between the coordinate point on the ROC curve and the target coordinate point, wherein the abscissa of the target coordinate point is 0 and the ordinate of the target coordinate point is 1;

[0028] The determination unit is further used to determine a first coordinate point on the ROC curve, where the first coordinate point is a coordinate point on the ROC curve with the smallest distance from the target coordinate point;

[0029] The acquisition unit is further used to acquire a classification threshold of the classification model according to the first coordinate point.

[0030] In a possible design, in an implementation of the second aspect of the embodiments of the present application,

[0031] The acquisition unit is also used to obtain the sensitivity corresponding to the coordinate point on the ROC curve;

[0032] The acquisition unit is further used to acquire the classification threshold of the classification model according to the sensitivity corresponding to the coordinate point on the ROC curve.

[0033] In a possible design, in an implementation of the second aspect of the embodiments of the present application,

[0034] The determination unit is further used to determine a third probability according to a plurality of second probabilities corresponding to a plurality of classification categories in the classification model, wherein the third probability is a probability with the largest probability value among the plurality of second probabilities;

[0035] The determination unit is further used to determine the information classification result according to the classification category to which the third probability belongs.

[0036] In a possible design, in an implementation of the second aspect of the embodiments of the present application,

[0037] The acquisition unit is further used to acquire the medical image to be classified, the medical image including the image with the area where the lesion is marked;

[0038] The generating unit is further used to generate a lesion classification result of the medical image according to the second probability.

[0039] A third aspect of an embodiment of the present application provides a server, including: a memory, a transceiver, a processor, and a bus system; wherein the memory is used to store a program; and the processor is used to execute the program in the memory, including the following steps:

[0040] Get the information to be classified;

[0041] Obtaining a first probability corresponding to the information to be classified through a classification model;

[0042] The first probability is modified based on the information classification rule to obtain a second probability, wherein the information classification rule represents a corresponding relationship between an original probability set and a modified probability set, the original probability set includes M original probabilities, the modified probability set includes M modified probabilities, M is an integer greater than 1, the original probability and the modified probability have a one-to-one corresponding relationship, and the modified probability in the modified probability set and the original probability in the original probability set satisfy the same distribution order;

[0043] generating an information classification result according to the second probability;

[0044] The bus system is used to connect the memory and the processor so that the memory and the processor can communicate with each other.

[0045] A fourth aspect of the embodiments of the present application provides a computer-readable storage medium, in which instructions are stored. When the computer-readable storage medium is run on a computer, the computer executes any of the methods described above.

[0046] A fifth aspect of the embodiments of the present application provides a computer program product comprising instructions, which, when executed on a computer or a processor, enables the computer or the processor to execute any of the methods described above.

[0047] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0048] The embodiments of the present application provide a method for information classification and related devices and servers. After obtaining the predicted probability output by the classification model, the predicted probability corresponding to each category is re-corrected according to the information classification rules, so that the predicted probability corresponding to each category can be reconstructed and aligned according to its distribution in the original probability corresponding to the category to which it belongs, so that the distribution of each category is as similar as possible, and then classification is performed, which can effectively improve the accuracy of category prediction with less sample data. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 A schematic diagram of the architecture of the information classification system in the embodiment of the present application;

[0050] Figure 2 An example diagram of a method for information classification provided in an embodiment of the present application;

[0051] Figure 3 An example diagram of a ROC curve provided in an embodiment of the present application;

[0052] Figure 4 An example diagram of obtaining a first coordinate point on a ROC curve provided in an embodiment of the present application;

[0053] Figure 5 An example diagram of detection of a lesion area provided in an embodiment of the present application;

[0054] Figure 6 An example diagram of an apparatus for information classification provided in an embodiment of the present application;

[0055] Figure 7 A schematic diagram of a server structure provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] The embodiments of the present application provide a method for information classification and related devices and servers for improving the accuracy of category prediction with less sample data in a classification model.

[0057] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "corresponding to" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0058] Artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0059] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0060] Among them, machine learning is a multi-disciplinary cross-disciplinary subject, involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or realize human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all fields of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and teaching learning. At present, there are many classification methods for machine learning based on different emphasis, including classification based on learning strategies, classification based on learning methods, classification based on learning methods, classification based on data forms, and classification based on learning goals. Among them, the classification of machine learning based on learning strategies mainly includes statistical machine learning. Statistical machine learning is based on the preliminary understanding of data and the analysis of learning purposes, selecting appropriate mathematical models, formulating hyperparameters, and inputting sample data. According to certain strategies, the model is trained using appropriate learning algorithms, and finally the trained model is used to analyze and predict the data.

[0061] The information classification method provided in the embodiments of the present application involves technologies such as statistical machine learning of artificial intelligence, which will be specifically described through the following embodiments.

[0062] It should be understood that the embodiments of the present application are applied to scenarios where information classification is performed based on a classification model based on machine learning. Specifically, it can be applied to scenarios where text information such as news texts and comment texts are classified, or to scenarios where images are classified. For example, some medical images of brain hemorrhage are classified through a classification model to obtain the category of brain hemorrhage causes to which the medical image belongs, which can effectively assist doctors in making a diagnosis. Specifically, in this embodiment, the classification model based on machine learning can be a binary classification model or a multi-classification model, which is not specifically limited here.

[0063] It is understandable that before using a classification model to classify information, it is usually necessary to use a large amount of sample data to train the classification model, and finally use the trained classification model to classify the information. However, due to the difficulty in obtaining some sample data, for example, the lesion images corresponding to some rare diseases in medicine are difficult to obtain, which will result in more samples of some categories and fewer samples of some categories in the sample data used to train the classification model, that is, the sample data is unbalanced. At present, most of the classification models based on machine learning, after being trained with unbalanced sample data, will shift their prediction results to the categories with more sample data in the training data when predicting and classifying information, resulting in a very low prediction accuracy for the categories with less sample data. For example, when the medical image of brain hemorrhage is classified by the cause of bleeding through the classification model, it is assumed that there are six causes of bleeding (i.e., there are six classification categories in total), among which the first to third causes of bleeding are relatively common causes of bleeding. Therefore, there are more relevant medical images that can be provided clinically, that is, there are more medical images that are used to train the classification model and belong to the first to third causes of bleeding. In addition, the fourth to sixth causes of bleeding are relatively rare causes of bleeding. Therefore, there are fewer relevant medical images that can be provided clinically, that is, there are fewer medical images that are used to train the classification model and belong to the fourth to sixth causes of bleeding. In this way, the classification results predicted by the classification model will be biased towards the first to third causes of bleeding, that is, the prediction accuracy of the fourth to sixth causes of bleeding is low. In fact, since the first to third causes of bleeding are relatively common causes of bleeding, clinicians are often easier to distinguish, while the fourth to sixth causes of bleeding are relatively rare causes of bleeding, clinicians are often not easy to distinguish, and doctors usually pay more attention to the fourth to sixth causes of bleeding and hope to get better prediction results.

[0064] In view of this, an embodiment of the present application provides a method for information classification and related devices and servers. After obtaining the predicted probability output by the classification model, the predicted probability corresponding to each category is re-corrected according to the information classification rules, so that the predicted probability corresponding to each category can be reconstructed and aligned according to its distribution in the original probability corresponding to the category to which it belongs, so that the distribution of each category is as similar as possible, and then classification is performed, which can effectively improve the accuracy of category prediction with less sample data.

[0065] For ease of understanding, the following will introduce the system architecture used by the method provided in the embodiment of the present application. The present application proposes a method for information classification, which is applied to Figure 1 See the information classification system shown in Figure 1 , Figure 1 This is a schematic diagram of the architecture of the information classification system in the embodiment of the present application. As shown in the figure, the terminal establishes a communication connection with the server through the network, and a classification model based on machine learning is deployed on the server, which is used to classify information; when the user needs to classify text or images, the text or image to be classified can be sent to the server through the terminal, and the server classifies the text or image to be classified through the classification model, and corrects the probability of the original output of the classification model based on the information classification rule to obtain the corrected probability, and finally generates the corresponding information classification result based on the corrected probability. For example, after a clinician obtains a medical image through an imaging device (such as a computed tomography (CT) device), the medical image can be sent to the server through a terminal that has a communication connection with the server, and the server obtains the final image classification result based on the classification model and the information classification rule, and finally obtains the image classification result returned by the server.

[0066] It is worth noting that in this embodiment, a classification model based on machine learning can also be deployed on the terminal. When the terminal obtains information such as text or images that need to be classified, the classification model deployed on the terminal can be used to classify the text or image, and the probability of the original output of the classification model is corrected based on the information classification rule to obtain the corrected probability, and finally the corresponding information classification result is generated based on the corrected probability. For the sake of ease of description, the following will take the execution of information classification on the server as an example to introduce in detail the information classification method provided in the embodiment of the present application.

[0067] in, Figure 1 The terminal shown in the figure is only a simple illustration. In actual applications, the terminal includes but is not limited to terminal devices such as mobile phones, tablet computers, desktop computers, laptop computers, and handheld computers.

[0068] See also Figure 2 , Figure 2 An example diagram of a method for information classification provided in an embodiment of the present application. An example of a method for information classification provided in an embodiment of the present application includes:

[0069] 201. Obtain information to be classified;

[0070] In this embodiment, the server can obtain the information to be classified sent by the terminal, and the information to be classified may include but is not limited to text information or image information. For the sake of convenience, the following will take image information as an example for introduction. When the information classification method provided in the embodiment of the present application is used to classify medical images, the server can obtain the medical image to be classified, which includes an image with an area where the lesion is located, for example, the medical image may include an image with cerebral hemorrhage.

[0071] 202. Obtaining a first probability corresponding to the information to be classified through a classification model;

[0072] In this embodiment, the classification model refers to that each element in the data set is assigned to a known data category, that is, for each element in a data set, it can be assigned to a pre-set data category to achieve the classification of the elements. For example, according to the doctor's medical record, the examiner corresponding to the medical record is divided into healthy or sick. The examiner referred to here is the element in the data set, and healthy or sick can be understood as a pre-set data category; for another example, according to the medical image obtained by imaging equipment taking the patient's lesion site (such as the brain), the cause of the patient's lesion can be divided into cerebral hemorrhage caused by aneurysm rupture, hypertensive cerebral hemorrhage, or cerebral hemorrhage caused by arteriovenous malformation, etc. The medical image referred to here can be understood as an element in the data set, and the above three causes of cerebral hemorrhage can be understood as data categories.

[0073] Generally speaking, building a classification model requires the following three steps: 1) data preprocessing; 2) model training; 3) model performance evaluation. Data preprocessing refers to obtaining corresponding training data and preprocessing these training data according to the information to be classified by the classification model. For example, if the classification model wants to classify the causes of cerebral hemorrhage in medical images of cerebral hemorrhage, it is necessary to obtain a large number of medical images of cerebral hemorrhage as training data, and annotate these training data with specific causes of cerebral hemorrhage. Finally, these annotated training data are cleaned to obtain valid training data. Model training refers to inputting these valid training data into a pre-built classification model, training the classification model with these training data, so that the parameters in the classification model can be optimized, and finally the classification model can obtain the ability to predict the category to which new image data belongs by learning the classification results in the training data. Model performance evaluation refers to evaluating the predictive ability of the model based on test data similar to the training data. If the model's predictive ability is better, it means that the model's performance is better, and it can be determined that the model can be used for classification prediction. If the model's predictive ability is poor, it means that the model's performance is poor, and further optimization of the model's parameters is required.

[0074] Generally speaking, after obtaining the classification model, by inputting the medical image to be classified into the classification model, the classification model can automatically classify the medical image according to the input medical image and output the category to which the medical image belongs. In fact, for most classification models, when predicting the category to which the medical image belongs, the classification model first predicts the probability corresponding to each category of the medical image, and then determines the category to which the medical image belongs by comparing the probabilities corresponding to each category, that is, the category with the largest probability value is the category to which the medical image belongs. Therefore, in this embodiment, only the probability (i.e., the first probability) predicted by the classification model for the medical image can be obtained, wherein, for a classification model, the number of first probabilities predicted by the classification model is the same as the number of categories that can be classified in the classification model, that is, the classification model can predict multiple first probabilities, and each first probability corresponds to a category. For example, for a medical image of brain hemorrhage, the corresponding categories of the medical image are: brain hemorrhage caused by aneurysm rupture, hypertensive brain hemorrhage, brain hemorrhage caused by arteriovenous malformation, moyamoya disease, brain hemorrhage caused by cavernous hemangioma and other causes, a total of six categories. The classification model predicts six probability values ​​0.3, 0.2, 0.15, 0.15, 0.1 and 0.1, which correspond to the above six categories respectively.

[0075] It is worth noting that the classification model in the embodiments of the present application can be a binary classification model or a multi-classification model, wherein a binary classification model refers to a model with two categories in the classification task, for example, a binary classification model can be used to classify a certain examinee as sick or not sick; a multi-classification model refers to a model with multiple categories in the classification task, for example, a multi-classification model can be used to classify medical images into six categories, including cerebral hemorrhage caused by aneurysm rupture, hypertensive cerebral hemorrhage, cerebral hemorrhage caused by arteriovenous malformations, moyamoya disease, cerebral hemorrhage caused by cavernous hemangioma and other causes.

[0076] 203. Modify the first probability based on the information classification rule to obtain a second probability, wherein the information classification rule represents a correspondence between an original probability set and a modified probability set, the original probability set includes M original probabilities, the modified probability set includes M modified probabilities, M is an integer greater than 1, the original probability and the modified probability have a one-to-one correspondence, and the modified probability in the modified probability set and the original probability in the original probability set satisfy the same distribution order;

[0077] In this embodiment, after obtaining the first probability predicted by the classification model, the first probability corresponding to each category can be modified according to the information classification rule. For ease of understanding, how to obtain the information classification rule will be described in detail below.

[0078] First, in the process of training the classification model, the data used for training is divided into a training set and a validation set, where the data in the training set is used to train the classification model normally, and the data in the validation set is used to validate the trained classification model. Specifically, in the validation process, assuming that the classification model has S categories and there are M data in the validation set, then for each data, S first probabilities predicted by the classification model can be obtained; for the predicted probability p of the i-th category i , then there are M corresponding predicted probability values ​​[p i1 、p i2 …、p iM ]. First, for each category, a target probability threshold can be determined. For a piece of data, if the probability value corresponding to the data in a certain category is greater than the target probability threshold corresponding to the category, then the data can be considered to belong to the category. Therefore, for S categories, S target probability thresholds can be determined, which are [t1, t2, …, t S ]. At this time, the first probability corresponding to each data in the validation set can be reconstructed and corrected. Assuming that the reconstructed and corrected first probability is F(p), then F(t i ) is 0.5, that is, the target probability threshold corresponding to each category is corrected to 0.5.

[0079] Assume that, in the validation set, the original probability of m1 data is less than t i , and the predicted probability of m2=N-m1-1 data is greater than t i , then the corresponding relationship between the original probability and the modified probability of M data in the i-th category is shown in Table 1:

[0080] Table 1

[0081]

[0082] Among them, the first row in Table 1 represents the original probabilities corresponding to M These M original probabilities are arranged from small to large, that is, p i1 is the original probability with the minimum probability value, and p i,M is the original probability with the largest probability value; the second row in Table 1 shows the modified probabilities corresponding to the M original probabilities. These M modified probabilities are also arranged from small to large, and p il to The corresponding correction probability is evenly distributed in the interval [0,0.5), and to p i,M The corresponding correction probabilities are evenly distributed in the interval (0.5,1].

[0083] As can be seen from Table 1, for the revised probability corresponding to any category, the revised probabilities corresponding to the M data are distributed in the interval [0,1], and the distribution of these revised probabilities is determined according to the distribution position of the original probability relative to the target threshold probability, that is, the modified probabilities in the revised probability set and the original probabilities in the original probability set meet the same distribution order, and are sorted from small to large. Obviously, after the original probability of the data in the validation set is revised, even if the original probability corresponding to a certain data in a category with less sample data is lower than the original probability corresponding to the data in another category with more sample data, as long as the original probability of the data in the category with less sample data is ranked relatively high, then the revised probability corresponding to the data in the category with less sample data may also be greater than the revised probability corresponding to the data in the category with more sample data. For example, assuming that for data A, the original probability corresponding to data A in category 1 is 0.5, the original probability corresponding to data A in category 2 is 0.3, and the original probability corresponding to data A in category 3 is 0.2. If the classification result of data A is determined according to the original probability, data A is classified into category 1; assuming that the target threshold probability corresponding to data A in category 1 is 0.6, the target threshold probability corresponding to data A in category 2 is 0.25, and the target threshold probability corresponding to data A in category 3 is 0.1; after correcting the original probability of data A, the corrected probability corresponding to data A in category 1 is 0.4, the corrected probability corresponding to data A in category 2 is 0.45, and the corrected probability corresponding to data A in category 3 is 0.6. If the classification result of data A is determined according to the corrected probability, data A is classified into category 3. In other words, even if the original probability of data A in a certain category is less than the original probability of another category, after the probability correction, the corrected probability of data A in this category can also be greater than that of another category.

[0084] The above is an introduction to the correspondence between the original probability set and the modified probability set. The following will introduce how to modify the first probability based on the correspondence between the original probability set and the modified probability set to obtain the second probability.

[0085] Since the revised probability is determined based on the distribution position of the corresponding original probability relative to the target threshold probability, the second probability corresponding to the first probability can also be obtained based on the same principle. It can be understood that if the first probability is the same as any of the original probabilities in Table 1 above, the second probability corresponding to the first probability can be obtained based on the revised probability corresponding to the original probability, that is, the second probability is the same as the revised probability. If the first probability is different from all the original probabilities in Table 1 above, the second probability corresponding to the first probability can be obtained by linear interpolation.

[0086] Among them, linear interpolation refers to the interpolation method in which the interpolation function is a first-order polynomial, and the interpolation error at the interpolation node is zero. The geometric meaning of linear interpolation is to use a straight line passing through two coordinate points to approximate the original function between the two coordinate points. In addition, linear interpolation can be used to approximate the original function, and can also be used to calculate the value that is not in the table during the table lookup process. For example, suppose x is located at the coordinate point (x o , y0) and (x1, y1), then for y = F(x), we have

[0087] In this embodiment, there are only M corresponding original probabilities [p i1 、p i2 …、p iM ], and the newly obtained first probability may be different from all the original probabilities in the above Table 1, that is, the original probability corresponding to the first probability cannot be found in Table 1. In this case, linear interpolation can be used to obtain the original probability corresponding to the first probability.

[0088] Specifically, first find the interval that the first probability actually falls into among the M original probabilities. Assume that the first probability falls into the interval [p i,k ,p i,k+1 ], that is, the first probability p i,k <p i ≤p i,k+1 , where p i,k The corresponding modified probability is F(p i,k ), p i,k+1 The corresponding modification probability is F(p i,k+1 ), that is, the first probability p i Located at the coordinate point (p i,k , F(p i,k )) and (p i,k+1 , F(p i,k+1 )), so for the first probability p i The corresponding modified probability (i.e., the second probability) F(p i ), can be obtained by the following formula:

[0089]

[0090] Obviously, it can be seen from the above formula that for any first probability, the interval it falls into can be obtained, and then the second probability corresponding to the first probability can be obtained based on the coordinate points corresponding to the interval.

[0091] In addition, since the above is a reconstruction and correction of the probability of each category, for multiple first probabilities of a medical image to be classified, the second probability corresponding to these first probabilities after correction may not be equal to 1, that is, F(p1)+F(p2)+…+F(p N ) may not be equal to 1. Therefore, the multiple second probabilities may be normalized, specifically, may be normalized by the following formula:

[0092]

[0093] in, is the normalized second probability.

[0094] It is understandable that for some classification models, the classification model may be a multi-model fusion model. In this case, the second probability of each single model can be The average is taken to obtain the final classification probability.

[0095] 204. Generate an information classification result according to the second probability.

[0096] In this embodiment, after obtaining multiple second probabilities corresponding to the medical image to be classified, a third probability can be determined based on the multiple second probabilities, wherein the third probability is a probability with the largest probability value among the multiple second probabilities, and then it is determined to which category the third probability specifically belongs, that is, the medical image can be classified into the category corresponding to the third probability. Specifically, when the information to be classified is a medical image, a lesion classification result of the medical image can be generated based on the second probability. For example, for a medical image of cerebral hemorrhage, one of the following lesion classification results can be obtained: cerebral hemorrhage caused by aneurysm rupture, hypertensive cerebral hemorrhage, cerebral hemorrhage caused by arteriovenous malformation, moyamoya disease, cerebral hemorrhage caused by cavernous hemangioma, or other causes.

[0097] In this embodiment, after obtaining the predicted probability output by the classification model, the predicted probability corresponding to each category is re-corrected according to the information classification rules, so that the predicted probability corresponding to each category can be reconstructed and aligned according to its distribution in the original probability corresponding to the category to which it belongs, so that the distribution of each category is as similar as possible, and then classification is performed, which can effectively improve the accuracy of category prediction with less sample data.

[0098] Optionally, in the above Figure 2On the basis of a corresponding embodiment, in an optional embodiment of the information classification method provided in an embodiment of the present application, before correcting the first probability based on the information classification rule, the method also includes: obtaining a classification threshold of the classification model, the classification threshold belongs to the original probability set; determining the target probability threshold according to the classification threshold; generating an information classification rule according to the target probability threshold, wherein the corrected probability set includes the target probability threshold, the target probability threshold belongs to the Nth corrected probability among the M corrected probabilities, the first corrected probability to the N-1th corrected probability are all less than the target probability threshold, the N+1th corrected probability to the Mth corrected probability are all greater than the target probability threshold, and N is an integer greater than 1 and less than M.

[0099] In this embodiment, before determining the target probability threshold, the classification threshold of the classification model can be first found in the original probability set. The classification threshold refers to a threshold that can achieve the optimal classification effect for a certain category. For example, for a category, if it is judged that its predicted probability is greater than 0.5, it belongs to the category, and less than 0.5, it does not belong to the category, and when the threshold is judged to be 0.5, the classification result with the highest accuracy can be obtained, then 0.5 can be regarded as the classification threshold corresponding to the category in the classification model. After obtaining the classification threshold, the target probability threshold can be determined according to the classification threshold. Specifically, the target probability threshold can be the same as the classification threshold.

[0100] Optionally, in the above Figure 2 On the basis of a corresponding embodiment, in an optional embodiment of the information classification method provided in an embodiment of the present application, before obtaining the classification threshold of the classification model, the method also includes: obtaining a receiver operating characteristic (ROC) curve of the classification model; obtaining the classification threshold of the classification model, including: obtaining the classification threshold of the classification model according to the ROC curve.

[0101] In this embodiment, the classification threshold of the classification model can be obtained specifically through the ROC curve. Among them, the ROC curve is a curve obtained by using the probability of each test data as the threshold to obtain the false positive rate (False PositiceRate, FPR) and true positive rate (True Positive Rate, TPR) of the test set as the coordinates of the point, and then connecting such coordinates in sequence. That is to say, in the ROC curve, the horizontal coordinate of each point is FPR and the vertical coordinate is TPR. For ease of understanding, the ROC curve will be introduced in detail below.

[0102] Generally speaking, binary classification is a very common problem in machine learning and is often used. The ROC curve is a tool specifically used to evaluate the pros and cons of a binary classification model. For a binary classification problem, instances can usually be divided into positive and negative classes. For example, in the problem of computer-aided identification of lung nodules, the presence of lung nodules in a lung image is considered positive, and the absence of lung nodules is considered negative. Therefore, in actual detection, there will be the following four situations:

[0103] (1) True Positive (TP): the number of nodules detected and actually present; the number of correct positive matches;

[0104] (2) False Positive (FP): a nodule is detected but there is no nodule in reality; false positive, the match given is incorrect;

[0105] (3) True Negative (TN): detected no nodules, and there are no nodules in fact; the number of non-matches correctly rejected;

[0106] (4) False Negative (FN): No nodules are detected, but nodules actually exist; missed reports, the number of matches that are not found correctly.

[0107] Based on the above four parameters, we can usually get several main parameter indicators for evaluating the binary classification model, as follows:

[0108]

[0109]

[0110] Among them, TPR can also be called sensitivity or recall, and FPR can also be called misdiagnosis rate. From the above formula, we can know that the horizontal axis of the ROC curve is actually the proportion of samples predicted to be positive but actually negative to all negative samples; the vertical axis of the ROC curve is actually the proportion of samples predicted to be positive and actually positive to all positive samples. In a two-classification model, assuming that a logistic regression classifier is used, which gives the probability of each instance being a positive class, then by setting a threshold such as 0.6, the probability greater than or equal to 0.6 is a positive class, and less than 0.6 is a negative class. Correspondingly, a set of (FPR, TPR) can be calculated, and the corresponding coordinate points can be obtained in the plane. As the threshold gradually decreases, more and more instances are classified as positive classes, but these positive classes are also mixed with real negative instances, that is, TPR and FPR will increase at the same time. When the threshold is maximum, the corresponding coordinate point is (0,0), and when the threshold is minimum, the corresponding coordinate point is (1,1). Obviously, ideally, TPR should be close to 1 and FPR should be close to 0. As Figure 3 As shown, Figure 3 An example diagram of a ROC curve provided in an embodiment of the present application. Figure 3 In the figure, there are three ROC curves, and different ROC curves represent different classification models. Generally speaking, the ROC curve on the top is better than the ROC curve on the bottom, because under the same false positive rate, the true positive rate of the ROC curve on the top is higher. Based on this, the area under the curve (AUC) can usually be used as a criterion for evaluating the model. The larger the AUC (indicating that the ROC curve is on the top), the better the model. However, although the ROC curve and AUC are a common way to evaluate models, they are usually limited to two-category classification, so the scope of application is relatively narrow, and the ROC curve itself cannot provide a suitable threshold. It can only be used to compare models, and the threshold cannot be directly calculated to give specific classification results.

[0111] Based on this, in the embodiment of the present application, when dealing with multiple classifications, a one-to-many strategy is used to obtain the corresponding ROC curve for each category. Specifically, for the i-th category, it can be considered that the corresponding binary classification probability is [p i , 1-p i ], based on this binary classification probability, the ROC curve corresponding to the i-th category can be obtained. Obviously, for each category in the multi-classification model, the corresponding ROC curve can be obtained by the above method, that is, in a multi-classification model, the number of ROC curves is the same as the number of categories included in the multi-classification model.

[0112] After obtaining the ROC curve, the optimal threshold for binary classification corresponding to each category (i.e., the above-mentioned classification threshold) can be obtained according to the ROC curve. The specific acquisition methods include but are not limited to the following:

[0113] A. Obtain the model accuracy value of the coordinate point on the ROC curve. The model accuracy value is determined by the precision and recall rate corresponding to the coordinate point; obtain the classification threshold of the classification model according to the model accuracy value.

[0114] Specifically, the model accuracy value of the coordinate point on the ROC curve can be the F1 score (F1 Score). The F1 score is an indicator used in statistics to measure the accuracy of a binary classification model. It takes into account both the precision and recall of the classification model. The F1 score can be regarded as a harmonic mean of the model's precision and recall. Its maximum value is 1 and its minimum value is 0. Among them, the F1 score is specifically Therefore, for each category of the ROC curve, we can find a coordinate point with the largest F1 score on the ROC curve, and then get the probability p corresponding to the coordinate point. i , and the probability p of the coordinate point i As the classification threshold corresponding to the category, in this way, by obtaining the F1 score on the ROC curve corresponding to each category and finding the coordinate point with the largest F1 score on each ROC curve, multiple classification thresholds of the classification model can be obtained.

[0115] B. Obtain the distance between the coordinate point on the ROC curve and the target coordinate point, where the horizontal coordinate of the target coordinate point is 0 and the vertical coordinate of the target coordinate point is 1; determine the first coordinate point on the ROC curve, where the first coordinate point is the coordinate point on the ROC curve with the smallest distance from the target coordinate point; obtain the classification threshold of the classification model according to the first coordinate point.

[0116] Specifically, the target coordinate point can be the coordinate point (0,1). For each coordinate point on each ROC curve, the distance d between these coordinate points and the target coordinate point can be calculated. Obviously, for each ROC curve corresponding to a category, a coordinate point with the smallest d on the ROC curve (i.e., the first coordinate point) can be obtained. Therefore, the probability p of the coordinate point can be calculated. i As the classification threshold corresponding to the category, by obtaining the first coordinate point on the ROC curve corresponding to each category, multiple classification thresholds of the classification model can be obtained. Figure 4 As shown, Figure 4 This is an example diagram of obtaining the first coordinate point on the ROC curve provided in an embodiment of the present application. Figure 4 As shown, the corresponding first coordinate point can be found for each ROC curve.

[0117] C. Obtain the sensitivity corresponding to the coordinate point on the ROC curve; obtain the classification threshold of the classification model according to the sensitivity corresponding to the coordinate point on the ROC curve.

[0118] It is understandable that due to the particularity of some classification problems, such as medical classification problems, doctors may hope that the classification model can have a higher sensitivity to certain categories (for example, sensitivity>k). Then, in this case, the coordinate point closest to the sensitivity k can be found on the ROC curve corresponding to the category (where the ordinate of the ROC curve is sensitivity, so the corresponding coordinate point can be found based on the ordinate of the coordinate point on the ROC curve), and finally the probability p of the coordinate point is i As the classification threshold corresponding to this category.

[0119] It is worth noting that when obtaining the classification threshold corresponding to each category in the classification model, it is necessary to obtain it under the same standard. That is to say, the classification threshold corresponding to all categories can be obtained by the above-mentioned acquisition method A, or the acquisition method B or acquisition method C can be used to obtain the classification threshold corresponding to all categories, that is, different acquisition methods are not used to obtain the classification thresholds corresponding to different categories in the classification model.

[0120] For ease of understanding, the information classification method provided by the embodiment of the present application is described in detail below with reference to specific examples. Specifically, some CT plain scan brain medical images are selected as data sets, which contain 6 categories of data, namely, cerebral hemorrhage caused by aneurysm rupture, hypertensive cerebral hemorrhage, cerebral hemorrhage caused by arteriovenous malformation, moyamoya disease, cerebral hemorrhage caused by cavernous hemangioma and other causes.

[0121] First, the data set is normalized. For lesion areas without fixed shape, size and directionality, data augmentation operations such as flipping, rotating, scaling or contrast enhancement of medical images can be performed to increase the number of data sets and increase the information value in terms of directionality and different sizes.

[0122] Secondly, there are two ways to extract features and classify the dataset: using traditional feature methods to extract features from the dataset and using the extreme gradient boosting (XGBoost) model for classification; and using a deep learning model for classification.

[0123] Specifically, the classification method using traditional features is as follows:

[0124] For medical images, the lesion area is first detected or segmented. Specifically, the methods of detecting the lesion area include but are not limited to manual annotation by doctors, detection or segmentation using existing detection algorithms or segmentation algorithms, or unsupervised algorithms such as threshold segmentation to distinguish the lesion area, and then the lesion area in the medical image is extracted by extracting radiomics features. For details, please refer to Figure 5 , Figure 5 This is an example diagram of detecting a lesion area provided in an embodiment of the present application. Figure 5 In the figure, (a) represents a medical image, and (b) represents a medical image of the lesion area (i.e., the cerebral hemorrhage area) detected by the above method. After the lesion area is obtained by manual labeling or algorithm labeling, the imaging genomics features are extracted from the lesion area to obtain the features used for subsequent classification.

[0125] Then, the whole data set is divided into five parts, three of which are used as training sets, one as validation set, and one as test set. First, XGBoost classification is used to obtain the classification probability results of the validation set and the ROC curve of each class (it is worth noting that the classifier here can use a variety of traditional classifiers, including but not limited to XGBoost, AdaBoost or decision tree, etc.), and then the F1 score is used as a threshold selector to select the classification threshold and construct the probability reconstruction function F;

[0126] The probability results of the final test set are reconstructed and the final classification results are obtained using the argmax function.

[0127] Specifically, the method of classification using the deep learning model is as follows:

[0128] First, the overall three-dimensional image is selected as input data;

[0129] Then the overall data is divided into five parts, three of which are used as training sets, one as validation set, and one as test set. First, the residual neural network (ResNet) is used to classify the validation set classification probability results and the ROC curve of each class. Then, the F1 score is used as the threshold selector to select the classification threshold and construct the probability reconstruction function F.

[0130] The probability results of the final test set are reconstructed and the final classification results are obtained using the argmax function.

[0131] Specifically, the experimental results are shown in Table 2:

[0132] Table 2

[0133]

[0134]

[0135] Among them, there are more data in categories 1 and 2, which are easier for clinicians to distinguish, and doctors pay more attention to categories 3 to 6. It can be seen that the classification accuracy of categories 3 to 6 has been significantly improved, which can have a significant impact on product decisions, and the overall accuracy has not decreased.

[0136] The above is an introduction to the information classification method provided in the embodiment of the present application. The following is an introduction to the information classification device provided in the embodiment of the present application. Figure 6 As shown, Figure 6 An example diagram of an apparatus for information classification provided in an embodiment of the present application.

[0137] An information classification device 60 provided in an embodiment of the present application includes:

[0138] An acquisition unit 601 is used to acquire information to be classified;

[0139] The acquisition unit 601 is further used to acquire a first probability corresponding to the information to be classified through a classification model;

[0140] A correction unit 602 is used to correct the first probability based on the information classification rule to obtain a second probability, wherein the information classification rule represents a correspondence between an original probability set and a modified probability set, the original probability set includes M original probabilities, the modified probability set includes M modified probabilities, M is an integer greater than 1, the original probability and the modified probability have a one-to-one correspondence, and the modified probability in the modified probability set and the original probability in the original probability set satisfy the same distribution order;

[0141] The generating unit 603 is configured to generate an information classification result according to the second probability.

[0142] Optionally, in the above Figure 6 On the basis of the corresponding embodiment, in another embodiment of the information classification device 60 provided in the embodiment of the present application, it further includes a determination unit 604;

[0143] The acquisition unit 601 is further used to acquire a classification threshold of the classification model, where the classification threshold belongs to the original probability set;

[0144] A determination unit 604 is used to determine a target probability threshold according to the classification threshold;

[0145] The generation unit 603 is also used to generate information classification rules according to the target probability threshold, wherein the modified probability set includes the target probability threshold, the target probability threshold belongs to the Nth modified probability among the M modified probabilities, the first modified probability to the N-1th modified probability are all less than the target probability threshold, the N+1th modified probability to the Mth modified probability are all greater than the target probability threshold, and N is an integer greater than 1 and less than M.

[0146] Optionally, in the above Figure 6 Based on the corresponding embodiment, in another embodiment of the information classification device 60 provided in the embodiment of the present application,

[0147] The acquisition unit 601 is further used to acquire a receiver operating characteristic (ROC) curve of the classification model;

[0148] The acquisition unit 601 is further used to acquire the classification threshold of the classification model according to the ROC curve.

[0149] Optionally, in the above Figure 6 Based on the corresponding embodiment, in another embodiment of the information classification device 60 provided in the embodiment of the present application,

[0150] The acquisition unit 601 is further used to acquire the model accuracy value of the coordinate point on the ROC curve, where the model accuracy value is determined by the precision and recall rate corresponding to the coordinate point;

[0151] The acquisition unit 601 is further used to acquire the classification threshold of the classification model according to the model accuracy value.

[0152] Optionally, in the above Figure 6 Based on the corresponding embodiment, in another embodiment of the information classification device 60 provided in the embodiment of the present application,

[0153] The acquisition unit 601 is further used to acquire the distance between the coordinate point on the ROC curve and the target coordinate point, wherein the abscissa of the target coordinate point is 0 and the ordinate of the target coordinate point is 1;

[0154] The determination unit 604 is further used to determine a first coordinate point on the ROC curve, where the first coordinate point is a coordinate point on the ROC curve with the smallest distance from the target coordinate point;

[0155] The acquisition unit 601 is further configured to acquire a classification threshold of the classification model according to the first coordinate point.

[0156] Optionally, in the above Figure 6 Based on the corresponding embodiment, in another embodiment of the information classification device 60 provided in the embodiment of the present application,

[0157] The acquisition unit 601 is also used to acquire the sensitivity corresponding to the coordinate point on the ROC curve;

[0158] The acquisition unit 601 is further used to acquire the classification threshold of the classification model according to the sensitivity corresponding to the coordinate point on the ROC curve.

[0159] Optionally, in the above Figure 6 Based on the corresponding embodiment, in another embodiment of the information classification device 60 provided in the embodiment of the present application,

[0160] The determining unit 604 is further configured to determine a third probability according to the plurality of second probabilities corresponding to the plurality of classification categories in the classification model, wherein the third probability is a probability with the largest probability value among the plurality of second probabilities;

[0161] The determining unit 604 is further configured to determine an information classification result according to the classification category to which the third probability belongs.

[0162] Optionally, in the above Figure 6 Based on the corresponding embodiment, in another embodiment of the information classification device 60 provided in the embodiment of the present application,

[0163] The acquisition unit 601 is further used to acquire a medical image to be classified, wherein the medical image includes an image with an area where a lesion is marked;

[0164] The generating unit 603 is further configured to generate a lesion classification result of the medical image according to the second probability.

[0165] Figure 7 A schematic diagram of a server structure provided for an embodiment of the present application, the server 700 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 722 (for example, one or more processors) and a memory 732, and one or more storage media 730 (for example, one or more mass storage devices) storing application programs 742 or data 744. Among them, the memory 732 and the storage medium 730 can be short-term storage or permanent storage. The program stored in the storage medium 730 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the server. Furthermore, the central processing unit 722 can be configured to communicate with the storage medium 730 to execute a series of instruction operations in the storage medium 730 on the server 700.

[0166] The server 700 may also include one or more power supplies 726, one or more wired or wireless network interfaces 750, one or more input and output interfaces 758, and / or one or more operating systems 741, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0167] In the embodiment of the present application, the CPU 722 included in the server 700 also has the following functions:

[0168] Get the information to be classified;

[0169] Obtaining a first probability corresponding to the information to be classified through a classification model;

[0170] The first probability is modified based on the information classification rule to obtain a second probability, wherein the information classification rule represents a corresponding relationship between an original probability set and a modified probability set, the original probability set includes M original probabilities, the modified probability set includes M modified probabilities, M is an integer greater than 1, the original probability and the modified probability have a one-to-one corresponding relationship, and the modified probability in the modified probability set and the original probability in the original probability set satisfy the same distribution order;

[0171] An information classification result is generated according to the second probability.

[0172] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0173] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0174] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0175] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0176] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0177] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for information classification, characterized in that: Applied to classify text information or images, including: Get the information to be classified; Acquire a first probability corresponding to the information to be classified through a classification model; Obtaining a classification threshold of the classification model, wherein the classification threshold belongs to an original probability set; Determining a target probability threshold according to the classification threshold, wherein the target probability threshold is the same as the classification threshold; generating information classification rules according to the target probability threshold; The first probability is modified based on the information classification rule to obtain a second probability, wherein the information classification rule represents the corresponding relationship between the original probability set and the modified probability set, the original probability set includes M original probabilities, the modified probability set includes M modified probabilities, M is an integer greater than 1, the original probability and the modified probability have a one-to-one corresponding relationship, and the modified probabilities in the modified probability set and the original probabilities in the original probability set satisfy the same distribution order, the modified probability set includes the target probability threshold, the target probability threshold belongs to the Nth modified probability among the M modified probabilities, the first modified probability to the N-1th modified probability are uniformly distributed in the interval [0, target probability threshold), the N+1th modified probability to the Mth modified probability are uniformly distributed in the interval (target probability threshold, 1], and N is an integer greater than 1 and less than M; An information classification result is generated according to the second probability.

2. The information classification method according to claim 1, characterized in that: Before obtaining the classification threshold of the classification model, the method further includes: Obtaining a receiver operating characteristic (ROC) curve of the classification model; The obtaining of the classification threshold of the classification model includes: A classification threshold of the classification model is obtained according to the ROC curve.

3. The information classification method according to claim 2, characterized in that: The step of obtaining the classification threshold of the classification model according to the ROC curve includes: Obtaining a model accuracy value of a coordinate point on the ROC curve, wherein the model accuracy value is determined by the precision and recall corresponding to the coordinate point; A classification threshold of the classification model is obtained according to the model accuracy value.

4. The information classification method according to claim 2, characterized in that: The step of obtaining the classification threshold of the classification model according to the ROC curve includes: Obtaining the distance between the coordinate point on the ROC curve and the target coordinate point, wherein the abscissa of the target coordinate point is 0 and the ordinate of the target coordinate point is 1; Determine a first coordinate point on the ROC curve, where the first coordinate point is a coordinate point on the ROC curve with a minimum distance from the target coordinate point; A classification threshold of the classification model is acquired according to the first coordinate point.

5. The information classification method according to claim 2, characterized in that: The step of obtaining the classification threshold of the classification model according to the ROC curve includes: Obtaining the sensitivity corresponding to the coordinate point on the ROC curve; The classification threshold of the classification model is obtained according to the sensitivity corresponding to the coordinate point on the ROC curve.

6. The information classification method according to claim 1, characterized in that: Generating an information classification result according to the second probability includes: Determine a third probability according to a plurality of second probabilities corresponding to a plurality of classification categories in the classification model, wherein the third probability is a probability with the largest probability value among the plurality of second probabilities; An information classification result is determined according to the classification category to which the third probability belongs.

7. The information classification method according to claim 1, characterized in that: The obtaining of the information to be classified includes: Acquire a medical image to be classified, wherein the medical image includes an image with an area where a lesion is marked; Generating an information classification result according to the second probability includes: A lesion classification result of the medical image is generated according to the second probability.

8. An information classification device, characterized in that: Applied to classify text information or images, including: An acquisition unit, used for acquiring information to be classified; The acquisition unit is further used to acquire the first probability corresponding to the information to be classified through a classification model; The acquisition unit is further used to acquire a classification threshold of the classification model, where the classification threshold belongs to an original probability set; a determining unit, configured to determine a target probability threshold according to the classification threshold, wherein the target probability threshold is the same as the classification threshold; A generating unit, configured to generate an information classification rule according to the target probability threshold; a correction unit, configured to perform correction processing on the first probability based on the information classification rule to obtain a second probability, wherein the information classification rule represents a correspondence between the original probability set and the revised probability set, the original probability set includes M original probabilities, the revised probability set includes M revised probabilities, M is an integer greater than 1, the original probability and the revised probability have a one-to-one correspondence, and the revised probabilities in the revised probability set and the original probabilities in the original probability set satisfy the same distribution order, the revised probability set includes the target probability threshold, the target probability threshold belongs to the Nth revised probability among the M revised probabilities, the first revised probability to the N-1th revised probability are uniformly distributed in the interval [0, target probability threshold), the N+1th revised probability to the Mth revised probability are uniformly distributed in the interval (target probability threshold, 1], and N is an integer greater than 1 and less than M; A generating unit is used to generate an information classification result according to the second probability.

9. A server, characterized in that: Applied to classify text information or images, including: memory, transceiver, processor and bus system; Wherein, the memory is used to store programs; The processor is used to execute the program in the memory, comprising the following steps: Get the information to be classified; Acquire a first probability corresponding to the information to be classified through a classification model; Obtaining a classification threshold of the classification model, wherein the classification threshold belongs to an original probability set; Determining a target probability threshold according to the classification threshold, wherein the target probability threshold is the same as the classification threshold; generating information classification rules according to the target probability threshold; The first probability is modified based on the information classification rule to obtain a second probability, wherein the information classification rule represents the corresponding relationship between the original probability set and the modified probability set, the original probability set includes M original probabilities, the modified probability set includes M modified probabilities, M is an integer greater than 1, the original probability and the modified probability have a one-to-one corresponding relationship, and the modified probabilities in the modified probability set and the original probabilities in the original probability set satisfy the same distribution order, the modified probability set includes the target probability threshold, the target probability threshold belongs to the Nth modified probability among the M modified probabilities, the first modified probability to the N-1th modified probability are uniformly distributed in the interval [0, target probability threshold), the N+1th modified probability to the Mth modified probability are uniformly distributed in the interval (target probability threshold, 1], and N is an integer greater than 1 and less than M; generating an information classification result according to the second probability; The bus system is used to connect the memory and the processor so that the memory and the processor can communicate with each other.

10. A computer program product, characterized in that The computer program product comprises instructions, and when the computer program product is run on a computer, the computer is caused to execute the method according to any one of claims 1 to 7.

Images