Data classification method and device based on mutual information out-of-domain data detection
Through the extra-domain data detection method based on mutual information, the problem of poor extra-domain data processing in the prior art is solved, and a higher accuracy, security and reliability of the classification model are achieved.
Patent Information
- Application Number
- CN202411123525.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-15
- Publication Date
- 2025-03-04
AI Technical Summary
The prior art is difficult to effectively process extraterritorial data, resulting in misjudgment in the model in practical applications, resulting in safety and reliability problems.
The extraterrestrial data detection method based on mutual information is adopted. By entering the data to be classified into the classification model to obtain the predicted value, and inputting it into the mutual information model to obtain the classification mutual information value, computing the detection score in combination with the extraterrestrial data detection function, determining the validity of the predicted value to generate the classification result.
Effectively distinguish extraterritorial data samples, reduce the risks caused by model misjudgment, and improve the accuracy, security and reliability of classification models.
Smart Images
Figure CN119559418A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer information processing, and more specifically, to a data classification method, device, electronic device, computer-readable medium and computer program product based on mutual information out-of-domain data detection. Background Art
[0002] Out-of-domain data refers to input data that does not fall within the data distribution range used when the model was trained. In the field of intent recognition, this usually means that the input text is not included in any predefined intent category, or its characteristics are significantly different from those of the known categories in the training set. Since the model only learns how to recognize and process data within a specific range during training, when it encounters out-of-domain data, it may not be able to classify or recognize it correctly, resulting in inaccurate prediction results.
[0003] For example, in the field of autonomous driving, if a model is trained with a training data set in advance, assuming that the training data set contains K categories [a1, a2, …, aK], and the model training results are verified and can be put into use online. During the autonomous driving process, the vehicle's camera obtains pictures of natural scenes in real time and transmits them to the model for classification, thereby controlling the vehicle's driving. However, in extreme cases, the pictures obtained by the vehicle's camera do not have a similar category when the model is trained. At this time, this picture belongs to the out-of-domain data of the model.
[0004] Assuming that the image belongs to the K+1 category and is not included in the K categories trained by the model, due to the blindness of the model, the model will still try to make an identification and give a result that it thinks is highly confident, but this result is definitely wrong, which may lead to serious security issues. In the existing technology, there is no good solution for processing out-of-domain data.
[0005] Therefore, the present application proposes a new data classification method, device, electronic device, computer-readable medium and computer program product based on mutual information out-of-domain data detection.
[0006] The above information disclosed in the Background section is only for enhancement of understanding of the background of the present application and therefore it may contain information that does not constitute the prior art that is already known to a person of ordinary skill in the art. Summary of the invention
[0007] In view of this, the present application provides a data classification method, device, electronic device, computer-readable medium and computer program product based on mutual information out-of-domain data detection, which can effectively distinguish out-of-domain data samples, greatly reducing the risks caused by model misjudgment in practical applications, thereby improving the accuracy, security and reliability of the classification model.
[0008] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by the practice of the present application.
[0009] According to one aspect of the present application, a data classification method based on mutual information out-of-domain data detection is proposed, the method comprising: inputting the data to be classified into a classification model to obtain a classification prediction value; inputting the classification prediction value into a mutual information model to obtain a classification mutual information value, the mutual information model being obtained by training in-domain data, out-of-domain data and a neural network model; obtaining a detection score through the classification mutual information value and an out-of-domain data detection function; when the detection score is greater than a threshold, determining that the classification prediction value is valid; and generating a classification result of the data based on the valid classification prediction value.
[0010] Optionally, it also includes: determining in-domain data and out-of-domain data through the classification type of the classification model; generating sample data through the in-domain data and the out-of-domain data; inputting the sample data into a neural network model for training, wherein the training goal is to maximize the mutual information value; after the training is completed, generating the mutual information model.
[0011] Optionally, the method further includes: determining a joint probability function of classification mutual information values; performing total variation mathematical improvement on the joint probability function to generate an out-of-domain data detection function.
[0012] Optionally, the data to be classified is input into a classification model to obtain a classification prediction value, including: preprocessing the data to be classified and then inputting it into the classification model; the classification model estimates the data based on a preset classification type to obtain a classification prediction value corresponding to each classification type.
[0013] Optionally, the classification prediction value is input into a mutual information model to obtain a classification mutual information value, including: inputting the classification prediction value into a mutual information model; the mutual information model generates a classification mutual information value between prediction values of every two classification types according to the classification type.
[0014] Optionally, generating a classification result of the data according to the valid classification prediction values includes: extracting a classification type corresponding to a maximum value or a minimum value from the valid classification prediction values as the classification result of the data.
[0015] Optionally, generating sample data through the in-domain data and the out-of-domain data includes: generating positive sample data through the in-domain data and its corresponding type label; generating negative sample data through the out-of-domain data and its corresponding out-of-domain label; generating the sample data through the positive sample data and the negative sample data.
[0016] Optionally, the sample data is input into a neural network model for training, and the training goal is to maximize the mutual information value, including: inputting the sample data into the neural network model one by one; the neural network model generates classification prediction values of the sample data one by one; generating mutual information values through the classification prediction values of the sample data; during the model training process, automatically adjusting the parameters of the neural network to maximize the mutual information value.
[0017] Optionally, generating mutual information values through classification prediction values of sample data includes: generating out-of-domain logic detection values through classification prediction values of multiple sample data and out-of-domain data detection functions; and generating the mutual information values through mathematical average of the out-of-domain logic detection values.
[0018] According to one aspect of the present application, a data classification device based on mutual information out-of-domain data detection is proposed, and the device includes: a classification module, used to input the data to be classified into a classification model to obtain a classification prediction value; an information module, used to input the classification prediction value into the mutual information model to obtain a classification mutual information value, and the mutual information model is obtained by training in-domain data, out-of-domain data and a neural network model; a detection module, used to obtain a detection score through the classification mutual information value and an out-of-domain data detection function; a judgment module, used to determine that the classification prediction value is valid when the detection score is greater than a threshold; and a result module, used to generate a classification result of the data according to the valid classification prediction value.
[0019] According to one aspect of the present application, an electronic device is proposed, which includes: one or more processors; a storage device for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described above.
[0020] According to one aspect of the present application, a computer-readable medium is provided, on which a computer program is stored. When the program is executed by a processor, the method described above is implemented.
[0021] According to one aspect of the present application, a computer program product is provided, comprising: a computer program / instruction, wherein the computer program / instruction implements the method described above when executed by a processor.
[0022] According to the data classification method, device, electronic device, computer-readable medium and computer program product based on mutual information out-of-domain data detection of the present application, the data to be classified is pre-processed and then input into the classification model to obtain a classification prediction value; the classification prediction value is input into the mutual information model to obtain a classification mutual information value, and the mutual information model is obtained by training the in-domain data, out-of-domain data and a neural network model; the detection score is obtained by the classification mutual information value and the out-of-domain data detection function; when the detection score is greater than a threshold, the classification prediction value is determined to be valid; the classification result of the data is generated according to the valid classification prediction value, which can effectively distinguish out-of-domain data samples, greatly reducing the risk caused by model misjudgment in practical applications, thereby improving the accuracy, security and reliability of the classification model.
[0023] It should be understood that the foregoing general description and the following detailed description are exemplary only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] The above and other objects, features and advantages of the present application will become more apparent by describing in detail the exemplary embodiments thereof with reference to the accompanying drawings. The accompanying drawings described below are only some embodiments of the present application, and it is clear to a person skilled in the art that other accompanying drawings can be obtained from these accompanying drawings without creative effort.
[0025] Figure 1 The present invention is a flowchart showing a data classification method based on mutual information out-of-domain data detection according to an exemplary embodiment.
[0026] Figure 2 It is a flowchart of a data classification method based on mutual information out-of-domain data detection according to another exemplary embodiment.
[0027] Figure 3 It is a block diagram of a data classification device based on mutual information out-of-domain data detection according to an exemplary embodiment.
[0028] Figure 4 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION
[0029] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this application will be comprehensive and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted.
[0030] The technical abbreviations of this application are explained as follows:
[0031] In-distribution Data: In the context of machine learning or deep learning, in-distribution data refers to data that belongs to the same distribution as the data used when the model was trained. In other words, this data is an instance of the type of data that the model has already learned during training.
[0032] Suppose there is a model for image classification, which is trained with image data containing categories such as "dog", "cat", "horse", etc. In actual applications, the "dog" picture input to the model belongs to the in-domain data because this category exists in the training data.
[0033] Out-of-distribution Data (OOD): In contrast to in-domain data, out-of-domain data refers to data that does not belong to the data distribution used when the model is trained. These data are not encountered by the model during training, so the model may not be able to recognize or process them correctly.
[0034] Continuing with the above example, if the image input to the image classification model is the image of "Milky Way", and this category does not appear in the training data, then this image is out-of-domain data.
[0035] Out-of-distribution Detection (OOD Detection): Out-of-domain data detection is a technique or method used to determine whether the input data belongs to the model's training data distribution (i.e., whether it is in-domain data) before or during the prediction process. If the input data is judged to be out-of-domain data, the model's prediction results may be untrustworthy or unreliable.
[0036] Mutual Information (MI): Mutual information is a basic concept in information theory that is used to quantify the amount of information shared between two or more random variables. It is a measure of the statistical correlation between two random variables and represents the amount of information contained in one random variable about another random variable.
[0037] Figure 1 The flowchart of a data classification method based on mutual information out-of-domain data detection according to an exemplary embodiment is shown. The data classification method 10 based on mutual information out-of-domain data detection includes at least steps S102 to S110.
[0038] like Figure 1As shown, in S102, the data to be classified is input into the classification model to obtain a classification prediction value. For example, the data to be classified can be pre-processed and then input into the classification model; the classification model estimates the data based on a preset classification type to obtain a classification prediction value corresponding to each classification type.
[0039] In actual application scenarios, the data to be classified may be, for example, image data, and the classification model may be, for example, an image recognition model for recognizing image content. The image data may be processed by feature extraction, segmentation, and matching, and then input into the image recognition model.
[0040] In actual application scenarios, the data to be classified may also be text information, for example, and the classification model may be an intent classification model for identifying user intent. The text information may be vectorized and then input into the intent classification model.
[0041] The intent classification model estimates the input data based on its built-in classification types (i.e. the categories learned during model training). Assume that the intent classification model is set to the categories: 'greeting', 'asking for name', 'asking for ability to operate', and the input text message is "The weather is nice today".
[0042] The intent classification model processes the text and then calculates the classification prediction value corresponding to each classification type.
[0043] More specifically, for example:
[0044] The classification prediction value corresponding to 'greeting' is 0.5;
[0045] The classification prediction value corresponding to 'ask name' is 0.2;
[0046] The corresponding classification prediction value for ‘asking whether one is capable of operating’ is 0.7;
[0047] These predictions typically represent the probability or score of the data belonging to each class.
[0048] In S104, the classification prediction value is input into a mutual information model to obtain a classification mutual information value, and the mutual information model is obtained by training the in-domain data, the out-of-domain data and the neural network model. In one embodiment, for example, the classification prediction value can be input into a mutual information model; the mutual information model generates a classification mutual information value between prediction values of each two classification types according to the classification type.
[0049] More specifically, following the application scenario above, the classification prediction values obtained in the previous step are input into the mutual information model. The mutual information model evaluates the correlation or independence between these prediction values by calculating the mutual information between each two classification type prediction values.
[0050] More specifically, the mutual information model is trained using in-domain data, out-of-domain data, and a neural network model, so it can capture subtle differences in data distribution. The classification mutual information value calculated by the mutual information model can reflect the degree of information correlation between classification predictions, which helps to identify abnormal or inconsistent predictions.
[0051] Among them, the technical content of training the mutual information model will be Figure 2 The corresponding embodiments are described in detail.
[0052] In S106, a detection score is obtained by using the classification mutual information value and the out-of-domain data detection function; wherein the out-of-domain data detection function has a linear splitting characteristic.
[0053] The classification mutual information value is input into the out-of-domain data detection function, and the detection score is calculated. The detection score is used to evaluate the overall effectiveness and credibility of the classification prediction.
[0054] In S108, when the detection score is greater than a threshold, the classification prediction value is determined to be valid. If the detection score is greater than the threshold, it is considered that the input data of this classification prediction is in-domain data of the classification model, and the classification prediction value is valid; otherwise, it can be considered that the input data is out-of-domain data, and the result given by the classification model is unreliable due to the influence of the out-of-domain data.
[0055] As mentioned above, in-domain data refers to a data set that meets the requirements of the current processing or analysis task, while out-of-domain data refers to a data set that does not meet or may interfere with the current task. In a specific embodiment, the detection score is used to quantitatively evaluate the degree to which a sample data belongs to in-domain data.
[0056] More specifically, in this application, the detection score can be, for example, a specific numerical value used to measure the similarity or conformity of the data input this time with the in-domain data set. The detection score directly reflects the probability that the input data belongs to the in-domain data of the classification model. The higher the score, the higher the similarity between the sample and the target category, that is, the more likely it is in-domain data; conversely, the lower the score, the greater the difference between the sample and the target category, and the more likely it is out-of-domain data.
[0057] In one embodiment, a threshold τ can also be set based on historical data or empirical data to distinguish whether the input data belongs to in-domain data or out-of-domain data. If the detection score of a certain input data is greater than τ, the sample can be considered to belong to in-domain data; if the score is less than τ, it can be considered to belong to out-of-domain data.
[0058] In one embodiment, the value of the threshold τ may also be tested and adjusted on the evaluation set to achieve an optimal value.
[0059] In S110, a classification result of the data is generated according to the valid classification prediction values. For example, according to a preset rule, the classification type corresponding to the maximum value or the minimum value is extracted from the valid classification prediction values as the classification result of the data. According to the preset rule (such as extracting the classification type corresponding to the maximum value or the minimum value), one of the valid classification prediction values is selected as the final classification result, that is, which category the data is classified into.
[0060] In a specific embodiment, following the above application scenario, assume that the categories set in the intent classification model are: 'greeting', 'asking for name', 'asking for ability to operate', and the input text information is "the weather is nice today".
[0061] The classification prediction value corresponding to 'greeting' is 0.5;
[0062] The classification prediction value corresponding to 'ask name' is 0.2;
[0063] The corresponding classification prediction value for ‘asking whether one is capable of operating’ is 0.7;
[0064] Under normal circumstances, the category corresponding to the text information can be identified as 'asking whether the user is capable of operating'.
[0065] However, according to the method in this application, the detection score corresponding to the text information is calculated to be 0.1. Assuming that the threshold τ is 0.5, the detection score is less than the threshold, then it can be judged that the data corresponding to the text information is out-of-domain data, and its classification result (asking whether it is capable of operating) is unreliable.
[0066] According to the data classification method based on mutual information out-of-domain data detection of the present application, the data to be classified is pre-processed and then input into the classification model to obtain a classification prediction value; the classification prediction value is input into the mutual information model to obtain a classification mutual information value, and the mutual information model is obtained by training in-domain data, out-of-domain data and a neural network model; a detection score is obtained by the classification mutual information value and the out-of-domain data detection function; when the detection score is greater than a threshold, the classification prediction value is determined to be valid; the classification result of the data is generated according to the valid classification prediction value, which can effectively distinguish out-of-domain data samples, greatly reducing the risk caused by model misjudgment in practical applications, thereby improving the accuracy, security and reliability of the classification model.
[0067] The data classification method based on mutual information out-of-domain data detection of the present application can avoid making erroneous and potentially dangerous predictions by identifying and refusing to process inputs that do not belong to the training data distribution, which greatly reduces the risks caused by model misjudgment in practical applications.
[0068] It should be clearly understood that the present application describes how to form and use specific examples, but the principles of the present application are not limited to any details of these examples. On the contrary, based on the teaching of the content disclosed in the present application, these principles can be applied to many other embodiments.
[0069] Figure 2 It is a flowchart of a data classification method based on mutual information out-of-domain data detection according to another exemplary embodiment. Figure 2 The process 20 shown is for Figure 1 Supplementary description of the process shown.
[0070] like Figure 2 As shown, in S202, the in-domain data and out-of-domain data are determined by the classification type of the classification model. For example, positive sample data can be generated by the in-domain data and its corresponding type label; negative sample data can be generated by the out-of-domain data and its corresponding out-of-domain label; and the sample data can be generated by the positive sample data and the negative sample data.
[0071] For example, suppose a classification model is an image classification model, and the classification types of the model include "dog", "cat", and "horse". In practical applications, images related to the above types can be obtained as in-domain data, and images completely unrelated to the above types can be obtained as out-of-domain data. The out-of-domain data can be, for example, pictures of types such as "bicycles" and "mobile phones".
[0072] In S204, sample data is generated by the in-domain data and the out-domain data. Furthermore, training samples can be generated based on these data and their corresponding labels:
[0073] Positive sample data: It consists of in-domain data and its corresponding type labels, which indicate which specific in-domain category the data belongs to. More specifically, it can be assumed that there are K categories of in-domain data, and the type label corresponding to each positive sample data is one of 1 to k.
[0074] Negative sample data: consists of out-of-domain data and its corresponding out-of-domain labels, which usually indicate that the data does not belong to any known in-domain category.
[0075] In one embodiment, the proportion of positive sample data and negative sample data can also be set, for example, the proportion of positive sample data and negative sample data can be set to 50%. More specifically, the out-of-domain data can be set as the k+1th category, and the label corresponding to each negative sample data is the k+1th category.
[0076] In S206, the sample data is input into the neural network model for training, and the training goal is to maximize the mutual information value. For example, the sample data can be input into the neural network model one by one; the neural network model generates the classification prediction value of the sample data one by one; the mutual information value is generated by the classification prediction value of the sample data; during the model training process, the parameters of the neural network are automatically adjusted to maximize the mutual information value.
[0077] Furthermore, mutual information values are generated through classification prediction values of sample data, including: generating out-of-domain logic detection values through classification prediction values of multiple sample data and out-of-domain data detection functions; and generating the mutual information values through mathematical average of the out-of-domain logic detection values.
[0078] In practical applications, sample data can be input into the neural network model Tθ one by one, and the model Tθ will generate a classification prediction value for each sample, indicating the probability that the sample belongs to the domain or outside the domain. The input of Tθ is sample X, and the output Y is a vector of K+1 logits (logical prediction values). Each logical prediction value corresponds to a category from 1 to K+1.
[0079] Assume that the classification model has 5 categories: A, B, C, D, E. For a certain text T, the corresponding classification logic values are calculated by the neural network model in this application: X_A, X_B, X_C, X_D, X_E.
[0080] Based on these classification prediction values, combined with the out-of-domain data detection function, the out-of-domain logic detection value is generated. The relevant content of the out-of-domain data detection function will be Figure 3 The corresponding embodiments can be described. In one embodiment, the mutual information value can also be obtained by calculating the mathematical average (or other statistics) of these detection values, but the present application is not limited thereto. The mutual information value reflects the overall performance of the model in distinguishing data within and outside the domain.
[0081] During the training process, the parameters of the neural network model are automatically adjusted to maximize the mutual information value to more accurately distinguish between in-domain and out-of-domain data.
[0082] In S208, after the training is completed, the mutual information model is generated. When the training process is completed, that is, the model reaches a predetermined performance standard on the validation set or the test set, a mutual information model is generated. This model can be used for subsequent tasks, such as real-time detection of whether a new data point belongs to the domain or outside the domain, thereby helping to identify abnormal or out-of-domain data.
[0083] In an embodiment of the present application, it also includes: performing total variation improvement on the joint probability function of the classification mutual information value to generate the out-of-domain data detection function.
[0084] Among them, mutual information represents the information contained in each other between random variables X and Y.
[0085] The expression of mutual information between random variables X and Y can be:
[0086]
[0087] Among them, I(x,y) is the mutual information, which represents the mutual information between random variables X and Y, and p(x,y) is the joint probability distribution, which represents the probability that random variables X and Y take the values x and y at the same time. p(x) is the marginal probability distribution of X, which represents the probability that random variable X takes the value x. p(y) is the marginal probability distribution of Y, which represents the probability that random variable Y takes the value y.
[0088] In this application, the joint probability function between two classification prediction values (also called logical values) X and Y can be expressed as: (2) in, is the mutual information value between x and y, is the expected value, f can be a neural network or any set of functions. Using the above formula, we can get the lower limit of the mutual information, which is the mutual information estimate.
[0089] More specifically, in the present application, total variation (TV) may be used to improve the joint probability function.
[0090] The improved joint probability function can be: (3) in, is a random variable, T is a deep model to be trained that satisfies specific conditions. More specifically, in this application, the specific condition generally refers to the absolute value of the output of T being less than or equal to 1. . are parameters to be trained, which can be trained in a deep learning manner.
[0091] The estimation method in Formula 2 is designed with exp and log functions, which are not linear and difficult to analyze and linearly split. However, for the improved estimation method of this application, as shown in Formula 3, all terms in Formula 3 are linearly separable, and Formula 3 has the advantages of being easy to analyze and implement algorithms.
[0092] Formula 3 is further sorted out to generate the following: (4) Among them, function Sc() is the out-of-domain data detection function. N is the number of samples, i, j are the numbers of the current variables x, y, K is the sample category, and when training model T, the training goal is to maximize L, that is, to maximize the out-of-domain data detection function.
[0098] In the data classification method based on mutual information out-of-domain data detection of the present application, the traditional mutual information estimation method is cleverly modified to have the characteristics of linear splitting. This improvement not only enhances the flexibility and practicality of mutual information estimation, but also provides an intuitive and interpretable data correlation measure by linearly integrating the different components of mutual information in the present application. This linear splitting capability enables the out-of-domain data detection function to effectively evaluate the degree of information sharing between variables in a variety of application scenarios, providing strong support for subsequent data analysis and decision making.
[0099] In the data classification method based on mutual information out-of-domain data detection of the present application, the modified mutual information estimation method (including out-of-domain data detection function) is applied to the field of out-of-domain data detection. In the training stage, through a carefully designed algorithm, a large amount of in-domain and out-of-domain data can be used to train the model so that it can learn the key features that distinguish these two types of data. After the training is completed, the model can perform fast and accurate out-of-domain detection on new data samples based on the mutual information estimation method.
[0100] Those skilled in the art will appreciate that all or part of the steps for implementing the above embodiments are implemented as a computer program executed by a CPU. When the computer program is executed by the CPU, the above functions defined by the above method provided in the present application are executed. The program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc.
[0101] In addition, it should be noted that the above figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present application, and are not intended to be limiting. It is easy to understand that the processes shown in the above figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.
[0102] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.
[0103] Figure 3FIG. 1 is a block diagram of a data classification device based on mutual information out-of-domain data detection according to an exemplary embodiment. Figure 3 As shown, the data classification device 30 based on mutual information out-of-domain data detection includes: a classification module 302 , an information module 304 , a detection module 306 , a determination module 308 , and a result module 310 .
[0104] The classification module 302 is used to input the data to be classified into the classification model to obtain a classification prediction value; the classification module 302 is also used to input the data to be classified into the classification model after preprocessing; the classification model estimates the data based on a preset classification type to obtain a classification prediction value corresponding to each classification type.
[0105] The information module 304 is used to input the classification prediction value into the mutual information model to obtain the classification mutual information value, and the mutual information model is obtained by training the in-domain data, out-of-domain data and neural network model; the information module 304 is also used to input the classification prediction value into the mutual information model; the mutual information model generates the classification mutual information value between the prediction values of each two classification types according to the classification type. The detection module 306 is used to obtain the detection score through the classification mutual information value and the out-of-domain data detection function; the out-of-domain data detection function has a linear splitting characteristic;
[0106] The determination module 308 is used to determine that the classification prediction value is valid when the detection score is greater than a threshold value; the result module 310 is used to generate the classification result of the data according to the valid classification prediction value. The result module 310 is also used to extract the classification type corresponding to the maximum value or the minimum value from the valid classification prediction value as the classification result of the data.
[0107] According to the data classification device based on mutual information out-of-domain data detection of the present application, the data to be classified is pre-processed and then input into the classification model to obtain a classification prediction value; the classification prediction value is input into the mutual information model to obtain a classification mutual information value, and the mutual information model is obtained by training the in-domain data, out-of-domain data and a neural network model; the detection score is obtained by the classification mutual information value and the out-of-domain data detection function; when the detection score is greater than a threshold, the classification prediction value is determined to be valid; the method of generating the classification result of the data according to the valid classification prediction value can effectively distinguish out-of-domain data samples, greatly reducing the risk caused by model misjudgment in practical applications, thereby improving the accuracy, security and reliability of the classification model.
[0108] like Figure 4 As shown, an embodiment of the present application provides an electronic device, including a processor 410, a memory 420 and a bus, wherein the processor 410 and the memory 420 communicate with each other through the bus 440;
[0109] Memory 420, for storing computer programs;
[0110] The processor 410 is configured to implement the data classification method based on mutual information out-of-domain data detection of any of the above embodiments when executing the program stored in the memory 420 .
[0111] The communication interface 420 is used for communication between the above electronic device and other devices.
[0112] The memory 420 may include a random access memory 420 (RAM), or may include a non-volatile memory 420 (non-volatile memory), such as at least one disk memory 420. Optionally, the memory 420 may also be at least one storage device located away from the processor 410.
[0113] If the above method in the present application is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc.
[0114] The embodiment of the present application provides a computer-readable storage medium, which stores one or more programs, and the one or more programs can be executed by one or more processors to implement the data classification method based on mutual information out-of-domain data detection of any of the above embodiments. For example, the data to be classified can be pre-processed and then input into the classification model to obtain a classification prediction value; the classification prediction value is input into the mutual information model to obtain a classification mutual information value, and the mutual information model is obtained by training the in-domain data, out-of-domain data and neural network model; the detection score is obtained by the classification mutual information value and the out-of-domain data detection function; when the detection score is greater than a threshold, it is determined that the classification prediction value is valid; and the classification result of the data is generated according to the valid classification prediction value.
[0115] The exemplary embodiments of the present application are specifically shown and described above. It should be understood that the present application is not limited to the detailed structures, configurations or implementations described herein; on the contrary, the present application is intended to cover various modifications and equivalent configurations included in the spirit and scope of the appended claims.
Claims
1. A data classification method based on mutual information out-of-domain data detection, characterized in that: include: Input the data to be classified into the classification model to obtain the classification prediction value; Inputting the classification prediction value into a mutual information model to obtain a classification mutual information value, wherein the mutual information model is obtained by training in-domain data, out-of-domain data and a neural network model; A detection score is obtained by using the classification mutual information value and the out-of-domain data detection function; the out-of-domain data detection function has a linear splitting characteristic; When the detection score is greater than a threshold, determining that the classification prediction value is valid; Generate a classification result of the data based on the effective classification prediction value.
2. The method according to claim 1, characterized in that Input the data to be classified into the classification model to obtain the classification prediction value, including: The data to be classified is preprocessed and then input into the classification model; The classification model estimates the data based on a preset classification type to obtain a classification prediction value corresponding to each classification type.
3. The method according to claim 1, characterized in that Inputting the classification prediction value into the mutual information model to obtain the classification mutual information value includes: Inputting the classification prediction value into a mutual information model; The mutual information model generates a classification mutual information value between prediction values of every two classification types according to the classification type.
4. The method according to claim 1, characterized in that Generating a classification result of the data according to the effective classification prediction value includes: The classification type corresponding to the maximum value or the minimum value is extracted from the valid classification prediction values as the classification result of the data.
5. The method according to claim 1, characterized in that Also includes: Determine in-domain data and out-of-domain data through the classification type of the classification model; Generate sample data using the in-domain data and the out-domain data; Inputting the sample data into a neural network model for training, wherein the training objective is to maximize the mutual information value of the sample data; After the training is completed, the mutual information model is generated.
6. The method according to claim 5, characterized in that Generating sample data by using the in-domain data and the out-domain data includes: Generate positive sample data through the in-domain data and its corresponding type label; Generate negative sample data using the out-of-domain data and its corresponding out-of-domain label; The sample data is generated by using the positive sample data and the negative sample data.
7. The method according to claim 5, characterized in that The sample data is input into a neural network model for training, and the training goal is to maximize the mutual information value of the sample data, including: Inputting the sample data into the neural network model one by one; The neural network model generates classification prediction values of the sample data one by one; Generate mutual information value of sample data through classification prediction value of sample data; During the model training process, the parameters of the neural network are automatically adjusted to maximize the mutual information value.
8. The method according to claim 7, characterized in that The mutual information value of the sample data is generated by the classification prediction value of the sample data, including: Generate an out-of-domain logic detection value through classification prediction values of multiple sample data and an out-of-domain data detection function; The mutual information value is generated by mathematically averaging the out-of-domain logical detection values.
9. A data classification device based on mutual information out-of-domain data detection, characterized in that: include: The classification module is used to input the data to be classified into the classification model to obtain the classification prediction value; An information module, used for inputting the classification prediction value into a mutual information model to obtain a classification mutual information value, wherein the mutual information model is obtained by training the in-domain data, the out-of-domain data and the neural network model; A detection module, used to obtain a detection score through the classification mutual information value and an out-of-domain data detection function; the out-of-domain data detection function has a linear splitting characteristic; A determination module, configured to determine that the classification prediction value is valid when the detection score is greater than a threshold; A result module is used to generate a classification result of the data according to the effective classification prediction value.
10. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 8.
11. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.
12. A computer program product, characterized in that The method comprises a computer program / instruction, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 8.