Data classification methods with integrated unified threshold loss functions

Abstract

This invention discloses a data classification method integrating a unified threshold loss function, comprising: acquiring a data set from N categories; extracting data features using a feature extraction model and normalizing the features; then constructing a cross-entropy loss function integrating a unified threshold based on the cross-entropy loss function and the unified threshold, and training the feature extraction model; classifying the data to be classified according to the data feature extraction model to obtain the classification results of each data. This invention can, in actual data processing, classify the data to be classified in the database according to the trained data feature extraction model, thereby quickly and accurately obtaining the classification results of each data.

Description

Technical Field

[0001] This invention relates to the field of information technology, and in particular to a data classification method that integrates a loss function with a unified threshold. Background Technology

[0002] In data classification, linear classifiers are often used to classify data features. The combination of a linear classifier and data features forms positive and negative sample-class pairs. If the model, after training, can use a uniform threshold to distinguish these positive and negative sample pairs, it will effectively improve the data classification accuracy. The loss function is crucial for training a data classification model, but existing loss functions learn the threshold for distinguishing positive and negative sample-class pairs on a sample-by-sample basis. They cannot effectively learn a uniform threshold for distinguishing positive and negative sample-class pairs across all samples, thus hindering the rapid and accurate selection of the required data from the database and the acquisition of classification results for each data point.

[0003] Therefore, existing technologies still need improvement. Summary of the Invention

[0004] The technical problem to be solved by the present invention is that, in view of the defects of the prior art, the present invention provides a data classification method with an integrated unified threshold loss function, so as to solve the problem that the prior art cannot quickly and accurately select the required data from the database and obtain the classification results of each data.

[0005] The technical solution adopted by this invention to solve the technical problem is as follows: In a first aspect, the present invention provides a data classification method with an integrated loss function that incorporates a unified threshold, comprising: Obtain a dataset of N categories; wherein, the dataset of N categories is used to train a data feature extraction model; Any feature data in the dataset is normalized to obtain a normalized data feature set; Using a unified threshold, a cross-entropy loss function integrating the unified threshold is constructed based on the cross-entropy loss function, and the data feature extraction model is trained. The trained data feature extraction model is used to classify the data to be classified, and the classification results of each data are obtained.

[0006] In one implementation, obtaining the data set of N categories includes: Obtain the data sets of the N categories; The data sets of the N categories are represented by features to obtain the corresponding feature datasets;

[0007] in, Indicates sample features, This represents the feature extraction model; M is the length of the feature vector; Indicates they come from the same category A subset of the data.

[0008] In one implementation, the normalization process for any feature data in the dataset includes: Normalize the weight matrix in the linear fully connected classifier; The normalized data feature set is obtained by normalizing any feature data in the feature dataset.

[0009] In one implementation, the step of using a unified threshold, constructing an integrated unified threshold cross-entropy loss function based on the cross-entropy loss function, and training the data feature extraction model includes: Based on the cross-entropy loss function and the unified threshold, construct the cross-entropy loss function with integrated unified threshold; The data feature extraction model is trained based on the cross-entropy loss function with the integrated unified threshold.

[0010] In one implementation, the unified threshold satisfies the following in the optimal state:

[0011] in, Let be any sample of class i; Let i be the class feature of the i-th class; Positive sample-class pair; Features The similarity between the class features of class i and class i; Let j be the class feature of the j-th class; Negative sample-class pair; Features The similarity between the class features of class j and the class features of class j; To standardize the threshold.

[0012] In one implementation, constructing a cross-entropy loss function integrating a unified threshold based on the cross-entropy loss function and the unified threshold includes: The cross-entropy loss function is transformed as follows:

[0013] Combining the transformed cross-entropy loss function with the unified threshold yields the following inequality regarding the cross-entropy loss function:

[0014] The cross-entropy loss function with the integrated unified threshold is obtained as follows:

[0015] in, These are trainable parameters.

[0016] Secondly, the present invention provides a data classification apparatus with an integrated loss function that incorporates a unified threshold, comprising: The data acquisition module is used to acquire a dataset of N categories; wherein, the dataset of N categories is used to train a data feature extraction model. The normalization processing module is used to normalize any feature data in the dataset to obtain a normalized data feature set; A unified threshold integration module is used to construct an integrated unified threshold cross-entropy loss function based on the cross-entropy loss function, and train the data feature extraction model using a unified threshold. The data classification module is used to classify the data to be classified according to the trained data feature extraction model, and obtain the classification results of each data.

[0017] Thirdly, the present invention provides a terminal, comprising: a processor and a memory, wherein the memory stores a data classification program with an integrated unified threshold loss function, and the data classification program with the integrated unified threshold loss function, when executed by the processor, is used to implement the operation of the data classification method with the integrated unified threshold loss function as described in the first aspect.

[0018] Fourthly, the present invention also provides a medium, which is a computer-readable storage medium, storing a data classification program with an integrated uniform threshold loss function, which, when executed by a processor, is used to implement the operation of the data classification method with the integrated uniform threshold loss function as described in the first aspect.

[0019] The present invention, by employing the above technical solution, has the following effects: This invention uses a uniform threshold to distinguish positive and negative sample-class pairs in all data samples, and explicitly integrates this uniform threshold into the cross-entropy loss function. When training a data classification model using this cross-entropy loss function with the integrated uniform threshold, the model will learn a uniform threshold to distinguish positive and negative sample-class pairs. In actual data processing, for data that needs to be classified in the database, the model will classify the data to be classified based on the trained data feature extraction model, thereby quickly and accurately obtaining the classification results of each data. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0021] Figure 1 This is a flowchart of a data classification method that integrates a loss function with a unified threshold in one implementation of the present invention.

[0022] Figure 2 This is a functional schematic diagram of the terminal in one implementation of the present invention.

[0023] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0025] Exemplary methods In data classification, linear classifiers are often used to classify data features. The combination of a linear classifier and data features forms positive and negative sample-class pairs. If the model, after training, can use a uniform threshold to distinguish these positive and negative sample pairs, it will effectively improve the data classification accuracy. The loss function is crucial for training a data classification model, but existing loss functions learn the threshold for distinguishing positive and negative sample-class pairs on a sample-by-sample basis. They cannot effectively learn a uniform threshold for distinguishing positive and negative sample-class pairs across all samples, thus hindering the rapid and accurate selection of the required data from the database and the acquisition of classification results for each data point.

[0026] To address the aforementioned issues, this invention provides a data classification method that integrates a unified threshold loss function. By using a unified threshold to distinguish positive and negative sample-class pairs among all data samples, and explicitly integrating this unified threshold into the cross-entropy loss function, the data classification model can learn a unified threshold for distinguishing positive and negative sample-class pairs when training the data classification model using this integrated unified threshold cross-entropy loss function. Furthermore, in actual data processing, for data in the database that needs to be classified, the model can classify the data to be classified based on the trained data feature extraction model, thereby quickly and accurately obtaining the classification results for each data set.

[0027] like Figure 1 As shown, this embodiment of the invention provides a data classification method with an integrated unified threshold loss function, including the following steps: Step S100: Obtain a dataset of N categories; wherein the dataset of N categories is used to train the data feature extraction model.

[0028] In this embodiment, the data classification method with an integrated unified threshold loss function is applied to a terminal, which includes, but is not limited to, devices such as computers.

[0029] In this embodiment, starting from the commonly used multi-class cross-entropy loss function (i.e., the SoftMax loss function), a cross-entropy loss function with an integrated unified threshold (Unified Cross-entropy loss, UCE loss) is derived. When using it to train a data classification model, it can learn a unified threshold to distinguish between positive and negative samples and class pairs, thereby effectively improving the data classification accuracy.

[0030] Specifically, in one implementation of this embodiment, step S100 includes the following steps: Step S101: Obtain the data sets of the N categories; Step S102: Perform feature representation on the data sets of the N categories to obtain the corresponding feature datasets.

[0031] In this embodiment, N categories of data sets are obtained, wherein the data sets can be image data sets of different categories or text datasets of different categories; of course, these different categories of data sets can all be obtained from public databases.

[0032] Suppose M is a data feature extraction model trained on dataset D, where:

[0033] Dataset It is a collection containing data from N categories. This represents a subset of data from the same category i. For any data sample The feature dataset is as follows:

[0034] in, Indicates sample characteristics; This represents the feature extraction model; M is the length of the feature vector; Indicates they come from the same category A subset of the data.

[0035] like Figure 1 As shown, in one implementation of this invention, the data classification method integrating a loss function with a unified threshold further includes the following steps: Step S200: Normalize any feature data in the dataset to obtain a normalized data feature set.

[0036] In this embodiment, a fully connected linear (FC) classifier is typically used in data classification research. It consists of a weight matrix W and a bias vector b, and classifies data samples X based on features x, where:

[0037] In the weight matrix W, For the first Class characteristics or class proxy.

[0038] Specifically, in one implementation of this embodiment, step S200 includes the following steps: Step S201: Normalize the weight matrix in the linear fully connected classifier; Step S202: Normalize any feature data in the feature dataset to obtain the normalized data feature set.

[0039] In this embodiment, for ease of discussion, the weight matrix W is normalized and the bias is temporarily ignored, that is:

[0040] and arbitrary data features Standardize to .

[0041] In this embodiment, N samples are arbitrarily selected from the normalized dataset to determine the corresponding cross-entropy loss function, thereby constructing a loss function that integrates a unified classification threshold.

[0042] In this embodiment, N samples are arbitrarily selected from dataset D. , where for any i, we have So for a given sample Then its corresponding multi-class cross-entropy loss function is:

[0043] in:

[0044] Represent two vectors , The inner product of.

[0045] For those from the Any sample of the class In this embodiment, It is called its positive sample-class pair. This is called its negative sample-class pair. Common data classification loss functions, including the cross-entropy loss function, are used to correctly classify samples. These are achieved by training the model M and the classifier weight matrix W, such that for any sample from class i... Its characteristics With the The class features of a class have the highest similarity, that is:

[0046] This is actually sample by sample. Seek its corresponding classification threshold Distinguish between positive and negative samples – class pairs:

[0047] Seeking satisfaction threshold This loss function, which seeks the classification threshold sample by sample, is not suitable for open set classification tasks such as face image recognition.

[0048] like Figure 1 As shown, in one implementation of this invention, the data classification method integrating a loss function with a unified threshold further includes the following steps: Step S300: Using a unified threshold, construct a cross-entropy loss function that integrates the unified threshold based on the cross-entropy loss function, and train the data feature extraction model.

[0049] In this embodiment, a loss function integrating a unified classification threshold can be constructed, wherein the unified threshold satisfies the following in the optimal state:

[0050] in, For the first Any sample of the class; For the first Class characteristics; Positive sample-class pair; Features With the Similarity between class features; For the first Class characteristics; Negative sample-class pair; Features With the Similarity between class features; To standardize the threshold.

[0051] Specifically, in one implementation of this embodiment, step S300 includes the following steps: Step S301: Construct the cross-entropy loss function with integrated unified threshold based on the cross-entropy loss function and the unified threshold.

[0052] Specifically, in one implementation of this embodiment, step S301 includes the following steps: Step S301a: Transform the cross-entropy loss function.

[0053] In this embodiment, from the cross-entropy loss function We then construct an integrated, unified threshold cross-entropy (UCE) loss function.

[0054] First, the cross-entropy loss function can be transformed into:

[0055] Step S301b: Combine the transformed cross-entropy loss function with the unified threshold to obtain an inequality regarding the cross-entropy loss function.

[0056] After transforming the cross-entropy loss function and applying a uniform threshold t, we obtain:

[0057] Step S301c yields the cross-entropy loss function with the integrated unified threshold.

[0058] The constructed integrated unified threshold UCE loss function is as follows: .

[0059] in, For is in use A learnable constant during the training of model M, which will eventually learn a uniform threshold to distinguish positive and negative sample-class pairs for all samples. .

[0060] Step S302: Train the data feature extraction model based on the cross-entropy loss function with the integrated unified threshold.

[0061] It is worth mentioning that the UCE loss function constructed in this embodiment integrates a unified threshold t that can distinguish positive and negative sample-class pairs for all samples. It effectively establishes a mathematical relationship between the sample-class similarity bias and this unified threshold, and can learn this unified threshold by learning the bias during the training of the feature extraction model M. Previous literature commonly used data classification loss functions, such as the cross-entropy loss function (i.e., the SoftMax loss function),... These methods explicitly integrate or learn this unified threshold. The closest loss function to the UCE loss function in this embodiment is the Binary Cross-Entropy (BCE) loss function:

[0062] The key difference between the BCE loss function and the UCE loss function in this embodiment is that the BCE loss function uses different trainable parameters for different classes. Furthermore, it does not provide a relationship between this bias and the classification threshold; while the UCE loss function in this embodiment not only uses uniform trainable parameters Furthermore, it was explicitly stated that this parameter is related to the unified classification threshold. The connection between them.

[0063] like Figure 1 As shown, in one implementation of this invention, the data classification method integrating a loss function with a unified threshold further includes the following steps: Step S400: Classify the data to be classified according to the trained data feature extraction model to obtain the classification results of each data.

[0064] In this embodiment, by setting the UCE loss function, the threshold for uniformly distinguishing positive and negative sample-class pairs can be learned during the training of the data feature extraction model, thereby effectively improving the data classification accuracy. Furthermore, in the actual data processing process, for the data to be classified in the database, the trained data feature extraction model can be used to classify the data to be classified, thereby quickly and accurately obtaining the classification results of each data.

[0065] The key point of the UCE loss function in this embodiment is to use a uniform threshold t to distinguish between positive and negative sample-class pairs in all data samples, and to explicitly integrate this threshold into the UCE loss function. If a general similarity function is used... Measuring sample features With class features The similarity between them can be used to derive the general UCE loss function:

[0066] Choosing different similarity measurement functions Different variations of the UCE loss function in this embodiment can be obtained, all of which are data classification loss functions that integrate a unified threshold.

[0067] This embodiment achieves the following technical effects through the above technical solution: This embodiment uses a uniform threshold to distinguish positive and negative sample-class pairs in all data samples, and explicitly integrates this uniform threshold into the cross-entropy loss function. When training the data classification model using the cross-entropy loss function with the integrated uniform threshold, the model will learn the uniform threshold for distinguishing positive and negative sample-class pairs. In actual data processing, for data in the database that needs to be classified, the model will classify the data to be classified based on the trained data feature extraction model, thereby quickly and accurately obtaining the classification results of each data.

[0068] Exemplary device Based on the above embodiments, the present invention also provides a data classification apparatus with an integrated unified threshold loss function, comprising: The data acquisition module is used to acquire a dataset of N categories; wherein, the dataset of N categories is used to train a data feature extraction model. The normalization processing module is used to normalize any feature data in the dataset to obtain a normalized data feature set; A unified threshold integration module is used to construct an integrated unified threshold cross-entropy loss function based on the cross-entropy loss function, and train the data feature extraction model using a unified threshold. The data classification module is used to classify the data to be classified according to the trained data feature extraction model, and obtain the classification results of each data.

[0069] Based on the above embodiments, the present invention also provides a terminal, the principle block diagram of which can be as follows: Figure 2 As shown.

[0070] The terminal includes: a processor, a memory, an interface, a display screen, and a communication module connected via a system bus; wherein, the processor of the terminal provides computing and control capabilities; the memory of the terminal includes a storage medium and internal memory; the storage medium stores the operating system and computer programs; the internal memory provides an environment for the operation of the operating system and computer programs in the storage medium; the interface is used to connect to external devices, such as mobile terminals and computers; the display screen is used to display relevant information; and the communication module is used to communicate with a cloud server or mobile terminal.

[0071] When executed by a processor, this computer program is used to implement a data classification method that integrates a loss function with a uniform threshold.

[0072] It will be understood by those skilled in the art that Figure 2 The block diagram shown is merely a partial structural diagram related to the present invention and does not constitute a limitation on the terminal to which the present invention is applied. A specific terminal may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0073] In one embodiment, a terminal is provided, comprising: a processor and a memory, the memory storing a data classification program with an integrated uniform threshold loss function, the data classification program with the integrated uniform threshold loss function being executed by the processor to implement the operation of the data classification method with the integrated uniform threshold loss function as described above.

[0074] In one embodiment, a storage medium is provided, wherein the storage medium stores a data classification program with an integrated uniform threshold loss function, which, when executed by a processor, is used to implement the operation of the data classification method with the integrated uniform threshold loss function as described above.

[0075] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory.

[0076] In summary, this invention provides a data classification method integrating a unified threshold loss function, comprising: obtaining a dataset of N categories; wherein the dataset of N categories is used to train a data feature extraction model; normalizing any feature data in the dataset to obtain a normalized data feature set; using a unified threshold, constructing a cross-entropy loss function integrating the unified threshold based on the cross-entropy loss function, and training the data feature extraction model; classifying the data to be classified according to the trained data feature extraction model to obtain the classification result of each data; this invention can, in actual data processing, classify the data to be classified in the database according to the trained data feature extraction model, thereby quickly and accurately obtaining the classification result of each data.

[0077] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A data classification method integrating a unified threshold loss function, characterized in that, include: Obtain a dataset of N categories; wherein, the dataset of N categories is used to train a data feature extraction model; the dataset is an image dataset or a text dataset of different categories; Any feature data in the dataset is normalized to obtain a normalized data feature set; Using a unified threshold, a cross-entropy loss function integrating the unified threshold is constructed based on the cross-entropy loss function, and the data feature extraction model is trained. The trained data feature extraction model is used to classify the data to be classified, and the classification results of each data are obtained. The acquisition of the data set of N categories includes: Obtain the data sets of the N categories; The data sets of the N categories are represented by features to obtain the corresponding feature datasets; in, Indicates sample features, This represents the feature extraction model; M is the length of the feature vector; Indicates they come from the same category A subset of data; The step of using a unified threshold, constructing an integrated unified threshold cross-entropy loss function based on the cross-entropy loss function, and training the data feature extraction model includes: Based on the cross-entropy loss function and the unified threshold, construct the cross-entropy loss function with integrated unified threshold; The data feature extraction model is trained based on the cross-entropy loss function with the integrated unified threshold. The step of constructing a cross-entropy loss function integrating a unified threshold based on the cross-entropy loss function and the unified threshold includes: The cross-entropy loss function is transformed as follows: Combining the transformed cross-entropy loss function with the unified threshold yields the following inequality regarding the cross-entropy loss function: The cross-entropy loss function with the integrated unified threshold is obtained as follows: in, These are trainable parameters; For the data in the database that needs to be classified, the data to be classified is classified according to the data feature extraction model after training, and the classification results of each data are obtained.

2. The data classification method with an integrated unified threshold loss function according to claim 1, characterized in that, The normalization process for any feature data in the dataset includes: Normalize the weight matrix in the linear fully connected classifier; The normalized data feature set is obtained by normalizing any feature data in the feature dataset.

3. The data classification method with an integrated unified threshold loss function according to claim 1, characterized in that, The unified threshold satisfies the following in the optimal state: in, For the first Any sample of the class; For the first Class characteristics; Positive sample-class pair; Features With the Similarity between class features; For the first Class characteristics; Negative sample-class pair; Features With the Similarity between class features; To standardize the threshold.

4. A data classification apparatus with an integrated unified threshold loss function, used to implement the data classification method with an integrated unified threshold loss function as described in any one of claims 1-3, characterized in that, include: The data acquisition module is used to acquire a dataset of N categories; wherein, the dataset of N categories is used to train a data feature extraction model. The normalization processing module is used to normalize any feature data in the dataset to obtain a normalized data feature set; A unified threshold integration module is used to construct an integrated unified threshold cross-entropy loss function based on the cross-entropy loss function, and train the data feature extraction model using a unified threshold. The data classification module is used to classify the data to be classified according to the trained data feature extraction model, and obtain the classification results of each data.

5. A terminal, characterized in that, include: The processor and memory, the memory storing a data classification program with an integrated uniform threshold loss function, the data classification program with the integrated uniform threshold loss function being executed by the processor to implement the operation of the data classification method with the integrated uniform threshold loss function as described in any one of claims 1-3.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a data classification program with an integrated uniform threshold loss function, which, when executed by a processor, is used to implement the operation of the data classification method with an integrated uniform threshold loss function as described in any one of claims 1-3.

Images