Classification model training method and device, classification method and device, equipment and storage medium

Abstract

The invention provides a training method and device of a classification model, a classification method and device, equipment and a storage medium, and relates to the technical field of computers, in particular to the technical fields of artificial intelligence, large models, data classification and the like. According to the specific implementation scheme, a first sample set is obtained, the first sample set comprises a plurality of first training samples, and the first training samples comprise first input data and label data; determining second input data corresponding to the first input data based on the first input data and the label data; performing first training on the first classification model by using the second input data and the label data to obtain a second classification model; performing second training on the second classification model by using the third input data and the label data to obtain a target classification model; in the second training process, preset feature noise is injected into the data features extracted by the second classification model; the third input data includes the first input data and the second input data. The robustness of the classification model can be improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, and particularly relates to the technical field of artificial intelligence, large model, data classification, etc. BACKGROUND

[0002] In recent years, with the development of artificial intelligence, classification models have become the core support for key tasks such as image recognition and natural language processing. However, in actual deployment, classification models face serious robustness problems. For example, classification models are highly sensitive to input noise, are vulnerable to adversarial attacks, and their performance drops significantly with slight changes in data distribution, which restricts the reliability and generalization ability of classification models in real scenarios. Therefore, how to improve the robustness of classification models has become a problem to be solved. SUMMARY

[0003] The present disclosure provides a training method of a classification model, a classification method, an apparatus, a device and a storage medium.

[0004] According to an aspect of the present disclosure, a training method of a classification model is provided, comprising: obtaining a first sample set, the first sample set comprising a plurality of first training samples, each first training sample comprising first input data and label data; determining second input data corresponding to the first input data based on the first input data and the label data; performing first training on a first classification model using the second input data and the label data to obtain a second classification model; performing second training on the second classification model using third input data and label data to obtain a target classification model; wherein during the second training, data features extracted from the second classification model are injected with preset feature noise; the third input data comprises the first input data and the second input data.

[0005] According to another aspect of the present disclosure, a classification method is provided, comprising: obtaining data to be classified; inputting the data to be classified into a target classification model to output class information of the data to be classified by the target classification model; wherein the target classification model is trained by the training method proposed in the present disclosure.

[0006] According to another aspect of the present disclosure, a training apparatus of a classification model is provided, comprising: a first obtaining module configured to obtain a first sample set, the first sample set comprising a plurality of first training samples, each first training sample comprising first input data and label data; The data determination module is configured to determine second input data corresponding to the first input data based on the first input data and the label data. The first training module is configured to perform first training on the first classification model by using the second input data and the label data, to obtain a second classification model. The second training module is configured to perform second training on the second classification model by using third input data and the label data, to obtain a target classification model; wherein, in the second training process, data features extracted from the second classification model are injected with preset feature noise; and the third input data includes the first input data and the second input data.

[0007] According to another aspect of the present disclosure, a classification device is provided, comprising: The second acquisition module is configured to acquire to-be-classified data. The data classification module is configured to input the to-be-classified data into a target classification model, and output class information of the to-be-classified data by the target classification model. The target classification model is trained by the training device provided in the present disclosure.

[0008] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any of the embodiments of the present disclosure.

[0009] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to perform the method according to any of the embodiments of the present disclosure.

[0010] According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the method according to any of the embodiments of the present disclosure.

[0011] The disclosure adopts a multi-stage training manner to train the classification model. Specifically, in the first stage, the first classification model is trained based on the second input data and the label data corresponding to the first input data, to obtain the second classification model, so that the second classification model has the ability to process semantic variants, learns semantic common features, and enhances the adaptability to semantic differences; in the second stage, the second classification model is trained in combination with the third input data and the label data, in which stage, the extracted data features are injected with preset feature noise, simulating the interference in the category prediction process, so that the second classification model adjusts the model parameters under this interference. After multi-stage training, the target classification model has both semantic recognition and feature anti-interference capabilities, thereby improving the generalization ability and robustness of the model.

[0012] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the disclosure, nor is it used to limit the scope of the disclosure. Other features of the disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0013] The accompanying drawings are used to better understand the present scheme and do not constitute a limitation on the present disclosure. Among them: Figure 1 is an application scenario diagram according to an embodiment of the present disclosure; Figure 2 is an implementation flowchart of a training method of a classification model according to an embodiment of the present disclosure; Figure 3 is a flowchart of training a target classification model according to an embodiment of the present disclosure; Figure 4 is an implementation flowchart of a classification method according to an embodiment of the present disclosure; Figure 5 is a structural schematic diagram of a training device 500 of a classification model according to an embodiment of the present disclosure; Figure 6 is a structural schematic diagram of a classification device 600 according to an embodiment of the present disclosure; Figure 7 shows a schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure. DETAILED DESCRIPTION

[0014] Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, which include various details of the embodiments of the present disclosure to help understanding, and should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of the present disclosure. Also, in order to be clear and concise, the description below omits the description of well-known functions and structures.

[0015] The term "and / or" in this disclosure indicates that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. The term "at least one" in this document means any combination of at least two of a plurality of options, such as including at least one of A, B, and C, which can mean including any one or more elements selected from the set of A, B, and C. The terms "first" and "second" in this document refer to and distinguish multiple similar technical terms, and do not imply a specific order or a limitation to only two. For example, "first feature" and "second feature" refer to two types / two features; the first feature can be one or more, and the second feature can also be one or more.

[0016] With the widespread adoption and application of artificial intelligence (AI) technology, classification models play a crucial role in key tasks such as image recognition and natural language processing. However, in practical deployments, classification models often face the problem of insufficient robustness. For example, they are sensitive to input noise, vulnerable to adversarial attacks, and their performance deteriorates significantly when data distribution changes. These limitations restrict the reliability and generalization ability of classification models in real-world scenarios.

[0017] Although pre-trained classification models (such as Bidirectional Encoder Representations from Transformers, BERT) learn rich feature representations from large-scale data, they are prone to overfitting limited labeled data when fine-tuning to adapt to downstream classification tasks. This leads to the neglect of the original pre-training knowledge and ultimately reduced robustness. Existing methods to improve robustness, such as data augmentation and adversarial training, while improving robustness to some extent, often rely on single-level optimization (e.g., processing only the input data), resulting in limited improvement.

[0018] The following is a brief introduction to the existing technology in this field: (1) Data augmentation techniques: This technique uses manually set rules or simple transformations, such as rotating images or replacing synonyms in text, to expand the training data and expose the classification model to more diverse samples. This technique is simple to operate and easy to implement. However, the data generated by this technique is often not realistic or complex enough to cover extreme or adversarial scenarios. Moreover, its effect on improving the robustness of the classification model's internal representation is also relatively limited.

[0019] (2) Adversarial training technique: This technique generates adversarial examples and adds them to the training set, enabling the classification model to learn robust representations that maintain stable performance when faced with adversarial perturbations. This technique can enhance the classification model's ability to resist adversarial attacks, but it requires a large amount of computational resources. At the same time, this technique may also focus too much on the optimization of adversarial scenarios, thereby sacrificing the accuracy of the classification model in predicting normal data. In addition, adversarial training mostly focuses only on the input layer of the training samples, ignoring the potential optimization possibilities of the feature layer.

[0020] In summary, most existing technologies only address the robustness of classification models, failing to systematically optimize across multiple levels, thus offering limited improvement in robustness. This disclosure innovatively injects noise into both the input and feature extraction layers of the classification model, and employs a multi-stage training strategy to progressively enhance the model's adaptability to noise and distribution variations. Specifically, this disclosure first injects noise into the input layer of the classification model to simulate data variation in real-world scenarios, and then perturbs the intermediate representation (or feature data) in the feature extraction layer to strengthen the feature processing robustness of the classification model. This multi-level, multi-stage noise injection mechanism effectively improves the generalization ability and anti-interference capability of the classification model, achieving a balance between robustness and accuracy.

[0021] Figure 1 This is a schematic diagram illustrating an application scenario according to an embodiment of this disclosure, such as... Figure 1 As shown in the illustration, the application scenario diagram of this disclosure may include, but is not limited to, a model training device 110 and a classification model 120. The model training device 110 and the classification model 120 can communicate via any type of wired or wireless network. Specifically, the model training device 110 can be used to receive training samples and train an initial model based on the training samples to obtain a trained classification model 120. The training samples may include data to be classified and category information (i.e., label data) for the data to be classified. The data to be classified may include data of multiple modalities such as text, images, and audio. In this disclosure embodiment, the model training device 110 may include an electronic device or server for providing backend management for the classification model 120. Furthermore, this disclosure embodiment does not impose a specific limitation on the number of model training devices 110. For example, the application scenario diagram of this disclosure embodiment may include one or more model training devices 110.

[0022] Figure 2 This is a flowchart illustrating the implementation of a training method for a classification model according to an embodiment of the present disclosure, including: S210. Obtain a first sample set, which includes multiple first training samples, and the first training samples include first input data and label data; S220. Based on the first input data and the label data, determine the second input data corresponding to the first input data; S230. Using the second input data and label data, perform the first training on the first classification model to obtain the second classification model; S240. Using the third input data and label data, the second classification model is trained a second time to obtain the target classification model; wherein, during the second training process, preset feature noise is injected into the data features extracted by the second classification model; the third input data includes the first input data and the second input data.

[0023] In this embodiment of the disclosure, the first sample set can be a data collection containing multiple first training samples used to train the model. In one example, these first training samples may share common features or belong to the same type of task scenario. For example, in an image classification task, the first sample set may contain a large number of images of different categories.

[0024] In this embodiment of the disclosure, the first input data can be the input content used to train the classification model, i.e., the object that the classification model needs to process and analyze. The form or format of the first input data may differ in different tasks. For example, in an image classification task, the first input data may be a feature vector of an image; in a natural language processing task, the first input data may be a sequence of words or characters in text.

[0025] Label data can be the correct classification or annotation information corresponding to the first input data. For example, if the first input data is an image of an animal, the label data is the category of that animal; if the first input data is text expressing a positive emotion, the label data might be "positive". Here, the label data provides the learning target and feedback for model training. In one example, the model can adjust its parameters by comparing the differences between its own output and the label data, thereby continuously improving its performance.

[0026] Furthermore, this disclosure can utilize the first input data and label data to determine the second input data corresponding to the first input data. In one example, this disclosure can generate new input data, i.e., the second input data, by injecting semantic noise without changing the category (i.e., label data) to which the first input data belongs.

[0027] Here, semantic noise injection can refer to changing and interfering with the expression of data to a certain extent while keeping the semantics of the first input data unchanged. In other words, semantic noise injection focuses on operating at the semantic level, so that the generated new data is similar to the original data in semantics, but different in expression.

[0028] For example, for an image showing a "cat" (i.e., the first input data, with the corresponding label "cat"), without affecting the overall recognition as a "cat," non-critical parts of the image are occluded to generate a new image as the second input data. For a text "The plot of this movie is very exciting" (i.e., the first input data, with the corresponding label "positive reviews"), without affecting the category of the text, semantic noise is injected into the text to obtain the second input data, which could be "The storyline of this movie is excellent, the plot is full of twists and turns, exceeding expectations and bringing many surprises to the viewers."

[0029] It is understandable that the first input data corresponds to the label data. Since the second input data is generated based on the first input data, the second input data and the label data also correspond to each other, that is, the label data can represent the category of the second input data.

[0030] Furthermore, this disclosure can utilize the second input data and label data to perform a first training (or first-stage training) on ​​the first classification model. In one example, the first classification model can predict the classification result of the input second input data based on its existing parameters and structure. Then, the predicted classification result is compared with the actual label data to determine the difference between the two. This difference reflects the accuracy of the first classification model's classification of the second input data. Based on the calculated difference, this disclosure can use an optimization algorithm to adjust its parameters, thereby reducing the error in category prediction. By repeating the above process of data input, category prediction, difference calculation, and parameter adjustment until the difference between the category result predicted by the first classification model and the label data is less than a preset value, or until a preset number of training rounds is reached, the training process ends, and the second classification model is obtained.

[0031] In this embodiment of the disclosure, a second training (or second-stage training) is performed on the second classification model using third input data (i.e., including the first and second input data) and label data. During the second training process, this disclosure can inject preset feature noise into the data features extracted by the second classification model. Here, the preset feature noise can be generated according to specific rules or statistical characteristics, used to simulate interference and changes in the data features. In one example, the purpose of injecting preset feature noise can be to expose the second classification model to interfering features during training, thereby learning how to accurately classify under such suboptimal conditions.

[0032] It is understandable that, since the third input data contains the first and second input data, and assuming that both the first and second input data correspond to the label data, the third input data also corresponds to the label data.

[0033] After performing a second training on the second classification model, the target classification model can be obtained. The specific methods for the second training will be described in detail later.

[0034] This disclosure employs a multi-stage training approach to train the classification model. Specifically, in the first stage, a first classification model is trained based on the second input data and label data corresponding to the first input data, resulting in a second classification model. This second classification model is capable of handling semantic variations, learning common semantic features, and enhancing its adaptability to semantic differences. In the second stage, the second classification model is trained by combining the third input data and label data. In this stage, pre-set feature noise is injected into the extracted data features to simulate interference in the category prediction process, allowing the second classification model to adjust its parameters under this interference. After multi-stage training, the target classification model possesses both semantic recognition and feature anti-interference capabilities, thereby improving the model's generalization ability and robustness.

[0035] This disclosure proposes a training method based on multi-stage noise injection to improve the robustness of classification models. The core of this method lies in injecting controllable noise in stages from both the model input layer and the feature extraction layer, while simultaneously employing a multi-stage joint training strategy to train the model. This disclosure is applicable to various classification model training scenarios based on pre-trained models (such as BERT, Vision Transformer (ViT), etc.) and can be integrated into existing training workflows.

[0036] Figure 3 This is a flowchart illustrating a training target classification model according to an embodiment of the present disclosure, as shown below. Figure 3 As shown, the training process of the target classification model includes the following steps.

[0037] S301. Obtain the first sample set.

[0038] In this embodiment of the disclosure, the first sample set may include multiple first training samples, and the first training samples may include first input data and label data.

[0039] S302. Using the first sample set, train the basic classification model to obtain the first classification model.

[0040] In this embodiment, a pre-trained model can be selected as the base classification model. For example, BERT can be used for text classification tasks; and Residual Neural Network (ResNet) or ViT can be used for image classification tasks. Further, this disclosure can utilize multiple first training samples (e.g., sentiment analysis classification samples, image classification samples) from the first sample set, combined with the cross-entropy loss function, to train the base classification model, thereby obtaining the first classification model. In this embodiment, the loss function calculation formula for training the base classification model is: (1) In this formula, The loss function used to train the base classification model (which can be called the base loss function); The cross-entropy loss function; x ∈D, where D is the first sample set. x y represents the first input data in the first sample set; y represents the label data corresponding to the first input data. f (.) represents the basic classification model.

[0041] It is understood that this disclosure allows the first input data to be input into a base classification model, and the base classification model to output a base classification result (i.e., f ( x ), thereby determining the basic classification results and label data (i.e. y The cross-entropy loss function between the two is used to train the base classification model to obtain the first classification model.

[0042] The first classification model exhibits good classification accuracy on common training samples (such as the first training sample), but its classification accuracy is relatively low on noisy training samples (such as training samples containing second input data, adversarial samples, etc.). The main goal of subsequent work in this disclosure is to improve the classification performance of the model on noisy samples while ensuring that its classification performance on common samples is not affected.

[0043] S303, Determine the second input data.

[0044] In some implementations, determining second input data corresponding to the first input data based on the first input data and the tag data includes: Based on the first input data and label data, construct the prompt information; The prompt information is input into the large model, and the large model injects semantic noise into the first input data to obtain the second input data; The injected semantic noise includes at least one of style transfer, colloquial rewriting, and context reconstruction; the semantic similarity between the second input data and the first input data is less than or equal to a preset threshold; and the label data corresponding to the second input data is the same as the label data corresponding to the first input data.

[0045] This step involves providing noise-injected samples—training samples containing the second input data—to enhance the robustness of the model during training.

[0046] To address the issue of semantic uniformity in traditional data augmentation, this disclosure utilizes the powerful generative capabilities of Large Language Model (LLM) or Vision-Language Model (VLM) to inject semantic noise into the first input data. Here, the semantic noise is not random characters or random interference, but rather, while keeping the label data unchanged, it involves at least one of the following: style transfer, colloquial rewriting, and background reconstruction of the first input data.

[0047] In one example, style transfer can refer to converting the style of the first input data to another style to generate the second input data while preserving the core semantic content of the first input data; colloquial rewriting can refer to converting the more formal and written expressions in the first input data into everyday colloquial expressions to generate the second input data. Generally, colloquial expressions can contain more slang, colloquialisms, and abbreviations; context reconstruction can include changing the background information of the first input data while ensuring that its core semantics are not affected to obtain the second input data.

[0048] In this embodiment of the disclosure, in order to inject semantic noise into the first input data using a large model, the disclosure requires constructing a prompt message input into the large model.

[0049] In one example, this disclosure can design a prompt text that includes a classification task definition, original samples, and rewriting requirements. The original samples may include first input data that needs to be injected with semantic noise and label data used to constrain the categories. Furthermore, this disclosure can use the prompt text to require a large model to generate second input data that is semantically similar to the first input data but has significantly different expressions. In one example, this disclosure further determines the second input data by using a preset threshold for semantic similarity. This preset threshold can be a pre-defined semantic similarity threshold used to measure the degree of semantic similarity between the first and second input data. In this example, the semantic similarity between the first and second input data must be less than or equal to the preset threshold to determine the second input data that is semantically similar to the first input data. Based on this, the label data corresponding to the second input data can be considered the same as the label data corresponding to the first input data. It is understood that the process of generating the second input data can be considered as a process of injecting semantic noise.

[0050] In this embodiment, a prompt message containing first input data and label data can be input into a large model, which then outputs second input data corresponding to the first input data. Furthermore, this disclosure can utilize the second input data and the corresponding label data to construct an enhanced training sample D' with noise injection, i.e. ,in Indicates the first i The second input data, Indicates the first i Individual label data.

[0051] For example, in a text classification task, there can be two categories: "positive reviews" and "negative reviews" (i.e., label data). For a negative review text "This store is really expensive" (i.e., the first input data), the prompt text input to the large model can be "Generate a variant that is semantically similar to the negative review text 'This store is really expensive,' but with a huge difference in expression." The output content of the large model (i.e., the second input data) can be "Using this store's coupon, the price is cheaper than the original price, but it is still not worth the discounted price."

[0052] Understandably, if the first input data is "positive", then the second input data will also be "positive"; if the first input data is "negative", then the second input data will also be "negative".

[0053] By using the above method, the injection of semantic noise simulates various complex changes that may occur in the data in the application scenario, increases the difficulty of classifying the category of the classification model, enables the model to deeply explore the essential features of the data during the training process, and the classification model trained in this way can adapt to input data with different styles, expressions and backgrounds, enhances the ability to recognize semantic variants, and thus improves the classification stability and robustness of the model in complex and ever-changing environments.

[0054] In this embodiment of the disclosure, Figure 3 There are no specific restrictions on the execution order of steps S302 and S303. For example, step S302 can be executed before S303, or step S302 can be executed after S303, or step S302 can be executed synchronously with S303.

[0055] To improve the robustness and stability of the target classification model, this disclosure adopts a phased training strategy, such as... Figure 3 As shown, the training process of the target classification model also includes the following steps.

[0056] S304. Based on the second input data, train the first classification model to obtain the second classification model.

[0057] During the training process of the basic classification model, the first classification model can show a relatively ideal classification effect for ordinary samples, but its classification performance is poor when faced with difficult samples.

[0058] In this training phase, based on the first classification model, we incorporate hard samples (i.e., training samples containing the second input data) generated by semantic noise injection using a large model to further train the first classification model. The core purpose of this is to help the first classification model adapt to the input data with semantic noise first, thereby improving the classification ability of the second classification model obtained after training the first classification model on hard samples.

[0059] In some implementations, the first classification model is trained using the second input data and label data, including: The second input data is input into the first classification model, and the first classification model outputs the first classification result. Based on the label data corresponding to the first classification result and the second input data, determine the first loss function; The first classification model is trained using the first loss function.

[0060] In one example, this disclosure may input second input data into a first classification model, which uses a pre-set algorithm and parameters to extract, analyze and process features from the second input model, and finally output a first classification result.

[0061] In this example, the label data can be known correct classification information, which provides a supervision signal for training the first classification model. The first loss function is a function used to measure the degree of difference between the first classification result and the label data. The greater the degree of difference, the larger the value of the first loss function; the smaller the degree of difference, the smaller the value of the first loss function.

[0062] This disclosure can calculate a specific loss value by substituting the comparison result between the first classification result and the label data corresponding to the second input data into the corresponding calculation formula, based on a pre-selected loss function type. This loss value constitutes the first loss function. In one example, this disclosure can use cross-entropy loss to calculate the first loss function. (2) In this formula, The first loss function; The second input data, where The aforementioned augmented training sample D', i.e. ; The first classification result is represented by y; the second input data is represented by y. This represents the cross-entropy loss.

[0063] Furthermore, this disclosure allows the first loss function to be fed back to the optimization algorithm. Based on the gradient information of the first loss function, the optimization algorithm calculates the direction and magnitude (or adjustment method) of adjusting the parameters of the first classification model. According to the calculated adjustment method, the parameters of the first classification model are updated, enabling the first classification model to output a first classification result closer to the true label (i.e., the label data) when faced with similar second input data in the next iteration. This process can be repeated multiple times until the value of the first loss function converges to a preset requirement or a preset number of training epochs are reached, ultimately yielding the second classification model.

[0064] By employing the above method, the first classification model is exposed to more challenging and diverse second input data during training, simulating various complex changes that may occur in input data in real-world application scenarios. This training phase enables the second classification model to deeply mine the essential characteristics of the data, reducing its reliance on homogenized data. Consequently, it can stably and reliably classify data even when faced with noise interference and complex data distributions, thus improving the robustness of the second classification model.

[0065] like Figure 3 As shown, the training process of the target classification model also includes the following steps.

[0066] S305. Train the second classification model to obtain the target classification model.

[0067] In this training phase, the present disclosure can use third input data (including first input data (ordinary input data or input data without semantic noise) and second input data (input data with semantic noise)) and label data to train the second classification model. At the same time, during the training process, preset feature noise is injected into the data features extracted by the second classification model, forcing the second classification model to learn a robust representation that is insensitive to feature variation.

[0068] In some implementations, a second training of the second classification model is performed using third input data and label data, including: The third input data is input into the second classification model, and the second classification model extracts the data features of the third input data. Based on data features, a second classification model is used to determine a second predicted classification result for the third input data; and, Based on data features and preset feature noise, the third predicted classification result is determined for the third input data using the second classification model; The second classification model is trained a second time using the second and third predicted classification results and the label data corresponding to the third input data.

[0069] In this embodiment of the disclosure, the third input data can be input into the second classification model and passed sequentially along the direction of the input layer and hidden layer of the second classification model. The neurons of each hidden layer perform nonlinear transformation on the third input data (which may be the first input data or the second input data) according to the preset activation function to extract the corresponding features. These features are continuously passed and combined between the hidden layers to gradually form data features that can represent the essential information of the input data.

[0070] In one example, this disclosure can utilize neurons in the output layer of a second classification model to make classification decisions based on data features. Generally, the number of neurons in the output layer corresponds to the number of categories in the classification task; each neuron can represent a category, and its output value can represent the probability or score of the input data belonging to that category. The second classification model can determine the final predicted classification result (i.e., the second predicted classification result for the third input data) based on these output values ​​using preset decision rules (such as the maximum probability principle, threshold judgment, etc.).

[0071] In another example, this disclosure can enhance the robustness of the second classification model by injecting preset feature noise into the feature extraction layer during the forward data propagation. For example, a preset random perturbation (i.e., preset feature noise) can be superimposed on the data features output by the feature extraction layer (such as a convolutional layer or attention mechanism layer), enabling the second classification model to learn robust feature representations that are invariant to feature variations during the training phase, thereby improving its generalization ability under noise interference.

[0072] Based on the foregoing, in this example, the present disclosure can inject preset feature noise into the data features, and then input the data features with preset feature noise back into the second classification model, and then determine the third predicted classification result for the third input data according to the classification decision inside the second classification model.

[0073] In some implementations, the preset feature noise includes the product of a noise matrix and the standard deviation of the data features, wherein the noise matrix has the same dimension as the data features.

[0074] In this embodiment of the disclosure, the preset feature noise can be composed of a noise matrix and the standard deviation of the data features. The dimension of the noise matrix is ​​the same as the dimension of the data features to be injected with noise; for example, if the data features are a 16×8 matrix, then the dimension of the noise matrix should also be 16×8. The standard deviation of the data features reflects the degree of dispersion of the data features in each dimension. Multiplying it by the noise matrix makes the amplitude of the generated noise proportional to the fluctuation range of the data features themselves.

[0075] In one example, the process of injecting pre-defined feature noise into data features can be formulated as follows: (3) In this formula, F These are the data features extracted by the second classification model from the input data (either the first or second input data); These are the data features after injecting preset characteristic noise; It is a noise matrix; This is the standard deviation of the data features. The data features and the preset feature noise are represented by matrices of the same dimension, and the resulting data features after injecting the preset feature noise also have the same dimension.

[0076] In some implementations, the noise matrix includes multiple noise elements, which are uniformly distributed noise within a preset range.

[0077] In this embodiment of the disclosure, the noise matrix is ​​composed of multiple noise elements arranged according to a preset dimension. Each noise element is generated independently and follows a uniform distribution within a preset range.

[0078] Specifically, the noise element is the basic building block of the noise matrix, and its value is obtained through uniformly distributed random sampling. That is, the matrix elements are uniformly distributed noise within a preset range. Here, the characteristic of uniformly distributed noise is that within the preset range (b1, b2), the probability density of all possible values ​​is equal, where b1 and b2 are the upper and lower limits of the possible values ​​of the noise element, respectively.

[0079] In this embodiment of the disclosure, That is, the values ​​of the noise elements in the noise matrix range from -a to a. In one example, a can be 0.1. U (.) represents uniformly distributed noise. For example, if the noise element is set to uniformly distributed noise in the range (-0.1, 0.1), then each noise element will fall within this interval with equal probability, forming unbiased random interference.

[0080] In formula (3), the product of the noise matrix and the standard deviation of the data features (i.e., the preset feature noise) can be understood as the result of multiplying each noise element in the noise matrix with the standard deviation of the data features.

[0081] Using the above method, noise intensity is directly controlled by independent parameters. By adjusting uniformly distributed parameters, the interference amplitude can be linearly adjusted to meet the differentiated noise intensity requirements at different training stages. Simultaneously, by dynamically optimizing the distribution of noise elements based on the statistical properties of data features (such as standard deviation), a linear mapping relationship between noise intensity and data features can be established by quantifying the dispersion of feature dimensions. This allows the model to exhibit adaptive characteristics to feature perturbations during training, improving the robustness of model training.

[0082] Furthermore, this disclosure can utilize the second predicted classification result, the third predicted classification result, and the label data corresponding to the third input data to perform a second training on the second classification model to obtain the target classification model.

[0083] By employing the above method, this disclosure utilizes the degree of difference between the second predicted classification result and the true label (i.e., label data) to optimize the basic classification ability of the second classification model. At the same time, by utilizing the degree of difference between the third predicted classification result and the true label, the second classification model can be equipped with the ability to distinguish between key features and noisy features, reducing the dependence of the second classification model on data features, improving the robustness of the second classification model, and thus improving the reliability and accuracy of the trained target classification model in classifying data.

[0084] The following content details the process of performing a second training on the second classification model.

[0085] In some implementations, the second classification model is trained a second time using the second predicted classification result, the third predicted classification result, and the label data corresponding to the third input data, including: Based on the second and third predicted classification results, and the label data corresponding to the third input data, the second loss function is determined. The second loss function is used to perform a second training on the second classification model.

[0086] In this embodiment, the second loss function can be a composite loss function. For example, this disclosure can calculate one loss function using the second predicted classification result and label data, and calculate another loss function using the third predicted classification result and label data; or, this disclosure can calculate one loss function using the second predicted classification result and label data, and calculate another loss function using the second and third predicted classification results. Furthermore, this disclosure can use the above two loss functions, and the weights corresponding to each loss function, to calculate the second loss function.

[0087] In this embodiment of the disclosure, during the second training of the second classification model using the second loss function, gradient updates can be applied simultaneously to the feature extraction layer and the classification head of the second classification model, thereby obtaining the target classification model.

[0088] By adopting the above method, the second loss function can simultaneously constrain the performance of the second classification model in terms of classification accuracy and noise handling robustness, so that the trained target classification model has basic classification ability and noise interference resistance, thereby improving the robustness of the target classification model.

[0089] In some implementations, a second loss function is determined based on the second predicted classification result, the third predicted classification result, and the label data corresponding to the third input data, including: Using the label data corresponding to the second predicted classification result and the third input data, determine the first sub-loss; and, The second sub-loss is determined using the second and third predicted classification results; The second loss function is determined based on the first and second sub-losses.

[0090] In this embodiment of the disclosure, the first sub-loss can be used to measure the degree of difference between the second predicted classification result and the label data; the second sub-loss can be used to measure the degree of difference between the second predicted classification result and the third predicted classification result.

[0091] In some implementations, the first sub-loss is determined using the label data corresponding to the second predicted classification result and the third input data, including: Determine the cross-entropy loss between the second predicted classification result and the label data; The cross-entropy loss is used as the first sub-loss.

[0092] In the second training, in order to maintain the stability of the training without losing the performance of the second classification model on ordinary samples (i.e., training samples containing the first input data), this disclosure can increase the relative entropy loss (also known as Kullback-Leibler (KL) divergence) to constrain the classification difference of the second classification model before and after injecting preset feature noise into the data features.

[0093] In some implementations, the second sub-loss is determined using the second and third predicted classification results, including: Determine the relative entropy loss between the second and third predicted classification results; The relative entropy loss is used as the second sub-loss.

[0094] It should be noted that this disclosure can utilize specific functions to determine the cross-entropy loss between the second predicted classification result and the label data, as well as the relative entropy loss between the second predicted classification result and the third predicted classification result.

[0095] Based on this, the formula for calculating the second loss function can be: (4) In this formula, This is the second loss function; (.) represents the cross-entropy loss, i.e., the first sub-loss; This is the relative entropy loss (or KL divergence), i.e., the second sub-loss; n The input data is either the first or the second input data, i.e., n∈(D∪D'), in other words, n can be any of the aforementioned... x It can also be the aforementioned x’ ; y for n Corresponding tag data; It is the parameter that harmonics the cross-entropy loss and the relative entropy loss, and can also be considered as the weight of the relative entropy loss; f ( n The second predicted classification result is the classification result that does not inject preset feature noise into the data features. f’ ( n The third prediction classification result is the classification result corresponding to the pre-defined feature noise injected into the data features.

[0096] Using the above methods, cross-entropy loss can directly constrain the consistency between the second predicted classification result and the label data, providing a stable direction for training the basic classification ability of the second classification model; relative entropy loss, by constraining the consistency between the second and third predicted classification results, provides a training objective for noise processing in the second classification model. The combined effect of these two losses improves the stability of training the second classification model, thereby enhancing the robustness of the trained target classification model.

[0097] This disclosure determines the target distribution of the classification task through a first sub-loss and introduces adversarial noise signals through a second sub-loss, which improves the stability of the classification boundary under the second loss function and enhances the robustness and generalization ability of the target classification model in complex scenarios.

[0098] This disclosure proposes a training method for classification models to improve their robustness, the advantages of which include: (1) Multi-level robustness enhancement: By injecting noise into the input layer (semantic level) and the feature layer (abstract level), the classification model can not only resist data-level perturbations, but also learn more robust features from a higher level.

[0099] (2) Large model empowerment: The semantic noise data generated by the large model can be closer to the real scene than the traditional data augmentation method, which improves the classification model's ability to identify "difficult" samples.

[0100] (3) Stable and efficient training: The multi-stage training strategy reduces the training non-convergence caused by excessive noise, so that the classification model can improve its robustness without reducing the accuracy of classifying ordinary samples.

[0101] Through the aforementioned multi-stage noise injection, the classification model maintains high classification accuracy on ordinary samples while improving its robustness to input noise and adversarial attacks. For example, in text classification tasks, this disclosure improves the accuracy by 15% on samples to be classified that are all noisy. Furthermore, the training method for the classification model proposed in this disclosure can be extended to various types of tasks such as image classification and speech recognition.

[0102] This disclosure also proposes a classification method. Figure 4 This is a flowchart illustrating the implementation of a classification method according to an embodiment of the present disclosure, including: S410. Obtain the data to be classified; S420. Input the data to be classified into the target classification model, and the target classification model outputs the category information of the data to be classified. The target classification model is trained using the training method proposed in this disclosure.

[0103] In this embodiment of the disclosure, the data to be classified may include raw data samples that have not yet been assigned explicit category labels, and are essentially the processing objects of the target classification model. In one example, the data to be classified may include the following properties: (1) Unlabeled, that is, the category has not been pre-labeled by human or algorithm (such as "cat", "bird" or "good review", "bad review" etc.); (2) Multimodality, that is, the data to be classified can be data in multiple modalities such as text, image, and audio.

[0104] Furthermore, this disclosure allows the data to be classified to be input into a target classification model, which then outputs the category information of the data. In one example, the target classification model can convert the input data into a high-dimensional feature vector using structures such as convolutional layers and attention mechanisms; then, based on fully connected layers and a classification head, the high-dimensional feature vector is used to calculate the scores of each category of the input data, thereby determining the category information of the data.

[0105] Because this disclosure employs a hierarchical noise injection strategy in the input layer and feature extraction layer during the training phase, the target classification model learns the dual adversarial ability against data perturbation and feature anomalies. Therefore, using the target classification model to determine the category information of the data to be classified can improve the accuracy and stability of data classification.

[0106] This disclosure also proposes a training device for a classification model. Figure 5 This is a schematic diagram of the structure of a classification model training device 500 according to an embodiment of the present disclosure, including: The first acquisition module 510 is used to acquire a first sample set, which includes multiple first training samples, and the first training samples include first input data and label data. The data determination module 520 is used to determine the second input data corresponding to the first input data based on the first input data and the label data; The first training module 530 is used to perform first training on the first classification model using the second input data and label data to obtain the second classification model; The second training module 540 is used to perform a second training on the second classification model using the third input data and label data to obtain the target classification model; wherein, during the second training process, preset feature noise is injected into the data features extracted by the second classification model; the third input data includes the first input data and the second input data.

[0107] In some implementations, the first training module 530 is used for: The second input data is input into the first classification model, and the first classification model outputs the first classification result. Based on the label data corresponding to the first classification result and the second input data, determine the first loss function; The first classification model is trained using the first loss function.

[0108] In some implementations, the second training module 540 is used for: The third input data is input into the second classification model, and the second classification model extracts the data features of the third input data. Based on data features, a second classification model is used to determine a second predicted classification result for the third input data; and, Based on data features and preset feature noise, the third predicted classification result is determined for the third input data using the second classification model; The second classification model is trained a second time using the second and third predicted classification results and the label data corresponding to the third input data.

[0109] In some implementations, the second training module 540 is used for: Based on the second and third predicted classification results, and the label data corresponding to the third input data, the second loss function is determined. The second loss function is used to perform a second training on the second classification model.

[0110] In some implementations, the second training module 540 is used for: Using the label data corresponding to the second predicted classification result and the third input data, determine the first sub-loss; and, The second sub-loss is determined using the second and third predicted classification results; The second loss function is determined based on the first and second sub-losses.

[0111] In some implementations, the second training module 540 is used for: Determine the cross-entropy loss between the second predicted classification result and the label data; The cross-entropy loss is used as the first sub-loss.

[0112] In some implementations, the second training module 540 is used for: Determine the relative entropy loss between the second and third predicted classification results; The relative entropy loss is used as the second sub-loss.

[0113] In some implementations, the preset feature noise includes the product of the noise matrix and the standard deviation of the data feature, wherein the noise matrix and the data feature have the same dimension.

[0114] In some implementations, the noise matrix includes multiple noise elements, which are uniformly distributed noise within a preset range.

[0115] In some implementations, the data determination module 520 is used for: Based on the first input data and label data, construct the prompt information; The prompt information is input into a large model, which injects semantic noise into the first input data to obtain the second input data. The injected semantic noise includes at least one of style transfer, colloquial rewriting, and background reconstruction. The semantic similarity between the second input data and the first input data is less than or equal to a preset threshold. The label data corresponding to the second input data is the same as the label data corresponding to the first input data.

[0116] This disclosure also proposes a classification device. Figure 6 This is a schematic diagram of the structure of a sorting device 600 according to an embodiment of the present disclosure, including: The second acquisition module 610 is used to acquire the data to be classified. The data classification module 620 is used to input the data to be classified into the target classification model, and the target classification model outputs the category information of the data to be classified. The target classification model is trained using the training device proposed in this disclosure.

[0117] The specific functions and examples of each module and submodule of the apparatus in this disclosure can be found in the relevant descriptions of the corresponding steps in the above method embodiments, and will not be repeated here.

[0118] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0119] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0120] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0121] likeFigure 7 As shown, device 700 includes a computing unit 701, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 702 or a computer program loaded into random access memory (RAM) 703 from storage unit 708. The RAM 703 may also store various programs and data required for the operation of device 700. The computing unit 701, ROM 702, and RAM 703 are interconnected via bus 704. Input / output (I / O) interface 705 is also connected to bus 704.

[0122] Multiple components in device 700 are connected to I / O interface 705, including: input unit 706, such as keyboard, mouse, etc.; output unit 707, such as various types of monitors, speakers, etc.; storage unit 708, such as disk, optical disk, etc.; and communication unit 709, such as network card, modem, wireless transceiver, etc. Communication unit 709 allows device 700 to exchange / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0123] The computing unit 701 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as training methods and classification methods for classification models. For example, in some embodiments, the training methods and classification methods for classification models can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed on device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of the training methods and classification methods for classification models described above can be performed. Alternatively, in other embodiments, the computing unit 701 may be configured by any other suitable means (e.g., by means of firmware) to perform a training method or a classification method for a classification model.

[0124] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0125] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0126] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0127] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0128] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0129] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0130] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0131] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for training a classification model, comprising: Obtain a first sample set, which includes multiple first training samples, and the first training samples include first input data and label data; Based on the first input data and the label data, determine the second input data corresponding to the first input data; Using the second input data and the label data, the first classification model is trained to obtain the second classification model; Using the third input data and the label data, the second classification model is trained a second time to obtain the target classification model; wherein, during the second training process, preset feature noise is injected into the data features extracted by the second classification model; The third input data includes the first input data and the second input data.

2. The method according to claim 1, wherein, The step of using the second input data and the label data to perform a first training on the first classification model includes: The second input data is input into the first classification model, and the first classification model outputs the first classification result. Based on the label data corresponding to the first classification result and the second input data, a first loss function is determined; The first classification model is trained using the first loss function.

3. The method according to claim 1, wherein, The second training of the second classification model using the third input data and the label data includes: The third input data is input into the second classification model, and the second classification model extracts the data features of the third input data. Based on the data features, using the second classification model, a second predicted classification result is determined for the third input data; and, Based on the data features and the preset feature noise, the second classification model is used to determine the third predicted classification result for the third input data; The second classification model is trained a second time using the second predicted classification result, the third predicted classification result, and the label data corresponding to the third input data.

4. The method according to claim 3, wherein, The step of using the second predicted classification result, the third predicted classification result, and the label data corresponding to the third input data to perform a second training on the second classification model includes: Based on the second predicted classification result, the third predicted classification result, and the label data corresponding to the third input data, a second loss function is determined; The second classification model is then trained a second time using the second loss function.

5. The method according to claim 4, wherein, The step of determining the second loss function based on the second predicted classification result, the third predicted classification result, and the label data corresponding to the third input data includes: Using the label data corresponding to the second predicted classification result and the third input data, a first sub-loss is determined; and, The second sub-loss is determined using the second and third prediction classification results; The second loss function is determined based on the first sub-loss and the second sub-loss.

6. The method according to claim 5, wherein, The step of determining the first sub-loss using the label data corresponding to the second predicted classification result and the third input data includes: Determine the cross-entropy loss between the second predicted classification result and the label data; The cross-entropy loss is used as the first sub-loss.

7. The method according to claim 5, wherein, The step of determining the second sub-loss using the second prediction classification result and the third prediction classification result includes: Determine the relative entropy loss between the second predicted classification result and the third predicted classification result; The relative entropy loss is used as the second sub-loss.

8. The method according to any one of claims 3-7, wherein, The preset feature noise includes the product of the noise matrix and the standard deviation of the data feature, wherein the noise matrix and the data feature have the same dimension.

9. The method according to claim 8, wherein, The noise matrix includes multiple noise elements, which are uniformly distributed noise within a preset range.

10. The method according to claim 1, wherein, The step of determining the second input data corresponding to the first input data based on the first input data and the tag data includes: Based on the first input data and the label data, a prompt message is constructed; The prompt information is input into a large model, which injects semantic noise into the first input data to obtain the second input data. The injected semantic noise includes at least one of style transfer, colloquial rewriting, and background reconstruction. The semantic similarity between the second input data and the first input data is less than or equal to a preset threshold. The label data corresponding to the second input data is the same as the label data corresponding to the first input data.

11. A classification method, comprising: Obtain the data to be classified; The data to be classified is input into the target classification model, and the target classification model outputs the category information of the data to be classified. The target classification model is trained by any of the training methods described in claims 1-10.

12. A training device for a classification model, comprising: The first acquisition module is used to acquire a first sample set, the first sample set including multiple first training samples, the first training samples including first input data and label data; The data determination module is used to determine the second input data corresponding to the first input data based on the first input data and the label data; The first training module is used to perform a first training on the first classification model using the second input data and the label data to obtain a second classification model; The second training module is used to perform a second training on the second classification model using the third input data and the label data to obtain the target classification model; wherein, during the second training process, preset feature noise is injected into the data features extracted by the second classification model; The third input data includes the first input data and the second input data.

13. A sorting device, comprising: The second acquisition module is used to acquire the data to be classified. The data classification module is used to input the data to be classified into the target classification model, and the target classification model outputs the category information of the data to be classified. The target classification model is trained by the training device described in claim 12.

14. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-11.

15. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-11.

16. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-11.

Images