An enhanced processing method, device, and equipment for image classification and a medium

Abstract

The present application relates to the technical field of artificial intelligence, and in particular to an image classification enhancement processing method and device, equipment and medium. The above method is applied to the medical field, obtains a pre-training model corresponding to a previous task, a pre-training generator, real data corresponding to a current task, and real label values corresponding to the real data, uses pseudo data generated based on the pre-training generator and the real data as training data, trains a model in the current task using the training data, obtains an initial pre-training model in the current task, performs enhancement training on the initial pre-training model, and determines a trained model as a target pre-training model. In the present application, training with historical pseudo data ensures the stability of the feature distribution of the model, knowledge distillation is used to minimize the difference between the pseudo data in the adjacent task models, the feature deviation caused by the pseudo data is reduced, and the output accuracy of the target pre-training model in the current task obtained by training is improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to an enhanced processing method, apparatus, device, and medium for image classification. Background Technology

[0002] With the development of computer technology, automatic image recognition and classification through software algorithms has been initially achieved. In the medical field, neural networks are generally used for the detection and classification of medical images. Image classification neural networks typically consist of two parts: a feature extraction subnetwork and a classification subnetwork. The feature extraction network takes the input image and feeds it through a series of feature extraction layers, ultimately outputting the extracted features. The classification network is usually a fully connected layer that takes the features extracted by the feature extraction network and inputs them, outputting the class probability. In conventional incremental learning, the structure of the feature extraction network is usually not changed; only the number of output classes in the fully connected layer is increased based on the new data's class. Since the model is no longer trained on old class data but only on new classes, the fully connected layer tends to assign high prediction values ​​to the new classes. This leads to the model assigning high scores to the new classes when predicting data from the old classes, resulting in prediction failure. Meanwhile, the features extracted by the feature extractor are more inclined to extract features that are more discriminative to new categories. Therefore, the discriminative power of newly learned features in old categories decreases as incremental learning progresses, resulting in lower accuracy of the new model's output. Thus, improving the accuracy of the new model's output in incremental learning has become an urgent problem to be solved. Summary of the Invention

[0003] Therefore, it is necessary to provide an image classification enhancement processing method, apparatus, device, and medium to address the aforementioned technical problems, in order to solve the problem of low accuracy of the new model output results under enhancement processing.

[0004] A first aspect of this application provides an image classification enhancement processing method, the enhancement processing method comprising:

[0005] Obtain the pre-trained model and pre-trained generator corresponding to the previous image classification task, as well as the real image and the real label value corresponding to the real image for the current image classification task.

[0006] The pseudo data generated based on the pre-trained generator and the real image are used as training data. The training data is used to train the model in the current image classification task to obtain the initial pre-trained model in the current image classification task.

[0007] The pseudo data is input into the initial pre-trained model, and a corresponding first result is output. The pseudo data is input into the pre-trained model, and a corresponding second result is output. The first loss between the first result and the second result is calculated.

[0008] The real image is input into the initial pre-trained model, and a corresponding third result is output. The second loss between the third result and the real label value corresponding to the real image is calculated.

[0009] Based on the first loss and the second loss, the initial pre-trained model is enhanced and trained to determine the trained model as the target pre-trained model. The target pre-trained model is then used to classify the image to be processed to obtain the image classification result.

[0010] A second aspect of this application provides an image classification enhancement processing apparatus, the enhancement processing apparatus comprising:

[0011] The acquisition module is used to acquire the pre-trained model and pre-trained generator corresponding to the previous image classification task, as well as the real image and the real label value corresponding to the real image for the current image classification task.

[0012] The initial pre-trained model determination module is used to use the pseudo data generated based on the pre-training generator and the real image as training data, and use the training data to train the model in the current image classification task to obtain the initial pre-trained model in the current image classification task.

[0013] The first loss determination module is used to input the pseudo data into the initial pre-trained model, output a corresponding first result, input the pseudo data into the pre-trained model, output a corresponding second result, and calculate a first loss between the first result and the second result;

[0014] The second loss determination module is used to input the real image into the initial pre-trained model, output a corresponding third result, and calculate the second loss between the third result and the real label value corresponding to the real image.

[0015] The enhancement training module is used to enhance the initial pre-trained model based on the first loss and the second loss, determine the trained model as the target pre-trained model, and use the target pre-trained model to classify the image to be processed to obtain the image classification result.

[0016] Thirdly, embodiments of the present invention provide a computer device, the computer device including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the enhanced processing method as described in the first aspect.

[0017] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the enhanced processing method as described in the first aspect.

[0018] The advantages of this invention compared to the prior art are:

[0019] This invention obtains the pre-trained model and pre-trained generator corresponding to the previous image classification task, as well as the real image and its corresponding real label value for the current image classification task. Pseudo-data generated by the pre-trained generator and the real image are used as training data to train the model for the current image classification task, resulting in an initial pre-trained model. Pseudo-data is then input into the initial pre-trained model, outputting a first result. Pseudo-data is then input into the pre-trained model, outputting a second result. A first loss is calculated between the first and second results. The real image is then input into the initial pre-trained model, outputting a third result. A second loss is calculated between the third result and the corresponding real label value for the real image. Based on the first and second losses, the initial pre-trained model is augmented to determine the trained model as the target pre-trained model. The target pre-trained model is then used to classify the image to be processed, obtaining the image classification result. In this invention, training with historical pseudo-data ensures the stability of the model's feature distribution. Knowledge distillation minimizes the differences between these pseudo-data and models in adjacent tasks, reducing feature bias caused by pseudo-data and thus improving the output accuracy of the target pre-trained model for the current image classification task. In the medical field, to improve the classification accuracy of image classification models, it is necessary to update and train the classification models with new medical images so that the models can adapt to the classification of images of various new symptoms. Due to user privacy, more new images are not available when training the image classification models. Using a target pre-trained model trained with pre-trained models, pseudo data, and real data for image classification reduces the feature bias caused by pseudo data, thereby improving the output accuracy of the target pre-trained model in the current image classification task. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1This is a schematic diagram of an application environment for an image classification enhancement processing method provided in an embodiment of the present invention;

[0022] Figure 2 This is a schematic flowchart of an image classification enhancement processing method provided in an embodiment of the present invention;

[0023] Figure 3 This is a schematic flowchart of an image classification enhancement processing method provided in an embodiment of the present invention;

[0024] Figure 4 This is a schematic diagram of the structure of an image classification enhancement processing device provided in an embodiment of the present invention;

[0025] Figure 5 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0028] It should also be understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0029] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0030] Furthermore, in the description of this invention and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0031] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of the invention include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0032] It should be understood that the sequence number of each step in the following embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0033] To illustrate the technical solution of the present invention, specific embodiments are described below.

[0034] An embodiment of the present invention provides an image classification enhancement processing method, which can be applied to, for example... Figure 1 In this application environment, the client communicates with the server. Clients include, but are not limited to, PDAs, desktop computers, laptops, ultra-mobile personal computers (UMPCs), netbooks, and personal digital assistants (PDAs). The server can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0035] In this embodiment, a pre-trained model and a pre-trained generator for image classification are obtained from the medical system. For example, a model for classifying gastroscopy images and a generator trained using historical image data are used. The pre-trained generator generates pseudo-data, which is data similar to historical data in the classification task of the pre-trained model. The model in the current image classification task is trained using the pseudo-data and real images, where the real images are the gastroscopy images to be classified at the current moment. An initial pre-trained model for the current image classification task is obtained. The pseudo-data is input into the initial pre-trained model, and a first result is output. The pseudo-data is input into the pre-trained model, and a second result is output. A first loss between the first and second results is calculated. The real image is input into the initial pre-trained model, and a third result is output. A second loss between the third result and the real label value corresponding to the real image is calculated. Based on the first and second losses, the initial pre-trained model is enhanced and trained to determine the trained model as the target pre-trained model. The target pre-trained model is used to classify the images to be processed, and the classification result of the gastroscopy image to be classified is obtained.

[0036] See Figure 2 This is a schematic flowchart of an image classification enhancement processing method provided in an embodiment of the present invention. The above-described image classification enhancement processing method can be applied to... Figure 1 The server in the above-mentioned configuration connects to the corresponding client, such as... Figure 2 As shown, the image classification enhancement processing method may include the following steps.

[0037] S201: Obtain the pre-trained model and pre-trained generator corresponding to the previous image classification task, as well as the real image and the real label value of the corresponding real image for the current image classification task.

[0038] In step S201, since the training data cannot be obtained all at once or cannot be stored in its entirety, training needs to be carried out in batches step by step. Each batch is called a task, and each task is interconnected. The pre-trained model and pre-trained generator corresponding to the previous image classification task are obtained. The pre-trained model is the model that processes the data in the previous image classification task in the previous task. The pre-trained generator is used to generate pseudo data similar to the data in the previous image classification task. The real image corresponding to the previous task and the real label value of the corresponding real image are also obtained so that the model in the current image classification task can be trained using the real image in the current image classification task.

[0039] In this embodiment, image classification is performed. Due to the different pre-trained models in each task, the image classification results vary. To continuously improve the model's accuracy in image classification, this embodiment uses augmentation. Augmentation is a learning paradigm that continuously learns new knowledge from new samples while retaining most of the previously learned knowledge; it can be considered a form of continuous or lifelong learning. From the perspective of whether or not data from previous tasks is retained, augmentation can be divided into memory-based augmentation and data-free augmentation. Memory-based augmentation retains a portion of the real images from previous tasks, called the core set or memory register. While learning new tasks, this historical data is re-recorded to avoid forgetting old knowledge. Data-free augmentation does not store any historical data and typically uses synthetic pseudo-data or generated data for transfer learning of old knowledge. This embodiment uses data-free augmentation, obtaining more training samples by acquiring the pre-trained generator from the previous image classification task. To maintain learning for the current image classification task, it is also necessary to obtain the corresponding real images and their corresponding real label values ​​from the current image classification task.

[0040] S202: Using pseudo-data generated by the pre-trained generator and real images as training data, the model in the current image classification task is trained using the training data to obtain the initial pre-trained model in the current image classification task.

[0041] In step S202, when training the model in the current image classification task, it is necessary to determine the training data. The pseudo data generated by the pre-training generator and the real images are used as training data. Using the pseudo data generated by the pre-training generator in the previous image classification task as part of the training data can effectively avoid the feature distribution shift problem and diversify the training data.

[0042] In this embodiment, a pre-training generator is used to generate pseudo data, which is data similar to historical data in the previous image classification task. Directly using the pre-training generator to generate pseudo data reduces the storage of historical data. The pseudo data and the real images in the current image classification task are used as training data. The training data is used to train the model in the current image classification task to obtain the initial pre-trained model in the current image classification task.

[0043] It should be noted that if the current image classification task is the first task, there is no pre-trained generator, and the training data is the real images in the current image classification task.

[0044] Optionally, pseudo-data generated by the pre-trained generator and real images are used as training data. The training data is then used to train the model for the current image classification task, resulting in an initial pre-trained model for the current image classification task, including:

[0045] The pseudo data generated by the pre-trained generator is input into the pre-trained model, and the pseudo label value corresponding to the pseudo data is output. The pseudo label value corresponding to the pseudo data is labeled.

[0046] Using pseudo-labeled data and real images as training data, the model in the current image classification task is trained using the training data to obtain the initial pre-trained model in the current image classification task.

[0047] In this embodiment, when training the model for the current image classification task using training data, it is necessary to obtain the label value information corresponding to the training data so as to iteratively train the model for the current image classification task by using the loss between the label value and the predicted value. When obtaining the pseudo-label value in the pseudo data, the pseudo data generated based on the pre-training generator is input into the pre-training model, and the pseudo-label value corresponding to the pseudo data is output. The pre-training model is the model that classified historical data in the previous image classification task. The pseudo data generated based on the pre-training generator is similar to the historical data in the previous image classification task, so inputting the pseudo data into the pre-training model can obtain the accurate pseudo-label value corresponding to the pseudo data, as shown in the following formula:

[0048]

[0049] Where z is random noise, For a pre-trained generator, θ n-1 Here, S represents the parameters in the pre-trained model. These are the pseudo-label values ​​corresponding to the pseudo-data. This is the generated pseudo data.

[0050] Using pseudo-labeled data and real images as training data, the model in the current image classification task is trained using the training data to obtain the initial pre-trained model in the current image classification task.

[0051] S203: Input pseudo-data into the initial pre-trained model and output the corresponding first result; input pseudo-data into the pre-trained model and output the corresponding second result; calculate the first loss between the first result and the second result.

[0052] In step S203, when training the model in the current image classification task, it is necessary to obtain the training loss corresponding to the training. Pseudo-data is input into the initial pre-trained model to output the corresponding first result. Pseudo-data is input into the pre-trained model to output the corresponding second result. The first loss between the first result and the second result is calculated, wherein the first loss is used as part of the loss in model training.

[0053] In this embodiment, when training the model for the current image classification task, pseudo-data generated by the pre-training generator is used. This pseudo-data contains more diverse features and is closer to the decision boundary of image classification than real images. The pseudo-data is input into both the pre-training model and the initial pre-training model, respectively, outputting a first result and a second result. A first loss is then calculated between the first and second results.

[0054] Optionally, pseudo-data is input into the initial pre-trained model, and the corresponding first result is output, including:

[0055] The pseudo data is input into the initial pre-trained model. Based on the hidden layer in the initial pre-trained model, the first feature information corresponding to the pseudo data is output. Based on the fully connected layer in the initial pre-trained model, the first numerical information corresponding to the pseudo data is output.

[0056] The first feature information and the first numerical information are used as the first result.

[0057] In this embodiment, the pseudo-data generated by the pre-trained generator is more diverse, and the features of these samples are more aggressive, closer to the decision boundary of image classification than real data. Therefore, better knowledge distillation can be achieved on these pseudo-data samples. To further reduce the feature distribution bias, we use pseudo-data to perform feature distillation on the initial pre-trained model. The pseudo-data is input into the initial pre-trained model, and based on the hidden layers in the initial pre-trained model, the first feature information corresponding to the pseudo-data is output. Based on the fully connected layers in the initial pre-trained model, the first numerical information corresponding to the pseudo-data is output. The first feature information and the first numerical information are used as the first result.

[0058] Optionally, pseudo-data is input into the pre-trained model, and the corresponding second result is output, including:

[0059] The pseudo data is input into the pre-trained model. Based on the hidden layer in the pre-trained model, the second feature information corresponding to the pseudo data is output. Based on the fully connected layer in the pre-trained model, the second numerical information corresponding to the pseudo data is output.

[0060] The second feature information and the second numerical information are used as the second result.

[0061] In this embodiment, in order to calculate the loss in knowledge distillation and feature distillation, pseudo data is input into the pre-trained model to obtain the second feature information corresponding to the pseudo data output by the hidden layer in the pre-trained model, and the second numerical information corresponding to the pseudo data output by the fully connected layer.

[0062] Optionally, calculating the first loss between the first result and the second result includes:

[0063] Based on the first and second results, calculate the knowledge distillation loss and feature distillation loss between the initial pre-trained model and the pre-trained model;

[0064] The sum of knowledge distillation loss and feature distillation loss is used as the first loss.

[0065] In this embodiment, the characteristic distillation loss is calculated based on the first feature information in the first result and the second feature information in the second result. The calculation formula is as follows:

[0066]

[0067] Among them, L feaKD (θ n ) represents the feature distillation loss between the pre-trained model and the initial pre-trained model, x g G is based on pseudo-data generated by a pre-trained generator. n-1 To train the generator, S f (x g ;θ n ) is the second feature information, S f (x g ;θ n-1 ) is the second feature information.

[0068] The knowledge distillation loss is calculated based on the first and second numerical information in the second result, using the following formula:

[0069]

[0070] Among them, L logKD (θ n ) represents the knowledge distillation loss between the pre-trained model and the initial pre-trained model, x g G is based on pseudo-data generated by a pre-trained generator. n-1 To train the generator, G n-1 For training the generator, KL is the divergence value, S(x) g ;θ n S(x) is the initial pre-trained model. g ;θ n-1 ) is a pre-trained model.

[0071] S204: Input the real image into the initial pre-trained model, output the corresponding third result, and calculate the second loss between the third result and the real label value corresponding to the real image.

[0072] In step S204, the initial pre-trained model is trained using real images from the current image classification task, and the corresponding predicted value, i.e. the third result, is output. The second loss is calculated based on the difference between the predicted value and the corresponding real label value.

[0073] In this embodiment, the real image is input into the initial pre-trained model, and the corresponding third result is output. The third result is the prediction value of the real image by the initial pre-trained model. The second loss between the predicted value and the real label value corresponding to the real image is calculated. The formula for calculating the second loss is as follows:

[0074] S205: Based on the first loss and the second loss, perform augmentation training on the initial pre-trained model, determine the trained model as the target pre-trained model, use the target pre-trained model to classify the image to be processed, and obtain the image classification result.

[0075] In step S205, the initial pre-trained model is augmented by replacing the loss function in the initial pre-trained model, and the trained model is determined as the target pre-trained model. The initial pre-trained model is augmented based on the first loss and the second loss, which can improve the training accuracy of the initial pre-trained model.

[0076] In this embodiment, the loss between the training output result and the label value corresponding to the training data during the initial pre-training model training process is used as the first loss and the second loss. The initial pre-training model is then enhanced based on this loss until the loss converges, resulting in a trained model. The trained model is then determined as the target pre-training model. The target pre-training model is used to classify the image to be processed, and the image classification result is obtained.

[0077] Optionally, based on the first loss and the second loss, the initial pre-trained model is subjected to augmentation training to determine the trained model as the target pre-trained model, including:

[0078] By setting weight hyperparameters for the first loss and the second loss, the corresponding target loss can be obtained;

[0079] Based on the target loss, the initial pre-trained model is augmented and trained to determine the trained model as the target pre-trained model.

[0080] In this embodiment, weighted hyperparameters are set for the first loss and the second loss to obtain the corresponding target loss. The calculation formula for the target loss is as follows:

[0081] L s2 (θ n )=βlogKD L logKD (θ n )+β feaKD L feaKD (θ n )+β tn L tn (θ n )

[0082] Among them, L tn (θ n ) is the second loss, L feaKD (θ n L represents the feature distillation loss between the pre-trained model and the initial pre-trained model. logKD (θ n ) represents the knowledge distillation loss between the pre-trained model and the initial pre-trained model, β logKD ,β feaKD and β tn L represents the corresponding weight hyperparameter. s2 (θ n The target loss is used to perform augmentation training on the initial pre-trained model based on the target loss, and the trained model is determined as the target pre-trained model.

[0083] This invention obtains the pre-trained model and pre-trained generator corresponding to the previous image classification task, as well as the real image and its corresponding real label value for the current image classification task. Pseudo-data generated by the pre-trained generator and the real image are used as training data to train the model for the current image classification task, resulting in an initial pre-trained model. Pseudo-data is then input into the initial pre-trained model, outputting a first result. Pseudo-data is then input into the pre-trained model, outputting a second result. A first loss is calculated between the first and second results. The real image is then input into the initial pre-trained model, outputting a third result. A second loss is calculated between the third result and the corresponding real label value for the real image. Based on the first and second losses, the initial pre-trained model is augmented to determine the trained model as the target pre-trained model. The target pre-trained model is then used to classify the image to be processed, obtaining the image classification result. In this invention, training with historical pseudo-data ensures the stability of the model's feature distribution. Knowledge distillation minimizes the differences between these pseudo-data and models in adjacent tasks, reducing feature bias caused by pseudo-data and thus improving the output accuracy of the target pre-trained model for the current image classification task.

[0084] See Figure 3 This is a schematic flowchart of an image classification enhancement processing method provided in an embodiment of the present invention. The above-described image classification enhancement processing method can be applied to... Figure 1The server in the above-mentioned configuration connects to the corresponding client, such as... Figure 3 As shown, the image classification enhancement processing method may include the following steps.

[0085] S301: Train the pseudo data generator corresponding to the previous image classification task using the pre-trained model, and calculate the loss between the pseudo data generated by the pseudo data generator and the real image in the previous image classification task.

[0086] S302: Train the pseudo-data generator based on the loss between the pseudo-data generated by the pseudo-data generator and the real image in the previous image classification task to obtain the pre-trained generator corresponding to the previous image classification task.

[0087] In this embodiment, to improve the diversity of pseudo-data generated by the pre-trained generator and ensure that the probability of the corresponding synthetic data for each class is the same, we expect the categories to be uniformly distributed. Therefore, we input the pseudo-data into the pre-trained model and maximize the information entropy of the model output to train the pseudo-data generator. During the training process, when constructing the loss function, to ensure the diversity of pseudo-data generated by the trained pre-trained generator, a diversity loss function is constructed as follows:

[0088]

[0089] Among them, L div (φ) is the diversity loss function. Let t represent the information entropy function, where t i Represents the number of categories included in the previous image classification task, and φ indicates that the model parameters are the parameters in the pseudo-data generator. For data generated by the pseudo-data generator, θ n-1 Here are the parameters in the pre-trained model, and S(·) represents the pre-trained model.

[0090] A diversity loss function might result in relatively flat outputs from fully connected layers, making the generated pseudo-data unhelpful for initial pre-training. Therefore, a content loss function needs to be constructed to ensure that the outputs of fully connected layers have high confidence in only one category. This requires a corresponding content loss function, as shown in the following equation:

[0091]

[0092] Among them, L con (φ) is the content loss function, L CE (·) represents the cross-entropy loss function, where τ represents a temperature constant used to evaluate S(θ). n-1 The output of the fully connected layer is scaled. These are the classification labels output by the pre-trained model.

[0093] To offset the feature distribution of the generated pseudo-data, the image feature distribution is treated as a high-dimensional Gaussian distribution, with its mean and variance represented by the mean and variance of the input information of the batch normalization layer (BNS) of the pre-trained model. Then, the parameters of the pseudo-data generator are updated by minimizing the KL divergence between the feature distribution of the pseudo-data in the corresponding network layer and the original feature distribution. The corresponding statistical loss function is as follows:

[0094]

[0095] in, Indicates a Gaussian distribution. and μ represents the mean and standard deviation of the features of the pseudo-data in layer l. l and ρ l L represents the mean and standard deviation of the features in the l-th layer of the pre-trained model. sta (φ) is the statistical loss function.

[0096] The historical training data for the pre-trained model contains real images, which are locally smoother in pixel space. Therefore, a prior loss is provided to make the generated pseudo-data more closely resemble real images. The prior loss function is shown below:

[0097]

[0098] Among them, G k Represents the Gaussian kernel. This indicates that the pseudo-data is convolved with a Gaussian kernel, which is essentially a smoothing process for the pseudo-data. L pri (φ) is the prior loss function.

[0099] The pseudo-data generator corresponding to the previous image classification task is trained based on the various loss functions. The total loss between the pseudo-data generated by the pseudo-data generator and the real images in the previous image classification task is then calculated. The calculation formula is as follows:

[0100] L inv =α div L div (φ)+α con L con (φ)+α sta L sta (φ)+α pri L pri (φ)

[0101] Among them, L inv For the total loss, L div (φ) is the diversity loss function, where L con(φ) is the content loss function, L sta (φ) is the statistical loss function, L pri (φ) is the prior loss function. α div α con α sta and α pri This represents the corresponding weight hyperparameters. The pseudo-data generator is trained based on the total loss between the pseudo-data generated by the pseudo-data generator and the real images in the previous image classification task, resulting in the pre-trained generator for the previous image classification task.

[0102] S303: Obtain the pre-trained model and pre-trained generator corresponding to the previous image classification task, as well as the real image and the real label value of the corresponding real image for the current image classification task.

[0103] S304: Using pseudo-data generated by the pre-trained generator and real images as training data, the model in the current image classification task is trained using the training data to obtain the initial pre-trained model in the current image classification task.

[0104] S305: Input pseudo-data into the initial pre-trained model and output the corresponding first result; input pseudo-data into the pre-trained model and output the corresponding second result; calculate the first loss between the first result and the second result.

[0105] S306: Input the real image into the initial pre-trained model, output the corresponding third result, and calculate the second loss between the third result and the real label value corresponding to the real image.

[0106] S307: Based on the first loss and the second loss, perform augmentation training on the initial pre-trained model, determine the trained model as the target pre-trained model, use the target pre-trained model to classify the image to be processed, and obtain the image classification result.

[0107] The steps S303 to S307 are the same as the steps S201 to S205, and can be referred to the description of steps S201 to S205, which will not be repeated here.

[0108] Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of an image classification enhancement processing device provided in an embodiment of the present invention. In this embodiment, the terminal includes units used for performing... Figures 2 to 3 The steps in the corresponding embodiments. Please refer to the details. Figures 2 to 3 as well as Figures 2 to 3 The relevant descriptions in the corresponding embodiments are shown below. For ease of explanation, only the parts relevant to this embodiment are shown. See also... Figure 4The enhancement processing device 40 includes: an acquisition module 41, an initial pre-trained model determination module 42, a first loss determination module 43, a second loss determination module 44, and an enhancement training module 45.

[0109] The acquisition module 41 is used to acquire the pre-trained model and pre-trained generator corresponding to the previous image classification task, as well as the real image and the real label value of the corresponding real image corresponding to the current image classification task.

[0110] The initial pre-trained model determination module 42 is used to use pseudo data generated by the pre-training generator and real images as training data, and to train the model in the current image classification task using the training data to obtain the initial pre-trained model in the current image classification task.

[0111] The first loss determination module 43 is used to input pseudo data into the initial pre-trained model and output the corresponding first result, input pseudo data into the pre-trained model and output the corresponding second result, and calculate the first loss between the first result and the second result.

[0112] The second loss determination module 44 is used to input the real image into the initial pre-trained model, output the corresponding third result, and calculate the second loss between the third result and the real label value corresponding to the real image.

[0113] The enhancement training module 45 is used to enhance the initial pre-trained model based on the first loss and the second loss, determine the trained model as the target pre-trained model, and use the target pre-trained model to classify the image to be processed to obtain the image classification result.

[0114] Optionally, the aforementioned enhancement processing device 40 further includes:

[0115] The generator loss determination module is used to train the pseudo-data generator corresponding to the previous image classification task using a pre-trained model, and to calculate the loss between the pseudo-data generated by the pseudo-data generator and the real image in the previous image classification task.

[0116] The pre-trained generator is determined and used to train the pseudo-data generator based on the loss between the pseudo-data generated by the pseudo-data generator and the real images in the previous image classification task, so as to obtain the pre-trained generator corresponding to the previous image classification task.

[0117] Optionally, the initial pre-trained model determination module 42 mentioned above includes:

[0118] The pseudo-label value determination unit is used to input pseudo-data generated by the pre-training generator into the pre-training model, output the pseudo-label values ​​corresponding to the pseudo-data, and label the pseudo-data with the corresponding pseudo-label values.

[0119] The initial training unit is used to train the model in the current image classification task using pseudo-data with pseudo-label values ​​and real images as training data, and to obtain the initial pre-trained model in the current image classification task.

[0120] Optionally, the first loss determination module 43 mentioned above includes:

[0121] The first output unit is used to input pseudo data into the initial pre-trained model, output the first feature information corresponding to the pseudo data based on the hidden layer in the initial pre-trained model, and output the first numerical information corresponding to the pseudo data based on the fully connected layer in the initial pre-trained model.

[0122] The first result determination unit is used to take the first feature information and the first numerical information as the first result.

[0123] Optionally, the first loss determination module 43 mentioned above includes:

[0124] The second output unit is used to input pseudo-data into the pre-trained model, output the second feature information corresponding to the pseudo-data based on the hidden layer in the pre-trained model, and output the second numerical information corresponding to the pseudo-data based on the fully connected layer in the pre-trained model.

[0125] The second result determination unit is used to take the second feature information and the second numerical information as the second result.

[0126] Optionally, the first loss determination module 43 mentioned above includes:

[0127] The distillation loss determination unit is used to calculate the knowledge distillation loss and feature distillation loss between the initial pre-trained model and the pre-trained model based on the first result and the second result.

[0128] The first loss calculation unit is used to use the sum of knowledge distillation loss and feature distillation loss as the first loss.

[0129] Optionally, the above-mentioned enhanced training module 45 includes:

[0130] The weight setting unit is used to set the weight hyperparameters for the first loss and the second loss to obtain the corresponding target loss.

[0131] The training unit is used to enhance the initial pre-trained model based on the target loss, and determine the trained model as the target pre-trained model.

[0132] It should be noted that the information interaction and execution process between the above-mentioned units are based on the same concept as the method embodiments of the present invention. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0133] Figure 5 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Figure 5 As shown, the computer device of this embodiment includes: at least one processor ( Figure 5 Only one is shown in the diagram), a memory, and a computer program stored in the memory and executable on at least one processor, which, when executed by the processor, implements the steps in the embodiments of the enhanced processing methods for any of the above image classifications.

[0134] This computer device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 5 The examples of computer devices are merely examples and do not constitute a limitation on computer devices. Computer devices may include more or fewer components than shown in the illustration, or combinations of certain components, or different components, such as network interfaces, displays, and input devices.

[0135] The processor referred to can be a CPU, but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0136] Memory includes readable storage media, internal memory, etc., wherein internal memory can be the RAM of a computer device, providing an environment for the operation of the operating system and computer-readable instructions stored in the readable storage media. The readable storage media can be the hard drive of a computer device, or in other embodiments, it can be an external storage device of the computer device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, memory can include both internal storage units and external storage devices of a computer device. Memory is used to store the operating system, applications, bootloader, data, and other programs, such as program code for computer programs. Memory can also be used to temporarily store data that has been output or will be output.

[0137] Those skilled in the art will understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this invention. The specific working process of the units and modules in the above device can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention can implement all or part of the processes in the methods of the above embodiments by instructing related hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the above method embodiments. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium can include at least: any entity or device capable of carrying computer program code, a recording medium, a computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0138] The present invention can implement all or part of the processes in the methods of the above embodiments, or it can be accomplished by a computer program product. When the computer program product is run on a computer device, the computer device executes the steps in the above method embodiments.

[0139] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0140] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0141] In the embodiments provided by this invention, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0142] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0143] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method of enhanced processing of image classification, characterized in that, The enhanced processing method comprises: obtaining a pre-training model corresponding to a previous image classification task, a pre-training generator, and real images corresponding to a current image classification task, and real label values corresponding to the real images; using the real images and pseudo data generated based on the pre-training generator as training data, training a model in the current image classification task using the training data, and obtaining an initial pre-training model in the current image classification task; inputting the pseudo data into the initial pre-training model to output a first result corresponding to the pseudo data, inputting the pseudo data into the pre-training model to output a second result corresponding to the pseudo data, and calculating a first loss between the first result and the second result; inputting the real images into the initial pre-training model to output a third result corresponding to the real images, and calculating a second loss between the third result and real label values corresponding to the real images; performing enhanced training on the initial pre-training model according to the first loss and the second loss, determining a trained model as a target pre-training model, and using the target pre-training model to classify a to-be-processed image to obtain an image classification result.

2. The method of claim 1, wherein the enhanced processing is performed by a processor of the electronic device. Before the obtaining a pre-training model corresponding to a previous image classification task, a pre-training generator, and real images corresponding to a current image classification task, and real label values corresponding to the real images, the method further comprises: training a pseudo data generator corresponding to the previous image classification task by using a pre-training model, and calculating a loss between pseudo data generated by the pseudo data generator and real images in the previous image classification task; training the pseudo data generator according to the loss between the pseudo data generated by the pseudo data generator and the real images in the previous image classification task, and obtaining a pre-training generator corresponding to the previous image classification task.

3. The method of claim 1, wherein the enhanced processing is performed by a processor of the electronic device. The using the real images and pseudo data generated based on the pre-training generator as training data, training a model in the current image classification task using the training data, and obtaining an initial pre-training model in the current image classification task comprises: inputting the pseudo data generated based on the pre-training generator into the pre-training model to output pseudo label values corresponding to the pseudo data, and labeling the pseudo data with the pseudo label values; using the pseudo data labeled with the pseudo label values and the real images as training data, training a model in the current image classification task using the training data, and obtaining an initial pre-training model in the current image classification task.

4. The enhanced processing method of claim 1, wherein, The inputting the pseudo data into the initial pre-training model to output a first result corresponding to the pseudo data comprises: inputting the pseudo data into the initial pre-training model, outputting first feature information corresponding to the pseudo data based on a hidden layer in the initial pre-training model, and outputting first numerical information corresponding to the pseudo data based on a full connection layer in the initial pre-training model; using the first feature information and the first numerical information as the first result.

5. The enhanced processing method of claim 1, wherein, The inputting the pseudo data into the pre-training model to output a second result corresponding to the pseudo data comprises: input the pseudo data into the pre-training model, output second feature information corresponding to the pseudo data based on a hidden layer in the pre-training model, and output second numerical information corresponding to the pseudo data based on a fully connected layer in the pre-training model; use the second feature information and the second numerical information as the second result.

6. The enhanced processing method of claim 1, wherein, The calculation of the first loss between the first result and the second result includes: According to the first result and the second result, the knowledge distillation loss and the feature distillation loss between the initial pre-training model and the pre-training model are calculated; The sum of the knowledge distillation loss and the feature distillation loss is used as the first loss.

7. The enhanced processing method of claim 1, wherein, The enhancement training of the initial pre-training model according to the first loss and the second loss determines the trained model as the target pre-training model, including: Setting a weight hyperparameter for the first loss and the second loss to obtain a corresponding target loss; According to the target loss, the initial pre-training model is enhanced and trained to determine the trained model as the target pre-training model.

8. An enhanced processing apparatus for image classification, characterized by, The enhancement processing device includes: An acquisition module is configured to acquire a pre-training model corresponding to a previous image classification task, a pre-training generator, real images corresponding to a current image classification task, and real label values corresponding to the real images; An initial pre-training model determination module is configured to use pseudo data generated based on the pre-training generator and the real images as training data, train a model in the current image classification task using the training data, and obtain an initial pre-training model in the current image classification task; A first loss determination module is configured to input the pseudo data into the initial pre-training model to output a first result, input the pseudo data into the pre-training model to output a second result, and calculate a first loss between the first result and the second result; A second loss determination module is configured to input the real images into the initial pre-training model to output a third result, and calculate a second loss between the third result and real label values corresponding to the real images; An enhancement training module is configured to enhance and train the initial pre-training model according to the first loss and the second loss, determine a trained model as a target pre-training model, and use the target pre-training model to classify a to-be-processed image to obtain an image classification result.

9. A computer device, comprising: The computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, and the processor implements the enhancement processing method of any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 9. The computer program is executed by the processor to implement the enhancement processing method of any one of claims 1 to 7.

Images