Method, system, medium and device for generating positive image samples to implement image classification

Abstract

The application discloses a kind of generation image positive sample realizes the method, system, medium and equipment of image classification, belong to computer vision field.The application is first extracted and rewrites text semantics by multimodal model, enhances visual prompt, and then guide denoising diffusion model to generate the positive sample consistent with original image semantics, and uses contrast learning to carry out unsupervised training to image encoder, so that it accurately obtains semantic information;Using labeled feature vector trains linear classifier, to build image classification model for image classification task, finally can complete image class determination without a large number of labeled data.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of computer vision, and particularly relates to a method, system, medium and device for generating image positive samples to realize image classification. BACKGROUND

[0002] Generally, image classification algorithms in certain professional fields such as automatic driving need to perform a large number of fine annotations on images in a training data set. However, in actual scenarios, large-scale professional annotation data requires professional personnel to invest a large amount of effort for a long time, which not only increases the difficulty and cost of obtaining training data, but also greatly limits the possibility of applying image classification algorithms to automatic driving actual scenarios. Considering this problem, the current self-supervised contrast learning method has shown exciting prospects in the research of image classification.

[0003] In the field of automatic driving, contrast learning as an efficient self-supervised learning method relies on its ability to improve the robustness and generalization ability of the model, and has been widely used in the training process of the perception system. Typical contrast learning methods such as SimCLR, MoCo and VICReg generate positive samples from original images through data augmentation techniques, so as to maximize the similarity between positive samples and minimize the difference between negative samples. This strategy greatly depends on the generation mechanism of positive and negative samples, which has a direct and significant impact on the quality of learned representations.

[0004] Although contrast learning has shown significant effects, its limitations in sample generation are also obvious. First, the current widely used data augmentation methods such as cropping, rotating and flipping, although they can increase the number of samples, often cannot effectively expand the diversity of samples. These traditional methods are difficult to guarantee the semantic consistency between samples in the process of generating samples, and are prone to domain shift between samples, resulting in insufficient generalization of learned representations. Second, although the use of generative models such as GAN can create visually diverse new samples, these methods still have deficiencies in controlling the semantic consistency of generated samples, which may lead to semantic deviation between the generated positive samples and the original samples.

[0005] In addition, although existing research has begun to focus on using multi-modal large models (MLLM) to improve the quality of positive samples, this method shows new possibilities in controlling the consistency and diversity of positive samples. However, research in this field is still insufficient, and how multi-modal models effectively integrate different modal data in practical applications, and how to increase sample diversity while maintaining semantic consistency, are technical problems that need to be solved. Solving these problems requires in-depth technical innovation and comprehensive model optimization in order to achieve more widespread application of self-supervised learning methods in automatic driving systems. SUMMARY

[0006] The present application aims to solve the problem of how to increase sample diversity while maintaining semantic consistency in the field of automatic driving in the prior art, and provide a method, system, medium and equipment for generating image positive samples to realize image classification. The method of the present application can be applied to scenes where the training data is insufficient or the label is scarce in the field of automatic driving. By generating semantic consistent positive sample pairs through the framework, the data diversity can be enhanced, and the representation ability of the model can be improved.

[0007] In order to achieve the above-mentioned application purpose, the present application specifically adopts the following technical solutions:

[0008] In a first aspect, the present application provides a method for generating image positive samples to realize image classification, comprising the following steps:

[0009] S1. Generating positive samples based on the obtained original image guided denoising diffusion model, and unsupervised training the image encoder in a contrastive learning manner based on the generated positive samples;

[0010] S2. Cascading the image encoder obtained by unsupervised training with a linear classifier to be trained to form an image classification model, training the image classification model in a supervised manner, fixing the parameters of the image encoder in the image classification model, and optimizing the parameters of the linear classifier to enable it to realize image class judgment;

[0011] S3. Inputting the image to be classified into the trained image classification model, and outputting the classification result of the image to be classified.

[0012] On the basis of the above-mentioned scheme, each step can be realized in the following preferred specific manner.

[0013] As a preferred embodiment of the above-mentioned first aspect, in step S1, the specific process of generating positive samples is as follows:

[0014] S11. Inputting the original image and a sentence for prompting semantic extraction into a pre-trained multi-modal large model to generate a first feature description of the original image in a text space;

[0015] S12. Sending the first feature description and a sentence for prompting semantic rewriting into a large language model to perform semantic rewriting on the first feature description, and generating a second feature description of the original image in a text space;

[0016] S13. Adding Gaussian noise to the original image to generate a random noise image, obtaining a mask image of the original image and a target image, weighting and mixing the mask image of the original image and the target image according to pixel values to generate a mask mixed image, taking one of the original image, the random noise image, the mask image of the original image and the mask mixed image as a first visual prompt, and performing data enhancement on the first visual prompt to obtain a second visual prompt;

[0017] S14. Inputting the second feature description and the second visual prompt into the denoising diffusion model to generate a semantic perception synthesized image, and taking the semantic perception synthesized image as a positive sample consistent with semantics of the original image.

[0018] As a preferred embodiment of the first aspect, in step S1, the specific process of unsupervised training of the image encoder is as follows:

[0019] In each iteration round, the original image is subjected to data enhancement to generate a first processing image, the semantic perception synthesized image is subjected to data enhancement to generate a second processing image, the first processing image is input into the image encoder in the MoCo contrast learning framework for feature extraction to obtain a first feature map, the second processing image is input into the momentum image encoder in the MoCo contrast learning framework for feature extraction to obtain a second feature map, a contrast loss is calculated based on the first feature map and the second feature map, and the training is iteratively performed until a preset iteration round is reached, and the training of the image encoder is completed.

[0020] As a preferred embodiment of the first aspect, in step S2, in the process of training the linear classifier, the original image is input into the trained image encoder, the features output by the image encoder are subjected to the linear classifier, the output dimension of the linear classifier is set to the number of categories, the image category prediction result is output by the linear classifier, the real label corresponding to the original image is obtained, the first cross-entropy loss between the real label and the image category prediction result is calculated, the parameters of the linear classifier are updated based on minimizing the first cross-entropy loss, the training is iteratively performed until a preset iteration round is reached, and the linear classifier converges to obtain the trained linear classifier.

[0021] As a preferred embodiment of the first aspect, the linear classifier adopts a full connection layer.

[0022] As a preferred embodiment of the first aspect, the output dimension of the linear classifier is set to 100.

[0023] Secondly, the present application provides a system for generating image positive samples to realize image classification, which comprises:

[0024] The first model training module is configured to guide a denoising diffusion model to generate positive samples from the obtained original images, and to perform unsupervised training on an image encoder in a contrast learning manner based on the generated positive samples.

[0025] The second model training module is configured to cascade the image encoder obtained through the unsupervised training and a linear classifier to be trained to form an image classification model, to perform supervised training on the image classification model, to fix the parameters of the image encoder in the image classification model, and to optimize the parameters of the linear classifier to enable the image classification.

[0026] The result obtaining module is configured to input the image to be classified into the trained image classification model, and to output a classification result of the image to be classified.

[0027] In a third aspect, the present application provides a computer program product, which comprises computer programs / instructions, and the computer programs / instructions can implement the method for generating positive samples of images and realizing image classification according to any one of the solutions of the first aspect when executed by a processor.

[0028] In a fourth aspect, the present application provides a computer readable storage medium, which stores a computer program, and the computer program can implement the method for generating positive samples of images and realizing image classification according to any one of the solutions of the first aspect when executed by a processor.

[0029] In a fifth aspect, the present application provides a computer electronic device, which comprises a memory and a processor.

[0030] The memory is configured to store a computer program.

[0031] The processor is configured to implement the method for generating positive samples of images and realizing image classification according to any one of the solutions of the first aspect when executing the computer program.

[0032] Compared with the prior art, the present application has the following beneficial effects:

[0033] The present application improves the way of generating positive samples in contrast learning by introducing a positive sample generation mechanism with semantic perception, effectively solves the shortcomings of traditional data enhancement methods in terms of sample diversity and semantic consistency. Compared with existing data enhancement techniques such as cropping, rotating and flipping, the present application generates positive sample pairs with higher semantic consistency, significantly reduces the domain bias between samples, and improves the robustness and generalization ability of the learned representation. In addition, the present application uses a multi-modal large model (MLLM) to increase the diversity of samples while maintaining the semantic consistency between positive samples, overcoming the limitations of existing generation methods in semantic control, optimizing the smoothness of the sample space, and facilitating the seamless integration of generated samples with various mainstream contrast learning frameworks and traditional hand-designed data enhancement methods. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of the steps of the method of the present invention;

[0035] Figure 2 This is a schematic diagram of the statement used to prompt semantic rewriting in the method of the present invention;

[0036] Figure 3 This is a schematic diagram illustrating the training of the image encoder and linear classifier using the method of the present invention.

[0037] Figure 4 This is a system block diagram of the present invention;

[0038] Figure 5 This is a schematic diagram of the components of a computer electronic device. Detailed Implementation

[0039] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. Technical features in the various embodiments of the present invention can be combined accordingly without mutual conflict.

[0040] In the description of this invention, it should be understood that the terms "first" and "second" are used only for descriptive purposes and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first" and "second" may explicitly or implicitly include at least one of those features.

[0041] like Figure 1 As shown, in a preferred embodiment of the present invention, the method for generating positive image samples to achieve image classification includes the following steps S1 to S3. The specific implementation process of each step will be described in detail below.

[0042] S1. Based on the acquired original image, guide the denoising diffusion model to generate positive samples, and use the generated positive samples to perform unsupervised training on the image encoder in a contrastive learning manner.

[0043] It should be noted that in step S1 of the present invention, an image dataset consisting of original images is first obtained. The image dataset used in this embodiment is Car196. The main application of this dataset in the field of autonomous driving is to support vehicle recognition and classification tasks.

[0044] It should be noted that in step S1 of the present application, after obtaining the image data set, the guided denoising diffusion model is started to generate positive samples, and each generation of positive samples includes two steps of text semantic extraction and visual prompt enhancement. Taking the generation of a positive sample as an example, the specific process is as follows:

[0045] 1) Text semantic extraction

[0046] S11. In each iteration of generating positive samples, the original image and a sentence for prompting semantic extraction are input into the pre-trained multi-modal large model (MLLM) to generate a first feature description of the original image in the text space.

[0047] It should be noted that in step S11 of the present application embodiment, as shown in Figure 3 , a sentence for prompting semantic extraction (i.e. Figure 3 semantic extraction prompt) is used as an example: “Please describe this image in detail”, and the multi-modal large model specifically adopts the BLIP-2 model to finally generate the first feature description of the original image in the text space.

[0048] The implementation of the BLIP-2 model is prior art, and in this embodiment, the model is briefly described to facilitate understanding by those skilled in the art. The BLIP-2 model is a multi-modal model that can combine image and text data to understand visual scenes and generate natural language descriptions. The input of the model includes image and text prompts (optional), and the output is a text description or extracted feature vector. In this embodiment, the input to the BLIP-2 model is an image and a text prompt, and the output of the model is a text description.

[0049] 2) Text semantic rewriting

[0050] S12. The first feature description and a sentence for prompting semantic rewriting are input into the large language model (LLM), and the first feature description is semantically rewritten to generate a second feature description of the original image in the text space.

[0051] It should be noted that step S12 of the present application embodiment enhances sample diversity by adding several examples for guiding semantic rewriting in the above sentence. A sentence for prompting semantic rewriting (i.e. Figure 3 semantic rewriting prompt) used in this step is shown in Figure 2 . In this embodiment, the large language model used is the product LLaMA publicly released by Meta, and the first feature description is rewritten by the model to output the second feature description.

[0052] In this embodiment, 100 samples are prepared, and 5 samples are randomly selected each time to construct a sentence for prompting semantic rewriting, so as to ensure the consistency of the semantic consistency before and after the enhancement of the sample diversity, thereby enhancing the richness of the sample.

[0053] 3) Visual prompt enhancement

[0054] S13. Add Gaussian noise to the original image to generate a random noise image, obtain a mask image of the original image and a target image, weight and mix the mask image of the original image and the target image according to pixel values to generate a mask mixing image, use one of the original image, the random noise image, the mask image of the original image and the mask mixing image as a first visual prompt, and perform data enhancement on the first visual prompt to obtain a second visual prompt.

[0055] It should be noted that, in step S13 of the embodiment of the present application, in order to more flexibly control the generation type of the positive sample, four different visual prompts are designed, which are the original image, the random noise image, the mask image of the original image and the mask mixing image, and then one of them is selected as the first visual prompt, and Gaussian noise is additionally added to the first visual prompt and random cropping is performed to generate the second visual prompt. The mask image of the original image is generated by YOSO (You Only Segment Once). YOSO is a model for image segmentation, mainly used for generating a mask image. The mask image (Mask Image) is an image used for marking or highlighting a specific area in computer vision. Generally, the mask image is composed of a binary image, which is used to distinguish the target area and the background area.

[0056] 4) Positive sample generation

[0057] S14. Input the second feature description and the second visual prompt into the denoising diffusion model to generate a semantic perception synthesized image, and use the semantic perception synthesized image as a positive sample with semantic consistency with the original image.

[0058] It should be noted that in step S14 of the present application, the basic principle of the denoising diffusion model (Denoising Diffusion Probabilistic Models, DDPM) is mainly used to generate image samples. The denoising diffusion model mentioned here refers to a class of denoising diffusion models, and the common point of this class of models is to realize image generation and other tasks through a gradual noise adding and denoising process. The input of the DDPM model is a random noise image and conditional information (optional), and the output is a high-quality image after multiple denoising. In this embodiment, the input of the DDPM model is an image and a text semantic description, and the output is a high-quality image after multiple denoising. As for the structure of the denoising diffusion model, it can include different deformation structures such as Stable Diffusion and ControlNet deformation structure, so the present application does not make too many limitations on its specific structure. The denoising diffusion model used in this embodiment is ControlNet-sd15-seg, and its implementation belongs to the prior art and will not be described here.

[0059] It should be noted that in step S1 of the present application, after generating the positive sample, the image encoder is trained in an unsupervised manner by contrast learning. The training process of the image encoder is as follows:

[0060] In each iteration round, the original image is data-augmented to generate a first processed image, and the semantic perception synthesized image is data-augmented to generate a second processed image. The first processed image is input into the image encoder in the MoCo contrast learning framework for feature extraction to obtain a first feature map. The second processed image is input into the momentum image encoder in the MoCo contrast learning framework for feature extraction to obtain a second feature map. The contrast loss is calculated based on the first feature map and the second feature map, and the training is iteratively performed until the preset iteration round is reached, and the training of the image encoder is completed.

[0061] In the process of training the image encoder of the present application, the existing contrast learning framework MoCo is adopted. Other contrast learning methods can also be selected here, and the corresponding contrast loss is designed, so the present application is not limited. In this embodiment, the MoCo-v2 contrast learning framework is adopted.

[0062] S2. The image encoder obtained by unsupervised training is cascaded with a linear classifier to be trained to form an image classification model, and the image classification model is trained in a supervised manner. The parameters of the image encoder in the image classification model are fixed, and the parameters of the linear classifier are optimized to enable the image classification model to realize image class judgment.

[0063] It should be noted that in step S2 of the present application, the image encoder trained by contrastive learning is used as a feature extractor, and the image classification model is composed of the feature extractor and a linear classifier in cascade. Meanwhile, the parameters of the image encoder are fixed, and a linear classifier is trained through supervised fine-tuning.

[0064] In the process of training the linear classifier, the original image is input into the trained image encoder, the features output by the image encoder are input into the linear classifier, the output dimension of the linear classifier is set as the number of categories, the image category prediction result is output by the linear classifier, the real label corresponding to the original image is obtained, the first cross-entropy loss between the real label and the image category prediction result is calculated, the parameters of the linear classifier are updated based on minimizing the first cross-entropy loss, and the training is iteratively performed until the preset iteration round is reached, the linear classifier converges, and the trained linear classifier is obtained.

[0065] It should be noted that in the training of the image encoder, a part of the original images is used, and the real labels corresponding to the part of the original images are not required. When the image encoder is trained, another part of the original images is used and the real labels of the part of the original images are used at the same time. A linear classifier is trained through supervised fine-tuning, aiming to improve the relevance and interpretability of the prediction.

[0066] It should be noted that the linear classifier of the present application can adopt various network structures, and can use existing models or be designed by a person skilled in the art according to actual needs. In the present embodiment, a full connection layer is used as the linear classifier, and the output dimension thereof is set as 100.

[0067] S3. Inputting the image to be classified into the trained image classification model to output the classification result of the image to be classified.

[0068] It should be noted that in step S3 of the present application, when the linear classifier is trained, the image classification can be performed by the image classification model, and the specific process is not described again.

[0069] The present application will be described below through a specific example to show the application effect of the method for generating image positive samples to implement image classification described in S1-S3 of the above embodiment on a specific data set, so as to understand the essence of the present application.

[0070] Embodiment

[0071] The specific implementation process of the method for generating image positive samples to implement image classification in the present embodiment is as described above, and will not be described again.

[0072] In order to show the technical effect of the embodiment, the present application carries out ablation experiment based on MoCo-v2 on ImageNet-100 dataset. The ImageNet-100 dataset is a subset of ImageNet, which is a large image dataset composed of 100 classes. The ImageNet-100 dataset is divided into "train" training set and "val" validation set to evaluate the classification performance of the model on different classes, and the linear classification accuracy is used as the evaluation index. In the method of the present application, the contrastive learning training of the image encoder is modified to only train with the original image, and the rest of the process remains unchanged. The image classification model finally obtained in this way is used as the baseline model, that is, the baseline model does not use semantic perception synthetic images, nor does it contain semantic extraction prompts and visual prompts.

[0073] The present embodiment designs three different semantic extraction prompt sentences: (a) "Question: What does this image describe? Answer:" (b) "Please describe this image in detail." (c) "What objects are included in this image?", and applies the three semantic extraction prompt sentences in the method of the present application respectively, and compares them with the baseline model without prompt, and the results are shown in Table 1.

[0074] Table 1. Ablation experiment of semantic extraction prompt component of the present application

[0075]

[0076] From Table 1, it can be observed that: 1) Compared with the baseline model, the accuracy of the present application is significantly improved regardless of the selected prompt; 2) Providing more detailed image description is beneficial to the accuracy.

[0077] In addition to the above semantic extraction prompts, the present embodiment designs four visual prompts (i.e. original image, random noise image, mask image of the original image or mask mixed image), and applies the three visual prompts in the method of the present application respectively, and compares them with the baseline model without prompt, and the results are shown in Table 2.

[0078] Table 2. Ablation experiment of visual prompt component of the present application

[0079]

[0080]

[0081] As shown in Table 2, the highest accuracy can be achieved when using the mask mixed image as the visual cue. In contrast, using the random noise image as the visual cue, due to the loss of the basic information of the original image, the introduction of unreasonable positive samples leads to the decrease of the accuracy. However, the accuracy of the present application is higher than that of the baseline model without using the visual cue in any of the above cases.

[0082] The present embodiment also provides an ablation experiment on the influence of the model in the case of generating a plurality of semantic perception synthetic images, and the samples are enhanced by the semantic extraction cue and the visual cue respectively. The results are shown in Table 3. In Table 3, "x1" and "x2" respectively represent twice and three times the number of semantic perception synthetic images. The experiment before "-" only generates one semantic perception synthetic image for each original image. As shown in Table 3, during the training process, one of the plurality of semantic perception synthetic images belonging to the same image is randomly selected to ensure the constancy of the calculation load. With the increase of the number of semantic perception synthetic images, the accuracy of the model is further improved, indicating the great potential of the present application. The accuracy can be improved by using semantic enhancement or visual cue enhancement alone, which proves the rationality of the present application.

[0083] Table 3. Ablation experiment of semantic enhancement and visual cue enhancement of the present application

[0084] Semantic enhancement Visual enhancement Linear classification accuracy - - 78.4 ×1 - 80.1 - ×1 80.4 ×1 ×1 81.2 ×2 ×2 81.9

[0085] In addition, it should be noted that the method of generating image positive samples to realize image classification in the above embodiment can be essentially executed by a computer program or module. Therefore, based on the same inventive concept, another preferred embodiment of the present application also provides a system for generating image positive samples to realize image classification, which corresponds to the method of generating image positive samples to realize image classification provided by the above embodiment, as shown in Figure 4 which comprises:

[0086] a first model training module for generating positive samples by guiding the denoising diffusion model with the obtained original image, and unsupervised training the image encoder in a contrast learning manner based on the generated positive samples;

[0087] a second model training module for cascading the image encoder obtained by unsupervised training with a linear classifier to be trained to form an image classification model, training the image classification model in a supervised manner, fixing the parameters of the image encoder in the image classification model, and optimizing the parameters of the linear classifier to enable the image classification.

[0088] a result obtaining module for inputting the image to be classified into the trained image classification model, and outputting the classification result of the image to be classified.

[0089] It can be understood that the method for generating the image positive sample to realize the image classification described in S1-S3 is essentially realized by a computer program. Therefore, based on the same inventive concept, another preferred embodiment of the present application also provides a computer program product corresponding to the method for generating the image positive sample to realize the image classification provided by the above-mentioned embodiment, which includes a computer program / instruction, which can realize the method for generating the image positive sample to realize the image classification as described in the above-mentioned embodiment when executed by a processor.

[0090] Similarly, based on the same inventive concept, another preferred embodiment of the present application also provides a computer electronic device corresponding to the method for generating the image positive sample to realize the image classification provided by the above-mentioned embodiment, as shown in Figure 5 which includes a memory and a processor;

[0091] The memory is used to store a computer program;

[0092] The processor is used to realize the method for generating the image positive sample to realize the image classification in the above-mentioned embodiment when executing the computer program.

[0093] In addition, the logical instructions in the above-mentioned memory can be realized in the form of a software functional unit and sold or used as an independent product, which can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part of the prior art that contributes to the present application or part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in the embodiments of the present application.

[0094] Therefore, based on the same inventive concept, another preferred embodiment of the present application also provides a computer readable storage medium corresponding to the method for generating the image positive sample to realize the image classification provided by the above-mentioned embodiment, which stores a computer program, and when the computer program is executed by a processor, the method for generating the image positive sample to realize the image classification in the above-mentioned embodiment can be realized.

[0095] It can be understood that the above-mentioned storage medium can include a random access memory (RAM) and can also include a non-volatile memory (NVM), such as at least one disk memory. Meanwhile, the storage medium can also be a U disk, a mobile hard disk, a magnetic disk or an optical disk, etc. various media that can store program codes.

[0096] It can be understood that the processor described above can be a general processor, including a central processing unit (CPU), a network processor (NP), etc.; can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0097] In addition, it should be noted that the skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system described above can refer to the corresponding process in the foregoing method embodiments, which will not be described here. In the embodiments provided in the present application, the division of steps or modules in the system and method described is only a logical functional division, and there can be another division mode in actual implementation, for example, multiple modules or steps can be combined or integrated together, or a module or step can be split.

[0098] The above-described embodiments are only a preferred scheme of the present application, and are not intended to limit the present application. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present application. Therefore, any technical scheme obtained by equivalent replacement or equivalent transformation falls within the protection scope of the present application.

Claims

1. A method for implementing image classification by generating a positive image sample, characterized in that, The method comprises the following steps: S1. generating positive samples based on the obtained original image and guiding the denoising diffusion model, and unsupervised training the image encoder in a contrast learning manner based on the generated positive samples; S2. cascading the image encoder obtained by unsupervised training and a linear classifier to be trained to form an image classification model, and training the image classification model in a supervised manner, fixing the parameters of the image encoder in the image classification model, and optimizing the parameters of the linear classifier to enable the image classification model to judge the image category; S3. inputting the image to be classified into the trained image classification model, and outputting the classification result of the image to be classified. In step S1, the specific process of generating positive samples is as follows: S11. inputting the original image and a sentence for prompting semantic extraction into a pre-trained multi-modal large model to generate a first feature description of the original image in a text space; S12. inputting the first feature description and a sentence for prompting semantic rewriting into a large language model to perform semantic rewriting on the first feature description, and generating a second feature description of the original image in the text space; S13. adding Gaussian noise to the original image to generate a random noise image, obtaining a mask image of the original image and a target image, generating a mask mixed image by weighting and mixing the mask image of the original image and the target image according to pixel values, taking one of the original image, the random noise image, the mask image of the original image and the mask mixed image as a first visual prompt, and performing data enhancement on the first visual prompt to obtain a second visual prompt; S14. inputting the second feature description and the second visual prompt into the denoising diffusion model to generate a semantic perception synthesized image, and taking the semantic perception synthesized image as a positive sample consistent with the semantics of the original image.

2. The method of claim 1, wherein the positive image is generated by a process comprising: In step S1, the specific process of unsupervised training the image encoder is as follows: In each iteration round, the original image is data enhanced to generate a first processed image, the semantic perception synthesized image is data enhanced to generate a second processed image, the first processed image is input into the image encoder in the MoCo contrast learning framework for feature extraction to obtain a first feature map, the second processed image is input into the momentum image encoder in the MoCo contrast learning framework for feature extraction to obtain a second feature map, a contrast loss is calculated based on the first feature map and the second feature map, and the training is iteratively performed until a preset iteration round is reached, and the training of the image encoder is completed.

3. The method of claim 1, wherein the positive image is generated by a process comprising: In step S2, in the process of training the linear classifier, the original image is input into the trained image encoder, the features output by the image encoder are input into the linear classifier, the output dimension of the linear classifier is set as the number of categories, the image category prediction result is output by the linear classifier, the real label corresponding to the original image is obtained, the first cross-entropy loss between the real label and the image category prediction result is calculated, the parameters of the linear classifier are updated based on minimizing the first cross-entropy loss, and the training is iteratively performed until a preset iteration round is reached, and the linear classifier converges to obtain the trained linear classifier.

4. The method of claim 1 or 3, wherein the positive sample of the generated image is implemented by the image classification. The linear classifier adopts a full connection layer.

5. The method of claim 3, wherein the positive image is generated by a process comprising: The output dimension of the linear classifier is set as 100.

6. A system for implementing image classification by generating a positive image sample, characterized in that, The method comprises the following steps: The first model training module is configured to guide a denoising diffusion model to generate a positive sample from the obtained original image, and to perform unsupervised training on an image encoder in a contrast learning manner based on the generated positive sample. The second model training module is configured to cascade the image encoder obtained through the unsupervised training and a linear classifier to be trained to form an image classification model, to train the image classification model in a supervised manner, to fix the parameters of the image encoder in the image classification model, and to optimize the parameters of the linear classifier to enable the image classification. The result obtaining module is configured to input an image to be classified into the trained image classification model, and to output a classification result of the image to be classified. In the first model training module, the specific process of generating the positive sample is as follows: S11. inputting an original image and a sentence for prompting semantic extraction into a pre-trained multi-modal large model to generate a first feature description of the original image in a text space; S12. inputting the first feature description and a sentence for prompting semantic rewriting into a large language model to perform semantic rewriting on the first feature description, and generating a second feature description of the original image in the text space; S13. adding Gaussian noise to the original image to generate a random noise image, obtaining a mask image of the original image and a target image, generating a mask mixed image by weighting and mixing the mask image of the original image and the target image according to pixel values, taking one of the original image, the random noise image, the mask image of the original image and the mask mixed image as a first visual prompt, and performing data enhancement on the first visual prompt to obtain a second visual prompt; S14. inputting the second feature description and the second visual prompt into the denoising diffusion model to generate a semantic perception synthesized image, and taking the semantic perception synthesized image as a positive sample consistent with the semantics of the original image.

7. A computer program product comprising computer programs / instructions, characterized in that, The computer program / instruction is executed by the processor to implement the method for generating an image positive sample to realize image classification according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The storage medium stores the computer program, and when the computer program is executed by the processor, the method for generating an image positive sample to realize image classification according to any one of claims 1-5 is implemented.

9. A computer electronic device, characterized by, The storage medium stores the computer program, and when the computer program is executed by the processor, the method for generating an image positive sample to realize image classification according to any one of claims 1-5 is implemented. The storage medium stores the computer program, and when the computer program is executed by the processor, the method for generating an image positive sample to realize image classification according to any one of claims 1-5 is implemented. The storage medium stores the computer program, and when the computer program is executed by the processor, the method for generating an image positive sample to realize image classification according to any one of claims 1-5 is implemented.

Images