Image amplification method and system based on controllable background mixing

Abstract

The present invention discloses an image augmentation method and system based on controllable background mixing, the method comprising: obtaining the description text of the foreground object and background information of each image in the original image data set based on a large visual language model to obtain a multimodal data set; using the large language model to classify the foreground objects and extract keywords from the background information according to the multimodal data set to obtain background keywords corresponding to the foreground objects of each category; performing a balance analysis on the background keywords of each category of foreground objects to determine a background balance scheme; generating an augmented image based on the background balance scheme, adding the augmented image to the original image data set to supplement the missing background information of each category of foreground objects, and obtaining an extended data set with background balance. The present invention can make full use of the contextual information of the background, complete data augmentation by processing the image background, achieve zero label pollution of the data set, and help improve the generalization ability of the model.

Description

Technical Field

[0001] The present invention relates to the technical field of data enhancement, and in particular to an image amplification method and system based on controllable background mixing. Background Art

[0002] In order to prevent deep networks from overfitting the training data, data augmentation has been widely studied. Data augmentation is used to expand the training data set so that the model is exposed to more sample variants during training and reduce the overfitting phenomenon of the model, that is, the phenomenon of performing well on the training data but poorly on new data. Current data augmentation methods include Mixup, CutMix, and AugMix. Mixup generates a synthetic image by linearly interpolating pixel values ​​from two randomly selected images; CutMix pastes random blocks from one image into another; AugMix is ​​a data augmentation operation that uses random combinations of input images; SaliencyMix uses saliency image enhancement to generate new data samples by combining the original image and its saliency map.

[0003] However, whether it is the linear interpolation Mix method based on salient regions or the Mix method based on label preservation, the foreground information will be changed to a certain extent, resulting in label contamination. For example, Mixup linearly interpolates the pixel values ​​of two images, including the foreground and background, which causes the foreground content to be changed and may no longer be consistent with the original label. Similarly, CutMix pastes image blocks into another image. Although the foreground information may be partially retained, the mixture of foreground and background may cause the label to no longer accurately match the actual image content, resulting in label contamination. In addition, these methods mainly focus on how to improve the feature performance of the foreground or generate new foreground samples, but not on how to maintain or optimize the consistency and usefulness of background information. The background processing is not comprehensive enough, which ultimately leads to unbalanced and uncontrollable background information in the enhanced samples.

[0004] Therefore, it is necessary to propose an image augmentation method and system based on controllable background mixing, which can make full use of the contextual information of the background, complete data augmentation by processing the image background, achieve zero label contamination of the data set, and help improve the generalization ability of the model. Summary of the invention

[0005] In view of this, the present invention provides an image amplification method and system based on controllable background mixing, so as to solve the technical problems that the existing data enhancement technology will change the foreground information to a certain extent, and the background processing is not comprehensive enough, and there is an imbalance and uncontrollable background information in the enhanced sample.

[0006] In order to achieve the above technical objectives, the present invention adopts the following technical solutions:

[0007] In a first aspect, the present invention provides an image amplification method based on controllable background mixing, comprising:

[0008] Based on the visual language model, the description text of the foreground object and background information of each image in the original image dataset is obtained to obtain a multimodal dataset containing multiple image-text pairs;

[0009] Using a large language model to classify foreground objects and extract keywords from background information according to the multimodal data set, to obtain background keywords corresponding to foreground objects of each category;

[0010] Perform a balance analysis on the background keywords of each type of foreground object, and determine the background balance solution based on the analysis results and the preset background balance threshold;

[0011] An augmented image is generated based on the background equalization scheme, and the augmented image is added to the original image data set to supplement the missing background information of each type of foreground object, thereby obtaining an extended data set with background equalization.

[0012] Further, generating an augmented image based on the background equalization scheme includes:

[0013] Determine the missing background of any foreground object according to the background equalization scheme, and generate background extension prompt words based on the missing background;

[0014] The background extension prompt word is input into a preset diffusion model to guide the diffusion model to generate an augmented image, so that the foreground object of the augmented image is consistent with that in the original image, and the background is the background in which the foreground object is missing in the original image data set.

[0015] Furthermore, generating an augmented image based on the background equalization scheme further includes:

[0016] Determine the missing background of any foreground object according to the background equalization scheme, and generate background extension prompt words based on the missing background;

[0017] Select any image containing the foreground object in the original image data set, perform image segmentation on the image, and obtain a foreground object mask image;

[0018] A background image in which the foreground object is missing is selected from a preset background data set according to the background extension prompt word, and the foreground object mask image and the background image are spliced ​​to obtain an augmented image.

[0019] Furthermore, the background extension prompt word is input into a preset diffusion model to guide the diffusion model to generate an amplified image, including:

[0020] Select any image containing the foreground object in the original image data set, perform image segmentation on the image, and obtain a foreground object mask image;

[0021] A background image in which the foreground object is missing is selected from a preset background data set, and the foreground object mask image and the background image are spliced ​​to obtain an augmented image.

[0022] Further, performing image segmentation on the image to obtain a foreground object mask image includes:

[0023] The image is input into a preset SAM model for image segmentation to generate a binary image of the foreground object, and the binary image of the foreground object is used as a foreground object mask image.

[0024] Furthermore, the method for generating the preset background data set is:

[0025] According to the background keywords, download the corresponding background images through the crawler tool;

[0026] Abnormal images are deleted and duplicate removal is performed on the downloaded background images, and the processed images are classified and stored to obtain a background data set.

[0027] Furthermore, based on the visual language model, the description text of the foreground object and background information of each image in the original image dataset is obtained, including:

[0028] Create image information prompt rules, input the image information prompt rules into the pre-trained visual language model, and batch generate text descriptions of the foreground objects and background information of each image in the original image data set.

[0029] Furthermore, the large language model is used to classify the foreground objects and extract keywords from the background information according to the multimodal data set to obtain background keywords corresponding to the foreground objects of each category, including:

[0030] Traversing each image-text pair in the multimodal dataset, using a large language model to analyze the text description of the background information, and extracting corresponding background vocabulary;

[0031] Based on the preset background dictionary, the background words are mapped to corresponding background keywords to obtain the background keywords corresponding to the foreground objects of each category.

[0032] Furthermore, a balance analysis is performed on the background keywords of each type of foreground object, and a background balance scheme is determined according to the analysis result and a preset background balance threshold, including:

[0033] Traverse each type of foreground object, count the frequency of each background keyword, and obtain the background keyword distribution of each type of foreground object;

[0034] Determining, according to the background keyword distribution, the missing background types whose number is less than a preset background equalization threshold;

[0035] The background type to be supplemented and the number of image amplifications to be added are determined based on the missing background type.

[0036] In a second aspect, the present invention provides an image amplification system based on controllable background mixing, comprising:

[0037] A text description module is used to obtain the description text of the foreground object and background information of each image in the original image dataset based on the visual language model, and obtain a multimodal dataset containing multiple image-text pairs;

[0038] A keyword extraction module is used to classify the foreground objects using a large language model and extract keywords from the background information description text to obtain background keywords corresponding to each category of foreground objects;

[0039] A program formulation module is used to analyze the balance of background keywords of each type of foreground object and determine the background balance program based on the analysis results and the preset background balance threshold;

[0040] The data set generation module is used to generate new image samples based on the background equalization scheme, add the new image samples to the original image data set to supplement the missing background information of each type of foreground object, and obtain an extended data set with background equalization.

[0041] In a third aspect, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the image amplification method based on controllable background mixing as described in any of the above technical solutions is implemented.

[0042] Compared with the prior art, the method and system provided by the present invention firstly describe the foreground object and background information of each image in the original image data set through a large visual language model; secondly, extract keywords from the background information using a large language model, perform a balance analysis on the background keywords of each type of foreground object and determine a background balance scheme; finally, generate an augmented image based on the background balance scheme, add the augmented image to the original image data set to supplement the missing background information of each type of foreground object, and finally create a high-quality extended data set. The method of the present invention makes full use of the rich context information provided by the background information for image recognition, more accurately completes image recognition and semantic segmentation, and generates a background-controllable and balanced augmented data set based on the background information of the current data set by using a prompt engineering-guided diffusion model, which can realize data augmentation by processing the background without changing the foreground information, thereby achieving zero label pollution and improving the effect of foreground object classification. The background-balanced controllable data set generated by the present method can improve the generalization ability of the model while ensuring the quality of the data set, making it more stable and accurate in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A schematic diagram of the process of the image amplification method based on controllable background mixing provided by the present invention;

[0044] Figure 2 A schematic diagram of the image amplification process provided by the present invention;

[0045] Figure 3 A schematic diagram of a process of generating a foreground object and background information description text through VLLM provided by the present invention;

[0046] Figure 4 A schematic diagram of using a large language model to extract background keywords from a text description provided by the present invention;

[0047] Figure 5 A schematic diagram of the frequency of background keywords of various foreground objects before the equalization process provided by the present invention;

[0048] Figure 6 A schematic diagram of a process for generating an amplification image through a diffusion model provided by the present invention;

[0049] Figure 7 A schematic diagram of the structure of the background data set provided by the present invention;

[0050] Figure 8 A schematic diagram of a method for image amplification using a SAM model and crawler technology provided by the present invention;

[0051] Fig. 9A schematic diagram of the frequency of background keywords of various foreground objects after the equalization processing provided by the present invention;

[0052] Fig.10 A schematic diagram of the structure of an image amplification system based on controllable background mixing provided by the present invention. DETAILED DESCRIPTION

[0053] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.

[0054] The present invention provides an image amplification method and system based on controllable background mixing, which are respectively described below.

[0055] Combination Figure 1 As shown, a specific embodiment of the present invention discloses an image amplification method based on controllable background mixing, comprising:

[0056] Step S101: obtaining a description text of the foreground object and background information of each image in the original image dataset based on the visual language macro model, and obtaining a multimodal dataset containing multiple image-text pairs;

[0057] Step S102: using a large language model to classify foreground objects and extract keywords from background information according to the multimodal data set, to obtain background keywords corresponding to foreground objects of each category;

[0058] Step S103: performing a balance analysis on the background keywords of each type of foreground object, and determining a background balance solution according to the analysis result and a preset background balance threshold;

[0059] Step S104: generating an augmented image based on the background equalization scheme, and adding the augmented image to the original image data set to supplement the missing background information of each type of foreground object, so as to obtain an extended data set with background equalization.

[0060] The method of this embodiment firstly describes the foreground object and background information of each image in the original image data set through a large visual language model; secondly, extracts keywords from the background information using a large language model, performs a balance analysis on the background keywords of each type of foreground object, and determines a background balance scheme; finally, generates an augmented image based on the background balance scheme, adds the augmented image to the original image data set to supplement the missing background information of each type of foreground object, and finally creates a high-quality extended data set. The method of this embodiment makes full use of the rich contextual information provided by the background information for image recognition, and more accurately completes image recognition and semantic segmentation. According to the background information of the current data set, a background-controllable and balanced augmented data set is generated using a prompt engineering-guided diffusion model, which can realize data augmentation by processing the background without changing the foreground information, thereby achieving zero label pollution and improving the effect of foreground object classification. The background-balanced and controllable data set generated by this method can improve the generalization ability of the model while ensuring the quality of the data set, making it more stable and accurate in practical applications.

[0061] Combine the following Figure 2-9 The above method is specifically described through a specific embodiment.

[0062] Figure 2 The data processing flow chart of this embodiment is shown, which mainly includes three steps: multimodal data set generation, background vocabulary analysis and data set expansion.

[0063] First, generate a multimodal dataset: use the pre-trained Visual Language Large Model (VLLM) to extract information from each image in the original dataset. Specifically:

[0064] Create image information prompt rules, such as: "Please provide a detailed description of this image", "Please provide a detailed description of the main object in this image", "Please provide a detailed description of the background information of this image", etc. Figure 3 As shown, Figure 3 The flowchart of generating description text of foreground objects and background information through VLLM is shown. By inputting prompt rules, VLLM can generate structured text descriptions that meet the prompt rules, thereby forming description texts of foreground objects and background information in batches, and obtaining an image-text multimodal dataset containing all image background information in the dataset.

[0065] It should be noted that the visual language model selected here can be OpenAI's CLIP, Google's BigGAN and DALL-E, Meta's Segment Anything Model (SAM), etc. These visual language models can not only understand and generate text, but also process visual information (such as pictures and videos). They learn to associate the content in the image with the text description to achieve the fusion of vision and language. In this embodiment, the main task of the model is image description, that is, answering questions generated by preset prompt rules based on the image content, and outputting natural language descriptions related to the image content.

[0066] Secondly, perform background vocabulary analysis: traverse each image-text pair in the multimodal dataset, use the large language model (LLM) to process the text description, extract the corresponding background vocabulary, and map the background vocabulary to standardized background keywords based on the preset background dictionary, so as to obtain the background keywords corresponding to the foreground objects of each category. The background dictionary here can be manually constructed or obtained based on a previous classification or labeling system. Figure 4 As shown, Figure 4 The schematic diagram of using a large language model to extract background keywords from text descriptions is shown. The extracted background words are converted into background keywords through background dictionary mapping, and these keywords are used for image classification and data analysis. By traversing each type of foreground object, the frequency of each background keyword of the foreground object is counted, such as Figure 5 As shown, the background keyword distribution of various foreground objects is obtained. Figure 5 In the background distribution corresponding to each image in the dataset, the code on the horizontal axis represents the type of background (such as a for branches, b for sky, c for beach, d for fence, e for snow, etc.), and the vertical axis represents the number of backgrounds of each type. Finally, the background distribution of all images in the dataset can be displayed.

[0067] Finally, a data set with controllable background balance is generated: Here, two methods are included to generate augmented images based on the background balance scheme:

[0068] The first method is to determine the background extension prompt word for the missing background, input the background extension word into a preset diffusion model, and guide the diffusion model to generate an augmented image so that the foreground object of the augmented image is consistent with that in the original image, and the background is the background where the foreground object is missing in the original image data set. For example, if the original data set lacks a picture with a snowy background, then input the prompt word "snow" into the diffusion model, cut out the picture through the diffusion model, and then replace the background. The effect of the augmented image is as follows: Figure 6As shown. The augmented image uses a pre-trained diffusion model to traverse all categories of foreground objects and obtain the background distribution corresponding to the foreground category. For image categories in the dataset where the background count is below the threshold, the diffusion model is used to generate augmented images based on the prompts, and then these images are added to the augmented dataset to achieve a balance in the number of backgrounds. The diffusion model here is a generative model used to create high-quality images. It generates images by gradually adding noise to the data and then gradually removing the noise by the reverse process. In this process, the model learns how to convert random noise into a clear image.

[0069] Another method is: after determining the background extension prompt words of the missing background, use the SAM technology to generate the MASK image corresponding to the dataset, and then use the crawler to build a background dataset containing all background keywords, such as Figure 7 shown.

[0070] Traverse each type of foreground object, count the frequency of each background keyword, and obtain the background keyword distribution of each type of foreground object; determine the missing background types whose number is less than the preset background equalization threshold based on the background keyword distribution; determine the background type to be supplemented and the number of image augmentations based on the missing background types. Use the prompt engineering isomorphic SAM technology and crawler technology to generate augmented images, which can ensure that the foreground remains unchanged and the background is controllable.

[0071] like Figure 8 As shown, Figure 8 The method flow of amplifying images with less than a preset threshold according to the background equalization scheme is demonstrated. First, a picture of a certain type of foreground object in the original data set is randomly selected, and then the mask (MASK) of the foreground object is cut out using the SAM technology, where the white area represents the object pixels corresponding to the area, and the black area represents the background or other non-object pixels; then a background of the corresponding type is randomly selected in the preset background data set, and finally the selected background is spliced ​​with the foreground object obtained by cutting out the image to form a new sample containing the missing background knowledge in the original data set.

[0072] The generated new samples are added to the original data set to supplement the missing background information and achieve a balanced background type. The distribution after equalization is as follows Fig. 9 As shown, Fig. 9 and Figure 5 Compared with the original background distribution map in ( Fig. 9 and Figure 5 The physical meaning of the horizontal and vertical coordinates of the background distribution is consistent), it can be seen that the background distribution after equalization is more uniform, and the controllable replacement of the background is achieved while ensuring that the foreground remains unchanged. This method ultimately creates a high-quality extended dataset.

[0073] In order to verify the effect of the equalized data set, we analyze it from a Bayesian perspective.

[0074] The background categories such as "sky" and "ocean" are regarded as attributes of the image, denoted by C k . Xc indicates whether the sample prediction is correct. P (Xc|C k ) indicates that in a given background category C k The probability of predicting the result Xc under the condition of . The posterior probability P(C k |Xc) represents the probability of a specific background category being predicted when the prediction result is Xc, then:

[0075]

[0076] Before background equilibrium, the prior probability P(C k ) is calculated based on the frequency of each background in the dataset, where N 天空 and N 总 Respectively represent the number of images with the background of "sky" and the total number of images, that is: It can be seen that when the number of samples of the background category is unbalanced, the prior probability of the background attribute will directly affect the learning process and the results of the model.

[0077] Through the method of the present invention, more samples are generated to achieve a balance in the number of samples between different background categories, thereby ensuring that the model handles all background attributes fairly. Specifically:

[0078] Assume that the equilibrium number of background categories is M. After background balancing, each category has N 均衡 samples, then the balanced prior probability P(C k ) is calculated as:

[0079]

[0080] Therefore, the prior probability P(C k ) are equal, which ensures that each category is considered equally when the model calculates the posterior probability, making the classifier more fair to different background categories.

[0081] The overall probability of the predicted outcome Xc can be recalculated as:

[0082]

[0083] Table 1 shows a lot of experimental data. Each value in Table 1 represents the probability that the background is Ck when the sample prediction is correct. We calculate the average value of the shadow data of the first K backgrounds in each column to approximate the expansion of P(Ck) when there are M backgrounds. k|Xc=1). The specific values ​​are shown in the last row of Table 1. According to the data, P(C k |Xc=1) and M, P(C k |Xc=1) can be expressed as:

[0084] ≈ -1.704M + 21.952

[0085] Let λ1(M) = (-1.704M+21.952)·M be a function of M. Similarly, when Xc = 0, there is also a λ2(M).

[0086] Finally, we can further simplify the probability formula to:

[0087] P(Xc) ≈ λ(M) · P(Xc|Ck)

[0088] This formula shows that a classifier with a balanced background can avoid interference from the background category distribution. By adjusting the prior probability of the background category, the impact of the background category on model training can be reduced, thereby reducing the model bias and making it more focused on the features of the foreground category, ensuring that the model can learn the features of each foreground category fairly and accurately, making the model better generalized to practical applications and improving its performance on unseen samples.

[0089] Table 1 The probability that the background is Ck when the sample prediction is correct

[0090]

[0091] like Fig.10 As shown, this embodiment also provides an image amplification system 1000 based on controllable background mixing, including:

[0092] A text description module 1001 is used to obtain a description text of the foreground object and background information of each image in the original image dataset based on the visual language macro model, and obtain a multimodal dataset containing multiple image-text pairs;

[0093] A keyword extraction module 1002 is used to classify the foreground objects using a large language model and extract keywords from the background information description text to obtain background keywords corresponding to each category of foreground objects;

[0094] A scheme formulation module 1003 is used to perform a balance analysis on the background keywords of each type of foreground object, and determine a background balance scheme according to the analysis result and a preset background balance threshold;

[0095] The data set generation module 1004 is used to generate new image samples based on the background equalization scheme, and add the new image samples to the original image data set to supplement the missing background information of each type of foreground object, so as to obtain an extended data set with background equalization.

[0096] The image augmentation method and system based on controllable background mixing provided in this embodiment, firstly, a text description is performed on the foreground object and background information of each image in the original image data set through a large visual language model; secondly, a large language model is used to extract keywords from the background information, and the background keywords of each type of foreground object are balanced and a background balance scheme is determined; finally, an augmented image is generated based on the background balance scheme, and the augmented image is added to the original image data set to supplement the missing background information of each type of foreground object, and finally a high-quality extended data set is created. The method of the present invention makes full use of the rich contextual information provided by the background information for image recognition, and more accurately completes image recognition and semantic segmentation. According to the background information of the current data set, a background-controllable and balanced augmented data set is generated by using a prompt engineering-guided diffusion model, which can realize data augmentation by processing the background without changing the foreground information, thereby achieving zero label pollution and improving the effect of foreground object classification. The background-balanced controllable data set generated by this method can improve the generalization ability of the model while ensuring the quality of the data set, so that it performs more stably and accurately in practical applications.

[0097] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. An image amplification method based on controllable background mixing, characterized in that: include: Based on the visual language model, the description text of the foreground object and background information of each image in the original image dataset is obtained to obtain a multimodal dataset containing multiple image-text pairs; Using a large language model to classify foreground objects and extract keywords from background information according to the multimodal data set, to obtain background keywords corresponding to foreground objects of each category; Perform a balance analysis on the background keywords of each type of foreground object, and determine the background balance solution based on the analysis results and the preset background balance threshold; Generating an augmented image based on the background equalization scheme includes: determining a missing background of any foreground object according to the background equalization scheme, and generating a background extension prompt word based on the missing background; Inputting the background extension prompt word into a preset diffusion model to guide the diffusion model to generate an augmented image, so that the foreground object of the augmented image is consistent with that in the original image, and the background is the background in which the foreground object is missing in the original image data set; The augmented image is added to the original image data set to supplement the missing background information of each type of foreground object, thereby obtaining an extended data set with balanced background.

2. The image amplification method based on controllable background mixing according to claim 1, characterized in that: Generating an augmented image based on the background equalization scheme also includes: Determine the missing background of any foreground object according to the background equalization scheme, and generate background extension prompt words based on the missing background; Select any image containing the foreground object in the original image data set, perform image segmentation on the image, and obtain a foreground object mask image; A background image in which the foreground object is missing is selected from a preset background data set according to the background extension prompt word, and the foreground object mask image and the background image are spliced ​​to obtain an augmented image.

3. The image amplification method based on controllable background mixing according to claim 2, characterized in that: Performing image segmentation on the image to obtain a foreground object mask image includes: The image is input into a preset SAM model for image segmentation to generate a binary image of the foreground object, and the binary image of the foreground object is used as a foreground object mask image.

4. The image amplification method based on controllable background mixing according to claim 2, characterized in that: The method for generating the preset background data set is: According to the background keywords, download the corresponding background images through the crawler tool; Abnormal images are deleted and duplicate removal is performed on the downloaded background images, and the processed images are classified and stored to obtain a background data set.

5. The image amplification method based on controllable background mixing according to claim 1, characterized in that: Based on the visual language model, the description text of the foreground object and background information of each image in the original image dataset is obtained, including: Create image information prompt rules, input the image information prompt rules into the pre-trained visual language model, and batch generate text descriptions of the foreground objects and background information of each image in the original image data set.

6. The image amplification method based on controllable background mixing according to claim 1, characterized in that: Using a large language model to classify foreground objects and extract keywords from background information based on the multimodal data set, obtaining background keywords corresponding to foreground objects of each category, including: Traversing each image-text pair in the multimodal dataset, using a large language model to analyze the text description of the background information, and extracting corresponding background vocabulary; Based on the preset background dictionary, the background words are mapped to corresponding background keywords to obtain the background keywords corresponding to the foreground objects of each category.

7. The image amplification method based on controllable background mixing according to claim 1, characterized in that: Perform a balance analysis on the background keywords of each type of foreground object, and determine the background balance solution based on the analysis results and the preset background balance threshold, including: Traverse each type of foreground object, count the frequency of each background keyword, and obtain the background keyword distribution of each type of foreground object; Determining, according to the background keyword distribution, the missing background types whose number is less than a preset background equalization threshold; The background type to be supplemented and the number of image amplifications to be added are determined based on the missing background type.

8. An image amplification system based on controllable background mixing, characterized in that: include: A text description module is used to obtain the description text of the foreground object and background information of each image in the original image dataset based on the visual language model, and obtain a multimodal dataset containing multiple image-text pairs; A keyword extraction module is used to classify the foreground objects using a large language model and extract keywords from the background information description text to obtain background keywords corresponding to each category of foreground objects; A program formulation module is used to analyze the balance of background keywords of each type of foreground object and determine the background balance program based on the analysis results and the preset background balance threshold; A data set generation module, used to generate new image samples based on the background equalization scheme, including: determining the missing background of any foreground object according to the background equalization scheme, and generating background extension prompt words based on the missing background; The background extension prompt words are input into a preset diffusion model to guide the diffusion model to generate an augmented image, so that the foreground object of the augmented image is consistent with that in the original image and the background is the background that is missing from the foreground object in the original image data set; the new image samples are added to the original image data set to supplement the missing background information of each type of foreground object, so as to obtain an extended data set with balanced background.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the image amplification method based on controllable background mixing as described in any one of claims 1 to 7 is implemented.

Images