Method, system and device for constructing portrait matting dataset based on diffusion model

Abstract

The application discloses a portrait matting dataset construction method based on a diffusion model, compared with an existing dataset construction method, and is suitable for solving the feasibility problem of portrait matting data generation; the method is based on a prompt construction strategy and a plurality of collaborative generation diffusion models, and is supplemented by a matting annotation optimization layer and a data screening module to improve the portrait matting dataset construction efficiency, is used for automatically constructing a large-scale dataset, and filters abnormal portrait matting data; based on a data enhancement module, the diversity of the portrait matting data is enriched, and is closer to a real scene, improves the generalization of the model on the portrait matting task, effectively alleviates the problem of insufficient scale of the real portrait matting dataset, and improves the portrait matting dataset construction efficiency. The application also provides a portrait matting dataset construction system and equipment based on the diffusion model, which is used for constructing a large-scale high-precision portrait matting dataset.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of portrait matting dataset construction, and particularly relates to a portrait matting dataset construction method, system and device based on a diffusion model. BACKGROUND

[0002] Portrait matting is a basic task in the field of computer vision, and the task definition is to predict the transparency mask of a portrait given a portrait picture containing a background. The transparency value of each pixel position ranges from 0 to 1, 0 indicates that the pixel position is completely transparent and is determined by the background color, 1 indicates that the pixel position is completely opaque and is determined by the foreground color, and 0 to 1 indicates that the pixel position is determined by the background color and the foreground color in a certain proportion. Patent application CN 118411285 A discloses a video portrait matting method, device, storage medium and equipment, which adopts a SAM (Segment Anything Model) and an artificial interaction optimization matting method to perform matting on the first frame image of the target video, and then performs fusion matting using two segmentation models (DeAOT and XMem) to obtain the matting result of each frame image in the target video. The portrait matting method provided in the application is based on a model and an artificial interaction optimization method, which requires high labeling accuracy and has high artificial labeling cost, and is difficult to expand on a large scale. Therefore, the current open source portrait matting dataset is small in size. With the continuous improvement of the learning ability of deep neural networks, the importance of dataset size to model performance improvement is becoming more and more obvious, and small-scale datasets have already failed to meet the needs of network learning.

[0003] Invention application CN 118675009 A discloses an image dataset construction method and device for data diversity enhancement. The method enhances the diversity of data during data generation, and simultaneously uses an efficient labeling module, a screening module and an enhancement module to further tap the potential of the generation model, so that the generated image dataset can be expanded on a large scale while maintaining the ability to improve the performance of the model, meeting the needs of network learning.

[0004] With the continuous development of diffusion model, the realism and controllability of the image generated by the diffusion model are also increasing. For example, reference 1 (Wu W, Zhao Y, Shou M Z, et al. Diffumask: Synthesizing images with pixel-level annotations for semantic segmentation using diffusion models [C], Proceedings of the IEEE / CVF International Conference on Computer Vision, 2023, 1206-1217) explores the cross-attention map of the diffusion model, uses an adaptive threshold strategy to extract the semantic information contained therein, uses the diffusion model to generate images and corresponding semantic segmentation annotations at the same time, and improves the performance of the downstream semantic segmentation task. Reference 2 (Zhao H, Sheng D, Bao J, et al. X-Paste: revisiting scalable copy-paste for instance segmentation using CLIP and stable diffusion [C], International Conference on Machine Learning. PMLR, 2023, 42098-42109) mixes network crawling data and generated data, and uses multiple pre-trained segmentation models to annotate the mixed data set, and combines the copy-paste strategy to improve the performance of the downstream instance segmentation task.

[0005] Although the existing method proposes to use diffusion model to generate large-scale image data set, the image visual task based on instance segmentation solves the problem of data diversity, but cannot perform high-precision image visual task processing, which limits the construction of large-scale high-precision image visual data set. SUMMARY

[0006] The purpose of the present application is to provide a diffusion model-based portrait matting data set construction method, system and device, which constructs a large-scale high-precision portrait matting data set by designing a prompt sentence construction strategy and using multiple portrait matting collaborative generation diffusion models, and simultaneously using matting annotation optimization, data screening and data enhancement modules, effectively alleviates the problem of insufficient real portrait matting data set scale, and improves the efficiency of portrait matting data set construction.

[0007] To achieve the above-mentioned purpose of the application, the diffusion model-based portrait matting data set construction method provided by the embodiment comprises the following steps:

[0008] Step 1: constructing a prompt construction module based on a prompt template and an attribute list, randomly intercepting a pre-defined prompt template, and using a pre-defined attribute list to randomly fill the intercepted prompt template to obtain a prompt set;

[0009] Step 2: constructing a collaborative generation module of portrait cutout, generating portrait cutout data containing portrait pictures and cutout labels based on the prompt set and portrait data through the collaborative generation module, and performing text segmentation, dilation and erosion operations on the portrait cutout data to obtain optimized portrait data;

[0010] Step 3: constructing a data screening module, performing image-text similarity calculation on the optimized portrait data and the abnormal screening prompt set in the data screening module, and screening the portrait data below the set threshold to obtain screened portrait data;

[0011] Step 4: constructing a data enhancement module, inputting the screened portrait data into a random data enhancer, and the random data enhancer will randomly select a pre-defined data enhancement method to obtain enhanced portrait data.

[0012] In one embodiment, the form of the prompt template T in step 1 is as follows:

[0013] T=Concate(Attr1,Conc1,Attr2,Conc2,...,Attr n-1 ,Conc n-1 ,Attr n )

[0014] Wherein, Concate(·) represents a string concatenation operation;

[0015] The random interception operation receives a pre-defined prompt template T as input, each attribute Attr in T and the corresponding connector Conc are deleted from the prompt template T with a probability value p, and the remaining attributes and connectors are concatenated to obtain the intercepted prompt template T drop ;

[0016] The attribute list Ls contains a pre-defined n attribute value list L, each list contains no more than k attribute values att; the attribute list random filling operation receives the intercepted prompt template T drop and the attribute list Ls as input, for the attribute Attr i in the pre-defined prompt template, the corresponding attribute value list L i is selected from the attribute list Ls i , and an attribute value att j is randomly selected from L i , which is concatenated with the corresponding connector, and T is traverseddrop after all the attributes in the prompt set Ts.

[0017] In one embodiment, the collaborative generation module in step 2 comprises a portrait cutout collaborative generation diffusion model layer based on text-to-image, a portrait cutout collaborative generation diffusion model layer based on image-to-image, and a cutout annotation optimization layer based on text-guided segmentation and dilation erosion, which specifically comprises the following steps:

[0018] (1) input the prompt set Ts into the portrait cutout collaborative generation diffusion model layer based on text-to-image to obtain first original portrait data D1 comprising portrait picture I te and cutout annotation A te ;

[0019] (2) input the portrait data Ps in the open source data set into the portrait cutout collaborative generation diffusion model layer based on image-to-image to obtain second original portrait data D2 comprising portrait picture I im and cutout annotation A im ;

[0020] (3) input the first original portrait data D1 and the second original portrait data D2 into the cutout annotation optimization layer based on text-guided segmentation and dilation erosion, input the original portrait picture I i in the original portrait data D1 and D2 and the segmentation prompt B into the text-guided segmentation model to obtain the segmentation mask M i ; then perform dilation operation on the segmentation mask M i to obtain the dilated mask MD i , set the pixel position corresponding to the mask MD i background area (pixel value is 0) in the cutout annotation A i in the original portrait data D1 and D2 to 0 to obtain the background-optimized cutout annotation AD i ; then perform erosion operation on the segmentation mask M i to obtain the eroded mask ME i , set the pixel position corresponding to the mask ME i foreground area (pixel value is 1) in the background-optimized cutout annotation AD i to 1 to obtain the foreground-optimized cutout annotation AE i , and the portrait picture I i and the optimized cutout annotation AE i in the original portrait data D1 and D2 as the final data pair to obtain the optimized portrait data D rfn .

[0021] In one embodiment, the abnormal screening prompt word set H described in step 3 is composed of h prompt words describing abnormal situations, and is used to screen abnormal portrait data.

[0022] In one embodiment, the image-text similarity calculation in step 3 is performed by using the optimized portrait data D rfn The abnormal screening prompt set H is input into the image encoder and text encoder respectively, and the encoding embeddings of the two are obtained, which are e i With e t , and then calculate the image-text similarity S(e i , e t ), the calculation formula is as follows:

[0023]

[0024] Keep the portrait data with lower image-text similarity as the filtered portrait data D flt For use in subsequent steps.

[0025] In one embodiment, the random data enhancer in step 4 is composed of a plurality of predefined enhancement modes. The random data enhancer receives the filtered portrait data D flt As input, randomly select 0 to r data enhancement methods to obtain the enhanced portrait data D aug .

[0026] In order to clearly demonstrate the method for constructing a portrait cutout dataset, an embodiment of the present invention further provides a system for constructing a portrait cutout dataset based on a diffusion model. The system uses the method for constructing a portrait cutout dataset in the process of constructing the portrait cutout dataset.

[0027] A portrait cutout dataset construction system based on a diffusion model, the system comprising: a prompt construction module, a collaborative generation module, a data screening module, and a data enhancement module;

[0028] The prompt construction module is used to obtain a prompt set for data generation, by randomly intercepting a predefined prompt template and randomly filling the intercepted prompt template with a predefined attribute list to obtain a prompt set;

[0029] The collaborative generation module is used to construct a collaborative generation module for portrait cutouts, including a diffusion model layer for collaborative generation of portrait cutouts based on text-generated images, a diffusion model layer for collaborative generation of portrait cutouts based on image-generated images, and a cutout annotation optimization layer based on text-guided segmentation and dilation and erosion. Based on the prompt set and portrait data, the collaborative generation module generates portrait cutout data including a portrait image and cutout annotations, and performs text segmentation, dilation, and erosion operations on the portrait cutout data to obtain optimized portrait data.

[0030] The data screening module is used to construct an image-text similarity-based data screening module to screen portrait data, and in the screening module, image-text similarity calculation is performed on the optimized portrait data and an abnormal screening prompt set, and portrait data below a set threshold is screened as screened portrait data.

[0031] The data enhancement module is used to construct a portrait matting-oriented data enhancement module, and the screened portrait data is input into a random data enhancer, the random data enhancer randomly selects a pre-defined data enhancement mode to obtain enhanced portrait data as a portrait matting data set for model training.

[0032] To achieve the above-mentioned purposes, the embodiment of the present application further provides a portrait matting data set construction device based on a diffusion model, comprising a memory and a processor, the memory is used to store a computer program, and the processor is used to realize the portrait matting data set construction method based on the diffusion model when the computer program is executed.

[0033] Compared with the prior art, the present application has at least the following beneficial effects:

[0034] The portrait matting data set construction method based on the diffusion model provided by the present application is suitable for solving the feasibility problem of portrait matting data generation compared with the existing data set construction method; the method is based on a prompt construction strategy and a plurality of collaborative generation diffusion models, and is supplemented by a matting annotation optimization layer and a data screening module to improve the portrait matting data set construction efficiency, for automatically constructing a large-scale data set and filtering abnormal portrait matting data; based on the data enhancement module, the diversity of the portrait matting data is enriched, and the model is closer to the real scene, improves the generalization of the model on the portrait matting task, effectively alleviates the problem of insufficient scale of the real portrait matting data set, and improves the portrait matting data set construction efficiency.

[0035] The present application also provides a portrait matting data set construction system and device based on a diffusion model, for constructing a large-scale high-precision portrait matting data set. BRIEF DESCRIPTION OF DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description.

[0037] Figure 1 The flowchart of the portrait matting data set construction method based on the diffusion model provided by the embodiment of the present application;

[0038] Figure 2A structural schematic diagram of the prompt construction module based on the prompt template and the attribute list is provided for the embodiment of the present application.

[0039] Figure 3 A structural schematic diagram of the collaborative generation module of the portrait matting is provided for the embodiment of the present application.

[0040] Figure 4 A structural schematic diagram of the matting annotation optimization layer based on the text-guided segmentation and the expansion and corrosion is provided for the embodiment of the present application.

[0041] Figure 5 A structural schematic diagram of the data screening module based on the image text similarity is provided for the embodiment of the present application.

[0042] Figure 6 A structural schematic diagram of the data enhancement module of the portrait matting is provided for the embodiment of the present application.

[0043] Figure 7 A structural schematic diagram of the portrait matting dataset construction system based on the diffusion model is provided for the embodiment of the present application. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the protection scope of the present application.

[0045] In order to realize the automatic construction of large-scale high-precision portrait matting dataset and effectively alleviate the problem of insufficient real portrait matting dataset, the embodiment provides a portrait matting dataset construction method based on a diffusion model. The method is aimed at portrait matting data and is suitable for solving the feasibility problem of portrait matting data generation.

[0046] As shown in Figure 1 , the embodiment provides a portrait matting dataset construction method based on a diffusion model, which includes the following steps:

[0047] S1, a prompt construction module based on a prompt template and an attribute list is constructed, the pre-defined prompt template is randomly intercepted, and the pre-defined attribute list is used to randomly fill the intercepted prompt template to obtain a prompt set.

[0048] As shown in Figure 2 , in the embodiment, the prompt template T is spliced by n attributes Attr and n-1 connectors Conc, and the form of the prompt template T is as follows:

[0049] T = Concate(Attr1, Conc1, Attr2, Conc2,..., Attrn, Conc n-1 , Conc n-1 , Attr n )

[0050] wherein Concate(·) denotes a string concatenation operation. In this embodiment, n = 18.

[0051] The random truncation operation receives the prompt template T as input, and for each attribute Attr and corresponding connector Conc in the prompt template T, it deletes them from the prompt template T with a certain probability p, and only concatenates the remaining attributes and connectors to obtain the prompt template T drop . In this embodiment, p = 0.3.

[0052] The attribute list Ls contains a predefined list of n attribute values L, each containing no more than k attribute values att. In this embodiment, k = 22.

[0053] The attribute list random filling operation receives the truncated prompt template T drop and the attribute list Ls as input, for the attribute Attr i in the predefined prompt template, selects the corresponding attribute value list L i from the attribute list Ls i , and randomly selects an attribute value att j from L drop , concatenates it with the corresponding connector, and traverses all attributes in T te to obtain a prompt; repeat the above operation to obtain a prompt set Ts containing t prompts. In this embodiment, t = 1000000.

[0054] S2, construct a collaborative generation module for portrait cutout, generate portrait cutout data containing portrait pictures and cutout labels based on the prompt set and portrait data through the collaborative generation module, and perform text segmentation, dilation and erosion operations on the portrait cutout data to obtain optimized portrait data, wherein the collaborative generation module includes a portrait cutout collaborative generation diffusion model layer based on text-to-image, a portrait cutout collaborative generation diffusion model layer based on image-to-image, and a cutout label optimization layer based on text-guided segmentation and dilation and erosion, and specifically includes the following steps:

[0055] (1) input the prompt set Ts into the portrait cutout collaborative generation diffusion model layer based on text-to-image to obtain the first original portrait data D1 containing portrait pictures I te and cutout labels A te ;

[0056] As Figure 3As shown in the example, in this embodiment, the collaborative generation diffusion model of the portrait cutout based on the Wensheng graph adopts LayerDiffusion, receives the prompt set Ts as input, and collaboratively generates the portrait image I te And cutout mark A te , portrait picture I te And cutout mark A te As the first original portrait data D1, it can be expressed as the following formula:

[0057] D1=I te ,At e =T2I(Ts) (1)

[0058] Among them, T2I(·) represents the generation process of the collaborative generation diffusion model of portrait cutout based on Wensheng graph.

[0059] (2) Input the portrait data Ps in the open source dataset into the collaborative generation diffusion model layer based on the image-based portrait cutout to obtain the portrait image I im And cutout mark A im The second original portrait data D2;

[0060] like Figure 3 As shown in the figure, in this embodiment, the portrait cutout collaborative generation diffusion model based on the Wensheng map adopts LayerDiffusion, receives the portrait data Ps in the open source dataset as input, and the portrait data in the open source dataset adopts the training set of the P3M-10K dataset to collaboratively generate the portrait image I im And cutout mark A im , portrait picture I im And cutout mark A im As the second original portrait data D2, it can be expressed as the following formula:

[0061] D2=I im , A im =I2I(Ps) (2)

[0062] Here, I2I(·) represents the generation process of the collaborative generation diffusion model of portrait cutout based on image-to-image.

[0063] (3) The first original portrait data D1 and the second original portrait data D2 are input into the cutout annotation optimization layer based on text-guided segmentation and dilation corrosion, and the segmentation model is used to perform the cutout annotation on the portrait image I. te and I im Predict the segmentation mask M i Then dilate and combine the cutout annotation to segment the mask M i The optimized portrait data D is obtained by corrosion rfn .

[0064] likeFigure 4 As shown in the embodiment, the first original portrait data D1 and the second original portrait data D2 are input to the cutout annotation optimization layer based on text-guided segmentation and dilation corrosion; first, the original portrait image I in the original portrait data D1 and D2 is i The predefined segmentation hint B is input into the text-guided segmentation model GroundingSAM to obtain the segmentation mask M i ; Then the segmentation mask M i Perform dilation operation to obtain dilation mask MD i , mark the original portrait data D1 and D2 with A i Middle Mask MD i The pixel position corresponding to the background area (pixel value is 0) is set to 0, and the cutout mark AD after background optimization is obtained i ; Then the segmentation mask M i Perform etching operation to obtain etching mask ME i , mark the background after optimization with AD i Middle Mask ME i The pixel position corresponding to the foreground area (pixel value is 1) is set to 1, and the foreground optimized cutout mark AE is obtained i , the portrait image I in the original portrait data D1 and D2 i And optimized cutout annotation AE i As the final data pair, the optimized portrait data D is obtained. rfn .

[0065] S3. Construct a data screening module, calculate the image-text similarity between the optimized portrait data and the abnormal screening prompt set in the data screening module, and screen the portrait data below the set threshold to obtain the screened portrait data.

[0066] like Figure 5 As shown, in this embodiment, the predefined abnormal screening prompt word set H is composed of h prompt words describing abnormal situations. In this embodiment, h=5; the optimized portrait data D rfn and the predefined abnormal screening prompt set H are input into the CLIP image encoder and CLIP text encoder respectively, and the encoding embeddings of the two are obtained, which are e i With e t , and then calculate the image-text similarity S(e i , e t ), the calculation formula is as follows:

[0067]

[0068] The high image-text similarity indicates that the generated portrait data is more related to an abnormal situation, and is considered to be low-quality portrait data. Since low-quality portrait data affects the learning efficiency of the model, the optimized portrait data D rfn The medium and low-quality portrait data is filtered out in this step, and the high-quality portrait data is retained as the filtered portrait data D flt for subsequent steps.

[0069] S4, a data enhancement module is constructed, and the filtered portrait data is input into a random data enhancer. The random data enhancer randomly selects a predefined data enhancement mode to obtain enhanced portrait data.

[0070] As shown in Figure 6 , the random data enhancer is composed of a plurality of predefined enhancement modes. The random data enhancer receives the filtered portrait data D flt as input, and randomly selects 0 to r data enhancement modes to apply to the portrait data. In this embodiment, r = 4, which are Gaussian noise enhancement, grayscale enhancement, gamma correction enhancement, and similar color background enhancement. The enhanced portrait data D aug is obtained through the random data enhancer, and a portrait matting data set is constructed for model training.

[0071] In order to clearly show the portrait matting data set construction method, the embodiment of the present application also provides a portrait matting data set construction system based on a diffusion model. The system uses the portrait matting data set construction method in the process of constructing the portrait matting data set.

[0072] As shown in Figure 7 , the portrait matting data set construction system in the embodiment includes a prompt construction module, a collaborative generation module, a data filtering module, and a data enhancement module.

[0073] The prompt construction module is used to obtain a prompt set for data generation. The pre-defined prompt template is randomly intercepted, and the pre-defined attribute list is used to randomly fill the intercepted prompt template to obtain the prompt set.

[0074] The collaborative generation module is used to construct a portrait matting oriented collaborative generation module, which includes a portrait matting collaborative generation diffusion model layer based on text-to-image, a portrait matting collaborative generation diffusion model layer based on image-to-image, and a matting annotation optimization layer based on text guided segmentation and dilation erosion. The portrait matting data containing the portrait picture and the matting annotation is generated based on the prompt set and the portrait data through the collaborative generation module, and the portrait matting data is subjected to text segmentation, dilation and erosion operations to obtain the optimized portrait data.

[0075] The data screening module is configured to construct an image-text similarity-based data screening module to perform portrait data screening. In the screening module, image-text similarity calculation is performed on the optimized portrait data and the abnormal screening prompt set, and portrait data below a set threshold is screened as screened portrait data.

[0076] The data enhancement module is configured to construct a portrait matting-oriented data enhancement module. The screened portrait data is input into a random data enhancer. The random data enhancer randomly selects a pre-defined data enhancement mode to obtain enhanced portrait data as a portrait matting data set for model training.

[0077] Based on the above inventive concept, the embodiments also provide a portrait matting data set construction device based on a diffusion model, including a memory and a processor. The memory is configured to store a computer program, and the processor is configured to implement the above portrait matting data set construction method based on a diffusion model when executing the computer program.

[0078] The specific embodiments described above have explained the technical solutions and beneficial effects of the present application. It should be understood that the above description is only the most preferred embodiment of the present application and is not intended to limit the present application. Any modifications, supplements, and equivalent replacements made within the principle range of the present application shall be included in the protection scope of the present application.

Claims

1. A method for constructing a portrait cutout dataset based on a diffusion model, characterized in that: The following steps are involved: Step 1: Construct a prompt construction module based on prompt templates and attribute lists, randomly intercept the pre-defined prompt templates, and randomly fill the intercepted prompt templates with the pre-defined attribute lists to obtain a prompt set; Step 2: Construct a collaborative generation module for portrait cutouts. Based on the prompt set and the portrait data, the collaborative generation module generates portrait cutout data including a portrait image and cutout annotations. The portrait cutout data is subjected to text segmentation, dilation, and erosion operations to obtain optimized portrait data. The collaborative generation module comprises a diffusion model layer for collaborative generation of portrait cutouts based on text-generated images, a diffusion model layer for collaborative generation of portrait cutouts based on image-generated images, and a cutout annotation optimization layer based on text-guided segmentation and dilation and erosion. The module specifically comprises the following steps: (1) Collect the prompts Input into the portrait cutout collaborative generation diffusion model layer based on the Wensheng graph to obtain a portrait image and cutout annotation The first original portrait data ; (2) The portrait data in the open source dataset Input to the portrait cutout collaborative generation diffusion model layer based on the image raw image to obtain a portrait image and cutout annotation The second original portrait data ; (3) The first original portrait data and the second original portrait data Input to the text-guided segmentation and dilation-erosion based cutout annotation optimization layer, and convert the original portrait data and Original portrait image in With segmentation prompt Input to the text-guided segmentation model to obtain the segmentation mask ; Then the segmentation mask Perform dilation operation to obtain dilation mask , the original portrait data and Cutout annotation in Middle mask The pixel position corresponding to the background area is set to 0, and the cutout annotation after background optimization is obtained ; Then the segmentation mask Perform etching operation to obtain etching mask , mark the cutout after background optimization Middle mask The pixel position corresponding to the foreground area is set to 1, and the foreground optimized cutout annotation is obtained. , the original portrait data and Portrait images in And optimized cutout annotation As the final data pair, the optimized portrait data is obtained ; Step 3: Construct a data screening module, in which the image-text similarity between the optimized portrait data and the abnormal screening prompt set is calculated, and portrait data below the set threshold is screened to obtain the screened portrait data; Step 4: Build a data enhancement module and input the filtered portrait data into the random data enhancer. The random data enhancer will randomly select a pre-defined data enhancement method to obtain enhanced portrait data.

2. The method for constructing a portrait cutout dataset based on a diffusion model according to claim 1, characterized in that: The prompt template described in step 1 The form is as follows: ， in, Represents string concatenation operation; Random interception operation accepts predefined prompt templates As input, Each attribute in The corresponding connector , with a certain probability value From the prompt template Delete it, and concatenate the remaining attributes and connectors to get the intercepted prompt template ; Property List Contains predefined List of attribute values , each list contains no more than attribute values ; The attribute list random filling operation receives the intercepted prompt template With property list As input, for the attributes in the predefined prompt template , from the property list Select the corresponding attribute value list , and randomly from Select a property value , concatenate it with the corresponding connector and traverse After completing all the properties in the above steps, you will get a prompt. Repeat the above steps to get Prompt collection of prompts .

3. The method for constructing a portrait cutout dataset based on a diffusion model according to claim 1, characterized in that: The set of exception screening prompts described in step 3 Depend on It is composed of a prompt describing the abnormal situation and is used to filter abnormal portrait data.

4. The method for constructing a portrait cutout dataset based on a diffusion model according to claim 1, characterized in that: The image-text similarity calculation described in step 3 is performed by taking the optimized portrait data Collection of exception filtering prompts Input them into the image encoder and text encoder respectively to get the encoding embedding of the two, which are and , and then calculate the image-text similarity between the two , the calculation formula is as follows: ， Keep the portrait data with lower image-text similarity as the filtered portrait data For use in subsequent steps.

5. The method for constructing a portrait cutout dataset based on a diffusion model according to claim 1, characterized in that: The random data enhancer described in step 4 is composed of a variety of predefined enhancement methods. The random data enhancer receives the filtered portrait data. As input, randomly select 0 to Data enhancement methods to obtain enhanced portrait data .

6. A portrait cutout dataset system based on a diffusion model, characterized in that: Using the method for constructing a portrait cutout dataset according to any one of claims 1 to 5, the system comprises: a prompt construction module, a collaborative generation module, a data screening module, and a data enhancement module; The prompt construction module is used to obtain a prompt set for data generation, by randomly intercepting a predefined prompt template and randomly filling the intercepted prompt template with a predefined attribute list to obtain a prompt set; The collaborative generation module is used to construct a collaborative generation module for portrait cutouts, including a diffusion model layer for collaborative generation of portrait cutouts based on text-generated images, a diffusion model layer for collaborative generation of portrait cutouts based on image-generated images, and a cutout annotation optimization layer based on text-guided segmentation and dilation and erosion. Based on the prompt set and portrait data, the collaborative generation module generates portrait cutout data including a portrait image and cutout annotations, and performs text segmentation, dilation, and erosion operations on the portrait cutout data to obtain optimized portrait data. The data screening module is used to construct a data screening module based on image-text similarity to screen portrait data. In the screening module, image-text similarity is calculated between the optimized portrait data and the abnormal screening prompt set, and portrait data below a set threshold is screened as screened portrait data. The data enhancement module is used to construct a data enhancement module for portrait cutouts. The filtered portrait data is input into a random data enhancer. The random data enhancer randomly selects a predefined data enhancement method to obtain enhanced portrait data as a portrait cutout dataset for model training.

7. A device for constructing a portrait cutout dataset based on a diffusion model, comprising a memory and a processor, wherein the memory is used to store a computer program, characterized in that: The processor is configured to implement the method for constructing a portrait cutout dataset based on a diffusion model according to any one of claims 1 to 5 when executing the computer program.

Images