A method and system for automatically generating a portrait cutout of a three-dimensional image

Abstract

The present invention discloses a method and system for automatically generating a trimap of a portrait image. First, a Parsing‑Net network for obtaining four semantic categories of a portrait image, a Mask‑Net network for obtaining a preliminary trimap result, and a Trimap‑Net network for obtaining a final trimap result are trained; then, the portrait image to be processed is input into Parsing‑Net to obtain four semantic categories of the portrait image, the four semantic categories of the portrait image are processed to obtain AOI, and the original portrait image to be processed and the corresponding mask are input into Mask‑Net to obtain a preliminary trimap result, the result is processed to generate a trimap of a trimap, and the portrait image to be processed and the trimap of the trimap are input into Trimap‑Net to obtain the final trimap result. The present invention uses a human body parsing method to solve the problem of automatically generating trimap using semantic information, and uses the automatically generated trimap to achieve the correct processing of hollowed-out hair, and can more specifically perform trimap on the hair and edge parts, thereby making the portrait cutout more accurate.

Description

Technical Field

[0001] The present invention relates to the technical fields of computer vision and image processing, and in particular to a method and system for automatically generating a portrait image of a three-dimensional image. Background Art

[0002] Cutting out refers to the problem of accurately estimating the opacity of foreground objects in images or video sequences. It is widely used in the fields of filmmaking and digital image editing. Cutting out algorithms can be divided into multiple categories, such as animal cutouts, transparent object cutouts, and portrait cutouts. For portrait cutouts, the characteristic is that the portrait occupies the majority of the image, and there is no need to consider transparent objects. The objects that need to be cut out also include some people's accessories or personal belongings. The difficulty lies mainly in the hair part. Because the characteristics of the hair category are very complex, under similar colors or different lighting conditions, the features learned by the network in the hair part will be very similar to the background, and it is difficult to separate the hair to obtain a fine segmentation result. In addition, there is currently no finely labeled data set specifically for portrait cutouts, which increases the difficulty of solving the cutout problem.

[0003] The cutout model can regard the image as a linear combination of the foreground and the background. That is, given a picture, it is necessary to predict the opacity of the foreground and the background as well as the boundary area between the foreground and the background. The absolute foreground outputs 1, the absolute background outputs 0, and the transition area outputs the opacity of the foreground in the background, with a value between 0 and 1. Depending on whether the trimap is input at the same time as the original image, the current methods for solving the cutout problem can be divided into two categories.

[0004] One of the current existing technologies is a cutout method that requires trimap, and this type of method includes the DIM method and the GCA method. Specifically, for a color image represented by RGB, the three RGB values ​​of the input image are known to find seven unknowns: rgb of the foreground Fg, rgb of the background Bg, and α. This is an ill-posed problem. Inputting the trimap of the trigraph as a hint while inputting the RGB image can reduce the solution space of the ill-posed problem. The trimap of the trigraph divides the image into a certain foreground Fg, a certain background Bg, and an unknown unknown area. After knowing more foreground and background feature information, in the cutout task, it is only necessary to estimate the opacity of the foreground in the unknown area, which reduces the solution space of the problem and can obtain a precise and accurate cutout result. The disadvantage of this type of technology is that the user needs to provide a precisely marked trimap of the trigraph. However, manually making a trimap of the trigraph is a very labor-intensive and time-consuming task, so using this type of method requires first solving the problem of automatically generating an accurate trimap.

[0005] At present, the methods for generating trimap can be divided into three categories. The first category uses special equipment to shoot to obtain trimap. In the papers such as "Automatic Natural Video Matting with Depth", "Automatic Trimap Generation and Consistent Matting for Light-Field Images", and "Image matting with color and depth information", the authors proposed using a special camera to obtain the depth information of the photo to generate trimap. However, since most photos in daily life are not taken with such special cameras, it is not universal. The second category dilates and corrodes a part of the picture to obtain the unknown part. In the paper "Automatic trimap generation for digital image matting", the trimap is generated by dilating and corroding the result of image segmentation; in the paper "Automatic trimap generation for image matting", the author classifies the significant superpixel foreground, and then dilates and corrodes the foreground part to obtain the trimap. This type of method does not distinguish between the difficult hair area and the easy clothing body area. It only dilates and corrodes the segmentation results. If it dilates too much, the foreground and background will be over-segmented into the unknown area. If it dilates too little, the hair part will not be included in the unknown area. The third category is the method using machine learning. In the papers "Deepautomatic portrait matting" and "Salient Image Matting", a convolutional neural network is used to classify the image into three categories to directly obtain the trimap. The unknown area in the trimap is a mixture of the foreground and background. The backgrounds of different images vary greatly, and it is difficult to learn the common features within this class. In addition, there is currently no data set that annotates the trimap of portrait data, so the three-category classification is difficult to achieve and obtain good results.

[0006] The second existing technology is the matting method without trimap. Since trimap is not automatically generated, there are some methods for matting without trimap. According to the different prompts input at the same time as the original image, it can be roughly divided into three categories: input background, directly input image, and input semantic segmentation. The first category uses the background as a prompt, including Background matting, inputting the original image, background, soft segmentation prompts, and action prompts for video segmentation. The second category inputs a single image for matting, including SHM and MODNet. SHM first classifies the image into three categories to generate a trimap, and then finely segments the unknown area in the trimap. MODNET first predicts the position of the portrait to obtain the portrait segmentation result, and then another submodule repairs the edge hair part. The third category uses the semantic segmentation result mask as a prompt in MGMatting, inputs the original image and the image segmentation result mask, and outputs the matting result through the encoder and decoder. The decoder adds side outputs at a scale of 1 / 8 and 1 / 4 of the image feature size, and uses the output of the previous layer as guidance information for the current layer through the PRM (Progressive Refinement Module) to guide the current layer to output more accurate matting results. The disadvantages of this type of technology are: the first type uses the background as an additional input, which is very limited in practical application. First, it is necessary to take another background picture without foreground, so it cannot be applied to real-time matting. In addition, this algorithm has very strict requirements on the shooting environment. The lighting, camera position, and camera parameters of the background image and the original image must be as consistent as possible, which is difficult to ensure in practical applications. The second and third types of methods first process the image through machine learning to generate the prompts required for the next step, and generate a trimap or semantic segmentation mask. In addition, the method of using mask as a prompt does not reduce the solution space of the problem, and cannot perform fine matting on the hair part, resulting in the final prediction effect of the hair not being accurate enough. Summary of the invention

[0007] The purpose of the present invention is to overcome the shortcomings of the existing methods and propose a method and system for automatically generating a three-dimensional image of a portrait. The main problem solved by the present invention is how to automatically generate a three-dimensional image so that a high-precision cutout result can be obtained by only inputting the original image.

[0008] In order to solve the above problem, the present invention proposes a method for automatically generating a three-dimensional portrait image cutout, the method comprising:

[0009] Collect portrait image datasets including portrait images and corresponding alpha images, and use them to construct a portrait image semantic four-category dataset;

[0010] The improved Self-Human-Parsing network is trained using the portrait image dataset and the portrait image semantic four-classification dataset to obtain the portrait image semantic four-classification network Parsing-Net, where the input of Parsing-Net is the portrait image and the output is the corresponding portrait image semantic four-classification result;

[0011] Each image in the portrait image semantic four-classification dataset is processed to obtain an AOI dataset including the original image, non-hair and non-edge internal area AOI, and portrait semantic segmentation mask, which is used to train Mask-NET, and then the AOI dataset is used to input the trained Mask-NET to obtain a preliminary portrait cutout result dataset;

[0012] Processing each image in the preliminary portrait cutout result data set to obtain a trimap data set including an original image, a trimap, and an alpha image;

[0013] Using the trimap dataset, the cutout network DIM with the original image and the trimap as input is retrained to make it suitable for the trimap dataset. The trained network is named Trimap-Net.

[0014] Input the portrait image to be processed into the Parsing-Net to obtain four semantic classifications of the portrait image;

[0015] The portrait image is processed by the four semantic categories to obtain AOI, and the original portrait image to be processed and the corresponding mask are input into Mask-Net to obtain a preliminary cutout result, and the result is processed to generate a trimap;

[0016] The portrait image to be processed and the trimap are input into the Trimap-Net to obtain the final cutout result.

[0017] Preferably, the collection of a portrait image dataset including portrait images and corresponding alpha images, and the construction of a portrait image semantic four-category dataset based on the dataset, is specifically as follows:

[0018] The collected portrait image dataset including portrait images and corresponding alpha images is input into the existing human body parsing model and face parsing model to obtain human body parsing images and face parsing images respectively. As needed, the facial features and neck of face parsing are merged into the same category, and the clothes and body of human body parsing are merged into the same category to obtain the portrait semantic four-category map.

[0019] Preferably, the processing of each image in the portrait image semantic four-classification data set to obtain an AOI data set including the original image, the non-hair and non-edge internal area AOI, and the portrait semantic segmentation mask is specifically as follows:

[0020] For each image in the portrait image semantic four-category dataset, the hair, skin and clothing areas are merged to obtain a portrait semantic segmentation mask, the hair and edge parts are combined to obtain AOE, and the AOE is subtracted from the portrait semantic four-category dataset to obtain the non-hair and non-edge internal area AOI. All the original portrait images in the portrait image dataset and the corresponding masks and AOIs together form an AOI dataset.

[0021] Preferably, the training of Mask-NET is specifically:

[0022] Mask-Net is improved based on the MGMating network. The semantic information of the internal area AOI is combined in the first output of the decoder, so that the network can ensure the prediction accuracy of the internal area when updating, and reduce the calculation range of the loss function and the solution space.

[0023] Mask-Net’s loss function L final as follows:

[0024]

[0025]

[0026] Where l represents the output of the lth layer, L l represents the loss function of the output of the lth layer, ω l The output weights of the first, second and third layers are 1:2:3 respectively. The purpose is to give a larger weight to the third layer that contains more details. is the predicted value of the output of the lth layer, α is the true value of the final output, and g l Indicates the area where the prediction value is updated and the loss function is calculated. It refers to the area where the prediction value of the previous layer is between 0 and 1, which is equivalent to the unknown area of ​​the trimap.

[0027] Preferably, the processing of each image in the preliminary portrait cutout result data set to obtain a trimap data set including an original image, a trimap and an alpha image is specifically as follows:

[0028] For each image in the preliminary portrait cutout result dataset, the part with a predicted value of 1 is set as the foreground part in the trimap, the part with a predicted value of 0 is set as the background part in the trimap, and the part with a predicted value between 0 and 1 is set as the unknown area of ​​the trimap, and the final trimap is obtained. All the original portrait images and the corresponding trimaps and alpha images in the portrait image dataset together form a trimap dataset.

[0029] Accordingly, the present invention also provides a portrait image cutout system for automatically generating a three-dimensional image, comprising:

[0030] A portrait image dataset construction unit is used to collect a portrait image dataset including portrait images and corresponding alpha images, and to construct a portrait image semantic four-classification dataset based on the dataset;

[0031] A Parsing-Net network training unit is used to train an improved Self-Human-Parsing network using the portrait image data set and the portrait image semantic four-classification data set to obtain a portrait image semantic four-classification network Parsing-Net, wherein the input of Parsing-Net is the portrait image, and the output is the corresponding portrait image semantic four-classification result;

[0032] A trimap data set construction unit is used to process each image in the portrait image semantic four-classification data set to obtain an AOI data set including an original image, an internal area AOI of non-hair and non-edge, and a portrait semantic segmentation mask, which is used to train Mask-NET, and then use the AOI data set to input the trained Mask-NET to obtain a preliminary portrait cutout result data set, and then process each image in the preliminary portrait cutout result data set to obtain a trimap data set including an original image, a trimap, and an alpha image;

[0033] A Trimap-Net network training unit is used to use the trimap data set to retrain the cutout network DIM that takes the original image and the trimap as input, so that it is suitable for the trimap data set. The trained network is named Trimap-Net;

[0034] The portrait image cutout unit is used to input the portrait image to be processed into the Parsing-Net to obtain the portrait image semantic four categories, then process the portrait image semantic four categories to obtain AOI, and combine the original portrait image to be processed and the corresponding mask to input into the Mask-Net to obtain a preliminary cutout result, process the result to generate a trimap, and input the portrait image to be processed and the trimap into the Trimap-Net to obtain the final cutout result.

[0035] The implementation of the present invention has the following beneficial effects:

[0036] The present invention uses a human body analysis method to solve the problem of automatically generating trimap using semantic information, and uses the automatically generated trimap to achieve correct processing of hollowed-out hair, and can more specifically cut out the hair and edge parts, thereby making the portrait cutout more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 is an overall flow chart of a method for automatically generating a three-dimensional portrait cutout according to an embodiment of the present invention;

[0038] Figure 2 is a schematic diagram of four-category semantic classification of portraits according to an embodiment of the present invention;

[0039] Figure 3 is a schematic diagram of AOI generation according to an embodiment of the present invention;

[0040] Figure 4 Schematic diagram of Mask-Net according to an embodiment of the present invention;

[0041] Figure 5 is a schematic diagram of trimap generation according to an embodiment of the present invention;

[0042] Figure 6 It is a structural diagram of a portrait image cutout system for automatically generating a three-dimensional image according to an embodiment of the present invention. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] Figure 1 FIG. 1 is a general flow chart of a method for automatically generating a three-dimensional portrait cutout according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0045] S1, collects portrait image datasets including portrait images and corresponding alpha images, and uses them to construct a portrait image semantic four-category dataset;

[0046] S2, using the portrait image dataset and the portrait image semantic four-category dataset to train the improved Self-Human-Parsing network to obtain the portrait image semantic four-category network Parsing-Net, the input of Parsing-Net is the portrait image, and the output is the corresponding portrait image semantic four-category result;

[0047] S3, processing each image in the portrait image semantic four-classification dataset to obtain an AOI dataset including the original image, non-hair and non-edge internal area AOI, and portrait semantic segmentation mask, for training Mask-NET, and then using the AOI dataset to input the trained Mask-NET to obtain a preliminary portrait cutout result dataset;

[0048] S4, processing each image in the preliminary portrait cutout result data set to obtain a trimap data set including an original image, a trimap, and an alpha image;

[0049] S5, using the trimap dataset, retraining the cutout network DIM that takes the original image and the trimap as input, so that it is suitable for the trimap dataset, and the trained network is named Trimap-Net;

[0050] S6, inputting the portrait image to be processed into the Parsing-Net to obtain four semantic classifications of the portrait image;

[0051] S7, processing the portrait image semantic four categories to obtain AOI, and combining the original portrait image to be processed with the corresponding mask to be input into Mask-Net to obtain a preliminary cutout result, and processing the result to generate a trimap;

[0052] S8, inputting the portrait image to be processed and the trimap into the Trimap-Net to obtain a final cutout result.

[0053] Step S1 is as follows:

[0054] In view of the semantic needs of portrait cutout, the key is to distinguish the hair part that is difficult to cut out from the non-hair part, and segment the portrait image into the following four areas: hair, skin, clothes and background. Figure 2 shown.

[0055] Human body parsing refers to segmenting a person in an image into multiple semantically consistent regions, such as arms, legs, face, hair, clothes, etc. Face parsing refers to segmenting the facial features, hair, and neck regions of a person in an image.

[0056] The collected portrait image dataset, including 300 portrait images and corresponding alpha images, was input into the existing human body parsing model (paper "Self-Correction for Human Parsing") and face parsing model (github: Using modified BiSeNet for face parsing in PyTorch) to obtain human body parsing images and face parsing images respectively. As needed, the facial features and neck of the face parsing were merged into the same category, and the clothes and body of the human body parsing were merged into the same category to obtain the portrait semantic four-category map;

[0057] Step S2 is as follows:

[0058] After obtaining the portrait image semantic four-category dataset, this embodiment modifies the network parameters for training based on the Self-Human-Parsing network. The original network is trained based on the 19-category dataset in LIP. Therefore, to adapt to new applications, the four-category dataset of the present invention is used for retraining to obtain the portrait image semantic four-category network Parsing-Net, which realizes the output of the corresponding portrait image semantic four-category result by inputting a picture.

[0059] Step S3 is as follows:

[0060] For each image in the portrait semantic four-category dataset, the hair, skin and clothing areas are merged to obtain the portrait semantic segmentation mask, the hair and edge parts are combined to obtain the AOE (area of ​​external), and the AOE is subtracted from the portrait semantic four-category to obtain the non-hair and non-edge internal area AOI (area of ​​inner). The AOI generation process is as follows: Figure 3 All the original portrait images and the corresponding masks in the portrait image dataset together with the AOI form the AOI dataset.

[0061] Then use the AOI dataset to train Mask-NET:

[0062] Mask-Net is improved based on MGMatting network, such as Figure 4 When testing the model results of MGMatting, it was found that for some portrait images, the internal prediction of MGMatting would be missing. By analyzing its network structure, it was found that the decoder of MGMatting calculated a region g according to the output of the previous layer.l . g l One of its functions is to use the current layer output to update the previous layer output g l The predicted value of the region, another role is to supervise the calculation area of ​​information loss. The problem of missing internal prediction is that if the output α in the first layer 0 If the foreground (should be 1 value) is incorrectly predicted as the background (0 value), or the background (should be 0 value) is predicted as the foreground (1 value), it will be ignored in the subsequent loss calculation and prediction update, and the error will be passed to the final prediction result. Therefore, in view of the shortcomings of the MGMating network, it is improved by combining the semantic information of the internal area AOI in the first output of the decoder, so that the network can ensure the prediction accuracy of the internal area when updating, and narrow the calculation range of the loss function and reduce the solution space;

[0063] Mask-Net’s loss function L final as follows:

[0064]

[0065]

[0066] Where l represents the output of the lth layer, L l represents the loss function of the output of the lth layer, ω l The output weights of the first, second and third layers are 1:2:3 respectively. The purpose is to give a larger weight to the third layer that contains more details. is the predicted value of the output of the lth layer, α is the true value of the final output, and g l Indicates the area where the prediction value is updated and the loss function is calculated. It refers to the area where the prediction value of the previous layer is between 0 and 1, which is equivalent to the unknown area of ​​the trimap.

[0067] Step S4 is as follows:

[0068] like Figure 5 As shown in FIG. 1 , for each image in the preliminary portrait cutout result dataset, the part with a predicted value of 1 is set as the foreground part in the trimap, the part with a predicted value of 0 is set as the background part in the trimap, and the part with a predicted value between 0 and 1 is set as the unknown area of ​​the trimap to obtain the final trimap. All original portrait images and the corresponding trimaps and alpha images in the portrait dataset together form a trimap dataset.

[0069] Step S5 is as follows:

[0070] The cutout network used in this embodiment, which takes the original image and the trimap as input, is the DIM network. The training data trimap originally used by this network is directly generated using the alpha label (that is, the uncertain area only includes the hair and edge area). Therefore, the trained network is only for the hair part and is not suitable for the trimap data of the trimap of this method. Therefore, for the trimap generated by this method, the DIM network is retrained to adapt to the entire portrait cutout process of this method, so that it can better predict the uncertain area in the trimap data of the trimap of this method.

[0071] Accordingly, the present invention also provides a portrait image cutout system for automatically generating a three-dimensional image, such as Figure 6 As shown, including:

[0072] A portrait image data set construction unit 1 is used to collect a portrait image data set including portrait images and corresponding alpha images, and to construct a portrait image semantic four-classification data set based on the data;

[0073] The Parsing-Net network training unit 2 is used to train the improved Self-Human-Parsing network using the portrait image data set and the portrait image semantic four-classification data set to obtain the portrait image semantic four-classification network Parsing-Net, the input of Parsing-Net is the portrait image, and the output is the corresponding portrait image semantic four-classification result;

[0074] The trimap data set construction unit 3 is used to process each image in the portrait image semantic four-classification data set to obtain an AOI data set including the original image, the non-hair and non-edge internal area AOI, and the portrait semantic segmentation mask, which is used to train Mask-NET, and then use the AOI data set to input the trained Mask-NET to obtain a preliminary portrait cutout result data set, and then process each image in the preliminary portrait cutout result data set to obtain a trimap data set including the original image, the trimap, and the alpha image;

[0075] The trimap-Net network training unit 4 is used to use the trimap data set to retrain the cutout network DIM that takes the original image and the trimap as input, so that it is suitable for the trimap data set. The trained network is named Trimap-Net;

[0076] The portrait image cutout unit 5 is used to input the portrait image to be processed into the Parsing-Net to obtain the portrait image semantic four categories, then process the portrait image semantic four categories to obtain AOI, and combine the original portrait image to be processed and the corresponding mask to input into the Mask-Net to obtain a preliminary cutout result, process the result to generate a trimap, and input the portrait image to be processed and the trimap into the Trimap-Net to obtain the final cutout result.

[0077] Therefore, the present invention uses a human body analysis method to solve the problem of automatically generating trimap using semantic information, and uses the automatically generated trimap to achieve correct processing of hollowed-out hair, and can more specifically cut out the hair and edge parts, thereby making the portrait cutout more accurate.

[0078] The above is a detailed introduction to a method and system for automatically generating a three-dimensional portrait cutout provided by an embodiment of the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for automatically generating a portrait cutout of a three-dimensional image, characterized in that: The method comprises: Collect portrait image datasets including portrait images and corresponding alpha images, and use them to construct a portrait image semantic four-category dataset; The improved Self-Human-Parsing network is trained using the portrait image dataset and the portrait image semantic four-classification dataset to obtain the portrait image semantic four-classification network Parsing-Net, where the input of Parsing-Net is the portrait image and the output is the corresponding portrait image semantic four-classification result; Each image in the portrait image semantic four-classification dataset is processed to obtain an AOI dataset including the original image, non-hair and non-edge internal area AOI, and portrait semantic segmentation mask, which is used to train Mask-NET, and then the AOI dataset is used to input the trained Mask-NET to obtain a preliminary portrait cutout result dataset; Processing each image in the preliminary portrait cutout result data set to obtain a trimap data set including an original image, a trimap, and an alpha image; Using the trimap dataset, the cutout network DIM with the original image and the trimap as input is retrained to make it suitable for the trimap dataset. The trained network is named Trimap-Net. Input the portrait image to be processed into the Parsing-Net to obtain four semantic classifications of the portrait image; The portrait image is processed by the four semantic categories to obtain AOI, and the original portrait image to be processed and the corresponding mask are input into Mask-Net to obtain a preliminary cutout result, and the result is processed to generate a trimap; Inputting the portrait image to be processed and the trimap into the Trimap-Net to obtain the final cutout result; The collection of a portrait image dataset including portrait images and corresponding alpha images, and the construction of a portrait image semantic four-category dataset based on the dataset, is specifically: To meet the semantic needs of portrait cutout, the portrait image needs to be segmented into four regions: hair, skin, clothes, and background. Human body analysis is to segment the person in the image into multiple semantically consistent regions, including arms, legs, face, hair, and clothes; face analysis is to segment the facial features, hair, and neck regions of the face in the image; The collected portrait image dataset including portrait images and corresponding alpha images is input into the existing human body parsing model and face parsing model to obtain human body parsing images and face parsing images respectively. As needed, the facial features and neck of face parsing are merged into the same category, and the clothes and body of human body parsing are merged into the same category to obtain the portrait semantic four-category map.

2. A method for automatically generating a portrait image of a three-dimensional image according to claim 1, characterized in that: The processing of each image in the portrait image semantic four-classification data set to obtain an AOI data set including the original image, the non-hair and non-edge internal area AOI, and the portrait semantic segmentation mask is specifically as follows: For each image in the portrait image semantic four-category dataset, the hair, skin and clothing areas are merged to obtain a portrait semantic segmentation mask, the hair and edge parts are combined to obtain AOE, and the AOE is subtracted from the portrait semantic four-category dataset to obtain the non-hair and non-edge internal area AOI. All the original portrait images in the portrait image dataset and the corresponding masks and AOIs together form an AOI dataset.

3. The method for automatically generating a portrait image of a three-dimensional image according to claim 1, characterized in that: The training of Mask-NET is specifically as follows: Mask-Net is improved based on the MGMating network. The semantic information of the internal area AOI is combined in the first output of the decoder, so that the network can ensure the prediction accuracy of the internal area when updating, and reduce the calculation range of the loss function and the solution space. Mask-Net’s loss function L final as follows: Where l represents the output of the lth layer, L l represents the loss function of the output of the lth layer, ω l The output weights of the first, second and third layers are 1:2:3 respectively. The purpose is to give a larger weight to the third layer that contains more details. is the predicted value of the output of the lth layer, α is the true value of the final output, and g l The area representing the update of the prediction value and the calculation of the loss function refers to the area where the prediction value of the previous layer is between 0 and 1, which is equivalent to the unknown area of ​​the trimap.

4. The method for automatically generating a portrait image of a three-dimensional image according to claim 1, characterized in that: The processing of each image in the preliminary portrait cutout result data set to obtain a trimap data set including an original image, a trimap, and an alpha image is specifically as follows: For each image in the preliminary portrait cutout result dataset, the part with a predicted value of 1 is set as the foreground part in the trimap, the part with a predicted value of 0 is set as the background part in the trimap, and the part with a predicted value between 0 and 1 is set as the unknown area of ​​the trimap, and the final trimap is obtained. All the original portrait images and the corresponding trimaps and alpha images in the portrait image dataset together form a trimap dataset.

5. A portrait cutout system for automatically generating a three-dimensional image, characterized in that: The system comprises: A portrait image dataset construction unit is used to collect a portrait image dataset including portrait images and corresponding alpha images, and to construct a portrait image semantic four-classification dataset based on the dataset; A Parsing-Net network training unit is used to train an improved Self-Human-Parsing network using the portrait image data set and the portrait image semantic four-classification data set to obtain a portrait image semantic four-classification network Parsing-Net, wherein the input of Parsing-Net is the portrait image, and the output is the corresponding portrait image semantic four-classification result; A trimap data set construction unit is used to process each image in the portrait image semantic four-classification data set to obtain an AOI data set including an original image, an internal area AOI of non-hair and non-edge, and a portrait semantic segmentation mask, which is used to train Mask-NET, and then use the AOI data set to input the trained Mask-NET to obtain a preliminary portrait cutout result data set, and then process each image in the preliminary portrait cutout result data set to obtain a trimap data set including an original image, a trimap, and an alpha image; A Trimap-Net network training unit is used to use the trimap data set to retrain the cutout network DIM that takes the original image and the trimap as input, so that it is suitable for the trimap data set. The trained network is named Trimap-Net; A portrait image cutout unit is used to input the portrait image to be processed into the Parsing-Net to obtain the portrait image semantic four categories, then process the portrait image semantic four categories to obtain AOI, and combine the original portrait image to be processed and the corresponding mask to input into the Mask-Net to obtain a preliminary cutout result, process the result to generate a trimap, and input the portrait image to be processed and the trimap into the Trimap-Net to obtain the final cutout result; The portrait image dataset construction unit is used to collect a portrait image dataset including portrait images and corresponding alpha images, and construct a portrait image semantic four-category dataset based on the dataset, specifically: To meet the semantic needs of portrait cutout, the portrait image needs to be segmented into four regions: hair, skin, clothes, and background. Human body analysis is to segment the person in the image into multiple semantically consistent regions, including arms, legs, face, hair, and clothes; face analysis is to segment the facial features, hair, and neck regions of the face in the image; The collected portrait image dataset including portrait images and corresponding alpha images is input into the existing human body parsing model and face parsing model to obtain human body parsing images and face parsing images respectively. As needed, the facial features and neck of face parsing are merged into the same category, and the clothes and body of human body parsing are merged into the same category to obtain the portrait semantic four-category map.

Images