A portrait cutout method based on mixed labeled data

Abstract

The present invention proposes a portrait cutout method based on mixed annotated data, which relates to the technical field of image processing. It solves the problem that, in a portrait cutout method based on deep learning, the efficiency of portrait detail extraction is low when facing coarsely annotated data, and accurate portrait cutout cannot be achieved. First, a mixed annotated image dataset is preprocessed, and then a portrait cutout neural network model for identifying, extracting and fusing image features and updating a supervision signal of the mixed annotated image dataset is constructed. Then, the portrait cutout neural network model is trained, and the image to be cutout is further input into the trained portrait cutout neural network model, and a prediction result of a transparency mask of the image to be cutout is output. Finally, according to the prediction result of the transparency mask of the image to be cutout, the portrait image is cut out from the image to be cutout, thereby reducing the dependence on the finely annotated portrait dataset, improving the efficiency of portrait detail extraction, and achieving accurate portrait cutout.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a portrait cutout method based on mixed annotation data. Background Art

[0002] The portrait cutout technology predicts the transparency mask alphamatte. The white area in alphamatte represents the foreground of the image, and the black area represents the background of the image. By predicting alphamatte, the foreground or background of the portrait can be cut out to achieve portrait cutout.

[0003] Currently, portrait matting methods are mainly divided into traditional portrait matting and deep learning portrait matting. Traditional portrait matting methods typically use RGB images and other auxiliary inputs as input and achieve portrait matting by predicting alphamatte. These auxiliary inputs are generally trimap or scribbles. A trimap roughly segments an image into foreground, background, and transition regions, while scribbles represent a small number of pixels belonging to the foreground or background. To further improve the efficiency of portrait matting, deep learning techniques have been introduced, making them the most effective method for extracting portrait matting images. For example, in 2017, Xu et al. used a deep learning model similar to VGG to directly predict alphamatte using RGB images and trimap. This is considered a pioneering work in deep learning matting, but it requires trimap as auxiliary input, and the accuracy of alphamatte prediction is limited by the quality of the trimap. In 2018, Chen et al. proposed an end-to-end model that first uses a semantic segmentation network to predict the trimap, which is then concatenated with the RGB image as input to the matting network to predict alphamatte. However, the alphamatte prediction results are limited by the annotation quality of the matting data. Existing patent documents disclose a portrait cutout method based on deep learning. By inputting the portrait to be cutout into a trained deep portrait cutout model, various portrait foregrounds with different shapes, sizes and textures are identified from different categories of portraits, and portrait details are extracted from the blurred contextual background. However, the deep portrait cutout model is trained using a finely annotated portrait dataset. When facing a coarsely annotated portrait dataset, the portrait segmentation in the coarsely annotated portrait dataset is inaccurate, and the extraction efficiency of the portrait details is low, making it impossible to achieve accurate portrait cutout. Summary of the Invention

[0004] In order to solve the problem that the existing portrait cutout method using deep learning has low efficiency in extracting portrait details when facing coarsely annotated data and cannot achieve accurate portrait cutout, the present invention proposes a portrait cutout method based on mixed annotated data, which reduces the dependence on finely annotated portrait datasets, improves the efficiency of portrait detail extraction, and achieves accurate portrait cutout.

[0005] In order to achieve the above technical effects, the technical solutions of the present invention are as follows:

[0006] A portrait cutout method based on mixed annotated data includes the following steps:

[0007] S1. Obtain a mixed annotated image dataset with a supervisory signal;

[0008] S2. Preprocess the mixed annotated image dataset;

[0009] S3. Construct a portrait matting neural network model for identifying, extracting, and fusing image features and updating the supervisory signal of a mixed annotated image dataset;

[0010] S4. Divide the mixed annotated image dataset into a training set and a test set, use the training set to train the constructed portrait cutout neural network model, and use the test set to test the effectiveness of the portrait cutout neural network model, thereby obtaining a trained portrait cutout neural network model;

[0011] S5. Input the image to be cut out into the trained portrait cutout neural network model, identify, extract and fuse the image features of the cutout image, and output the predicted result of the transparency mask of the cutout image;

[0012] S6. Extracting a portrait image from the image to be cut out according to the prediction result of the transparency mask of the image to be cut out.

[0013] In this technical solution, the mixed annotated image dataset is first preprocessed, and then a portrait cutout neural network model is constructed. The portrait cutout neural network model plays the role of identifying, extracting and fusing image features and updating the supervision signal of the mixed annotated image dataset, avoiding the disadvantage that the mixed annotated image dataset lacks important detail features in the transition area. The training set in the mixed annotated image dataset is then used to train the portrait cutout neural network model to obtain a trained portrait cutout neural network model, reducing the dependence on the fine annotated image dataset and improving the cutout performance of the portrait cutout neural network model. Finally, the image to be cutout is input into the portrait cutout neural network model, and the prediction result of the transparency mask of the image to be cutout is output. End-to-end portrait cutout can be performed without trimap as auxiliary input and prior knowledge such as trimap, ensuring the compatibility of the portrait cutout neural network model with the input data, improving the efficiency of portrait detail extraction, and realizing end-to-end portrait accurate cutout.

[0014] Preferably, the mixed annotated image dataset includes a finely annotated image sub-dataset for detailed annotation of image features, a coarsely annotated image sub-dataset for coarsely annotating image features, and a background image sub-dataset. The finely annotated image sub-dataset and the coarsely annotated image sub-dataset both include a foreground portrait image and a transparency mask alphamatte corresponding to the foreground portrait image. The background image sub-dataset includes a background image with a natural scene as the background.

[0015] Preferably, the specific process of preprocessing the mixed annotated image dataset is as follows:

[0016] S21. Combining the foreground portrait image, the alpha matte corresponding to the foreground portrait image, and the random background image into an RGB image, and performing a dilation and erosion operation on the alpha matte to generate a trimap corresponding to the alpha matte;

[0017] S22. Perform data augmentation on foreground portrait images, alpha matte, background images, RGB images, and trimap by random cropping, random flipping, and unifying image sizes.

[0018] Preferably, the portrait matting neural network model includes a shared encoder, a pyramid pooling module, a Bridgeblock module, a semantic decoder, a detail decoder and a feature fusion module. The output end of the shared encoder is respectively connected to the input end of the pyramid pooling module and the input end of the Bridgeblock module. The RGB image is input to the shared encoder. The shared encoder outputs the first feature map of the RGB image to the pyramid pooling module and the Bridgeblock module respectively. The input end of the pyramid pooling module is connected to the input end of the semantic decoder. The pyramid pooling module extracts the global context features of the first feature map and outputs the second feature map to the semantic decoder. The semantic decoder identifies the overall semantic part of the second feature map and outputs the third feature map, the sematic map. The input end of the Bridgeblock module is connected to the input end of the detail decoder. The Bridgeblock module obtains the detail information of the first feature map and fuses the low-level image features, outputs the fourth feature map to the detail decoder. The detail decoder extracts the detail information of the fourth feature map and outputs the fifth feature map, the detail map. The output ends of the detail decoder and the semantic decoder are both connected to the input end of the feature fusion module. The feature fusion module fuses the third feature map, the sematic map, and the fifth feature map, the detail map, and outputs the predicted alpha matte of the RGB image.

[0019] Preferably, the shared encoder of the ResNet-34 network selected as the backbone network is composed of several first blocks connected in sequence. The resolution of the RGB image input to the shared encoder is reduced by half each time it passes through a first block for extracting the features of the second feature map.

[0020] Preferably, the semantic decoder is composed of a plurality of second blocks connected in sequence, and each second block for extracting the fifth feature map features is composed of three 3×3 convolutional layers and one upsampling layer.

[0021] Preferably, the trimap corresponding to the alpha matte is used as the supervisory signal of the semantic decoder, and the recognition process of the second feature map by the semantic decoder is calculated by the cross entropy L SD To constrain, cross entropy L SD The constraints are as follows:

[0022] L SD =-∑G g log(G p )

[0023] Among them, G p ∈[0,1] represents the third feature map output by the semantic decoder, G g∈[0,1] represents the ground truth corresponding to the third feature map sematic map, and the ground truth corresponding to the third feature map sematic map is the trimap corresponding to the alpha matte.

[0024] Preferably, the alpha matte corresponding to the foreground portrait image is used as the supervisory signal of the detail decoder. The detail decoder is composed of several third blocks connected in sequence. The process of extracting the detail information of the fourth feature map by the detail decoder is calculated by the loss function L DD Constraints are imposed, and the loss function L DD The constraints are as follows:

[0025] L DD =L α +L lap

[0026] L α The expression is as follows:

[0027]

[0028] L lap The expression is as follows:

[0029]

[0030] Where i represents the index of the pixel, W i Indicates whether the pixel belongs to the transition area, ε=10 -6 , α i Represents the ground truth corresponding to the fifth feature map detail map. The ground truth corresponding to the fifth feature map detail map is the alpha matte corresponding to the foreground portrait image. Represents the fifth feature map detailmap output by the detail decoder, LαP k represents the kth level of the Laplacian pyramid.

[0031] Preferably, the portrait cutout neural network model is constructed using the sampling strategy gradient algorithm pytorch, and the portrait cutout neural network model is trained using the sampling gradient descent method.

[0032] Preferably, if the coarsely annotated image sub-dataset is input into the portrait matting neural network model, it is necessary to construct a new supervisory signal for the detail map output by the detail decoder in the next epoch after the detail decoder. The specific construction process of the new supervisory signal is as follows: first, the supervisory signal alpha matte of the detail decoder in the coarsely annotated image sub-dataset is trained for the set training round epoch, and then the supervisory signal alpha matte is updated to obtain a new supervisory signal. The updated calculation expression is as follows:

[0033] K=m*K+(1-m)*Y out

[0034] Among them, K represents the new supervision signal, Y out Represents the detail map output by the detail decoder, and m represents the weight coefficient.

[0035] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0036] The present invention proposes a portrait matting method based on mixed annotated data. First, a mixed annotated image dataset is preprocessed, and then a portrait matting neural network model is constructed. The portrait matting neural network model plays the role of identifying, extracting and fusing image features and updating the supervision signal of the mixed annotated image dataset, thereby avoiding the disadvantage that the mixed annotated image dataset lacks important detail features in the transition area. Then, the portrait matting neural network model is trained using the training set in the mixed annotated image dataset to obtain a trained portrait matting neural network model, thereby reducing the dependence on the fine annotated image dataset and improving the matting performance of the portrait matting neural network model. Finally, the image to be matted is input into the portrait matting neural network model, and a prediction result of the transparency mask of the image to be matted is output. End-to-end portrait matting can be performed without using trimap as an auxiliary input and prior knowledge such as trimap, thereby ensuring the compatibility of the portrait matting neural network model with the input data, improving the efficiency of portrait detail extraction, and realizing end-to-end portrait accurate matting. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A schematic diagram showing a flow chart of a portrait cutout method based on mixed annotated data proposed in an embodiment of the present invention;

[0038] Figure 2 A flowchart showing the preprocessing of a hybrid annotated image dataset proposed in an embodiment of the present invention;

[0039] Figure 3 A structural diagram showing the neural network model for portrait cutout proposed in an embodiment of the present invention. DETAILED DESCRIPTION

[0040] The accompanying drawings are for illustrative purposes only and are not to be construed as limiting this patent;

[0041] In order to better illustrate the present embodiment, some parts of the drawings may be omitted, enlarged, or reduced, and do not represent actual sizes. The description of the directions of parts such as "upper" and "lower" does not limit this patent;

[0042] It is understandable to those skilled in the art that some well-known contents may be omitted in the drawings;

[0043] The positional relationships described in the drawings are for illustrative purposes only and should not be construed as limiting this patent.

[0044] The technical solution of the present invention is further described below with reference to the accompanying drawings and embodiments.

[0045] Example 1

[0046] like Figure 1 As shown, this embodiment proposes a portrait cutout method based on mixed annotation data, comprising the following steps:

[0047] S1S1. Obtain a mixed annotated image dataset with a supervisory signal;

[0048] In step S1, the mixed annotated image dataset includes a finely annotated image subdataset for performing detailed annotations on image features, a coarsely annotated image subdataset for performing coarse annotations on image features, and a background image subdataset. The finely annotated image subdataset adopts the PM (Portrait Matting) dataset, the coarsely annotated image subdataset adopts the DAPM (Deep Automatic Portrain Matting) dataset, and the background image subdataset adopts the BG20K dataset. Both the finely annotated image subdataset and the coarsely annotated image subdataset include foreground portrait images and transparency masks alpha matte corresponding to the foreground portrait images. The background image subdataset includes background images with natural scenes as the background. The PM dataset contains 500 high-resolution foreground portrait images and transparency masks alpha matte corresponding to the foreground portrait images, the DAPM dataset contains 3,400 coarsely annotated images, and the BG20K dataset contains 20,000 high-resolution clear background images.

[0049] S2. Preprocess the mixed annotated image dataset;

[0050] In step S2, see Figure 2 ,The specific process of preprocessing the mixed annotated image dataset is:

[0051] S21. synthesize the foreground portrait image, the alphamatte corresponding to the foreground portrait image, and the random background image into an RGB image, and generate a trimap corresponding to the alphamatte by performing dilation and erosion operations on the alphamatte;

[0052] In step S21, 450 foreground portrait images are selected from both the finely annotated image subdataset and the coarsely annotated image subdataset for processing. The 450 foreground portrait images in the finely annotated image subdataset are all provided with alphamatte corresponding to the foreground portrait images. The 450 foreground portrait images in the finely annotated image subdataset are synthesized with 30 background images to obtain 13,500 RGB images. The 450 foreground portrait images in the coarsely annotated image subdataset are synthesized with 30 background images to obtain 13,500 RGB images. Finally, 27,000 RGB images are obtained. The RGB image synthesis formula is as follows:

[0053] I i =α i F i +(1-α i )B i

[0054] Where I represents the synthesized RGB image, F represents the foreground portrait image, B represents the background image, α represents the alphamatte corresponding to the foreground portrait image, and its value range is [0,1]. i represents the index of the pixel. The alphamatte is used to generate the corresponding trimap through the expansion and corrosion operations as the supervision signal of the semantic encoder, and the alphamatte corresponding to the foreground portrait image is used as the supervision signal of the detail decoder.

[0055] S22. Perform data augmentation on foreground portrait images, alphamatte, background images, RGB images, and trimap by random cropping, random flipping, and unifying image sizes.

[0056] S3. Construct a portrait matting neural network model for identifying, extracting, and fusing image features and updating the supervisory signal of a mixed annotated image dataset;

[0057] S4. Divide the mixed annotated image dataset into a training set and a test set, use the training set to train the constructed portrait cutout neural network model, and use the test set to test the effectiveness of the portrait cutout neural network model, thereby obtaining a trained portrait cutout neural network model;

[0058] S5. Input the image to be cut out into the trained portrait cutout neural network model, identify, extract and fuse the image features of the cutout image, and output the predicted result of the transparency mask of the cutout image;

[0059] S6. Extracting a portrait image from the image to be cut out according to the prediction result of the transparency mask of the image to be cut out.

[0060] In step S6, the white area in the alphamatte of the image to be cut out represents the foreground part of the image, and the black area represents the background part. The foreground or background of the image can be cut out through alphamatte, thereby realizing the extraction of a portrait image from the image to be cut out.

[0061] In this embodiment, the mixed annotated image dataset is first preprocessed, and then a portrait cutout neural network model is constructed. The portrait cutout neural network model plays the role of identifying, extracting, and fusing image features and updating the supervision signal of the mixed annotated image dataset, thereby avoiding the disadvantage that the mixed annotated image dataset lacks important detail features in the transition area. The training set in the mixed annotated image dataset is then used to train the portrait cutout neural network model to obtain a trained portrait cutout neural network model, thereby reducing the dependence on the fine annotated image dataset and improving the cutout performance of the portrait cutout neural network model. Finally, the image to be cutout is input into the portrait cutout neural network model, and the prediction result of the transparency mask of the image to be cutout is output. End-to-end portrait cutout can be performed without using trimap as auxiliary input or prior knowledge such as trimap, thereby ensuring the compatibility of the portrait cutout neural network model with the input data, improving the efficiency of portrait detail extraction, and realizing end-to-end portrait accurate cutout.

[0062] Example 2

[0063] See also Figure 3The portrait cutout neural network model includes a shared encoder, a pyramid pooling module, a Bridgeblock module, a semantic decoder, a detail decoder and a feature fusion module. The output end of the shared encoder is respectively connected to the input end of the pyramid pooling module and the input end of the Bridgeblock module. The RGB image is input to the shared encoder. The shared encoder outputs the first feature map of the RGB image to the pyramid pooling module and the Bridgeblock module respectively. The input end of the pyramid pooling module is connected to the input end of the semantic decoder. The pyramid pooling module extracts the global context features of the first feature map and outputs the second feature map to the semantic decoder. The semantic decoder recognizes the overall semantic part of the second feature map and outputs the third feature map sematicmap. The input end of the Bridgeblock module is connected to the input end of the detail decoder. The Bridgeblock module obtains the detail information of the first feature map and fuses the low-level image features and outputs the fourth feature map to the detail decoder. The detail decoder extracts the detail information of the fourth feature map and outputs the fifth feature map detailmap. The output ends of the detail decoder and the semantic decoder are both connected to the input end of the feature fusion module. The feature fusion module combines the third feature map sematicmap and the fifth feature map detailmap. The fusion method mainly replaces the transition area in the semantic map with the detail map to obtain the final alphamatte. The predicted alphamatte of the RGB image is output. The white area in the alphamatte represents the foreground of the image, and the black area represents the background. The alphamatte can be used to cut out the foreground or background of the image, thereby realizing the extraction of a portrait image from the image to be cut out. On the one hand, existing portrait cutout models require trimap as auxiliary input, while the portrait cutout neural network model proposed in this embodiment only requires an RGB image as input for the portrait cutout neural network model, without the need for other prior knowledge such as trimap, and can perform end-to-end portrait cutout, which is more suitable for application in various aspects of real life. On the other hand, most existing technologies only use finely labeled data for cutout, but high-quality labeling requires huge manpower and material resources and also requires the labeler to have good skills. In comparison, coarsely labeled data is less difficult to obtain. Therefore, this embodiment utilizes both finely labeled image sub-datasets and coarsely labeled image sub-datasets to train the portrait cutout neural network model, thereby improving the efficiency of the portrait cutout neural network model in extracting portrait details.

[0064] Example 3

[0065] See also Figure 3The shared encoder with the ResNet-34 network as the backbone network consists of five first blocks connected in sequence, namely E0 to E4. The RGB image passes through E0 to E4 in sequence. After passing through E4, the first feature map will be output. The resolution of the RGB image input to the shared encoder is reduced by half every time it passes through a first block for extracting the features of the second feature map. In the shared encoder, the ResNet-34 pre-trained on the ImageNet training set is used as the backbone encoder;

[0066] The semantic decoder consists of five second The first feature map is extracted by downsampling through a shared encoder. The features of the first feature map are first extracted through a pyramid pooling module to extract the global context features of the first feature map. The extracted features are concat- ored with the input of each second block and then input into the next second block. Finally, the third feature map is output through upsampling through the semantic decoder. The trimap corresponding to the alpha matte is used as the supervision signal of the semantic decoder. The semantic decoder recognizes the second feature map through the cross entropy L. SD To constrain, cross entropy L SD The constraints are as follows:

[0067] L SD =-∑G g log(G p )

[0068] Among them, G p ∈[0,1] represents the third feature map output by the semantic decoder, G g ∈[0,1] represents the ground truth corresponding to the third feature map sematic map, and the ground truth corresponding to the third feature map sematic map is the trimap corresponding to the alpha matte;

[0069] The alpha matte corresponding to the foreground portrait image is used as the supervision signal of the detail decoder. The detail decoder consists of 5 third blocks DF4~DF0 connected in sequence, and the 5 third blocks DF4~DF0 are stacked symmetrically; the features of the first feature map of the shared encoder will first pass through a bridge block module to utilize the features of different scales of the first feature map. After passing through the bridge block module, the features of the fourth feature map output will be concat and then input into the detail decoder. Finally, the detail decoder outputs the prediction result of the detail map. The process of the detail decoder extracting the detail information of the fourth feature map is passed through the loss function L DD Constraints are imposed, and the loss function L DD The constraints are as follows:

[0070] L DD =L α +L lap

[0071] L α The expression is as follows:

[0072]

[0073] L lap The expression is as follows:

[0074]

[0075] Where i represents the index of the pixel, W i Indicates whether the pixel belongs to the transition area, ε=10 -6 , α i Represents the ground truth corresponding to the fifth feature map detail map. The ground truth corresponding to the fifth feature map detail map is the alpha matte corresponding to the foreground portrait image. Represents the fifth feature map detailmap output by the detail decoder, LαP k represents the kth level of the Laplacian pyramid.

[0076] Example 4

[0077] See also Figure 3 The portrait cutout neural network model sampling strategy gradient algorithm pytorch is constructed, and the portrait cutout neural network model is trained by sampling gradient descent method, and the number of training batches is set to 32, the learning rate is 1e-5, and the number of training epochs is 100.

[0078] If the sub-dataset of coarsely annotated images is input into the portrait matting neural network model, a new supervision signal for the detail map output by the detail decoder in the next epoch needs to be constructed after the detail decoder. The specific construction process of the new supervision signal is as follows:

[0079] First, set the number of training epochs for the alpha matte, which is used as the supervision signal for the detail decoder, in the sub-dataset of coarsely annotated images. At this time, epoch < cons_epoch, and we get:

[0080] K = Y g <000,0227>

[0081] where K represents the new supervision signal, and Y g is the ground truth corresponding to the sub-dataset of coarsely annotated images. This process is the initialization stage. During the training process of the portrait matting neural network model, when epoch < cons_epoch, it is in the initialization stage;

[0082] Then, update the supervision signal alpha matte to obtain a new supervision signal. The calculation expression for the update is as follows:

[0083] K = m * K + (1 - m) * Y out

[0084] where Y out represents the detail map output by the detail decoder, and m represents the weight coefficient. This process is the update stage. When epoch ≥ cons_epoch, it is in the update stage. From the update formula of K, it uses both K and the detail map. K is initialized with the original supervision signal, which can maintain semantic consistency before and after, ensuring that K does not update in a bad direction, while the detail map allows K to continuously learn good detail information during the update process. That is, in order to improve the detail quality of K based on the sub-dataset of coarsely annotated images, it should be noted that the updated new supervision signal does not directly act on the subsequent fusion process, but serves as the supervision signal for the detail map in the next epoch. This is because the difference between the sub-dataset of finely annotated images and the sub-dataset of coarsely annotated images is the quality of the detail information in the transition region, and constructing the new supervision signal is to improve the quality of the detail information on the basis of the original sub-dataset of coarsely annotated images. The new supervision signal can use the loss function to constrain, making the detail information in the detail map better. If the detail information of the detail map used for fusion is better, then the effect of the alpha matte obtained after fusion in the transition region will be better;

[0085] In this embodiment, although the quality of the details of the original supervisory signal is not high, the overall semantics is still reliable. Therefore, the constructed new supervisory signal can maintain the overall semantics and continuously learn good detail information during the updating process. If the alphamatte of the original coarsely labeled data is directly used as the supervisory signal, the performance of the method of using the mixed labeled image data set for cutout will be reduced.

[0086] Obviously, the above embodiments of the present invention are merely examples for the purpose of clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Those skilled in the art will appreciate that other variations or modifications can be made based on the above description. It is not necessary and impossible to enumerate all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the claims of the present invention.

Claims

1. A portrait cutout method based on mixed annotated data, characterized in that: The following steps are involved: S1. Obtain a mixed annotated image dataset with supervisory signals; the mixed annotated image dataset includes a fine annotated image sub-dataset for detailed annotation of image features, a coarse annotated image sub-dataset for rough annotation of image features, and a background image sub-dataset. The fine annotated image sub-dataset and the coarse annotated image sub-dataset each include a foreground portrait image and an alpha matte corresponding to the foreground portrait image; the background image sub-dataset includes a background image with a natural scene as the background. S2. Preprocess the mixed annotated image dataset; S3. Construct a portrait cutout neural network model for identifying, extracting and fusing image features and updating the supervisory signal of a mixed annotated image dataset; the portrait cutout neural network model includes a shared encoder, a pyramid pooling module, a Bridge block module, a semantic decoder, a detail decoder and a feature fusion module, the output end of the shared encoder is respectively connected to the input end of the pyramid pooling module and the input end of the Bridge block module, the RGB image is input to the shared encoder, the shared encoder outputs the first feature map of the RGB image to the pyramid pooling module and the Bridge block module respectively, the input end of the pyramid pooling module is connected to the input end of the semantic decoder, the pyramid pooling module extracts the global context feature of the first feature map, outputs the second feature map to the semantic decoder, the semantic decoder identifies the semantic part of the second feature map as a whole, and outputs the third feature map semantic map, the input end of the Bridge block module is connected to the input end of the detail decoder, The block module obtains the detail information of the first feature map and fuses the low-level image features, outputs the fourth feature map to the detail decoder, and the detail decoder extracts the detail information of the fourth feature map and outputs the fifth feature map detailmap. The output ends of the detail decoder and the semantic decoder are connected to the input end of the feature fusion module. The feature fusion module fuses the third feature map semantic map and the fifth feature map detail map and outputs the predicted alphamatte of the RGB image; S4. Divide the mixed annotated image dataset into a training set and a test set, use the training set to train the constructed portrait cutout neural network model, and use the test set to test the effectiveness of the portrait cutout neural network model, thereby obtaining a trained portrait cutout neural network model; S5. Input the image to be cut out into the trained portrait cutout neural network model, identify, extract and fuse the image features of the cutout image, and output the predicted result of the transparency mask of the cutout image; S6. Extracting a portrait image from the image to be cut out according to the prediction result of the transparency mask of the image to be cut out.

2. The portrait cutout method based on mixed annotation data according to claim 1, characterized in that: The specific process of preprocessing the mixed annotated image dataset is as follows: S21. Combining the foreground portrait image, the alpha matte corresponding to the foreground portrait image, and the random background image into an RGB image, and performing a dilation and erosion operation on the alpha matte to generate a trimap corresponding to the alpha matte; S22. Perform data augmentation on foreground portrait images, alpha matte, background images, RGB images, and trimap by random cropping, random flipping, and unifying image sizes.

3. The portrait cutout method based on mixed annotation data according to claim 2, characterized in that: The ResNet-34 network is selected as the shared encoder of the backbone network, which consists of several first blocks connected in sequence. The resolution of the RGB image input to the shared encoder is reduced by half every time it passes through a first block for extracting the features of the second feature map.

4. The portrait cutout method based on mixed annotation data according to claim 3, characterized in that: The semantic decoder is composed of a plurality of second blocks connected in sequence, and each second block for extracting the fifth feature map features is composed of three 3×3 convolutional layers and one upsampling layer.

5. The portrait cutout method based on mixed annotation data according to claim 4, characterized in that: The trimap corresponding to alphamatte is used as the supervision signal of the semantic decoder. The recognition process of the semantic decoder on the second feature map is performed by the cross entropy L SD To constrain, cross entropy L SD The constraints are as follows: L SD =-∑G g log(G p ) Among them, G p ∈[0,1] represents the third feature map output by the semantic decoder, G g ∈[0,1] represents the true value ground truth corresponding to the third feature map semantic map, and the true value ground truth corresponding to the third feature map semantic map is the trimap corresponding to the alpha matte.

6. The portrait cutout method based on mixed annotation data according to claim 5, characterized in that: The alpha matte corresponding to the foreground portrait image is used as the supervision signal of the detail decoder. The detail decoder consists of several third blocks connected in sequence. The process of extracting the detail information of the fourth feature map by the detail decoder is calculated through the loss function L DD Constraints are imposed, and the loss function L DD The constraints are as follows: L DD =L ɑ +L lap L α The expression is as follows: L lap The expression is as follows: Where i represents the index of the pixel, W i Indicates whether the pixel belongs to the transition area, ε=10 -6 , α i Represents the ground truth corresponding to the fifth feature map detail map. The ground truth corresponding to the fifth feature map detail map is the alpha matte corresponding to the foreground portrait image. Represents the fifth feature map detailmap output by the detail decoder, LαP k represents the kth level of the Laplacian pyramid.

7. The portrait cutout method based on mixed annotation data according to claim 6, characterized in that: The portrait cutout neural network model is constructed using the sampling strategy gradient algorithm pytorch, and the portrait cutout neural network model is trained using the sampling gradient descent method.

8. The portrait cutout method based on mixed annotation data according to claim 7, characterized in that: If the coarsely annotated image sub-dataset is input into the portrait matting neural network model, a new supervisory signal needs to be constructed after the detail decoder for the detail map output by the detail decoder in the next epoch. The specific construction process of the new supervisory signal is as follows: first, the supervisory signal alpha matte of the detail decoder in the coarsely annotated image sub-dataset is trained for the set training round epoch, and then the supervisory signal alpha matte is updated to obtain a new supervisory signal. The updated calculation expression is as follows: K=m*K+(1-m)*Y out Among them, K represents the new supervision signal, Y out Represents the detail map output by the detail decoder, and m represents the weight coefficient.

Images