Picture matting method and system based on deep learning

Abstract

The picture matting method based on deep learning comprises the steps of obtaining a to-be-processed picture; preprocessing the to-be-processed picture; inputting the preprocessed to-be-processed picture into a trained neural network model, performing semantic segmentation on the to-be-processed picture by the model, performing category label labeling on each pixel point in the to-be-processed picture, and outputting a foreground region, a background region and an unknown region in the segmented to-be-processed picture based on category labels; the alpha value of each pixel point in the unknown area is calculated, the alpha values are converted into transparency values, the transparency values are 0-1, the transparency values of the pixel points in the background area are 0, and the transparency values of the pixel points in the foreground area are 1; whether each pixel point in the unknown area belongs to a foreground or a background is calculated and analyzed, the transparency value of the pixel point belonging to the foreground is 1, and the transparency value of the pixel point belonging to the background is 0; and cutting out the region corresponding to the pixel point with the transparency value of 1 from the to-be-processed picture to obtain a foreground picture.

Description

Technical Field

[0001] The present invention relates to the technical field of picture cutout, and in particular to a picture cutout method and system based on deep learning. Background Art

[0002] At present, image cutting has become one of the most common operations in image processing, which requires segmenting the portrait in the image. For example, the implementation of some portrait beautification functions depends on the accurate segmentation of the background and the portrait. In the prior art, the image cutting solutions are mainly based on pixel clustering methods and graph partitioning algorithms. Such methods have poor segmentation effects when the background is complex. Summary of the invention

[0003] In view of the problems and shortcomings of the prior art, the present invention provides a method and system for image cutout based on deep learning.

[0004] The present invention solves the above technical problems through the following technical solutions:

[0005] The present invention provides a method for cutting out a picture based on deep learning, which is characterized in that it comprises the following steps:

[0006] S1. Get the image to be processed;

[0007] S2, preprocessing the image to be processed;

[0008] S3, input the pre-processed image to be processed into a pre-trained neural network model, the neural network model performs semantic segmentation on the image to be processed, and labels each pixel in the image to be processed with a category label, and outputs the segmented foreground area, background area and unknown area in the image to be processed based on the category label, and gives the coordinates and pixel value of each pixel in the foreground area, the number of pixels belonging to the foreground area, and the height and width of the foreground area, the coordinates and pixel value of each pixel in the background area, the number of pixels belonging to the background area, and the height and width of the background area, and the coordinates and pixel value of each pixel in the unknown area, the number of pixels belonging to the unknown area, and the height and width of the background area;

[0009] Among them, the category label is a human type label or a non-human type label, the foreground area is the label area corresponding to the human type label, the background area is the label area corresponding to the non-human type label, and the unknown area is the area without a label;

[0010] S4. Calculate the alpha value of each pixel in the unknown area using the Bayesian formula, and convert the alpha value into a transparency value for representation. The transparency value represents the probability that the pixel belongs to the foreground or the background. The transparency value ranges from 0 to 1, where 0 represents the background and 1 represents the foreground. The transparency value of each pixel in the background area is 0, and the transparency value of each pixel in the foreground area is 1.

[0011] S5, using the K-nearest neighbor algorithm to calculate and analyze whether each pixel in the unknown area belongs to the foreground or the background, the transparency value of the pixel belonging to the foreground is 1, and the transparency value of the pixel belonging to the background is 0;

[0012] S6. Cut out the area corresponding to the pixel point whose transparency value is 1 from the image to be processed to obtain a foreground image.

[0013] This embodiment provides a picture cutout system based on deep learning, which is characterized in that it includes an acquisition module, a preprocessing module, a semantic segmentation module, a first calculation and analysis module, a second calculation and analysis module and a cutout module;

[0014] The acquisition module is used to acquire the image to be processed;

[0015] The preprocessing module is used to perform preprocessing operations on the image to be processed;

[0016] The semantic segmentation module is used to input the pre-processed image to be processed into a pre-trained neural network model, the neural network model performs semantic segmentation on the image to be processed, and labels each pixel in the image to be processed with a category label, and outputs the segmented foreground area, background area and unknown area in the image to be processed based on the category label, and gives the coordinates and pixel value of each pixel in the foreground area, the number of pixels belonging to the foreground area, and the height and width of the foreground area, the coordinates and pixel value of each pixel in the background area, the number of pixels belonging to the background area, and the height and width of the background area, and the coordinates and pixel value of each pixel in the unknown area, the number of pixels belonging to the unknown area, and the height and width of the background area;

[0017] Among them, the category label is a human type label or a non-human type label, the foreground area is the label area corresponding to the human type label, the background area is the label area corresponding to the non-human type label, and the unknown area is the area without a label;

[0018] The first calculation and analysis module is used to calculate the alpha value of each pixel in the unknown area using the Bayesian formula, and convert the alpha value into a transparency value for representation, wherein the transparency value represents the probability that the pixel belongs to the foreground or the background, and the transparency value ranges from 0 to 1, where 0 represents the background and 1 represents the foreground. The transparency value of each pixel in the background area is 0, and the transparency value of each pixel in the foreground area is 1;

[0019] The second calculation and analysis module is used to calculate and analyze whether each pixel in the unknown area belongs to the foreground or the background by using the K-nearest neighbor algorithm, the transparency value of the pixel belonging to the foreground is 1, and the transparency value of the pixel belonging to the background is 0;

[0020] The cutout module is used to cut out the area corresponding to the pixel transparency value of 1 from the image to be processed to obtain a foreground image.

[0021] The positive and progressive effects of the present invention are:

[0022] The present invention inputs the pre-processed image to be processed into a pre-trained convolutional neural network model for semantic segmentation, obtains the foreground area, background area and unknown area in the image to be processed, calculates and analyzes the transparency value of each pixel in the foreground area, background area and unknown area, calculates and analyzes whether each pixel in the unknown area belongs to the foreground or the background, the transparency value of the pixel belonging to the foreground is 1, the transparency value of the pixel belonging to the background is 0, the transparency value of each pixel in the background area is 0, and the transparency value of each pixel in the foreground area is 1, so that the area corresponding to the transparency value of the pixel is 1 is cut out from the image to be processed to obtain the foreground image. The present invention can accurately segment the foreground image, and the foreground image segmentation effect is still very good when the background is relatively complex. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a flowchart of a deep learning-based image cutout method according to a preferred embodiment of the present invention.

[0024] Figure 2 This is a structural block diagram of a picture cutout method based on deep learning according to a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0026] like Figure 1 As shown, this embodiment provides a picture cutout method based on deep learning, which includes the following steps:

[0027] Step 101: Obtain the image to be processed.

[0028] Step 102: preprocess the image to be processed. The preprocessing operation includes adjusting the size of the image to be processed to a target size and converting the format of the image to be processed to a target format.

[0029] Step 103, input the preprocessed image to be processed into a pre-trained convolutional neural network model, the neural network model performs semantic segmentation on the image to be processed, and labels each pixel in the image to be processed with a category label, and outputs the segmented foreground area, background area and unknown area in the image to be processed based on the category label, and gives the coordinates and pixel value of each pixel in the foreground area, the number of pixels belonging to the foreground area, and the height and width of the foreground area, the coordinates and pixel value of each pixel in the background area, the number of pixels belonging to the background area, and the height and width of the background area, the coordinates and pixel value of each pixel in the unknown area, the number of pixels belonging to the unknown area, and the height and width of the background area.

[0030] Among them, the category label is a human type label or a non-human type label, the foreground area is the label area corresponding to the human type label, the background area is the label area corresponding to the non-human type label, and the unknown area is the area without a label.

[0031] Step 104, use the Bayesian formula to calculate the alpha value of each pixel in the unknown area, and convert the alpha value into a transparency value for representation. The transparency value represents the probability that the pixel belongs to the foreground or background. The transparency value is 0-1, where 0 represents the background and 1 represents the foreground. The transparency value of each pixel in the background area is 0, and the transparency value of each pixel in the foreground area is 1.

[0032] Step 105: Use the K-nearest neighbor algorithm to calculate and analyze whether each pixel in the unknown area belongs to the foreground or the background. The transparency value of the pixel belonging to the foreground is 1, and the transparency value of the pixel belonging to the background is 0.

[0033] Step 106: Cut out the area corresponding to the pixel point whose transparency value is 1 from the image to be processed to obtain a foreground image.

[0034] like Figure 2 As shown, this embodiment also provides a picture cutout system based on deep learning, which includes an acquisition module 1, a preprocessing module 2, a semantic segmentation module 3, a first computing and analysis module 4, a second computing and analysis module 5 and a cutout module 6.

[0035] The acquisition module 1 is used to acquire the image to be processed.

[0036] The preprocessing module 2 is used to perform preprocessing operations on the image to be processed, wherein the preprocessing operations include adjusting the size of the image to be processed to the target size and converting the format of the image to be processed to the target format.

[0037] The semantic segmentation module 3 is used to input the preprocessed image to be processed into a pre-trained convolutional neural network model. The neural network model performs semantic segmentation on the image to be processed, and labels each pixel in the image to be processed with a category label. Based on the category label, the segmented foreground area, background area and unknown area in the image to be processed are output, and the coordinates and pixel value of each pixel in the foreground area, the number of pixels belonging to the foreground area, and the height and width of the foreground area are given; the coordinates and pixel value of each pixel in the background area, the number of pixels belonging to the background area, and the height and width of the background area are given; the coordinates and pixel value of each pixel in the unknown area, the number of pixels belonging to the unknown area, and the height and width of the background area are given.

[0038] Among them, the category label is a human type label or a non-human type label, the foreground area is the label area corresponding to the human type label, the background area is the label area corresponding to the non-human type label, and the unknown area is the area without a label.

[0039] The first calculation and analysis module 4 is used to calculate the alpha value of each pixel in the unknown area using the Bayesian formula, and convert the alpha value into a transparency value for representation. The transparency value represents the probability that the pixel belongs to the foreground or background. The transparency value is 0-1, where 0 represents the background and 1 represents the foreground. The transparency value of each pixel in the background area is 0, and the transparency value of each pixel in the foreground area is 1.

[0040] The second calculation and analysis module 5 is used to calculate and analyze whether each pixel point in the unknown area belongs to the foreground or the background by using the K-nearest neighbor algorithm. The transparency value of the pixel point belonging to the foreground is 1, and the transparency value of the pixel point belonging to the background is 0.

[0041] The cutout module 6 is used to cut out the area corresponding to the pixel transparency value of 1 from the image to be processed to obtain a foreground image.

[0042] Although the specific embodiments of the present invention are described above, those skilled in the art should understand that these are only examples, and the protection scope of the present invention is defined by the appended claims. Those skilled in the art may make various changes or modifications to these embodiments without departing from the principles and essence of the present invention, but these changes and modifications all fall within the protection scope of the present invention.

Claims

1. A method for image cutout based on deep learning, characterized in that: It includes the following steps: S1. Get the image to be processed; S2, preprocessing the image to be processed; S3, input the pre-processed image to be processed into a pre-trained neural network model, the neural network model performs semantic segmentation on the image to be processed, and labels each pixel in the image to be processed with a category label, and outputs the segmented foreground area, background area and unknown area in the image to be processed based on the category label, and gives the coordinates and pixel value of each pixel in the foreground area, the number of pixels belonging to the foreground area, and the height and width of the foreground area, the coordinates and pixel value of each pixel in the background area, the number of pixels belonging to the background area, and the height and width of the background area, and the coordinates and pixel value of each pixel in the unknown area, the number of pixels belonging to the unknown area, and the height and width of the background area; Among them, the category label is a human type label or a non-human type label, the foreground area is the label area corresponding to the human type label, the background area is the label area corresponding to the non-human type label, and the unknown area is the area without a label; S4. Calculate the alpha value of each pixel in the unknown area using the Bayesian formula, and convert the alpha value into a transparency value for representation. The transparency value represents the probability that the pixel belongs to the foreground or the background. The transparency value ranges from 0 to 1, where 0 represents the background and 1 represents the foreground. The transparency value of each pixel in the background area is 0, and the transparency value of each pixel in the foreground area is 1. S5, using the K-nearest neighbor algorithm to calculate and analyze whether each pixel in the unknown area belongs to the foreground or the background, the transparency value of the pixel belonging to the foreground is 1, and the transparency value of the pixel belonging to the background is 0; S6. Cut out the area corresponding to the pixel point whose transparency value is 1 from the image to be processed to obtain a foreground image.

2. The image cutout method based on deep learning as claimed in claim 1, characterized in that: In step S2, the preprocessing operation includes adjusting the size of the image to be processed to a target size and converting the format of the image to be processed to a target format.

3. The image cutout method based on deep learning as claimed in claim 1, characterized in that: In step S3, the neural network model is a convolutional neural network model.

4. A picture cutout system based on deep learning, characterized in that: It includes an acquisition module, a preprocessing module, a semantic segmentation module, a first calculation and analysis module, a second calculation and analysis module and a cutout module; The acquisition module is used to acquire the image to be processed; The preprocessing module is used to perform preprocessing operations on the image to be processed; The semantic segmentation module is used to input the pre-processed image to be processed into a pre-trained neural network model, the neural network model performs semantic segmentation on the image to be processed, and labels each pixel in the image to be processed with a category label, and outputs the segmented foreground area, background area and unknown area in the image to be processed based on the category label, and gives the coordinates and pixel value of each pixel in the foreground area, the number of pixels belonging to the foreground area, and the height and width of the foreground area, the coordinates and pixel value of each pixel in the background area, the number of pixels belonging to the background area, and the height and width of the background area, and the coordinates and pixel value of each pixel in the unknown area, the number of pixels belonging to the unknown area, and the height and width of the background area; Among them, the category label is a human type label or a non-human type label, the foreground area is the label area corresponding to the human type label, the background area is the label area corresponding to the non-human type label, and the unknown area is the area without a label; The first calculation and analysis module is used to calculate the alpha value of each pixel in the unknown area using the Bayesian formula, and convert the alpha value into a transparency value for representation, wherein the transparency value represents the probability that the pixel belongs to the foreground or the background, and the transparency value ranges from 0 to 1, where 0 represents the background and 1 represents the foreground. The transparency value of each pixel in the background area is 0, and the transparency value of each pixel in the foreground area is 1; The second calculation and analysis module is used to calculate and analyze whether each pixel in the unknown area belongs to the foreground or the background by using the K-nearest neighbor algorithm, the transparency value of the pixel belonging to the foreground is 1, and the transparency value of the pixel belonging to the background is 0; The cutout module is used to cut out the area corresponding to the pixel transparency value of 1 from the image to be processed to obtain a foreground image.

5. The deep learning-based image cutout system according to claim 4, characterized in that: The preprocessing operation includes adjusting the size of the image to be processed to the target size and converting the format of the image to be processed to the target format.

6. The deep learning-based image cutout system according to claim 4, characterized in that: The neural network model is a convolutional neural network model.

Images