A head segmentation method based on position controllable
The method addresses the challenge of accurately segmenting a single head in multi-person images using a head key point detection and convolutional neural network, ensuring efficient and flexible head segmentation suitable for mobile devices.
Patent Information
- Application Number
- CN202110917750.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-08-11
- Publication Date
- 2024-12-13
- Estimated Expiration
- 2041-08-11
AI Technical Summary
Existing head segmentation technologies struggle to accurately segment a single head in multi-person scenarios, especially when using deep learning and instance segmentation, and require complex post-processing, making them unsuitable for end-side deployment.
A method involving a head key point detection module, position correction, and a fully convolutional neural network for head segmentation, which includes preprocessing, Gaussian blur, and a lightweight model design to accurately segment a single head by user-defined position, ignoring other individuals in the scene.
Enables efficient and flexible head segmentation in multi-person scenes, suitable for mobile deployment, by accurately isolating the target head and improving operational efficiency.
Smart Images
Figure CN113610818B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field related to image processing, and in particular to a method for segmenting a human head based on controllable position. Background Art
[0002] Head segmentation is a common function in current short video software. Its purpose is to provide a basis for users to create emoticons, head stickers, anime body replacement and other special effects.
[0003] Currently, most head segmentation technologies are developed based on deep learning technology, and semantic segmentation models are usually used to extract the head part of the image. However, when there are multiple people in the image, it is impossible to perform targeted extraction of a single head, which greatly reduces the playability. Although there is also instance segmentation technology that can perform differentiated segmentation of heads in multi-person scenarios, the current operation of instance segmentation technology is not considered high, and it also requires complex post-processing, which is not suitable for end-side deployment. Summary of the invention
[0004] The present invention aims to overcome the above-mentioned deficiencies in the prior art and provides a human head segmentation method based on position controllable with high operation efficiency and flexibility.
[0005] In order to achieve the above object, the present invention adopts the following technical solutions:
[0006] A method for head segmentation based on controllable position specifically comprises the following steps:
[0007] (1) Preprocess the input image, scale the resolution to 256x256, and then normalize it to control the pixel range between -1 and 1;
[0008] (2) Construct a head key point detection module. Input the input image in step (1) into the head key point detection module to obtain 1x256x256 key point features. Perform position analysis on the key point features to obtain the center point coordinates of each head. Assume that N = {N 1 ,N 2 ,N 3 ,…} represents the number of heads, where N i ={x i ,y i} indicates the specific position of each head in the picture;
[0009] (3) Assuming that the user clicks the position I = {x, y}, and there is a situation where the user clicks the position not completely on the head in the picture, the position correction module cyclically matches the position I with the positions of N heads of people at the closest distance, and obtains the head N closest to the user click position by comparing the sizes of the N closest distances. j, and the head is used as the head that the user needs to deduct from the picture;
[0010] (4) N for each person j The coordinates of the image are Gaussian blurred to obtain the conditions at a specific location. The calculation formula for Gaussian blur is as follows: Where σ = 2, and the size of the Gaussian kernel is set to 10 to increase the information range of the position condition, thereby ensuring better segmentation of the head at a specific position;
[0011] (5) Construct a head segmentation module, which is a fully convolutional neural network consisting of two modules: encoding and decoding. The encoding module consists of four feature extraction units, each of which consists of two convolutional layers and one downsampling layer. The multiple of each downsampling layer is 2, so the entire encoding module performs 16 times downsampling. In addition, in the decoding module, a convolutional layer and an upsampling layer are combined to restore the size of the feature, and the upsampling is performed by 2 times each time. At the same time, the feature is fused with the feature of the same size in the encoding module. This operation is performed four times in this way, and finally the output feature with the same size as the original image is obtained. Finally, the output feature is activated by the sigmoid function.
[0012] (6) The input image in step (1) and the head position condition information in step (4) are combined, and then input into the head segmentation network in step (5) to obtain a separate head mask at the corresponding position, thereby completing the head segmentation required by the user.
[0013] This method first uses the head key point detection network and the position correction module to obtain the head position that the user needs to segment, and then uses the position and the head segmentation network to accurately segment the head, ignoring the influence of other people in the picture. It can accurately segment a single head in a multi-person scene. It is a more flexible method, improves operating efficiency, and can be deployed on the mobile phone.
[0014] Preferably, in step (2), the head key point detection module refers to: a fully convolutional neural network, consisting of two modules, encoding and decoding, the encoding module consists of 5 feature extraction units, each feature extraction unit consists of two convolutional layers and one downsampling layer, and the multiple of each downsampling layer is 2, so the entire encoding module performs 32 times of downsampling; in the decoding module, a combination of two convolutional layers and one upsampling is used to restore the size of the feature, and 2 times of upsampling is performed each time, and the feature is fused with the feature of the same size in the encoding module, and the operation is performed 5 times in this way to finally obtain an output feature map with the same size as the original image; the key point positions in the output feature map are analyzed, and each key point position uses the expected method to obtain its specific coordinate information in the original image.
[0015] Preferably, in step (3), the calculation formula of the shortest distance dst between position I and the positions of N heads is as follows:
[0016]
[0017] The beneficial effects of the present invention are: it can accurately segment a single head in a multi-person scene, is more flexible, improves operating efficiency, and is deployed on a mobile phone. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1 It is a flow chart of the method of the present invention. DETAILED DESCRIPTION
[0019] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments.
[0020] like Figure 1 In the embodiment described above, a method for segmenting a head based on position controllable includes the following steps:
[0021] (1) Preprocess the input image, scale the resolution to 256x256, and then normalize it to control the pixel range between -1 and 1;
[0022] (2) Construct a head key point detection module. Input the input image in step (1) into the head key point detection module to obtain 1x256x256 key point features. Perform position analysis on the key point features to obtain the center point coordinates of each head. Assume that N = {N 1 ,N 2 ,N 3 ,…} represents the number of heads, where N i ={x i ,y i} indicates the specific position of each head in the picture; the head key point detection module refers to: a fully convolutional neural network, consisting of two modules, encoding and decoding. The encoding module consists of 5 feature extraction units, each of which consists of two convolutional layers and one downsampling layer. The multiple of each downsampling layer is 2, so the entire encoding module performs 32 times of downsampling; in the decoding module, a combination of two convolutional layers and one upsampling is used to restore the size of the feature, and 2 times of upsampling is performed each time. At the same time, the feature is fused with the feature of the same size in the encoding module. This operation is performed 5 times in this way, and finally an output feature map with the same size as the original image is obtained; the key point positions in the output feature map are parsed, and each key point position uses the expected method to obtain its specific coordinate information in the original image;
[0023] (3) Assuming that the user clicks the position I = {x, y}, and there is a situation where the user clicks the position not completely on the head in the picture, the position correction module cyclically matches the position I with the positions of N heads of people at the closest distance, and obtains the head N closest to the user click position by comparing the sizes of the N closest distances. j , and the head is taken as the head that the user needs to deduct from the picture; the calculation formula of the shortest distance dst between the position I and the position of N heads is as follows
[0024]
[0025] (4) N for each person j The coordinates of the image are Gaussian blurred to obtain the conditions at a specific location. The calculation formula for Gaussian blur is as follows: Where σ = 2, and the size of the Gaussian kernel is set to 10 to increase the information range of the position condition, thereby ensuring better segmentation of the head at a specific position;
[0026] (5) Construct a head segmentation module, which is a fully convolutional neural network consisting of two modules: encoding and decoding. The encoding module consists of four feature extraction units, each of which consists of two convolutional layers and one downsampling layer. The multiple of each downsampling layer is 2, so the entire encoding module performs 16 times downsampling. In addition, in the decoding module, a convolutional layer and an upsampling layer are combined to restore the size of the feature, and the upsampling is performed by 2 times each time. At the same time, the feature is fused with the feature of the same size in the encoding module. This operation is performed four times in this way, and finally the output feature with the same size as the original image is obtained. Finally, the output feature is activated by the sigmoid function.
[0027] (6) The input image in step (1) and the head position condition information in step (4) are combined, and then input into the head segmentation network in step (5) to obtain a separate head mask at the corresponding position, thereby completing the head segmentation required by the user.
[0028] The whole method consists of three modules: head key point detection module, position correction module and head segmentation module. The user click position is corrected by the position correction module to match the head key point position. The final head segmentation result is obtained by using the key point information and the head segmentation module. The whole system adopts a lightweight model design with high operation effect. This method first uses the head key point detection network and the position correction module to obtain the head position that the user needs to segment, and then uses the position and the head segmentation segmentation network to accurately segment the head, ignoring the influence of other people in the picture. It can accurately segment a single head in a multi-person scene. It is a more flexible method, improves operation efficiency, and is deployed on the mobile phone.
Claims
1. A method for head segmentation based on position controllable, characterized in that: The specific steps include: (1) Preprocess the input image, scale the resolution to 256x256, and then normalize it to control the pixel range between -1 and 1; (2) Construct a head key point detection module. Input the image obtained after preprocessing the input image in step (1) into the head key point detection module to obtain 1x256x256 key point features. Perform position analysis on the key point features to obtain the center point coordinates of each head. Assume that N = {N1, N2, N3, …} represents the number of heads, where N i ={x i , y i } indicates the specific position of each head in the picture; the head key point detection module refers to: a fully convolutional neural network, consisting of two modules, encoding and decoding. The encoding module consists of 5 feature extraction units, each of which consists of two convolutional layers and one downsampling layer. The multiple of each downsampling layer is 2, so the entire encoding module performs 32 times of downsampling; in the decoding module, a combination of two convolutional layers and one upsampling is used to restore the size of the feature, and 2 times of upsampling is performed each time. At the same time, the feature is fused with the feature of the same size in the encoding module. This operation is performed 5 times in this way, and finally an output feature map with the same size as the image obtained after preprocessing the input image is obtained; the key point positions in the output feature map are parsed, and each key point position uses the expected method to obtain its specific coordinate information in the image obtained after preprocessing the input image; (3) Assume that the user clicks on the position I = {x, y}, and there is a situation where the user clicks on the position that is not completely on the head of the person in the picture. The position correction module cyclically matches the position I with the positions of N heads of people at the closest distance, and obtains the head N closest to the user click position by comparing the sizes of the N closest distances. j , and the head is used as the head that the user needs to deduct from the picture; (4) N for each person j The coordinates of the image are Gaussian blurred to obtain the conditions at a specific location. The calculation formula for Gaussian blur is as follows: ,in , and the size of the Gaussian kernel is set to 10 to increase the information range of the position condition, thereby ensuring better segmentation of the head at a specific position; (5) Construct a head segmentation module, which is a fully convolutional neural network consisting of two modules: encoding and decoding. The encoding module consists of four feature extraction units, each of which consists of two convolutional layers and one downsampling layer. The multiple of each downsampling layer is 2, so the entire encoding module performs 16 times downsampling. In addition, in the decoding module, a convolutional layer and an upsampling layer are used to restore the size of the feature. The upsampling is performed by 2 times each time, and the feature is fused with the feature of the same size in the encoding module. This operation is performed four times in this way, and finally the output feature with the same size as the image obtained after preprocessing the input image is obtained. Finally, the output feature is activated by the sigmoid function. (6) The image obtained after preprocessing the input image in step (1) and the head position condition information in step (4) are combined, and then input into the head segmentation module in step (5) to obtain a separate head mask at the corresponding position, thereby completing the head segmentation required by the user.
2. The method for segmenting a head based on position controllable according to claim 1, characterized in that In step (3), the formula for calculating the shortest distance dst between position I and the positions of N heads is as follows: .