Multi-modal portrait segmentation method based on separation guide convolution

Abstract

The invention discloses a multi-modal portrait segmentation method based on separation guide convolution, which comprises the following steps of: inputting features output by an encoder into separation guide convolution through a decoder to carry out multi-scale learning, outputting a prediction probability graph of a portrait, and constructing a portrait segmentation model; inputting a to-be-detected figure image and a depth image thereof into the constructed network model for model training, adding depth supervision to a prediction image output from each side, calculating the difference between a manual annotation image and a predicted portrait segmentation image by using cross entropy loss, and feeding the errors back to the network to update the model parameters of the whole network; and inputting the original image needing to be tested and the depth image into the model for model testing. According to the method, the scale of the side output feature image is reduced, and the multi-scale feature expression capability is improved, so that the reasoning calculation speed is increased, and the detection accuracy in a complex scene is improved.

Description

technical field [0001] The invention relates to the technical field of computer vision and digital image processing, in particular to a multimodal portrait segmentation method based on separation-guided convolution. Background technique [0002] Portrait segmentation is actually to locate and segment the person information in the image from the original image, while multi-modal portrait segmentation is to add a depth map on the basis of the ordinary portrait segmentation algorithm. The depth map is similar to the grayscale image, but Each of its pixel values ​​is the actual distance from the sensor to the object. Multimodal portrait segmentation is more sensitive to the capture of high-level semantic information, which greatly improves the accuracy of portrait segmentation. [0003] Multi-modal portrait segmentation is a binary segmentation task in computer vision. The purpose is to separate the characters from the original image, which is beneficial to the background blur ...

AI Technical Summary

Problems solved by technology

The network model of the U-shaped network architecture is relatively complex, resulting in low computational efficiency

Although there are other solutions to solve the problem of low computational efficiency, the system cannot perform well due to the low resolution of the deep feature image

[0004] The traditional portrait segmentation algorithm needs to manually select the character area to assist it to achieve portrait segmentation. This algorithm is not only inefficient, but also has low segmentation accuracy in complex scenes.

The semantic segmentation method based on deep learning can solve the accuracy of portrait segmentation in complex scenes, but the existing network models are generally complex, the calculation time is long, and it is not easy to obtain high-resolution deep feature images

Images