Semantic segmentation method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a semantic segmentation method and device, electronic equipment and a storage medium method, and the method comprises the steps: extracting the original features of each frame in multiple frames of a video, the multiple frames comprising a current frame and at least one previous frame, and the previous frame being located before the current frame; the original features of each frame are input into a semantic segmentation model, a semantic segmentation result of the current frame is obtained, and the semantic segmentation model is configured to determine features, used for alignment, of each frame based on the original features of each frame; for each previous frame, based on the feature for alignment of the previous frame and the feature for alignment of the current frame, deformable convolution is performed on the feature for alignment of the previous frame to obtain an alignment spatial feature of the previous frame, and based on the feature for alignment of the current frame and the alignment spatial feature of each previous frame, deformable convolution is performed on the feature for alignment of the previous frame to obtain an alignment spatial feature of the previous frame. Obtaining an alignment motion feature of the current frame; and predicting a semantic segmentation result of the current frame based on the aligned motion features.

Description

technical field [0001] The present application relates to the field of computer vision, and specifically relates to a semantic segmentation method, device, electronic equipment and storage medium. Background technique [0002] Semantic segmentation is a computer vision technology that determines the semantic category of each pixel in an image, such as vehicle category and pedestrian category, and is widely used in the visual perception of vehicles and the construction of high-precision maps. When performing semantic segmentation on the current frame, in order to improve the consistency between the semantic segmentation results of the current frame and the semantic segmentation results of the previous frame before the current frame, it is necessary to perform feature alignment between the current frame and the previous frame to obtain the alignment of the current frame sporty features. Then, according to the aligned motion features of the current frame, the semantic segmenta...

AI Technical Summary

Problems solved by technology

On the one hand, calculating the optical flow between the current frame and the previous frame needs to calculate the optical flow between any two pixels, one of the two pixels is from the current frame, and the other is from the previous frame, resulting in The process of aligning motion features of the current frame consumes more computing resources, and semantic segmentation consumes more computing resources

On the other hand, in the current frame and / or in the previous frame, objects such as vehicles and pedestrians are partially occluded, and the camera that captures the current frame and the previous frame has a large change in viewing angle during the capture of the current frame and the previous frame. It will lead to low accuracy of the calculated optical flow between the current frame and the previous frame, and use the optical flow between the current frame and the previous frame with low accuracy to obtain the alignment motion feature of the current frame, resulting in the alignment motion of the current frame The accuracy of the features is low

Images