Video Semantic Segmentation Using Multi-Frequency Dynamic Atrous Convolution

Abstract

The invention discloses a video semantic segmentation method using multi-frequency dynamic hole convolution. The method of the present invention first enhances the sampling frame image of the video data, and extracts the shallow visual feature map through the encoder; then constructs the feature frequency separation module to obtain the multi-frequency feature map corresponding to the video frame, and inputs it into the dynamic hole convolution Module to obtain the corresponding multi-frequency high-level semantic feature map, and then obtain the segmentation mask of the video frame through the upsampling convolutional encoder; use the stochastic gradient descent algorithm to iteratively train the model until convergence, and input the new video into the model to obtain the semantic segmentation result. The method of the present invention separates the feature maps of video frames according to different frequencies to describe changes in different visual regions, can reduce redundant information in low-frequency visual space, reduce computational complexity, and adaptively expand the receptive field of multi-frequency feature maps through dynamic atrous convolution , to improve the ability to discriminate different semantic categories of videos, so as to obtain better video semantic segmentation results.

Description

technical field [0001] The invention belongs to the technical field of computer vision, in particular to the field of semantic segmentation in video processing, and relates to a video semantic segmentation method using multi-frequency dynamic hole convolution. Background technique [0002] With the increasing number of various types of vehicles, driving safety has become a very concerned aspect of the government and the public. Generally speaking, continuous driving for a long time will make people tired and distracted. At the same time, drivers of large vehicles are prone to visual blind spots, which brings great hidden dangers to driving safety. In recent years, autonomous driving technology has aroused strong interest in autonomous driving technology in the industry, and more and more research efforts have been invested in this field. Efficient visual understanding can guarantee the safety of autonomous driving, and video semantic segmentation is one of its core technolo...

AI Technical Summary

Problems solved by technology

[0004] There are still many shortcomings in the existing semantic segmentation methods: 1) The spatial pyramid pooling technology considers both local and global spatio-temporal structure information to make the segmentation results more reliable, but using the maximum average pooling operation for high-resolution feature maps will cause Poor fault tolerance, poor generalization ability, high computational complexity, etc.; 2) Although the long-term semantic dependency between feature maps is strengthened by using the attention mechanism, the model is bloated and takes up a lot of memory, which is not conducive to the real-time deployment of the model; 3) Transformer encoder, which is widely used in natural language processing as a feature extractor, takes the one-dimensional embedded feature representation sequence of two-dimensional images as input, and uses the self-attention mechanism and multi-layer perceptron stacking to capture the long-term dependencies, but the lack of weight sharing in the model leads to a huge amount of parameters, and the high computational complexity of self-attention makes it difficult to guarantee real-time performance

At the same time, the accuracy and real-time performance of most segmentation methods cannot be effectively balanced, resulting in the inability to effectively meet the needs of actual segmentation tasks

Therefore, in view of the high computational complexity and poor generalization ability of the segmentation model, there is an urgent need for a method that can not only guarantee the real-time performance of the segmentation model but also achieve high semantic segmentation accuracy.

Images