Time-frequency domain combined panoramic segmentation convolutional neural network and application

Abstract

The invention discloses a time-frequency domain combined panoramic segmentation convolutional neural network and application, and belongs to the field of deep learning image processing. Comprising: a frequency domain transformation network that transforms input into frequency domain information and extracts high and low frequency features of an image; the time domain transformation network that is used for transforming the input into time domain information and extracting instance features and semantic features in the image; the time-frequency domain joint network that is used for sequentially distributing product coefficients for the high-frequency features, the low-frequency features, the instance features and the semantic features correspondingly output by the frequency domain transformation network and the time domain transformation network respectively; and the segmentation fusion network that is used for fusing the foreground and background segmentation results to generate a panoramic segmentation result. The invention has the advantages that the invention can be applied to multiple fields such as autonomous automobiles, auxiliary driving, robots and public safety skyeye monitoring systems.

Description

technical field [0001] The invention belongs to the field of deep learning image processing, and specifically relates to a convolutional neural network that combines two perspective analysis panorama segmentation algorithms in the time domain and the frequency domain, and is suitable for unmanned driving and autonomous robot scenarios. Background technique [0002] In recent years, major breakthroughs have been made in the field of unmanned driving and robotics due to the rapid development of deep learning. Based on the powerful scene understanding function, panoramic segmentation technology has gradually become an important means of computer vision environment perception. However, the traditional time-domain convolutional neural network can only perform indiscriminate feature extraction on images from the perspective of the spatial domain, ignoring the significant difference between the foreground and background in the image due to different frequencies, resulting in the acc...

AI Technical Summary

Problems solved by technology

The patent "A Predictive Optimization Method for Image Panoramic Segmentation Based on Convolution" (Publication No.: CN109801297A) discloses a convolutional neural network based on time-domain panoramic segmentation. The network starts from the spatial relationship and occludes the priority of the instance target. Sorting solves the occlusion problem between instances, but this patent does not take into account the frequency characteristics of the panoramic image, and loses the high-frequency information of the instance object, which may limit the recognition accuracy of the network for the instance object

Images