Self-adaptive reinforced fusion real-time instance segmentation method based on camera and laser radar

Abstract

The invention discloses an adaptive reinforcement fusion real-time instance segmentation method based on a camera and a laser radar. The method comprises the steps: respectively extracting the image features of a camera image and a laser radar projection image of a target through employing a convolutional neural network, and respectively obtaining a first image feature and a second image feature;adaptively allocating weights of the first image feature and the second image feature, weighting the first image feature according to the allocated first weight to obtain a third image feature, weighting the second image feature according to the allocated second weight to obtain a fourth image feature, and performing enhanced fusion on the third image feature and the fourth image feature; According to the fused image features, outputting the category, confidence, bounding box and mask of the target by using a real-time instance segmentation network, and obtaining an instance segmentation result of the target. According to the method, target instance segmentation can be accurately and robustly realized in real time in a complex environment, and the method has a wide application prospect inthe field of intelligent network connection vehicle perception.

Description

technical field [0001] The invention belongs to the technical field of target instance segmentation of intelligent networked vehicles, in particular to an adaptive enhanced fusion real-time instance segmentation method based on a camera and a laser radar. Background technique [0002] Intelligent networked vehicles have great potential in improving road safety and traffic efficiency, and accurate perception of the traffic environment is the basis for planning, decision-making and control of intelligent networked vehicles. As the most commonly used sensor in the perception system of intelligent networked vehicles, the camera can obtain detailed shape and texture information of the surrounding environment. In recent years, vision-based deep learning algorithms have achieved impressive results in environmental perception. However, the camera is easily affected by light and weather conditions, causing the performance of the algorithm to seriously degrade or even fail. Therefor...

AI Technical Summary

Problems solved by technology

[0006] In short, the problems existing in the existing technology are: only relying on single-mode sensors cannot accurately and robustly perceive the external environment; and the current fusion sensing method based on cameras and lidars has low target positioning accuracy and poor real-time performance, and all The adopted fusion method cannot adaptively learn the complementary features of different modalities

Images