RGB-D multi-mode fusion person detection method based on asymmetric double-flow network

Abstract

The invention discloses an RGB-D multi-modal fusion person detection method based on an asymmetric double-flow network, and belongs to the field of computer vision and image processing. The method comprises the steps of RGBD image acquisition, depth image preprocessing, RGB feature extraction and Depth feature extraction, RGB multi-scale fusion and Depth multi-scale fusion, multi-modal feature channel reweighting and multi-scale personnel prediction. According to the method, an asymmetric RGBD double-flow convolutional neural network model is designed to solve the problem that a traditional symmetric RGBD double-flow network is prone to causing depth feature loss. Multi-scale fusion structures are designed for RGBD double-flow networks respectively, so multi-scale information complementation is achieved. A multi-modal reweighting structure is constructed, the RGB and Depth feature maps are combined, and weighted assignment is performed on each combined feature channel to realize modelautomatic learning contribution proportion. Person classification and frame regression are carried out by using multi-modal features, so the accuracy of personnel detection is improved while the real-time performance is ensured, and the robustness of detection under low illumination at night and personnel shielding is enhanced.

Description

technical field [0001] The invention belongs to the field of computer vision and image processing, and in particular relates to an RGB-D multi-modal fusion personnel detection method based on an asymmetric dual-stream network. Background technique [0002] In recent years, the fields of smart home, smart building, and smart security have developed rapidly. The wide application of video extraction and analysis technology has become a key driving force for its progress. Among them, the detection and statistics of people have gradually become the core of image video analysis and artificial intelligence. A hot research topic. In terms of smart home, by detecting the indoor personnel, the position of the person can be located, the behavior habits of the personnel can be recorded, and the intelligent equipment such as indoor lighting and air conditioning can be further adjusted to provide people with a more comfortable and intelligent home environment. In terms of smart buildings...

AI Technical Summary

Problems solved by technology

[0009] However, this method needs to manually extract the gradient direction histogram of the traditional RGBD image as an image feature, which is time-consuming and laborious and takes up a large storage space, and cannot realize end-to-end pedestrian detection; the gradient direction histogram feature is relatively simple, and it is difficult to extract RGB and depth Discriminatory features in the image are used for pedestrian detection; this method uses a simple fusion of RGB and depth image features, and it is difficult to fully and effectively mine the fine-grained information such as color and texture of the RGB image and the semantic information such as edge and depth provided by the depth image. , to realize the association and complementarity between multi-modal data, which has great limitations in improving the accuracy of pedestrian detection

Images