An object detection method based on cross-modal and multi-scale feature fusion

Abstract

The invention discloses an object detection method based on fusion of cross-modal and multi-scale features. The depth map detection network model is initialized by the network parameters of the RGB detection network model; and then based on the obtained RGB detection network model and the depth map detection network model, respectively Initialize the feature extraction weights of the fusion network model, and finally train a fusion network model that fuses multi-scale and cross-modal features. The present invention does not rely on a large number of marked depth image data sets, and can fuse depth image and RGB image features across modalities, real-time, efficient, and accurately complete object recognition, positioning and detection. The fusion network model designed by the present invention only needs a consumer-grade graphics card and a CPU as hardware to achieve real-time detection speed.

Description

technical field [0001] The present invention relates to the field of image recognition technology, in particular to an object detection method based on cross-modal multi-scale feature fusion, which simultaneously completes detection and positioning of objects in color depth images (RGB-D images, including color information and depth information) and precise identification tasks. Background technique [0002] In industry, faster, more accurate and more generalizable object detection methods are always in urgent need. RGB images will be severely affected in some special environments, such as motion or glare, which will degrade image data. Using RGB image features to complete detection often cannot achieve the expected accuracy. So it is necessary to utilize information from different sensors such as depth information to improve the performance of object detection. [0003] Since convolutional neural networks are used for object recognition and detection tasks, most high-prec...

AI Technical Summary

Problems solved by technology

However, the industry does not contain a sufficient number of categories, and a large-scale depth image dataset that has been labeled, so that it is impossible to directly obtain the general feature expression of depth information

[0004] On the other hand, the existing fusion feature detection methods have speed limitations, and often require high-performance GPUs to obtain results after long-term calculations, which cannot meet the rigid requirements for high real-time performance in industrial systems.

Images