Heterogenous data fusion method and device, and storage medium

Abstract

The invention relates to a heterogeneous data fusion method and device, and a storage medium. The heterogeneous data fusion method comprises the steps of obtaining two-dimensional image data and three-dimensional point cloud data of a target object; extracting semantic information of the two-dimensional image data to obtain color space features, and extracting point cloud features of the three-dimensional point cloud data to obtain geometric space features; fusing the color space features and the geometric space features to obtain pixel-level fusion features; pooling the pixel-level fusion features to obtain global features; splicing the pixel-level fusion features and the global features to obtain dense fusion features; wherein the dense fusion features are used for surface feature point detection of the target object. According to the technical scheme, the problem of pixel-level dense fusion of the three-dimensional point cloud data and the two-dimensional image data is solved, respective advantages of the three-dimensional depth information and the two-dimensional image information are fully utilized, and the surface feature points of the target object can be accurately detected by means of dense fusion features.

Description

technical field [0001] The invention relates to the technical field of image processing, in particular to a heterogeneous data fusion method and device, and a storage medium. Background technique [0002] In today's manufacturing industry, the assembly process takes a lot of time and money. In order to improve production efficiency and reduce labor costs, people have begun to explore the use of robots to realize automated assembly. As an indispensable and important link in the automated assembly process, part recognition and grasping position planning have a vital impact on the quality of assembly. Vision-based part pose determination and grasping position planning can significantly improve the automation of product assembly. Flexibility, reduce time-consuming and reduce costs, thereby improving manufacturing efficiency. Robotic automated assembly involves two key technologies: part recognition and automatic grasping. [0003] At present, pose recognition mainly uses two-d...

AI Technical Summary

Problems solved by technology

At present, the collected point cloud data often has many defects in complex environments, such as data invalidity, noise, holes and other defects. For the lack of environmental information caused by these defects, the existing methods are mostly through fitting and interpolation methods. In addition, the obtained information is only an estimate of environmental information, rather than real and accurate information. As a result, the existing point cloud data utilization technology cannot meet the requirements of real-time and accurate perception of the environment under complex conditions.

Images