Heterogenous data fusion method and device, and storage medium

A data fusion and data technology, applied in the field of image processing, can solve problems such as data invalidity, noise, holes, etc., and achieve the effects of reducing the amount of calculation, improving execution efficiency, and satisfying adaptability

Pending Publication Date: 2021-05-25
SHENZHEN HUAHAN WEIYE TECH
View PDF0 Cites 12 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

At present, the collected point cloud data often has many defects in complex environments, such as data invalidity, noise, holes and other defects. For the lack of environmental information caused by these defects, the existing methods are mostly through fitting and interpolation methods. In addition, the obtained information is only an estimate of environmental information, rather than real and accurate information. As a result, the existing point cloud data utilization technology cannot meet the requirements of real-time and accurate perception of the environment under complex conditions.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Heterogenous data fusion method and device, and storage medium
  • Heterogenous data fusion method and device, and storage medium
  • Heterogenous data fusion method and device, and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0038] Please refer to figure 1 , the present application discloses a heterogeneous data fusion method, which includes steps S110-S150, which will be described respectively below.

[0039] Step S110, acquiring 2D image data and 3D point cloud data of the target object.

[0040] It should be noted that the target object here may be a product on an industrial assembly line, a mechanical part in an object box, a tool on an operating table, and the like. Then, the two-dimensional image of the target object can be collected by optical imaging components such as cameras and video cameras and generate two-dimensional image data; and the target object can be obtained by such as contact or non-contact scanning equipment (such as laser scanning equipment) 3D point cloud data, and even use 3D mapping software to export the 3D point cloud data of the target object. The 3D point cloud data here can be a part of the appearance shape data of the target object surface.

[0041] In a specifi...

Embodiment 2

[0090] On the basis of the heterogeneous data fusion method disclosed in Embodiment 1, this application also discloses a feature detection device of a target object. For details, please refer to Figure 7 , the feature detection device 2 mainly includes a sensor 21 and a processing unit 22, which will be described separately below.

[0091] The sensor 21 is used to collect two-dimensional images of the target object to form two-dimensional image data, and collect three-dimensional images of the target object to form three-dimensional point cloud data. The sensor 21 may be a camera with 2D and 3D acquisition functions, so the sensor 21 may be used to acquire a two-dimensional image of the target object, or the sensor 21 may be used to acquire a three-dimensional point cloud of the target object. For example, a 3D camera using RGB-D data is used to collect relevant data, and output a color image and a depth image at the same time. At this time, the RGB image and the Depth image ...

Embodiment 3

[0109] On the basis of the heterogeneous data fusion method disclosed in Embodiment 1, this application also discloses a feature detection device of a target object. For details, please refer to Figure 8 , which mainly includes a memory 31 and a processor 32, which will be described separately below.

[0110] The memory 31 is used as a computer-readable storage medium for storing programs, and the programs may be program codes corresponding to the heterologous data fusion methods S110-S150 in Embodiment 1.

[0111] Of course, the memory 32 can also store some network weight parameters, network training process data, image detection process data, 2D image data, 3D point cloud data, training sample data, feature point detection results and other data.

[0112] The processor 32 is connected to the memory 31, and is used to execute the program stored in the memory 31 to realize the corresponding method. The functions implemented by the processor 31 can refer to the processing un...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a heterogeneous data fusion method and device, and a storage medium. The heterogeneous data fusion method comprises the steps of obtaining two-dimensional image data and three-dimensional point cloud data of a target object; extracting semantic information of the two-dimensional image data to obtain color space features, and extracting point cloud features of the three-dimensional point cloud data to obtain geometric space features; fusing the color space features and the geometric space features to obtain pixel-level fusion features; pooling the pixel-level fusion features to obtain global features; splicing the pixel-level fusion features and the global features to obtain dense fusion features; wherein the dense fusion features are used for surface feature point detection of the target object. According to the technical scheme, the problem of pixel-level dense fusion of the three-dimensional point cloud data and the two-dimensional image data is solved, respective advantages of the three-dimensional depth information and the two-dimensional image information are fully utilized, and the surface feature points of the target object can be accurately detected by means of dense fusion features.

Description

technical field [0001] The invention relates to the technical field of image processing, in particular to a heterogeneous data fusion method and device, and a storage medium. Background technique [0002] In today's manufacturing industry, the assembly process takes a lot of time and money. In order to improve production efficiency and reduce labor costs, people have begun to explore the use of robots to realize automated assembly. As an indispensable and important link in the automated assembly process, part recognition and grasping position planning have a vital impact on the quality of assembly. Vision-based part pose determination and grasping position planning can significantly improve the automation of product assembly. Flexibility, reduce time-consuming and reduce costs, thereby improving manufacturing efficiency. Robotic automated assembly involves two key technologies: part recognition and automatic grasping. [0003] At present, pose recognition mainly uses two-d...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06K9/62G06N3/08G06T3/40G06T7/11G06T7/90
CPCG06T3/4038G06T7/11G06T7/90G06N3/08G06T2207/10028G06T2207/20221G06T2207/20132G06T2207/20081G06T2207/20084G06T2207/10024G06F18/253
Inventor 杨洋
Owner SHENZHEN HUAHAN WEIYE TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap