Object detection method and device based on three-dimensional data and two-dimensional image

A two-dimensional image and object detection technology, applied in the field of image processing, can solve the problems of high price, inability to extract object point cloud, sparse data, etc., achieve recognition and accurate positioning, reduce pending data, and improve efficiency

Active Publication Date: 2020-08-28
汇智机器人科技(深圳)有限公司
View PDF4 Cites 16 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0002] At present, the processing of images only processes three-dimensional images or two-dimensional images separately, and two-dimensional images can recognize objects, but the objects obtained by two-dimensional images are flat, and the three-dimensional information of objects cannot be obtained; three-dimensional images can obtain objects At present, 3D image information can be obtained including 3D lidar. The number of lines of lidar is proportional to the density of point cloud data. The more lines, the higher the density of point cloud data, and the higher the number of lines The 3D lidar is expensive, and the 3D lidar with a low number of lines is limited by the number of lines. When the distance is long, the distance between the lines will increase, and the point cloud will become sparse, so conventional clustering and segmentation cannot be used. The method effectively extracts the point cloud of the object, resulting in sparse data, and the success rate and accuracy of object recognition are low

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Object detection method and device based on three-dimensional data and two-dimensional image
  • Object detection method and device based on three-dimensional data and two-dimensional image

Examples

Experimental program
Comparison scheme
Effect test

specific Embodiment 1

[0034] An object detection method based on three-dimensional data and two-dimensional images of the present invention, such as figure 1 shown, including the following steps:

[0035] S1. Acquire 2D images and 3D point cloud data of the environment;

[0036] S2. Recognize the object from the two-dimensional image, and obtain the position and bounding box of the object in the two-dimensional image;

[0037] S3. Map the data in the three-dimensional image to the two-dimensional image, and obtain the three-dimensional point cloud data within the bounding box of the object;

[0038] S4, filtering the three-dimensional point cloud data in the bounding box;

[0039] S5. Calculate the physical center of the filtered three-dimensional point cloud data, and obtain the three-dimensional coordinates of the central position of the identified object.

[0040] In a specific embodiment of the present application, a camera is used to obtain a two-dimensional image, and a three-dimensional l...

specific Embodiment 2

[0059] In a specific embodiment of the present application, including M groups of two-dimensional images of the environment, according to the positional relationship between the three-dimensional data and each group of two-dimensional images, the three-dimensional point cloud data is mapped to the two-dimensional image coordinates, and each group of two-dimensional Image, filtering points beyond the visual range corresponding to the two-dimensional image, combining all two-dimensional images, so that all three-dimensional point cloud data points are mapped to two-dimensional image coordinates. Wherein, M is a positive integer greater than or equal to 1.

[0060] Specifically, it includes three sets of environmental two-dimensional images, forming a full coverage of 360 degrees in the horizontal direction. The visual range of each group of environmental two-dimensional images corresponding to the horizontal direction is greater than or equal to 120 degrees. According to the posi...

specific Embodiment 3

[0063] An object detection device for three-dimensional data and two-dimensional images, including a camera, a three-dimensional laser radar, and a data processing center. The three-dimensional laser radar is set horizontally, and the relative positions of the camera and the three-dimensional laser radar are fixed to ensure that the coordinates of the two-dimensional data and the three-dimensional data can be realized. Mutual conversion, the horizontal shooting range of the camera is less than or equal to the scanning range of the 3D laser radar; the 3D laser radar generates 16 laser lines, collects point cloud data of each laser line, and the data processing center obtains the 2D image captured by the camera, the 3D laser radar The 3D point cloud data of 16 laser lines is taken, and the object detection method based on 3D lidar and vision is used to recognize the object in the 2D image and obtain the bounding box of the object. According to the relative position relationship be...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses an object detection method and device based on three-dimensional data and a two-dimensional image. The object detection method comprises the following steps: acquiring an environment two-dimensional image and three-dimensional point cloud data; identifying an object from the two-dimensional image, and obtaining the position and the boundary frame of the object in the two-dimensional image; mapping data in the three-dimensional image into the two-dimensional image to obtain three-dimensional point cloud data in the boundary frame range of the object; filtering the three-dimensional point cloud data in the boundary frame; and calculating the physical center of the filtered three-dimensional point cloud data, and obtaining the three-dimensional coordinates of the central position of the identified object, thereby realizing accurate positioning of the three-dimensional coordinates of the object. The object detection device comprises a plurality of cameras, a three-dimensional laser radar and a data processing center, wherein the relative positions of the cameras and the three-dimensional laser radar are fixed, and the visual range of the cameras in the horizontal direction is smaller than or equal to the scanning range of the three-dimensional laser radar. The object detection device adopts a detection method, and identifies an object and determines three-dimensional coordinates of the object.

Description

technical field [0001] The invention relates to the technical field of image processing, in particular to an object detection method and device based on three-dimensional data and two-dimensional images. Background technique [0002] At present, the processing of images only processes three-dimensional images or two-dimensional images separately, and two-dimensional images can recognize objects, but the objects obtained by two-dimensional images are flat, and the three-dimensional information of objects cannot be obtained; three-dimensional images can obtain objects At present, 3D image information can be obtained including 3D lidar. The number of lines of lidar is proportional to the density of point cloud data. The more lines, the higher the density of point cloud data, and the higher the number of lines The 3D lidar is expensive, and the 3D lidar with a low number of lines is limited by the number of lines. When the distance is long, the distance between the lines will in...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06T3/00G06T7/70G06T7/10
CPCG06T3/005G06T7/70G06T7/10G06T2207/10028G06T2207/10044
Inventor 潘何浩向良华罗方龙陈兆先张殿礼
Owner 汇智机器人科技(深圳)有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap