Monocular camera and millimeter wave radar fused object identification and positioning method and system

A technology of millimeter-wave radar and object recognition, applied in neural learning methods, character and pattern recognition, computer components, etc., can solve the problems of unapplied object recognition and positioning, to overcome sparse instability, increase positioning accuracy, and system The effect of the cost advantage

Pending Publication Date: 2019-08-16
ZHEJIANG UNIV
View PDF4 Cites 47 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Although it reduces the area and time of image processing and effectively completes the vehicle detection task, it has no application in object recognition and positioning

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Monocular camera and millimeter wave radar fused object identification and positioning method and system
  • Monocular camera and millimeter wave radar fused object identification and positioning method and system
  • Monocular camera and millimeter wave radar fused object identification and positioning method and system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0048] The specific implementation manners of the present invention will be described in further detail below in conjunction with the accompanying drawings. refer to figure 1 The object recognition and positioning method in the present invention will be further described.

[0049] An object recognition and positioning method based on the fusion of a monocular camera and a millimeter-wave radar, which specifically includes the following steps:

[0050] Step 1: Sensor data acquisition and frame synchronization;

[0051] 1a. Millimeter-wave radar and monocular camera data collection: After fixing the relative position of the millimeter-wave radar and monocular camera, keep the direction of the two in the same direction and overlap the field of view, and start collecting scene perception data.

[0052] 1b. Data sequence matching based on frame synchronization: After the data collection is turned on, the millimeter-wave radar and the monocular camera will generate a series of poi...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a monocular camera and millimeter wave radar fused object identification and positioning method and system, and belongs to the technical field of multi-sensor fusion. The method comprises the following steps: step 1, sensor data acquisition and frame synchronization; 1a, millimeter wave radar and monocular camera data acquisition; 1b, data sequence matching based on frame synchronization; 2, recognition of an object and a boundary based on the neural network processing image; 2a, training in advance an instance partition neural network based on Mask-RCNN; 2b, completingobject recognition and image instance segmentation by using a neural network; 3, coordinate system conversion and data fusion; 3a, under the condition of known external parameters between sensors, converting radar point cloud to an image coordinate system; 3b, completing object matching information fusion based on an instance segmentation result; and 3c, displaying the object category and the object spatial position under the three-dimensional coordinate system. The method is more stable, and the influence of severe weather on data acquisition can be overcome.

Description

technical field [0001] The invention relates to the technical field of multi-sensor fusion, in particular to an object recognition and positioning method and system for fusion of a monocular camera and a millimeter-wave radar. The monocular camera and the millimeter-wave radar are used for data fusion to complete object recognition and positioning. Background technique [0002] The perception technology of intelligent robots to the surrounding environment and objects has always been the core issue in the field of robotics research. With the rapid development and wide application of deep learning technology, the processing ability of sensor data has also made great progress. At present, inferring image information based on neural networks has been able to recognize obstacles with high accuracy, and even obtain precise object outline boundaries. Considering that the environmental information obtained by a single visual sensor is limited, and it is also susceptible to interfer...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06K9/62G06K9/34G06N3/04G06N3/08
CPCG06N3/08G06V10/267G06N3/045G06F18/251
Inventor 王滔祝义朋朱世强徐志伟
Owner ZHEJIANG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap