RTG visual obstacle detection method based on RGB and D information fusion

An obstacle detection and obstacle technology, which is applied in the field of RTG visual obstacle detection, can solve the problems of unrobustness, loss of distance information, and short detection distance.

Active Publication Date: 2019-03-12
NANJING UNIV
View PDF8 Cites 27 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Overcome the inaccuracy of using simple visual features to detect obstacles, the loss of distance information based on RGB images alone, the unrobustness of detecting obstacles using only Depth depth maps, the inability to perceive object types, and short detection distances, etc.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • RTG visual obstacle detection method based on RGB and D information fusion
  • RTG visual obstacle detection method based on RGB and D information fusion
  • RTG visual obstacle detection method based on RGB and D information fusion

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0098] In order to verify the effectiveness of the algorithm, videos containing different lighting, scenes, and various types of obstacles were actually collected for example verification, including situations with and without obstacles, and each frame of these videos was processed to detect whether There are obstacles and information about obstacles.

[0099] Taking the collected video clip as an example, for each frame of image in the video clip, follow the steps below to detect obstacles:

[0100] 1. Obtain the RGB image of the left camera in the binocular camera, the Depth depth image calculated by the binocular camera, and the ROI region of interest that needs to detect obstacles. The acquired image resolution is set to 640*480 pixels;

[0101] 2. Based on the Depth depth map, detect obstacles through the U / V parallax method, and obtain candidate obstacle object frames;

[0102] 3. Perform edge contour detection on the RGB image, detect obstacles, and obtain candidate ob...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a RTG vision obstacle detection method based on RGB and D information fusion. The method comprises the following steps: 1, acquiring an RGB image of a left camera in a binocular camera, a Depth depth image calculated by the binocular camera and a ROI interested region of an obstacle to be detected; 2, detecting an obstacle through a U/V parallax method based on Depth depthmap; 3, detecting that edge contour of the RGB map and detecting an obstacle; 4, detecting that trained object class by using a neural network model for target detection; 5, using the trained classification model to judge whether the detection result in the step 2 and the step 3 belongs to an obstacle or not; Step 6, merging and filtering the object frames in the step 4 and the step 5; 7, judgingwhether that object frame belong to the ROI detection area or not according to the positional relationship between the object frame and the ROI interest area; 8, calculate that distance of the obstacle by Depth depth map.

Description

technical field [0001] The invention relates to an RTG visual obstacle detection method based on RGB and D information fusion. Background technique [0002] Rubber-tyred container gantry crane (Rubber-Tyred Container Gantry Crane, RTG) is an important operating equipment for container port terminals. It plays a pivotal role in modern terminal operations. If you can maximize the use of RTG, it will greatly improve the efficiency of terminal work. However, RTG equipment is bulky and the height of the cab from the ground is high, which puts forward high operating requirements for the driver. Therefore, how to improve the safety of RTG equipment is an important research topic. In order to meet the above safety requirements, the anti-collision detection system has gradually received attention as an auxiliary driving device. [0003] At present, the obstacle detection methods of RTG anti-collision systems at home and abroad mainly include: obstacle detection methods based on ultr...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06K9/00G06K9/34G06T7/55
CPCG06T7/55G06V20/10G06V10/267
Inventor 申富饶韩峰黄羽佳姜少魁赵健李俊
Owner NANJING UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap