The method of rtg visual obstacle detection based on rgb and d information fusion

Abstract

The invention discloses an RTG visual obstacle detection method based on RGB and D information fusion, including: step 1, acquiring the RGB image of the left camera in the binocular camera, the Depth depth image obtained by binocular calculation, and the obstacle to be detected ROI region of interest; step 2, detect obstacles based on the Depth depth map through the U / V parallax method; step 3, perform edge contour detection on the RGB image, and detect obstacles; step 4, use the neural network model for target detection, detection training Step 5, use the trained classification model to judge whether the detection results in Step 2 and Step 3 belong to obstacles; Step 6, perform operations such as merging and filtering on the object frames in Step 4 and Step 5; Step 7 , judge whether it belongs to the ROI detection area according to the positional relationship between the object frame and the ROI region of interest; step 8, calculate the distance of the obstacle from the Depth depth map.

Description

technical field [0001] The invention relates to an RTG visual obstacle detection method based on RGB and D information fusion. Background technique [0002] Rubber-tyred container gantry crane (Rubber-Tyred Container Gantry Crane, RTG) is an important operating equipment for container port terminals. It plays a pivotal role in modern terminal operations. If you can maximize the use of RTG, it will greatly improve the efficiency of terminal work. However, RTG equipment is bulky and the height of the cab from the ground is high, which puts forward high operating requirements for the driver. Therefore, how to improve the safety of RTG equipment is an important research topic. In order to meet the above safety requirements, the anti-collision detection system has gradually received attention as an auxiliary driving device. [0003] At present, the obstacle detection methods of RTG anti-collision systems at home and abroad mainly include: obstacle detection methods based on ultr...

AI Technical Summary

Problems solved by technology

Overcome the inaccuracy of using simple visual features to detect obstacles, the loss of distance information based on RGB images alone, the unrobustness of detecting obstacles using only Depth depth maps, the inability to perceive object types, and short detection distances, etc.

Images