RTG visual obstacle detection method based on RGB and D information fusion

Abstract

The invention discloses a RTG vision obstacle detection method based on RGB and D information fusion. The method comprises the following steps: 1, acquiring an RGB image of a left camera in a binocular camera, a Depth depth image calculated by the binocular camera and a ROI interested region of an obstacle to be detected; 2, detecting an obstacle through a U / V parallax method based on Depth depthmap; 3, detecting that edge contour of the RGB map and detecting an obstacle; 4, detecting that trained object class by using a neural network model for target detection; 5, using the trained classification model to judge whether the detection result in the step 2 and the step 3 belongs to an obstacle or not; Step 6, merging and filtering the object frames in the step 4 and the step 5; 7, judgingwhether that object frame belong to the ROI detection area or not according to the positional relationship between the object frame and the ROI interest area; 8, calculate that distance of the obstacle by Depth depth map.

Description

technical field [0001] The invention relates to an RTG visual obstacle detection method based on RGB and D information fusion. Background technique [0002] Rubber-tyred container gantry crane (Rubber-Tyred Container Gantry Crane, RTG) is an important operating equipment for container port terminals. It plays a pivotal role in modern terminal operations. If you can maximize the use of RTG, it will greatly improve the efficiency of terminal work. However, RTG equipment is bulky and the height of the cab from the ground is high, which puts forward high operating requirements for the driver. Therefore, how to improve the safety of RTG equipment is an important research topic. In order to meet the above safety requirements, the anti-collision detection system has gradually received attention as an auxiliary driving device. [0003] At present, the obstacle detection methods of RTG anti-collision systems at home and abroad mainly include: obstacle detection methods based on ultr...

AI Technical Summary

Problems solved by technology

Overcome the inaccuracy of using simple visual features to detect obstacles, the loss of distance information based on RGB images alone, the unrobustness of detecting obstacles using only Depth depth maps, the inability to perceive object types, and short detection distances, etc.

Images