Environment semantic perception method based on visual information

Abstract

The invention provides an environmental semantic perception method based on visual information, which comprises the following steps: acquiring environmental image information by using a Kinect V1.0 camera to obtain a registered color image and depth image; resolving the three-dimensional pose of the camera according to the ORB feature points extracted from each frame through an ORB _ SLAM2 processbased on the registered color image and depth image to obtain camera pose information; performing semantic segmentation on each frame of image to generate semantic color information; synchronously generating a point cloud according to the input depth map and the internal reference matrix of the camera; registering the semantic color information into the point cloud to obtain a local semantic point cloud result; fusing the camera pose information with the local semantic point cloud result to obtain new global semantic point cloud information; and representing the fused global semantic point cloud information by using an octree map to obtain a final three-dimensional octree semantic map. According to the invention, deeper humanoid understanding is provided for environment detection of the extraterrestrial celestial body patroller.

Description

technical field [0001] The invention relates to an environmental semantic perception method based on visual information, and belongs to the field of artificial intelligence information technology. Background technique [0002] The effective perception of the mobile platform in the environment can be realized through synchronous positioning and drawing technology. It can not only know the obstacle information in the environment, but also obtain the relative relationship between itself and the environment. This is a key step to realize the autonomy of the platform. However, as With the continuous development of platforms and detection payloads, more mission scenarios and requirements have been proposed, and the recognition of target appearance and geometric features can no longer solve practical problems. In the process of patrolling extraterrestrial celestial bodies, if a piece of terrain with a similar appearance can only rely on traditional recognition methods, the three-di...

AI Technical Summary

Problems solved by technology

[0002] The effective perception of the mobile platform in the environment can be realized through synchronous positioning and drawing technology. It can not only know the obstacle information in the environment, but also obtain the relative relationship between itself and the environment. This is a key step to realize the autonomy of the platform. However, as With the continuous development of platforms and detection payloads, more mission scenarios and requirements have been proposed, and the recognition of target appearance and geometric features can no longer solve practical problems

In the process of patrolling extraterrestrial celestial bodies, if a piece of terrain with a similar appearance can only rely on traditional recognition methods, the three-dimensional reconstruction of the environment can be obtained, but it is difficult to distinguish the difference between the two terrains. For the previous detection missions, as long as It is sufficient to be able to identify whether there are obstacles ahead and whether they can pass through. However, with the increase of detection time and scale, the existing cognition is far from enough. It is necessary to have a semantic understanding of the environment, that is, not only to detect whether If there is a goal, it needs to be further analyzed to find out what it is, which will be the core of the patrol to become intelligent

In the research of unmanned driving, this problem is equally important. During the driving process, you will encounter random targets such as vehicles, pedestrians, and roadblocks. When effectively judging what is the obstacle, it highlights the importance of semantic understanding. After giving the understanding result that one is a person and the other is a haystack, it is easy to make a correct judgment

Images