Unlock instant, AI-driven research and patent intelligence for your innovation.

Dynamic three-dimensional scene reconstruction method based on image semantic segmentation

A semantic segmentation and 3D scene technology, applied in image analysis, image data processing, 3D modeling, etc., can solve problems such as increased camera pose error and inaccurate models, and achieve good positioning and mapping effects

Pending Publication Date: 2022-07-08
电子科技大学长三角研究院(湖州)
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, when there are objects (such as pedestrians) that move independently relative to the camera in the camera image, the error in the calculation of the camera pose will increase and the constructed model will be inaccurate.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Dynamic three-dimensional scene reconstruction method based on image semantic segmentation
  • Dynamic three-dimensional scene reconstruction method based on image semantic segmentation
  • Dynamic three-dimensional scene reconstruction method based on image semantic segmentation

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0028] The present invention will be further described below with reference to the accompanying drawings.

[0029] First, use the Deeplabv3+ image semantic segmentation network to perform pixel-level segmentation of the input image frame. Select Deelabv3+, the current top-notch semantic segmentation network, combine Deelabv3+ with ORB-SLAM2, and use Deelabv3+ semantic segmentation network to perform semantic segmentation on image frames. The segmentation process is as follows:

[0030] In the encoding part, the MobileNet lightweight deep convolutional network is used to convolve the image to obtain two feature layers: one low-level feature and one high-level feature. Then, for the high-level features, a 1×1 convolution, three 3×3 hole convolutions with different ratios, and an image pooling operation are performed for feature extraction and merged in a total of 5 operations.

[0031] In the decoding part, the low-level features obtained in the encoding part are combined with ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses an improved three-dimensional scene reconstruction technology fused with a semantic segmentation network. The method is mainly used for positioning and mapping in a dynamic scene, especially in an indoor dynamic scene with a moving object. Aiming at the problem that a moving object can influence a three-dimensional reconstruction effect in a scene reconstruction process, a relatively advanced semantic segmentation network DeeplabV < 3 + > at present is added in an ORB-SLAM2 framework, an MSCOCO data set is selected to train the semantic segmentation network, and static feature points and potential dynamic feature points are distinguished. And then screening is carried out by using a relationship between matched feature points and corresponding epipolar lines between front and back frames, and target feature points in a motion state in the potential feature points are determined and removed to obtain an accurate three-dimensional scene map. The ORB-SLAM2 model added with the semantic segmentation network can reduce the interference of a moving target on positioning and mapping in a dynamic scene to a certain extent, and the obtained three-dimensional map fuses part of semantic information.

Description

technical field [0001] The invention relates to the field of synchronous positioning and mapping in computer vision, and is aimed at positioning and mapping in dynamic scenes, especially in indoor dynamic scenes with moving objects. Background technique [0002] Simultaneous positioning and mapping, namely SLAM technology. Through the continuous motion of the robot in the unknown environment, the robot can acquire the three-dimensional structure of the scene with its own camera and update its own pose at the same time. SLAM technology is widely used in unmanned, intelligent UAV, AR and other fields. [0003] The current classical 3D reconstruction algorithms are based on static environments or slowly changing environments, so that the constructed scene can be gradually updated by the system. When an object (such as a pedestrian) that moves independently relative to the camera appears in the camera image, it will cause problems such as an increase in the error of the camera...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06T17/00G06T7/10
CPCG06T17/00G06T7/10G06T2207/10012
Inventor 贾海涛袁丁张博阳刘博文陈璐孙靖哲许文波
Owner 电子科技大学长三角研究院(湖州)