Method and system for reconstructing single image to three-dimensional point cloud model based on attention mechanism

Abstract

The invention provides a method and system for reconstructing a single image to a three-dimensional point cloud model based on an attention mechanism, and the method comprises the steps: carrying outfeature extraction of a to-be-processed image to acquire an image global feature matrix and an image local feature vector; processing the image global feature vector by using a model feature conversion network to obtain an initial sparse point cloud model feature, and generating a point cloud model; according to the features of the current point cloud model and the image local feature matrix, introducing an attention mechanism, and calculating an image association feature matrix of each sub-region in the current point cloud model; processing features of the current point cloud model and the corresponding image association feature matrix by using a next model feature conversion network to generate an optimized point cloud model; and repeating the optimization process, and sequentially polling each model feature conversion network to obtain a final dense point cloud model. The invention has good robustness for disturbance, and has good performance in an image generation point cloud task.

Description

technical field [0001] The invention belongs to the technical field of three-dimensional point cloud model generation, and in particular relates to a single image-to-three-dimensional point cloud model reconstruction method and system based on an attention mechanism. Background technique [0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art. [0003] With the in-depth research of computer vision and the continuous improvement of technical level, the application of 3D point cloud model is gradually widespread, involving various fields such as medical research, digital protection of cultural relics, game software development and engineering applications, and will be based on computer vision The combination of the 3D point cloud model and artificial intelligence can automatically generate a 3D point cloud model from a single image, which brings great convenience to research in v...

AI Technical Summary

Problems solved by technology

[0006] (1) 3D point cloud modeling technology based on professional software: this type is relatively professional, and users need to spend a lot of time learning how to use the software. Even only professional designers can use professional software well. The professional quality of personnel is high and difficult to master

[0007] (2) Modeling technology based on scanning equipment: low-cost scanning equipment often has low precision and small scanning scenes, while 3D scanning equipment with high precision and large scenes is very expensive, such as 3D laser scanners, which are only used in professional modeling domain, and lack of user participation and interaction in the modeling process

On the one hand, when extracting image features from a single image, since the image is two-dimensional, the extracted image features lack the information of the occluded part of the target, so the complete three-dimensional structure information cannot be obtained

On the other hand, since the 3D point cloud model is mostly used in the field of visualization, in order to facilitate modeling, mathematical parameters are often used to describe a 3D model, which reduces the consistency between the 3D model and the image, and increases the difficulty of generating the 3D model

Images