A geometric visual three-dimensional model texturing method and system

Abstract

The present application relates to the field of three-dimensional reconstruction technology, and specifically provides a method and system for texturing a geometric visual three-dimensional model, the method comprising: obtaining texture data and a textured reference image; mapping the texture data to a three-dimensional geometric model to generate a geometric structure; projecting the geometric structure to a plane corresponding to the textured reference image to generate a three-dimensional geometric model projection plane; back-projecting the three-dimensional geometric model projection plane to the three-dimensional geometric model to obtain an initial textured model. By back-projecting the three-dimensional geometric model projection plane to the three-dimensional geometric model, the purpose of providing great help to practical applications and ensuring the security of data and services is achieved.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional reconstruction technology, and in particular to a method and system for texturing a geometric visual three-dimensional model. Background Art

[0002] The point cloud model obtained based on multi-view stereo vision or laser point cloud scanning can represent the scene structure, and texture and lighting information are also the focus of visual attention. At present, processing is performed based on the texture system and lighting system based on the geometric model, or the texture and lighting information of the geometric structure is restored in the form of mapping. Structural texturing based on software systems requires a lot of manual operations. Based on mapping, the texturing of the main view of the model can be achieved, but the fineness of the mapping in the details is limited.

[0003] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0004] To this end, the technical problem to be solved by the present invention is to overcome the problem that structural texturing based on software systems in the prior art requires a lot of manual operations. Texturing of the main view of the model can be achieved based on mapping, but the fineness of the mapping in the details is limited.

[0005] In order to solve the above technical problems, the first aspect of the present invention provides a method for texturing a geometric visual three-dimensional model, the method comprising:

[0006] Get texture data and texturing reference images;

[0007] Mapping the texture data to a three-dimensional geometric model to generate a geometric structure;

[0008] Projecting the geometric structure onto a plane corresponding to the textured reference image to generate a three-dimensional geometric model projection plane;

[0009] The three-dimensional geometric model projection plane is back-projected onto the three-dimensional geometric model to obtain an initial textured model.

[0010] In one embodiment of the present invention, the step of acquiring a textured reference image comprises:

[0011] Acquire the collected image;

[0012] Determining whether the captured image is captured by mobile operation;

[0013] If yes, obtaining a preset number of images corresponding to adjacent coordinate points of the acquired image;

[0014] A preset number of images are subjected to feature matching and image fusion to generate a textured reference image.

[0015] In one embodiment of the present invention, the step of determining whether the captured image is captured by mobile means comprises:

[0016] If not, the plane of the captured image is the textured reference image plane;

[0017] A textured reference image is generated according to the textured reference image plane.

[0018] In one embodiment of the present invention, the preset number is 3.

[0019] In one embodiment of the present invention, the step of obtaining the collected image includes:

[0020] Determining whether the acquired image contains a multimodal image;

[0021] If yes, then obtain multimodal data;

[0022] The multimodal data are fused to generate a textured reference image.

[0023] In one embodiment of the present invention, the step of projecting the geometric structure onto a plane corresponding to the textured reference image to generate a three-dimensional geometric model projection plane includes:

[0024] Projecting the geometric structure onto a plane corresponding to the textured reference image according to a global projection or a local projection, to generate a model projection image;

[0025] Determining whether the model projection image and the textured reference plane have consistent scales;

[0026] If so, the texture data of the textured reference image is superimposed on the model projection image to generate a three-dimensional geometric model projection plane.

[0027] In one embodiment of the present invention, the step of determining whether the model projection image and the textured reference plane have consistent scales comprises:

[0028] If not, the model projection image and the textured reference image are matched, and the texture data of the matched textured reference image is superimposed on the matched model projection image to generate a three-dimensional geometric model projection plane.

[0029] A second aspect of the present invention provides a geometric visual three-dimensional model texturing system, the system comprising: a texture reference image determination module, a three-dimensional geometric model projection module and a texturing module;

[0030] The textured reference image determination module is configured to: acquire texture data and a textured reference image; map the texture data to a three-dimensional geometric model to generate a geometric structure;

[0031] The three-dimensional geometric model projection module is configured to: project the geometric structure onto a plane corresponding to the textured reference image to generate a three-dimensional geometric model projection plane;

[0032] The texturing module is configured to: back-project the three-dimensional geometric model projection plane onto the three-dimensional geometric model to obtain an initial textured model.

[0033] The third aspect of the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in the first aspect or any possible implementation of the first aspect is implemented.

[0034] A fourth aspect of the present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect or any possible implementation of the first aspect.

[0035] The above technical solution of the present invention has the following advantages compared with the prior art:

[0036] The geometric visual three-dimensional model texturing method and system described in the present invention, by back-projecting the projection plane of the three-dimensional geometric model to the three-dimensional geometric model, achieves the purpose of providing great help to practical applications while ensuring the security of data and business. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to make the contents of the present invention more clearly understood, the present invention is further described in detail below based on specific embodiments of the present invention in conjunction with the accompanying drawings.

[0038] Figure 1 It is a flow chart of a geometric visual three-dimensional model texturing method and system provided by the present invention;

[0039] Figure 2 It is a system architecture diagram of a geometric visual three-dimensional model texturing method and system provided by the present invention. DETAILED DESCRIPTION

[0040] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0041] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0042] Reference Figure 1 As shown, the present invention provides a method for texturing a geometric visual three-dimensional model, the method comprising:

[0043] S100, acquiring texture data and a textured reference image;

[0044] In step S100, the step of obtaining a textured reference image includes: obtaining a captured image; determining whether the captured image is a mobile capture; if so, obtaining a preset number of images corresponding to adjacent coordinate points of the captured image; performing feature matching and image fusion on the preset number of images to generate a textured reference image. The step of determining whether the captured image is a mobile capture includes: if not, the plane of the captured image is a textured reference image plane; generating a textured reference image based on the textured reference image plane. The preset number is 3. The step of obtaining the captured image includes: determining whether the captured image has a multimodal image; if so, obtaining multimodal data; fusing the multimodal data to generate a textured reference image.

[0045] In actual application scenarios, the coordinate position of the captured image captured by the fixed acquisition device is first determined. Since there will be an offset in the acquisition direction during actual operation, the plane of the captured image, that is, the direction of the camera, is determined as the plane of the textured reference image. The captured image is the textured reference image. For the captured image captured by mobile shooting, three adjacent coordinate points are determined based on the coordinate position of the captured image. The three adjacent coordinates can determine a fitting plane, which is determined as the plane of the textured reference image.

[0046] In the case of mobile acquisition, the textured reference image is obtained based on the corresponding images of three adjacent coordinate points. The textured reference image is obtained by performing feature matching and image fusion on the three images. Feature matching uses feature matching algorithms such as scale-invariant feature transformation. According to the results of feature matching, the geometric transformation and alignment of the image are completed. Image fusion algorithms such as Poisson fusion are used to fuse the images. The obtained image is the textured reference image.

[0047] When the scales of the fused images are inconsistent, it is necessary to resample the images to achieve scale consistency to avoid problems such as feature point matching errors and difficulty in alignment. Image resampling is I(x, y) = f(I′(x′, y′)), where I′(x′, y′) is the original image, I(x, y) is the resampled image, f(·) is the interpolation function (such as nearest neighbor interpolation, bilinear interpolation, etc.), and x, y and x′, y′ are the coordinates of the target image and the original image, respectively.

[0048] In order to better texture the 3D geometric model in the case of mobile acquisition, the 3D geometric model is layered. The layering is divided into several areas from low to high according to the elevation of the 3D geometric model. Each layer contains images of several coordinate points in the layer area. In order to reduce the impact of the repeated structure of the power equipment on the matching, the local texture reference image is selected and fused according to the layering situation.

[0049] In the case of multimodal images, the multimodal data can be fused, and the feature matching and fusion refer to the visible light image feature matching and fusion. In order to achieve better mapping of structure and texture, the source image of the textured reference image is a visible light image, and other modal data is used for subsequent superposition of other information (such as temperature of infrared images, discharge of ultraviolet images, etc.).

[0050] S200, mapping the texture data to a three-dimensional geometric model to generate a geometric structure;

[0051] In step S200, the direction of the three-dimensional geometric model can be obtained by multi-view stereo method or laser point cloud scanning. For the uncertainty of the direction caused by multi-view stitching, the main direction of the point cloud can be determined by principal component analysis. The specific calculation method is to calculate the centroid of the point cloud, that is, the mean (centroid) of all points, and subtract this mean from each point, construct a covariance matrix, calculate the eigenvalues ​​and eigenvectors of the covariance matrix, and the direction of the eigenvector corresponding to the maximum eigenvalue of the covariance matrix is ​​the main direction of the point cloud data, as well as the secondary direction (the second and third main maximum eigenvalues). After determining the main direction and the second and third directions, the direction of the three-dimensional geometric model can be determined. The geometric structure is obtained by mapping the texture data onto the three-dimensional geometric model.

[0052] S300, projecting the geometric structure onto a plane corresponding to the textured reference image to generate a three-dimensional geometric model projection plane;

[0053] In step S300, the step of projecting the geometric structure onto the plane corresponding to the textured reference image to generate a three-dimensional geometric model projection plane includes: projecting the geometric structure onto the plane corresponding to the textured reference image according to global projection or local projection to generate a model projection image; judging whether the scales of the model projection image and the textured reference plane are consistent; if so, superimposing the texture data of the textured reference image onto the model projection image to generate a three-dimensional geometric model projection plane. The step of judging whether the scales of the model projection image and the textured reference plane are consistent includes: if not, matching the model projection image with the textured reference image, superimposing the texture data of the matched textured reference image onto the matched model projection image to generate a three-dimensional geometric model projection plane.

[0054] In actual application scenarios, the geometric structure is then globally projected or locally projected onto the textured reference plane based on the layers. In the best case, the target positions in the model projection image and the textured reference image are basically consistent, and the texture data of the textured reference image can be directly superimposed on the model projection image. Texture mapping uses algorithms such as plane mapping. In the case of a high degree of inconsistency, the model projection image and the textured reference image can be matched again. After matching, the target positions and scales in the model projection image and the textured reference image will be basically consistent, and then the subsequent operations can be continued.

[0055] S400, back-projecting the three-dimensional geometric model projection plane onto the three-dimensional geometric model to obtain an initial textured model.

[0056] In step S400, all the coordinate points of the textured reference image are traversed to complete the above-mentioned steps of fitting the textured reference image, projecting the three-dimensional geometric model, and texture mapping. Then, the projection plane of the three-dimensional geometric model is back-projected into the three-dimensional geometric model to obtain an initial textured model. The texture of the power equipment is relatively simple, and the coverage of the material is not large. For local areas that still lack texture after the above operations, texture completion can be performed. When completing, the target is first connected to detect the connection, and the connected body can be filled with the existing texture for completion, so as to obtain a complete textured model.

[0057] A second aspect of the present invention provides a geometric visual three-dimensional model texturing system, the system comprising: a texture reference image determination module 100, a three-dimensional geometric model projection module 200 and a texturing module 300;

[0058] The textured reference image determination module 100 is configured to: acquire texture data and a textured reference image; map the texture data to a three-dimensional geometric model to generate a geometric structure;

[0059] The three-dimensional geometric model projection module 200 is configured to: project the geometric structure onto a plane corresponding to the textured reference image to generate a three-dimensional geometric model projection plane;

[0060] The texturing module 300 is configured to: back-project the three-dimensional geometric model projection plane onto the three-dimensional geometric model to obtain an initial textured model.

[0061] The effects of the above-mentioned system when applying the above-mentioned method can be found in the description of the above-mentioned method embodiment, which will not be repeated here.

[0062] The third aspect of the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in the first aspect or any possible implementation of the first aspect is implemented.

[0063] A fourth aspect of the present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect or any possible implementation of the first aspect.

[0064] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0065] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0066] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0067] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0068] Obviously, the above embodiments are merely examples for clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from these are still within the protection scope of the invention.

Claims

1. A method for texturing a geometric visual three-dimensional model, characterized in that: The method comprises: Get texture data and texturing reference images; Mapping the texture data to a three-dimensional geometric model to generate a geometric structure; Projecting the geometric structure onto a plane corresponding to the textured reference image to generate a three-dimensional geometric model projection plane; Back-projecting the projection plane of the three-dimensional geometric model onto the three-dimensional geometric model to obtain an initial textured model; The step of projecting the geometric structure onto a plane corresponding to the textured reference image to generate a three-dimensional geometric model projection plane includes: Projecting the geometric structure onto a plane corresponding to the textured reference image according to a global projection or a local projection, to generate a model projection image; Determining whether the model projection image and the textured reference plane have consistent scales; If yes, then the texture data of the textured reference image is superimposed on the model projection image to generate a three-dimensional geometric model projection plane; If not, the model projection image and the textured reference image are matched, and the texture data of the matched textured reference image is superimposed on the matched model projection image to generate a three-dimensional geometric model projection plane.

2. A geometric visual three-dimensional model texturing method according to claim 1, characterized in that: The steps to obtain a textured reference image include: Acquire the collected image; Determining whether the captured image is captured by mobile operation; If yes, obtaining a preset number of images corresponding to adjacent coordinate points of the acquired image; A preset number of images are subjected to feature matching and image fusion to generate a textured reference image.

3. A geometric visual three-dimensional model texturing method according to claim 2, characterized in that: The step of determining whether the captured image is captured by mobile means comprises: If not, the plane of the captured image is the textured reference image plane; A textured reference image is generated according to the textured reference image plane.

4. A geometric visual three-dimensional model texturing method according to claim 2, characterized in that: The preset number is 3.

5. A geometric visual three-dimensional model texturing method according to claim 2, characterized in that: The steps to obtain the acquired image then include: Determining whether the acquired image contains a multimodal image; If yes, then obtain multimodal data; The multimodal data are fused to generate a textured reference image.

6. A geometric visual three-dimensional model texturing system, characterized in that: The system comprises: a texture reference image determination module, a three-dimensional geometric model projection module and a texturing module; The textured reference image determination module is configured to: acquire texture data and a textured reference image; map the texture data to a three-dimensional geometric model to generate a geometric structure; The three-dimensional geometric model projection module is configured to: project the geometric structure onto a plane corresponding to the textured reference image to generate a three-dimensional geometric model projection plane; The texturing module is configured to: back-project the three-dimensional geometric model projection plane onto the three-dimensional geometric model to obtain an initial textured model; The step of projecting the geometric structure onto a plane corresponding to the textured reference image to generate a three-dimensional geometric model projection plane includes: Projecting the geometric structure onto a plane corresponding to the textured reference image according to a global projection or a local projection, to generate a model projection image; Determining whether the model projection image and the textured reference plane have consistent scales; If yes, then the texture data of the textured reference image is superimposed on the model projection image to generate a three-dimensional geometric model projection plane; If not, the model projection image and the textured reference image are matched, and the texture data of the matched textured reference image is superimposed on the matched model projection image to generate a three-dimensional geometric model projection plane.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of a geometric visual three-dimensional model texturing method as described in any one of claims 1 to 5 are implemented.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of a geometric visual three-dimensional model texturing method as described in any one of claims 1 to 5 are implemented.

Images