Three-dimensional model construction method, device, computer equipment and storage medium
By extracting and correcting for illumination and texture features in two-dimensional images, the method enhances the accuracy of three-dimensional model construction by mitigating the impact of lighting and texture variations.
Patent Information
- Application Number
- CN202011007477.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-09-23
- Publication Date
- 2024-12-17
- Estimated Expiration
- 2040-09-23
AI Technical Summary
The accuracy of three-dimensional models constructed from two-dimensional images is compromised by the influence of lighting and texture, leading to inconsistencies and reduced precision.
A method involving the extraction of illumination and texture feature information from the original image, followed by generating correction information to correct the image, and then constructing the three-dimensional model based on the corrected image to remove the influence of lighting and texture.
Improves the accuracy of the three-dimensional model by correcting for lighting and texture effects, resulting in a more precise representation of the target object's structure.
Smart Images
Figure CN112184884B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer vision technology, and in particular to a three-dimensional model construction method, device, computer equipment and storage medium. Background Art
[0002] In recent years, with the rapid development of intelligent robots, unmanned driving technology, virtual reality technology, etc., the construction of three-dimensional models based on two-dimensional images is a very important research direction in the field of computer vision.
[0003] In the prior art, the process of constructing a three-dimensional model based on a two-dimensional image is generally: the two-dimensional image is input into a trained neural network model, the neural network model can directly extract the feature vector representing the three-dimensional structural information of the target object in the two-dimensional image, and the target object is reconstructed in three dimensions based on the feature vector of the three-dimensional structural information of the target object.
[0004] However, in the above method, due to the influence of texture, lighting, etc., the accuracy of the reconstructed three-dimensional model of the target object is not high. Summary of the invention
[0005] Based on this, it is necessary to provide a three-dimensional model construction method, device, computer equipment and storage medium that can improve the accuracy of the three-dimensional model of the target object in order to solve the above technical problems.
[0006] A three-dimensional model construction method, the method comprising:
[0007] Obtaining an original image, and extracting illumination feature information and texture feature information from the original image;
[0008] Generate illumination correction information based on illumination feature information, and generate texture correction information based on texture feature information;
[0009] Correcting the original image based on the illumination correction information and the texture correction information to obtain a corrected target image;
[0010] Construct a three-dimensional model of the target object in the target image.
[0011] In one of the embodiments, the illumination feature information is an illumination feature matrix, and the illumination feature matrix is used to represent the brightness of each pixel in the original image;
[0012] The texture feature information is a texture feature matrix, and the texture feature matrix is used to represent pattern information in the original image.
[0013] In one embodiment, extracting illumination feature information and texture feature information from the original image includes:
[0014] The original image is input into the trained image element encoder to obtain the illumination feature information and texture feature information output by the image element encoder.
[0015] In one embodiment, the illumination correction information includes a first correction value of the three-channel value of each pixel of the original image; and the texture correction information includes a second correction value of the three-channel value of each pixel of the original image.
[0016] In one embodiment, generating illumination correction information based on illumination feature information and generating texture correction information based on texture feature information includes:
[0017] Inputting the illumination feature information into the trained illumination feature decoder to obtain illumination correction information output by the illumination feature decoder;
[0018] The texture feature information is input into a trained texture feature decoder to obtain texture correction information output by the texture feature decoder.
[0019] In one embodiment, the original image is corrected based on the illumination correction information and the texture correction information, including:
[0020] Establishing a first correction matrix according to the illumination correction information, and establishing a second correction matrix according to the texture correction information;
[0021] Get the original data matrix composed of three-channel values of each pixel of the original image;
[0022] The first correction matrix, the second correction matrix and the original data matrix are added bit by bit.
[0023] In one embodiment, constructing a three-dimensional model of a target object in a target image includes:
[0024] The trained 3D structural information encoder is used to extract the 3D structural features of the target object in the target image;
[0025] The trained 3D structural feature decoder is used to process the 3D structural features to obtain the 3D coordinates of multiple vertices of the target object;
[0026] A three-dimensional model of the target object is reconstructed according to the three-dimensional coordinates of a plurality of vertices of the target object.
[0027] A three-dimensional model building device, the device comprising:
[0028] An acquisition module is used to acquire the original image and extract illumination feature information and texture feature information from the original image;
[0029] A correction information generation module, used to generate illumination correction information based on illumination feature information, and to generate texture correction information based on texture feature information;
[0030] A correction module, used for correcting the original image based on the illumination correction information and the texture correction information to obtain a corrected target image;
[0031] The model building module is used to build a three-dimensional model of the target object in the target image.
[0032] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:
[0033] Obtaining an original image, and extracting illumination feature information and texture feature information from the original image;
[0034] Generate illumination correction information based on illumination feature information, and generate texture correction information based on texture feature information;
[0035] Correcting the original image based on the illumination correction information and the texture correction information to obtain a corrected target image;
[0036] Construct a three-dimensional model of the target object in the target image.
[0037] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:
[0038] Obtaining an original image, and extracting illumination feature information and texture feature information from the original image;
[0039] Generate illumination correction information based on illumination feature information, and generate texture correction information based on texture feature information;
[0040] Correcting the original image based on the illumination correction information and the texture correction information to obtain a corrected target image;
[0041] Construct a three-dimensional model of the target object in the target image.
[0042] The above-mentioned three-dimensional model construction method, device, computer equipment and storage medium obtain an original image, extract illumination feature information and texture feature information from the original image, then generate illumination correction information based on the illumination feature information, generate texture correction information based on the texture feature information, and perform correction processing on the original image based on the illumination correction information and the texture correction information to obtain a corrected target image. Then, a three-dimensional model of the target object is constructed based on the corrected target image. The embodiment of the present application corrects the original image through illumination correction information and texture correction information to remove the influence of illumination and texture in the original image on the target object, thereby improving the accuracy of the three-dimensional model. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] Figure 1 A schematic diagram of a process of constructing a three-dimensional model in one embodiment;
[0044] Figure 2 is a schematic diagram of the structure of a multi-layer perceptron MLP in one embodiment;
[0045] Figure 3 is a schematic flow chart of the steps of correcting an original image in one embodiment;
[0046] Figure 4 A schematic flow chart of the steps of constructing a three-dimensional model of a target object in one embodiment;
[0047] Figure 5 A structural block diagram of a three-dimensional model building device in one embodiment;
[0048] Figure 6 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION
[0049] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0050] In recent years, the construction of 3D models based on monocular 2D images has become a basic computer vision task with strong practical needs. In this task, it is necessary to restore the 3D model of the target object in the 2D image based on a monocular 2D image.
[0051] The prior art generally involves first directly extracting a feature vector that can characterize the three-dimensional structural information of a target object in a given monocular two-dimensional image through an encoder, and then using a decoder to reconstruct a three-dimensional model of the target object.
[0052] However, for the same target object, when obtaining a two-dimensional image of the target object, due to the influence of lighting, texture, etc., completely different two-dimensional images can be shown even at the same angle. Different two-dimensional images will result in different three-dimensional models of the target object obtained based on the two-dimensional images. This will lead to different three-dimensional models constructed from different two-dimensional images for the same target object. The reason for this is that the feature vector of the three-dimensional structural information of the target object directly extracted from a given monocular two-dimensional image is easily affected by the lighting and texture in the two-dimensional image. Lighting and texture have no effect on the actual structure of the target object, but in the process of building the three-dimensional model, they will interfere with the three-dimensional model, resulting in differences between the three-dimensional model of the target object and the actual structure of the target object, reducing the accuracy of the three-dimensional model.
[0053] The embodiment of the present application obtains an original image, extracts illumination feature information and texture feature information from the original image, then generates illumination correction information based on the illumination feature information, generates texture correction information based on the texture feature information, and corrects the original image based on the illumination correction information and the texture correction information to obtain a corrected target image. A three-dimensional model of the target object is then constructed based on the corrected target image. The embodiment of the present application corrects the original image using the illumination correction information and the texture correction information to remove the influence of illumination and texture in the original image on the target object, thereby improving the accuracy of the three-dimensional model.
[0054] In one embodiment, Figure 1 As shown, a three-dimensional model construction method is provided. This embodiment uses the method applied to a computer device as an example. The computer device may be a server, a computer, etc. In this embodiment, the method includes the following steps:
[0055] Step 101: A computer device obtains an original image, and extracts illumination feature information and texture feature information from the original image.
[0056] In the embodiment of the present application, the original image contains a target object, and the target object is an object for which a three-dimensional model is to be constructed.
[0057] In the embodiment of the present application, the texture feature information can be represented by a texture feature matrix, and the texture feature matrix is used to represent the pattern information in the original image. The texture feature refers to the feature of the pattern or text with or without color on the target object.
[0058] Optionally, the texture feature matrix can be a W×H×3 matrix, where W represents the width of the two-dimensional image, H represents the height of the two-dimensional image, and 3 represents the three-channel RGB value of each pixel in the two-dimensional image. Optionally, in an embodiment of the present application, the three-channel RGB values in the texture feature matrix include the three-channel RGB values of the pixels corresponding to the pattern and text areas on the target object.
[0059] In the embodiment of the present application, the illumination feature information can be represented by an illumination feature matrix, which is used to characterize the brightness of each pixel in the original image. The illumination feature refers to the feature of non-uniform illumination such as highlights and shadows on the target object.
[0060] Optionally, in an embodiment of the present application, the illumination feature matrix can be a W×H×3 matrix, where W represents the width of the two-dimensional image, H represents the height of the two-dimensional image, and 3 represents the three-channel RGB value of each pixel in the two-dimensional image. Optionally, in an embodiment of the present application, the three-channel RGB values in the illumination feature matrix include the three-channel RGB values of the pixels corresponding to the area where the non-uniform illumination is located.
[0061] Optionally, in the embodiment itself, the process of extracting illumination feature information and texture feature information from the original image by the computer device may include the following:
[0062] The computer device inputs the original image into the trained image element encoder to obtain the illumination feature information and texture feature information output by the image element encoder.
[0063] In the embodiment of the present application, the computer device can obtain a pre-trained image element encoder. Optionally, the network structure of the image element encoder can be a backbone network of ResNet50 (residual neural network). The image element encoder can extract features of the target object from the original image to obtain illumination feature information and texture feature information about the target object.
[0064] Step 102: The computer device generates illumination correction information based on illumination feature information and generates texture correction information based on texture feature information.
[0065] Optionally, the illumination correction information includes a first correction value of a three-channel value of each pixel of the original image, and the texture correction information includes a second correction value of a three-channel value of each pixel of the original image.
[0066] The illumination correction information is used to process the non-uniform illumination area of the target object in the original image into uniform illumination. The texture correction information is used to remove the colored or non-colored patterns or texts on the target object.
[0067] In the embodiment of the present application, the process of the computer device generating the illumination correction information based on the illumination feature information may include the following contents:
[0068] The computer device inputs the illumination feature information into a trained illumination feature decoder to obtain illumination correction information output by the illumination feature decoder.
[0069] Optionally, the illumination feature decoder may adopt MLP (English: Multi-Layer Perceptron, Chinese: Multi-layer Perceptron), wherein the structure of MLP may be as follows: Figure 2 The computer device can input the illumination feature matrix into the decoder to obtain a decoded matrix output by the decoder.
[0070] Continuing with the above example, the illumination feature matrix is a W×H×3 matrix. After being decoded by the decoder, the decoded matrix can be an illumination correction matrix of the same size of W×H×3, which is used to represent illumination correction information.
[0071] In the embodiment of the present application, the process of generating texture correction information based on texture feature information by a computer device may include the following:
[0072] The computer device inputs the texture feature information into a trained texture feature decoder to obtain texture correction information output by the texture feature decoder.
[0073] Optionally, the texture feature decoder may adopt MLP (English: Multi-Layer Perceptron, Chinese: Multi-layer Perceptron), wherein the structure of MLP may be as follows: Figure 2 The computer device can input the texture feature matrix into the decoder to obtain a decoded matrix output by the decoder.
[0074] Continuing with the above example, the texture feature matrix is a W×H×3 matrix. After being decoded by the decoder, the decoded matrix can be a texture correction matrix of the same size of W×H×3, and the texture correction matrix is used to represent texture correction information.
[0075] Step 103: The computer device corrects the original image based on the illumination correction information and the texture correction information to obtain a corrected target image.
[0076] In an embodiment of the present application, the computer device corrects the original image by: uniformizing the non-uniform illumination on the target object in the original image, and removing colored or non-colored patterns and text on the target object, to obtain a target image with removed illumination and texture.
[0077] Optionally, in an embodiment of the present application, the process of correcting the original image by the computer device based on the lighting correction information and the texture correction information may be: fusing the lighting correction information and the texture correction information into the original image to achieve correction processing of the original image.
[0078] Step 104: The computer device constructs a three-dimensional model of the target object in the target image.
[0079] In an embodiment of the present application, a computer device can directly extract a feature vector that can characterize the three-dimensional structural information of a target object in a two-dimensional image from a target image with illumination and texture removed, and then use a decoder to reconstruct a three-dimensional model of the target object.
[0080] The three-dimensional model construction method provided in the embodiment of the present application obtains the original image, extracts the illumination feature information and the texture feature information from the original image, then generates illumination correction information based on the illumination feature information, generates texture correction information based on the texture feature information, and corrects the original image based on the illumination correction information and the texture correction information to obtain a corrected target image. Then, a three-dimensional model is constructed for the target object based on the corrected target image. The embodiment of the present application corrects the original image through the illumination correction information and the texture correction information to remove the influence of illumination and texture in the original image on the target object, thereby improving the accuracy of the three-dimensional model.
[0081] In one embodiment of the present application, Figure 3 As shown, the process of the computer device correcting the original image based on the illumination correction information and the texture correction information in step 103 may include the following contents:
[0082] Step 301: The computer device establishes a first correction matrix according to illumination correction information, and establishes a second correction matrix according to texture correction information.
[0083] Optionally, in an embodiment of the present application, the lighting correction information includes a first correction value of the three-channel numerical value of each pixel of the original image. Therefore, a first correction matrix can be established based on the lighting correction information. Continuing with the above example, the first correction matrix can be a W×H×3 matrix.
[0084] The texture correction information includes the second correction value of the three-channel value of each pixel of the original image. Therefore, a second correction matrix can be established according to the texture correction information. Continuing with the above example, the second correction matrix can be a W×H×3 matrix.
[0085] Step 302: The computer device obtains an original data matrix composed of three-channel values of each pixel of the original image.
[0086] In the embodiment of the present application, for example, if the size of the original image is W×H, then the original data matrix is also a W×H×3 matrix.
[0087] Step 303: The computer device adds the first correction matrix, the second correction matrix and the original data matrix bit by bit.
[0088] In the embodiment of the present application, the computer device can perform bitwise addition and combination of the first correction matrix, the second correction matrix and the original data matrix to obtain a target image of the same size of W×H×3 with illumination information and texture structure information removed.
[0089] In an embodiment of the present application, the original image is corrected by bitwise addition of the first correction matrix, the second correction matrix and the original data matrix, thereby obtaining a target image with illumination information and texture structure information removed. In this way, when a three-dimensional model of the target object is constructed based on the target image, the influence of illumination and texture in the original image on the target object is removed, thereby improving the accuracy of the three-dimensional model.
[0090] In one embodiment of the present application, Figure 4 As shown, the process of constructing a three-dimensional model of a target object in a target image may include the following:
[0091] Step 401: A computer device extracts three-dimensional structural features of a target object in a target image using a trained three-dimensional structural information encoder.
[0092] The computer device can obtain a three-dimensional structure information encoder, wherein the three-dimensional structure information encoder can adopt a backbone network structure of ResNet50.
[0093] The computer device can input the target image into the three-dimensional structure information encoder to obtain the three-dimensional structure features of the target object to be reconstructed output by the three-dimensional structure information encoder.
[0094] Step 402: The computer device uses the trained three-dimensional structure feature decoder to perform feature processing on the three-dimensional structure feature to obtain the three-dimensional coordinates of multiple vertices of the target object.
[0095] The computer device can obtain a trained three-dimensional structure feature decoder, wherein the three-dimensional structure feature decoder can adopt an MLP network structure.
[0096] The computer device can input the three-dimensional structural features of the target object into a three-dimensional structural feature decoder to obtain the three-dimensional coordinates of multiple vertices of the target object output by the three-dimensional structural feature decoder.
[0097] Step 403: The computer device reconstructs a three-dimensional model of the target object according to the three-dimensional coordinates of the multiple vertices of the target object.
[0098] In an embodiment of the present application, a computer device may input the three-dimensional coordinates of a target image and multiple vertices of a target object into a mesh model to obtain a three-dimensional model of the target object output by the mesh model.
[0099] In an embodiment of the present application, the three-dimensional structural information of the target object directly extracted from the target image will not be affected by lighting and texture. Therefore, the three-dimensional structure features of the target object extracted from the target image can more accurately characterize the three-dimensional structure of the target object, thereby improving the accuracy of the three-dimensional coordinates of the target object.
[0100] It should be understood that although Figure 1-Figure 4 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1-Figure 4 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.
[0101] In one embodiment of the present application, Figure 5 As shown, a three-dimensional model construction device is provided, including: an acquisition module 501, a correction information generation module 502, a correction module 503 and a model construction module 504, wherein:
[0102] An acquisition module 501 is used to acquire an original image and extract illumination feature information and texture feature information from the original image;
[0103] A correction information generating module 502, used to generate illumination correction information based on illumination feature information, and to generate texture correction information based on texture feature information;
[0104] The correction module 503 is used to correct the original image based on the illumination correction information and the texture correction information to obtain a corrected target image;
[0105] The model building module 504 is used to build a three-dimensional model of the target object in the target image.
[0106] In one embodiment of the present application, the illumination feature information is an illumination feature matrix, which is used to characterize the brightness of each pixel in the original image; the texture feature information is a texture feature matrix, which is used to characterize the pattern information in the original image.
[0107] In one embodiment of the present application, the acquisition module 501 is further used to input the original image into a trained image element encoder to obtain the illumination feature information and texture feature information output by the image element encoder.
[0108] In one embodiment of the present application, the illumination correction information includes a first correction value of a three-channel value of each pixel of the original image; and the texture correction information includes a second correction value of a three-channel value of each pixel of the original image.
[0109] In one embodiment of the present application, the correction information generation module 502 is also used to input the lighting feature information into a trained lighting feature decoder to obtain the lighting correction information output by the lighting feature decoder; and input the texture feature information into a trained texture feature decoder to obtain the texture correction information output by the texture feature decoder.
[0110] In one embodiment of the present application, the correction module 503 is further used to establish a first correction matrix according to the illumination correction information and to establish a second correction matrix according to the texture correction information;
[0111] Get the original data matrix composed of three-channel values of each pixel of the original image;
[0112] The first correction matrix, the second correction matrix and the original data matrix are added bit by bit.
[0113] In one embodiment of the present application, the model building module 504 is further used to extract the three-dimensional structural features of the target object in the target image using the trained three-dimensional structural information encoder;
[0114] The trained 3D structural feature decoder is used to process the 3D structural features to obtain the 3D coordinates of multiple vertices of the target object;
[0115] Reconstruct the 3D model of the target object based on the 3D coordinates of multiple vertices of the target object
[0116] For the specific definition of the three-dimensional model construction device, please refer to the definition of the three-dimensional model construction method above, which will not be repeated here. Each module in the above-mentioned three-dimensional model construction device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
[0117] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 6As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store pre-trained image element encoders, illumination feature decoders, texture feature decoders, three-dimensional structure information encoders and three-dimensional structure feature decoders. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a three-dimensional model construction method is implemented.
[0118] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0119] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:
[0120] Obtaining an original image, and extracting illumination feature information and texture feature information from the original image;
[0121] Generate illumination correction information based on illumination feature information, and generate texture correction information based on texture feature information;
[0122] Correcting the original image based on the illumination correction information and the texture correction information to obtain a corrected target image;
[0123] Construct a three-dimensional model of the target object in the target image.
[0124] In one embodiment, when the processor executes the computer program, the following steps are also implemented: the illumination feature information is an illumination feature matrix, which is used to characterize the brightness of each pixel in the original image; the texture feature information is a texture feature matrix, which is used to characterize the pattern information in the original image.
[0125] In one embodiment, when the processor executes the computer program, the following steps are further implemented: inputting the original image into a trained image element encoder to obtain illumination feature information and texture feature information output by the image element encoder.
[0126] In one embodiment, when the processor executes the computer program, the following steps are also implemented: the illumination correction information includes a first correction value of the three-channel values of each pixel of the original image; the texture correction information includes a second correction value of the three-channel values of each pixel of the original image.
[0127] In one embodiment, when the processor executes the computer program, the following steps are also implemented: inputting the illumination feature information into a trained illumination feature decoder to obtain illumination correction information output by the illumination feature decoder; inputting the texture feature information into a trained texture feature decoder to obtain texture correction information output by the texture feature decoder.
[0128] In one embodiment, when the processor executes the computer program, the following steps are also implemented: establishing a first correction matrix based on the illumination correction information, and establishing a second correction matrix based on the texture correction information; obtaining an original data matrix composed of three-channel values of each pixel of the original image; and adding the first correction matrix, the second correction matrix and the original data matrix bit by bit.
[0129] In one embodiment, when the processor executes the computer program, the following steps are also implemented: using a trained three-dimensional structure information encoder to extract the three-dimensional structure features of the target object in the target image; using a trained three-dimensional structure feature decoder to perform feature processing on the three-dimensional structure features to obtain the three-dimensional coordinates of multiple vertices of the target object; and reconstructing the three-dimensional model of the target object based on the three-dimensional coordinates of the multiple vertices of the target object.
[0130] The computer device provided in the embodiment of the present application has similar implementation principles and technical effects to those of the above-mentioned method embodiment, and will not be described in detail here.
[0131] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:
[0132] Obtaining an original image, and extracting illumination feature information and texture feature information from the original image;
[0133] Generate illumination correction information based on illumination feature information, and generate texture correction information based on texture feature information;
[0134] Correcting the original image based on the illumination correction information and the texture correction information to obtain a corrected target image;
[0135] Construct a three-dimensional model of the target object in the target image.
[0136] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: the illumination feature information is an illumination feature matrix, which is used to characterize the brightness of each pixel in the original image; the texture feature information is a texture feature matrix, which is used to characterize the pattern information in the original image.
[0137] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: inputting the original image into a trained image element encoder to obtain illumination feature information and texture feature information output by the image element encoder.
[0138] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: the illumination correction information includes a first correction value of the three-channel value of each pixel of the original image; the texture correction information includes a second correction value of the three-channel value of each pixel of the original image.
[0139] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: inputting the illumination feature information into a trained illumination feature decoder to obtain illumination correction information output by the illumination feature decoder; inputting the texture feature information into a trained texture feature decoder to obtain texture correction information output by the texture feature decoder.
[0140] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: establishing a first correction matrix based on the illumination correction information, and establishing a second correction matrix based on the texture correction information; obtaining an original data matrix composed of three-channel values of each pixel of the original image; and adding the first correction matrix, the second correction matrix and the original data matrix bit by bit.
[0141] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented: using a trained three-dimensional structure information encoder to extract the three-dimensional structure features of the target object in the target image; using a trained three-dimensional structure feature decoder to perform feature processing on the three-dimensional structure features to obtain the three-dimensional coordinates of multiple vertices of the target object; and reconstructing the three-dimensional model of the target object based on the three-dimensional coordinates of the multiple vertices of the target object.
[0142] The computer-readable storage medium provided in the embodiment of the present application has similar implementation principles and technical effects to those of the above-mentioned method embodiment, and will not be described in detail here.
[0143] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).
[0144] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0145] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.
Claims
1. A three-dimensional model construction method, characterized in that: The method comprises: Acquire an original image, and extract illumination feature information and texture feature information from the original image; generating illumination correction information based on the illumination feature information, and generating texture correction information based on the texture feature information; Performing correction processing on the original image based on the illumination correction information and the texture correction information to obtain a corrected target image; Constructing a three-dimensional model of the target object in the target image; The step of generating illumination correction information based on the illumination feature information and generating texture correction information based on the texture feature information includes: Inputting the illumination feature information into a trained illumination feature decoder to obtain the illumination correction information output by the illumination feature decoder; Inputting the texture feature information into a trained texture feature decoder to obtain the texture correction information output by the texture feature decoder; The correcting the original image based on the illumination correction information and the texture correction information includes: Establishing a first correction matrix according to the illumination correction information, and establishing a second correction matrix according to the texture correction information; Obtaining an original data matrix consisting of three-channel values of each pixel of the original image; The first correction matrix, the second correction matrix and the original data matrix are added bit by bit.
2. The method according to claim 1, characterized in that: The illumination feature information is an illumination feature matrix, and the illumination feature matrix is used to represent the brightness of each pixel in the original image; The texture feature information is a texture feature matrix, and the texture feature matrix is used to represent pattern information in the original image.
3. The method according to claim 2, characterized in that The step of extracting illumination feature information and texture feature information from the original image includes: The original image is input into a trained image element encoder to obtain the illumination feature information and the texture feature information output by the image element encoder.
4. The method according to claim 2, characterized in that: The illumination correction information includes a first correction value of three-channel values of each pixel of the original image; the texture correction information includes a second correction value of three-channel values of each pixel of the original image.
5. The method according to claim 1, characterized in that The constructing of a three-dimensional model of the target object in the target image includes: Extracting the three-dimensional structural features of the target object in the target image using the trained three-dimensional structural information encoder; Using a trained three-dimensional structure feature decoder to perform feature processing on the three-dimensional structure feature to obtain three-dimensional coordinates of multiple vertices of the target object; A three-dimensional model of the target object is reconstructed according to the three-dimensional coordinates of the plurality of vertices of the target object.
6. A three-dimensional model construction device, characterized in that: The device comprises: An acquisition module, used to acquire an original image and extract illumination feature information and texture feature information from the original image; A correction information generating module, used to generate illumination correction information based on the illumination feature information, and to generate texture correction information based on the texture feature information; A correction module, used for performing correction processing on the original image based on the illumination correction information and the texture correction information to obtain a corrected target image; A model building module, used for building a three-dimensional model of the target object in the target image; The correction information generation module is further used to input the illumination feature information into a trained illumination feature decoder to obtain illumination correction information output by the illumination feature decoder; input the texture feature information into a trained texture feature decoder to obtain texture correction information output by the texture feature decoder; The correction module is also used to establish a first correction matrix according to the illumination correction information and a second correction matrix according to the texture correction information; obtain an original data matrix composed of three-channel values of each pixel of the original image; and add the first correction matrix, the second correction matrix and the original data matrix bit by bit.
7. The device according to claim 6, characterized in that The illumination feature information is an illumination feature matrix, and the illumination feature matrix is used to represent the brightness of each pixel in the original image; the texture feature information is a texture feature matrix, and the texture feature matrix is used to represent the pattern information in the original image.
8. The device according to claim 7, characterized in that The acquisition module is also used to input the original image into the trained image element encoder to obtain the illumination feature information and texture feature information output by the image element encoder.
9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.