Three-dimensional reconstruction method and related device

Abstract

The embodiment of the invention discloses a three-dimensional reconstruction method and a related device. When a three-dimensional reconstruction model used for three-dimensional reconstruction is trained, model parameters are adjusted through feature differences between point cloud features extracted by the model based on an image and point cloud features actually corresponding to an object to be reconstructed; therefore, on the basis of the characteristic that the overall contour of the object to be reconstructed in the three-dimensional space can be effectively represented based on the point cloud features, the three-dimensional reconstruction model can fully learn how to accurately analyze the overall contour structure of the object based on the image; therefore, the three-dimensional reconstruction model can be used for constructing a three-dimensional model which is relatively consistent with the whole to-be-reconstructed object in a three-dimensional space, the reconstruction precision of three-dimensional reconstruction is improved on the premise of relatively simple model input, and relatively accurate three-dimensional reconstruction can be carried out on each angle of the to-be-reconstructed object.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to three-dimensional reconstruction methods and related apparatus. Background Technology

[0002] 3D reconstruction is one of the most popular research directions in the field of 3D technology. Among them, image-based 3D reconstruction technology is the main technical implementation method. By inputting an image including the object, a 3D model corresponding to the object can be reconstructed.

[0003] In related technologies, 3D reconstruction can be performed using a reconstruction model. The input of the 3D reconstruction model is an image, and the output is a 3D model corresponding to the object in the image. In these technologies, the loss function used to train the reconstruction model is used to characterize the accuracy of the 3D model output by the reconstruction model in the 2D plane, ensuring that the 3D model output by the model in the 2D plane is closer to the actual 3D model in the 2D plane.

[0004] However, since image differences cannot represent the differences in the spatial structure of a 3D model, but only the differences at the visual level under some observation angles, the 3D model output by the reconstruction model obtained through this training method can only accurately construct the model part that is visible in the input image. The construction accuracy for the invisible parts of the image is low, resulting in poor 3D reconstruction effect. Summary of the Invention

[0005] To address the aforementioned technical issues, this application provides a three-dimensional reconstruction method that can enhance the spatial information extracted from images during model training, thereby improving the accuracy of three-dimensional reconstruction.

[0006] The embodiments of this application disclose the following technical solutions:

[0007] In a first aspect, embodiments of this application disclose a three-dimensional reconstruction method, the method comprising:

[0008] Acquire first point cloud data and sample image corresponding to the first object. The first point cloud data is used to identify the positions of multiple first surface points located on the surface of the first object in three-dimensional space, and the sample image is used to display the first object.

[0009] First point cloud features are extracted based on the first point cloud data. The first point cloud features are used to characterize the positional distribution of the plurality of first surface points in three-dimensional space. The second model part is used to generate first model construction data based on the first point cloud features. The first model construction data is used to construct the first three-dimensional model corresponding to the first object.

[0010] The first undetermined point cloud features are extracted from the sample image using the initial first model part;

[0011] Based on the feature differences between the first point cloud features to be determined and the first point cloud features, the model parameters corresponding to the initial first model part are adjusted to obtain the first model part. The first model part and the second model part are used to constitute a three-dimensional reconstruction model. The three-dimensional reconstruction model is used to generate target model construction data based on the image to be reconstructed. The image to be reconstructed is used to display the object to be reconstructed. The target model construction data is used to construct the three-dimensional model corresponding to the object to be reconstructed. The first model part is used to extract target point cloud features based on the image to be reconstructed. The target point cloud features are used to characterize the positional distribution of multiple surface points located on the surface of the object to be reconstructed in three-dimensional space. The second model part is used to generate the target model construction data based on the target point cloud features.

[0012] Secondly, embodiments of this application disclose a three-dimensional reconstruction method, the method comprising:

[0013] Acquire an image to be reconstructed, which is used to display the object to be reconstructed;

[0014] Using a 3D reconstruction model, target model construction data corresponding to the object to be reconstructed is generated based on the image to be reconstructed. This target model construction data is used to construct a 3D model corresponding to the object to be reconstructed. The 3D reconstruction model includes a first model part and a second model part. The first model part is used to extract target point cloud features from the image to be reconstructed. These target point cloud features characterize the positional distribution of multiple surface points on the surface of the object to be reconstructed in 3D space. The second model part is used to generate the target model construction data based on the target point cloud features. The first model part is trained in the following way:

[0015] Acquire first point cloud data and sample image corresponding to the first object. The first point cloud data is used to identify the positions of multiple first surface points located on the surface of the first object in three-dimensional space, and the sample image is used to display the first object.

[0016] First point cloud features are extracted based on the first point cloud data. The first point cloud features are used to characterize the positional distribution of the plurality of first surface points in three-dimensional space. The second model part is used to generate first model construction data based on the first point cloud features. The first model construction data is used to construct the first three-dimensional model corresponding to the first object.

[0017] The first undetermined point cloud features are extracted from the sample image using the initial first model part;

[0018] Based on the feature differences between the first undetermined point cloud features and the first point cloud features, the model parameters corresponding to the initial first model part are adjusted to obtain the first model part.

[0019] Thirdly, embodiments of this application disclose a three-dimensional reconstruction device, the device comprising a first acquisition unit, a first extraction unit, a second extraction unit, and a first adjustment unit:

[0020] The first acquisition unit is used to acquire first point cloud data and sample image corresponding to the first object. The first point cloud data is used to identify the positions of multiple first surface points located on the surface of the first object in three-dimensional space, and the sample image is used to display the first object.

[0021] The first extraction unit is used to extract first point cloud features based on the first point cloud data. The first point cloud features are used to characterize the positional distribution of the plurality of first surface points in three-dimensional space. The second model part is used to generate first model construction data based on the first point cloud features. The first model construction data is used to construct a first three-dimensional model corresponding to the first object.

[0022] The second extraction unit is used to extract first undetermined point cloud features from the sample image based on the initial first model part;

[0023] The first adjustment unit is used to adjust the model parameters corresponding to the initial first model part according to the feature difference between the first point cloud feature to be determined and the first point cloud feature, to obtain the first model part. The first model part and the second model part are used to constitute a three-dimensional reconstruction model. The three-dimensional reconstruction model is used to generate target model construction data according to the image to be reconstructed. The image to be reconstructed is used to display the object to be reconstructed. The target model construction data is used to construct the three-dimensional model corresponding to the object to be reconstructed. The first model part is used to extract target point cloud features according to the image to be reconstructed. The target point cloud features are used to characterize the positional distribution of multiple surface points located on the surface of the object to be reconstructed in three-dimensional space. The second model part is used to generate the target model construction data according to the target point cloud features.

[0024] In one possible implementation, the device further includes a second acquisition unit, a first generation unit, and a second adjustment unit:

[0025] The second acquisition unit is used to acquire the second point cloud data and the second model construction data corresponding to the second object. The second model construction data is used to construct the second three-dimensional model corresponding to the second object. The second point cloud data is used to identify the positions of multiple second surface points located on the surface of the second object in three-dimensional space.

[0026] The first generation unit is used to generate pending model construction data based on the second point cloud data through an initial generation model. The initial generation model includes an initial third model part and an initial second model part. The initial third model part is used to extract second pending point cloud features based on the second point cloud data, and the initial second model part is used to generate the pending model construction data based on the second pending point cloud features.

[0027] The second adjustment unit is used to adjust the model parameters corresponding to the initial generated model based on the data difference between the data to be constructed and the data to be constructed in the second model, so as to obtain a generated model. The generated model consists of a third model part and a second model part. The first extraction unit is specifically used for:

[0028] The third model part extracts the first point cloud features based on the first point cloud data.

[0029] In one possible implementation, the apparatus further includes a third acquisition unit, a construction unit, and a determination unit:

[0030] The third acquisition unit is used to acquire a target display image corresponding to the second three-dimensional model, the target display image corresponds to a target display angle, and the target display image is used to display the second three-dimensional model from the target display angle;

[0031] The construction unit is used to construct the undetermined three-dimensional model based on the undetermined model construction data.

[0032] The determining unit is used to determine a pending display image corresponding to the target display angle based on the target display angle and the pending 3D model. The pending display image is used to display the pending 3D model from the target display angle.

[0033] The second adjustment unit is specifically used for:

[0034] Based on the data differences between the pending model construction data and the second model construction data, and the image differences between the pending display image and the target display image, the model parameters corresponding to the initial generated model are adjusted to obtain the generated model.

[0035] In one possible implementation, the third acquisition unit is specifically used for:

[0036] Acquire multiple display images corresponding to the second 3D model. The multiple display images correspond to multiple display angles, and different display images correspond to different display angles. The target display image is any one of the multiple display images.

[0037] The second adjustment unit is specifically used for:

[0038] Based on the image difference between the pending display image corresponding to the target display angle and the target display image, determine the image difference corresponding to the target display angle;

[0039] Based on the data differences between the pending model construction data and the second model construction data, and the image differences corresponding to the multiple display angles, the model parameters corresponding to the initial generated model are adjusted to obtain the generated model.

[0040] In one possible implementation, the first model construction data includes relative position information corresponding to multiple spatial points in three-dimensional space. The relative position information corresponding to the target spatial point is used to identify the relative positional relationship between the target spatial point and the model surface of the first three-dimensional model. The target spatial point is any one of the multiple points. Generating the first model construction data based on the first point cloud features includes:

[0041] Based on the first point cloud features, the relative position features corresponding to the three dimensions of the three-dimensional space are determined respectively. The relative position features corresponding to the target dimension are used to identify the relative positional relationship between the spatial points in the three-dimensional space and the model surface of the first three-dimensional model under the target dimension. The target dimension is any one of the three dimensions.

[0042] Based on the relative position features corresponding to the three dimensions, the relative position information corresponding to the multiple spatial points is determined.

[0043] In one possible implementation, determining the relative position information corresponding to the plurality of spatial points based on the relative position features corresponding to the three dimensions includes:

[0044] Determine the position information of the target spatial point corresponding to the target dimension;

[0045] Based on the relative position features corresponding to the target dimension and the position information, the sub-position features corresponding to the target spatial point in the target dimension are determined. The sub-position features are used to characterize the relative positional relationship between the target spatial point and the model surface of the first three-dimensional model in the target dimension.

[0046] Based on the sub-positional features corresponding to the target spatial point in the three dimensions, the relative position information corresponding to the target spatial point is determined.

[0047] In one possible implementation, the plurality of first surface points have corresponding texture information, and the device further includes a third extraction unit:

[0048] The third extraction unit is used to extract sample texture distribution features based on the first point cloud data and the texture information corresponding to the plurality of first surface points respectively. The sample texture distribution features are used to characterize the distribution of the texture of the first object surface in three-dimensional space. The second model part is used to generate the first model construction data based on the first point cloud features and the sample texture distribution features.

[0049] The second extraction unit is specifically used for:

[0050] Through the initial first model part, first undetermined point cloud features and undetermined texture distribution features are extracted from the sample image;

[0051] The first adjustment unit is specifically used for:

[0052] Based on the feature differences between the first undetermined point cloud features and the first point cloud features, and the feature differences between the undetermined texture distribution features and the sample texture distribution features, the model parameters corresponding to the initial first model part are adjusted to obtain the first model part.

[0053] In one possible implementation, the first model construction data includes relative position information and texture information corresponding to multiple spatial points in three-dimensional space. The relative position information corresponding to the target spatial point is used to identify the relative positional relationship between the target spatial point and the model surface of the first three-dimensional model. The texture information corresponding to the target spatial point is used to characterize the texture representation of the target spatial point in three-dimensional space. The target spatial point is any one of the multiple spatial points. The step of generating the first model construction data based on the first point cloud features and the sample texture distribution features includes:

[0054] Based on the first point cloud features, determine the relative position information corresponding to the plurality of spatial points.

[0055] Based on the sample texture distribution features, the texture distribution features corresponding to the three dimensions of the three-dimensional space are determined respectively. The texture distribution features corresponding to the target dimension are used to characterize the distribution of the texture of the first object surface under the target dimension. The target dimension is any one of the three dimensions.

[0056] Based on the texture distribution features corresponding to the three dimensions, the texture information corresponding to the multiple spatial points is determined.

[0057] In one possible implementation, determining the texture information corresponding to the plurality of spatial points based on the texture distribution features corresponding to the three dimensions includes:

[0058] Determine the position information of the target spatial point corresponding to the target dimension;

[0059] Based on the texture distribution features corresponding to the target dimension and the position information, the sub-texture information corresponding to the target spatial point in the target dimension is determined, and the sub-texture information is used to characterize the texture representation of the target spatial point in the target dimension.

[0060] Based on the sub-texture information corresponding to the target spatial point in each of the three dimensions, the texture information corresponding to the target spatial point is determined.

[0061] In one possible implementation, the model parameters corresponding to the initial first model portion include feature extraction parameters and feature transformation parameters, and the second extraction unit is specifically used for:

[0062] Initial features are extracted from the sample image using the feature extraction parameters.

[0063] The feature format of the initial feature is transformed using the feature transformation parameters to obtain the first undetermined point cloud feature, which has the same feature format as the first point cloud feature.

[0064] The first adjustment unit is specifically used for:

[0065] Based on the feature differences between the first undetermined point cloud features and the first point cloud features, the feature extraction parameters and the feature transformation parameters are adjusted to obtain the first model part.

[0066] Fourthly, embodiments of this application disclose a three-dimensional reconstruction apparatus, the apparatus comprising a fourth acquisition unit and a second generation unit:

[0067] The fourth acquisition unit is used to acquire the image to be reconstructed, and the image to be reconstructed is used to display the object to be reconstructed;

[0068] The second generation unit is used to generate target model construction data corresponding to the object to be reconstructed based on the image to be reconstructed using a 3D reconstruction model. The target model construction data is used to construct a 3D model corresponding to the object to be reconstructed. The 3D reconstruction model includes a first model part and a second model part. The first model part is used to extract target point cloud features based on the image to be reconstructed. The target point cloud features are used to characterize the positional distribution of multiple surface points located on the surface of the object to be reconstructed in 3D space. The second model part is used to generate the target model construction data based on the target point cloud features. The first model part is trained in the following way:

[0069] Acquire first point cloud data and sample image corresponding to the first object. The first point cloud data is used to identify the positions of multiple first surface points located on the surface of the first object in three-dimensional space, and the sample image is used to display the first object.

[0070] First point cloud features are extracted based on the first point cloud data. The first point cloud features are used to characterize the positional distribution of the plurality of first surface points in three-dimensional space. The second model part is used to generate first model construction data based on the first point cloud features. The first model construction data is used to construct the first three-dimensional model corresponding to the first object.

[0071] The first undetermined point cloud features are extracted from the sample image using the initial first model part;

[0072] Based on the feature differences between the first undetermined point cloud features and the first point cloud features, the model parameters corresponding to the initial first model part are adjusted to obtain the first model part.

[0073] In one possible implementation, the first model part is used to extract target point cloud features and target texture distribution features based on the image to be reconstructed, wherein the target texture distribution features are used to characterize the distribution of the texture of the surface of the object to be reconstructed in three-dimensional space;

[0074] The second model part is used to generate the target model construction data based on the target point cloud features and the target texture distribution features;

[0075] The step of extracting first undetermined point cloud features from the sample image based on the initial first model portion includes:

[0076] Through the initial first model part, first undetermined point cloud features and undetermined texture distribution features are extracted from the sample image;

[0077] The step of adjusting the model parameters corresponding to the initial first model part based on the feature difference between the first undetermined point cloud feature and the first point cloud feature to obtain the first model part includes:

[0078] Based on the feature differences between the first undetermined point cloud feature and the first point cloud feature, and the feature differences between the undetermined texture distribution feature and the sample texture distribution feature, the model parameters corresponding to the initial first model part are adjusted to obtain the first model part. The sample texture distribution feature is used to characterize the distribution of the texture of the first object surface in three-dimensional space.

[0079] In one possible implementation, the fourth acquisition unit is specifically used for:

[0080] Acquire an initial image and first text information, wherein the initial image is used to display multiple objects, including the object to be reconstructed, and the first text information is used to describe the object to be reconstructed;

[0081] Based on the first text information, the image to be reconstructed is determined from the initial image, wherein the image to be reconstructed is the portion of the initial image used to display the image to be reconstructed.

[0082] In one possible implementation, the fourth acquisition unit is specifically used for:

[0083] Obtain second text information, which describes the object to be reconstructed;

[0084] The image to be reconstructed is generated based on the second text information.

[0085] Fifthly, embodiments of this application disclose a computer device, the computer device including a processor and a memory:

[0086] The memory is used to store computer programs and to transfer the computer programs to the processor;

[0087] The processor is configured to execute the three-dimensional reconstruction method according to any one of the first aspects, or to execute the three-dimensional reconstruction method according to any one of the second aspects, according to the instructions in the computer program.

[0088] In a sixth aspect, embodiments of this application disclose a computer-readable storage medium for storing a computer program, the computer program being used to execute the three-dimensional reconstruction method according to any one of the first aspects, or to execute the three-dimensional reconstruction method according to any one of the second aspects;

[0089] In a seventh aspect, embodiments of this application disclose a computer program product including a computer program, which, when run on a computer device, causes the computer device to execute the three-dimensional reconstruction method described in any one of the first aspects, or to execute the three-dimensional reconstruction method described in any one of the second aspects.

[0090] As can be seen from the above technical solution, when training the 3D reconstruction model for 3D reconstruction, this application can first acquire the first point cloud data and sample images corresponding to the first object. The first point cloud data is used to identify the positions of multiple first surface points located on the surface of the first object in 3D space, and the sample images are used to display the first object. Based on the first point cloud data, accurate first point cloud features can be extracted. The first point cloud features can be used to characterize the positional distribution of multiple first surface points in 3D space, thereby identifying the distribution of the surface of the first object in 3D space, that is, the overall structural characteristics of the object outline in 3D space. Therefore, the second model part can accurately generate first model construction data based on the first point cloud features, and the first model construction data is used to construct an accurate first 3D model corresponding to the first object. To achieve accurate 3D reconstruction based on images, the initial first model part can be used to analyze the overall structural features of the object contour represented by the sample image, extracting the first undetermined point cloud features. The difference between the first undetermined point cloud features and the first point cloud features can represent the accuracy of the analysis of the object contour structural features based on the image. Based on this difference, the parameters of the initial first model part can be adjusted, enabling the adjusted first model part to learn how to accurately analyze the overall contour features of the object contour based on the image, obtaining accurate point cloud features. Then, the second model part can generate accurate model construction data based on these point cloud features. Thus, the 3D reconstruction model composed of the first model part and the second model part can not only accurately reconstruct the object parts shown in the image, but also accurately reconstruct the object parts that are not visible in the image through accurate analysis of the overall contour of the object. Under the premise of relatively simple input information requirements, it improves the accuracy of 3D reconstruction, enabling 3D reconstruction technology to be applied to a wider range of application scenarios. Attached Figure Description

[0091] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0092] Figure 1 A schematic diagram illustrating a three-dimensional reconstruction method provided in an embodiment of this application;

[0093] Figure 2 A schematic diagram illustrating a three-dimensional reconstruction method in a practical application scenario provided by an embodiment of this application;

[0094] Figure 3A flowchart of a three-dimensional reconstruction method provided in this application embodiment;

[0095] Figure 4 A schematic diagram illustrating a three-dimensional reconstruction method provided in an embodiment of this application;

[0096] Figure 5 A schematic diagram illustrating a three-dimensional reconstruction method provided in an embodiment of this application;

[0097] Figure 6 A flowchart of a three-dimensional reconstruction method provided in this application embodiment;

[0098] Figure 7 A schematic diagram illustrating a three-dimensional reconstruction method provided in an embodiment of this application;

[0099] Figure 8 A schematic diagram illustrating a three-dimensional reconstruction method provided in an embodiment of this application;

[0100] Figure 9 A flowchart illustrating a 3D reconstruction method in a practical application scenario provided in this application embodiment;

[0101] Figure 10 A schematic diagram illustrating a three-dimensional reconstruction method in a practical application scenario provided by an embodiment of this application;

[0102] Figure 11 A structural block diagram of a three-dimensional reconstruction device provided in an embodiment of this application;

[0103] Figure 12 A structural block diagram of a three-dimensional reconstruction device provided in an embodiment of this application;

[0104] Figure 13 A structural diagram of a terminal provided in an embodiment of this application;

[0105] Figure 14 This is a structural diagram of a server provided in an embodiment of this application. Detailed Implementation

[0106] The embodiments of this application will now be described with reference to the accompanying drawings.

[0107] 3D reconstruction refers to the process of constructing a 3D model of an object to be reconstructed based on input information related to the object (such as images, scan data, etc.). The object to be reconstructed can be any 3D object, such as a character model in a 3D game or various objects in a real-world scene.

[0108] In related technologies, 3D reconstruction is achieved through a 3D reconstruction model. During model training, a sample image for displaying a first object can be obtained first, and the initial 3D reconstruction model can be constructed based on this image to build a 3D model corresponding to the first object. Then, a model image for displaying the 3D model is generated. The reconstruction accuracy of the model in 3D reconstruction is measured by the difference between the sample image and the model image, and the model training of the initial 3D reconstruction model can be completed based on this difference.

[0109] However, this training method can only enable the model to learn how to construct a 3D model that is relatively consistent with the object to be reconstructed in 2D image representation. On the one hand, 2D images are difficult to reflect the differences in 3D spatial structure; that is, the structural differences between the object and the 3D model in 3D space are reduced in 2D images. For example... Figure 1 As shown in the side view, model A has an added cuboid compared to object A, resulting in a significantly different structure in three-dimensional space. However, from the front view, only minor line differences exist, making it difficult to use these differences for model parameter adjustment during training. Furthermore, two-dimensional images can only effectively represent the structure of the visible parts. For the object parts not visible in two-dimensional images, the lack of analysis of the object's three-dimensional spatial features makes accurate 3D model construction difficult. Therefore, 3D reconstruction models trained based on image differences are unlikely to achieve accurate 3D reconstruction.

[0110] To address the aforementioned technical issues, this application provides a 3D reconstruction method. When training a 3D reconstruction model, the model parameters are adjusted based on the feature differences between point cloud features extracted from images and the actual point cloud features corresponding to the object to be reconstructed. This leverages the ability of point cloud features to effectively represent the overall contour of the object in 3D space, enabling the 3D reconstruction model to learn how to accurately analyze the overall contour structure of the object based on images. Consequently, the 3D reconstruction model can be used to construct a 3D model that closely matches the object in 3D space. This improves the reconstruction accuracy of 3D reconstruction while maintaining relatively simple model input, allowing for accurate 3D reconstruction from various angles of the object.

[0111] Understandably, this method can be applied to computer devices capable of performing data processing for 3D reconstruction, such as terminal devices or servers. The method can be executed independently by a terminal device or server, or it can be applied to network scenarios where the terminal device and server communicate, executing in cooperation. The terminal device can be a mobile phone, tablet, laptop, desktop computer, etc. The terminal device can also include various virtual reality devices, such as augmented reality (AR) devices like AR glasses and AR screens, and virtual reality (VR) devices like VR headsets. The server can be understood as an application server or a web server. In actual deployment, the server can be a standalone server, a cluster server, or a cloud server, etc.

[0112] This application can be applied to various 3D model construction scenarios. For example, it can be used to construct 3D models of game objects in a game based on game object design drawings, or to construct various animated characters in 3D animations based on animated character design drawings.

[0113] To facilitate understanding of the technical solutions provided in this application, the three-dimensional reconstruction method provided in the embodiments of this application will be introduced next in conjunction with a practical application scenario.

[0114] See Figure 2 , Figure 2 This is a schematic diagram of a three-dimensional reconstruction method in a practical application scenario provided by an embodiment of this application. In this practical application scenario, the computer device can be a server 101 with data processing function, and the first object used for training can be a cup.

[0115] like Figure 2As shown, server 101 first acquires the first point cloud data corresponding to the first object and a sample image for displaying the first object. The first point cloud data is used to identify multiple surface points located on the surface of the first object, and these multiple surface points constitute the point cloud corresponding to the first object. First point cloud features can be extracted from the first point cloud data. These first point cloud features characterize the positional distribution of multiple surface points in three-dimensional space. Since these surface points are located on the entire surface of the first object, this positional distribution can characterize the overall contour structure of the first object in three-dimensional space. Therefore, based on the first point cloud features, the overall contour structure of the first object can be accurately analyzed. Thus, through the second model part, first model construction data can be generated based on the first point cloud features to accurately construct the first three-dimensional model corresponding to the first object, making the first three-dimensional model closely resemble the first object in its overall structure, not limited to the expression on a two-dimensional image or the structure at a specific angle.

[0116] In order to generate accurate point cloud features based on images, server 101 can generate undetermined point cloud features based on sample images using the initial first model part. Since the first point cloud features are point cloud features extracted based on actual first point cloud data, the first point cloud features are accurate point cloud features corresponding to the first object. Thus, by the difference between the first point cloud features and the undetermined point cloud features, the accuracy of the initial first model part's analysis of the overall contour features of the first object based on the sample images can be characterized. Then, the model parameters corresponding to the initial first model part can be adjusted based on this difference to obtain the first model part, which can be used to extract point cloud features more accurately based on images.

[0117] Combining the first and second model parts yields a 3D reconstruction model for 3D reconstruction. When performing 3D reconstruction on an object to be reconstructed, the first model part extracts target point cloud features from the image to be reconstructed, accurately representing the overall contour structure of the object in 3D space. The second model part generates target model construction data based on these target point cloud features, accurately constructing the corresponding 3D model of the object. Since this application uses point cloud feature loss representing the overall contour structure in 3D space for model training, the 3D reconstruction model learns how to accurately analyze the overall contour structure of an object in 3D space based on images. This avoids the problems of narrow learning angle and poor learning accuracy caused by differences in 2D image features, resulting in a higher degree of matching between the reconstructed 3D model and the original object. Furthermore, it avoids complicating the model input and reduces the difficulty of 3D reconstruction.

[0118] The three-dimensional reconstruction method provided in this application will now be described in detail with reference to the accompanying drawings.

[0119] See Figure 3 , Figure 3 A flowchart of a three-dimensional reconstruction method provided in this application embodiment, wherein the computer device can be any of the above-mentioned computer devices with data processing functions, and the method includes:

[0120] S301: Obtain the first point cloud data and sample image corresponding to the first object.

[0121] The first point cloud data is used to identify the positions of multiple first surface points located on the surface of the first object in three-dimensional space. The sample image is used to display the first object. The multiple first surface points are points attached to the overall surface of the first object. The point cloud is a collection of multiple first surface points. The first object can be any three-dimensional object, such as a person, animal, or object in the display scene.

[0122] S302: Extract the features of the first point cloud based on the first point cloud data.

[0123] Computer equipment can analyze the positions of multiple first surface points in three-dimensional space, extracting first point cloud features. These features characterize the distribution of the multiple first surface points in three-dimensional space. Since these points cover the surface of the first object, their positions can represent the distribution of the object's surface in three-dimensional space, thus characterizing the overall contour structure of the first object. Therefore, point cloud features can effectively characterize the overall contour structure of an object in three-dimensional space.

[0124] The computer device can pre-train a second model. Since the first point cloud features can characterize the overall contour structure of the first object, the second model can be used to accurately analyze the contour structure of the first object based on the first point cloud features, thereby generating first model construction data. This first model construction data is used to construct the first 3D model corresponding to the first object. Because the first point cloud features can characterize the overall contour structure, the first 3D model determined in this way is relatively close to the first object in terms of overall contour structure, avoiding the problem of large differences between the unseen model parts and the first object, and has high model accuracy. The model construction data in this application can include various types of data, such as the Sign Distance Function (SDF), etc., which are not limited here.

[0125] S303: Extract the first undetermined point cloud features based on the sample image through the initial first model part.

[0126] As can be seen from the above, in image-based 3D reconstruction, the ability to generate accurate point cloud features from the image is crucial to the accuracy of the reconstructed 3D model. Based on this, a computer device can construct an initial first model, which is used to extract point cloud features from the image. The computer device can input sample images into the initial first model to extract first undetermined point cloud features. These first undetermined point cloud features characterize the positional distribution of multiple surface points on the surface of the first object, determined by the initial first model through analysis of the contour structure of the first object shown in the sample image.

[0127] S304: Based on the feature difference between the first undetermined point cloud feature and the first point cloud feature, adjust the model parameters corresponding to the initial first model part to obtain the first model part.

[0128] Since the first point cloud feature is determined based on the actual point cloud data corresponding to the first object, and the positions in the first point cloud data are the accurate positions of the first surface points in three-dimensional space, the positional distribution represented by the first point cloud feature is relatively accurate. Therefore, the more accurate the initial first model part is in analyzing the contour structure of the first object, the closer the extracted first undetermined point cloud feature should be to the first point cloud feature. Based on this, the computer device can use the feature difference between the first undetermined point cloud feature and the first point cloud feature as the loss for model training to adjust the model parameters corresponding to the initial first model part. This allows the first undetermined point cloud feature extracted by the initial first model part based on the sample image to gradually approach the first point cloud feature. Thus, the initial first model part can learn how to accurately analyze the overall contour structure characteristics of the first object based on the first object shown in the sample image, extract accurate point cloud features, and obtain the first model part. The first model part is the model part obtained by adjusting the parameters of the initial first model part.

[0129] The first and second model components are used to construct a 3D reconstruction model. Since the first model component can accurately determine point cloud features based on the image, and the second model component can accurately generate model construction data based on these point cloud features, this 3D reconstruction model can generate model construction data for building a corresponding 3D model of the object based on an image used to display the object. For example, in practical applications, this 3D reconstruction model can be used to generate target model construction data based on an image to be reconstructed. The image to be reconstructed can be any image including an object. The image to be reconstructed is used to display the object to be reconstructed, and the target model construction data is used to construct the 3D model corresponding to the object to be reconstructed. The first model component is used to extract target point cloud features from the image to be reconstructed. These target point cloud features characterize the positional distribution of multiple surface points on the surface of the object to be reconstructed in 3D space. The second model component is used to generate target model construction data based on these target point cloud features.

[0130] As can be seen from the above technical solution, when training the 3D reconstruction model for 3D reconstruction, this application adjusts parameters based on the difference between the point cloud features extracted by the model and the actual point cloud features corresponding to the object. The purpose is to enable the model to learn how to extract accurate point cloud features based on the image. Since point cloud features can accurately represent the overall contour structure of the object, the 3D model determined based on the point cloud features is closer to the object in terms of overall contour structure. It avoids the problem of low model accuracy in the invisible parts due to the perspective of the object displayed in the image. It can not only accurately reconstruct the object part displayed in the image, but also accurately reconstruct the object part that is not visible through the image by accurately analyzing the overall contour of the object. Under the premise of relatively simple input information requirements, it improves the accuracy of 3D reconstruction and enables 3D reconstruction technology to be applied to a wider range of application scenarios.

[0131] The technical solutions involved in this application will be described in detail below.

[0132] First, we introduce how to train the second model part, which has the function of generating data based on point cloud features.

[0133] In one possible implementation, the computer device can first acquire the second point cloud data and the second model construction data corresponding to the second object. The second model construction data is used to construct the second three-dimensional model corresponding to the second object, which is the accurate model construction data corresponding to the second three-dimensional model. The second point cloud data is used to identify the positions of multiple second surface points located on the surface of the second object in three-dimensional space.

[0134] The computer equipment constructs an initial generative model, which includes an initial third model and an initial second model. The initial third model can be any point cloud feature extraction model, such as a point cloud deep network (PointNet, PointBERT, PointMLP, PointNext, etc.). The initial second model can be any model structure based on the point cloud feature generation model; no restrictions are placed here. Using the initial generative model, data for constructing the desired model can be generated from the second point cloud data. For example... Figure 4As shown, the initial third model part is used to extract the second undetermined point cloud features based on the second point cloud data, and the initial second model part is used to generate the undetermined model construction data based on the second undetermined point cloud features. The second model construction data is the accurate model construction data corresponding to the second model. Therefore, by observing the data differences between the undetermined model construction data and the second model construction data, we can characterize, on the one hand, the accuracy of the second undetermined point cloud features extracted by the initial third model part in representing the distribution of the second surface points, and on the other hand, the accuracy of the undetermined model construction data generated by the initial second model part based on the second undetermined point cloud features.

[0135] Therefore, the computer device can adjust the model parameters corresponding to the initial generated model based on the data differences between the initial model construction data and the second model construction data to obtain the generated model. The generated model consists of a third model part and a second model part. The third model part can be used to accurately analyze the point cloud data and extract accurate point cloud features. The second model part can be used to generate accurate model construction data based on the point cloud features. Since the difference in parameter adjustment of the initial generated model is based on the difference in the dimension of the model construction data, and the model construction data is used to construct a complete 3D model, the third model part trained in this way can learn how to extract point cloud features that can represent the positional distribution of points on the overall surface of the object in 3D space, and enables the second model part to learn how to generate model construction data that can accurately construct a complete 3D model based on point cloud features, further improving the accuracy of the model contour structure in 3D space.

[0136] Furthermore, when performing step S302 to extract point cloud features, the computer device can execute step S3021 (not shown in the figure). Step S3021 is a possible implementation of step S302, including:

[0137] S3021: Extract the features of the first point cloud based on the first point cloud data through the third model part.

[0138] The computer device can accurately analyze the positions of multiple first surface points in the first point cloud data through the third model part, and obtain the positional distribution of multiple first surface points in three-dimensional space, thereby extracting accurate first point cloud features.

[0139] In order to more accurately adjust the parameters of the initial generated model, the computer device can construct the model training loss by combining differences in multiple dimensions.

[0140] Understandably, the closer two 3D models are, the more similar their visual representation will be when viewed from the same viewing angle. Therefore, to measure the accuracy of the initial model's generated data for constructing a pending model, the computer can construct a model loss by comparing the visual representation differences between the pending 3D model constructed based on the initial model's data and the second 3D model from the same viewing angle.

[0141] The computer device can first acquire the target display image corresponding to the second three-dimensional model. The target display image corresponds to the target display angle and is used to display the second three-dimensional model from the target display angle. For example, when the target display angle is a top view, the target display image is the top view corresponding to the second three-dimensional model.

[0142] Computer equipment can construct a 3D model based on the initial model construction data. Then, based on the target display angle and the 3D model, it determines a display image for that angle, used to display the 3D model from that angle. If the initial model construction data is accurate, the 3D model constructed based on it should be close to the second 3D model, resulting in a close resemblance between the display image and the target image. Therefore, when adjusting the model parameters of the initial model based on the differences between the initial and second model construction data to obtain the generated model, the computer equipment can adjust the parameters based on these differences, as well as the differences between the display image and the target image, to obtain the generated model. Data differences can express the accuracy of the data used to construct the model in three-dimensional space, and image differences can express the accuracy of the data used to construct the model in two-dimensional image space. This enriches the expression dimension of the accuracy of the parameters generated by the initial model, which is conducive to a more accurate analysis of the model effect of the initial model and thus improves the adjustment accuracy of the parameters of the initial model.

[0143] It is important to emphasize that this application uses the image differences between the two 3D models, rather than the image differences between the 3D model and the second object. The advantage of this is that the image differences between the 3D model and the object may be affected by more than just differences in the model's contour structure. For example, when the object's surface has texture, it may not be necessary to simulate that texture when constructing the 3D model. In this case, the image differences between the model and the object will still be affected by texture differences, and therefore, this difference alone cannot accurately express the differences in the model's contour structure. However, when there is no need for texture simulation, neither the second 3D model obtained in this application nor the constructed 3D model to be determined has texture parameters. In this case, adjusting parameters by using the differences between the images corresponding to the models can eliminate the influence of texture differences, intuitively expressing the differences in the model's contour structure, and better reflecting actual model construction needs.

[0144] Understandably, 3D models can be viewed from multiple angles in 3D space. The structure of a 3D model can be quite complex, and an image from a single viewing angle cannot accurately represent its structure. Figure 1 As shown. Based on this, in one possible implementation, in order to more accurately adjust model parameters based on two-dimensional images, the computer device can combine two-dimensional images from multiple display angles to more accurately represent the contour structure of the three-dimensional model in three-dimensional space.

[0145] When acquiring the target display image corresponding to the second 3D model, the computer device can acquire multiple display images corresponding to the second 3D model. These multiple display images correspond to multiple display angles, and different display images correspond to different display angles. The target display image is any one of these multiple display images. Therefore, by using multiple display images, the display angles of the model outline structure of the second 3D model can be enriched using 2D images, thereby enabling a more accurate representation of the model outline structure of the second 3D model based on multiple display images.

[0146] When adjusting the model parameters of the initial generated model based on the data differences between the data constructed from the undetermined model and the data constructed from the second model, and the image differences between the undetermined display image and the target display image, the computer device can determine the image difference corresponding to the target display angle based on the image differences between the undetermined display image and the target display image at the target display angle. This image difference at the target display angle is used to characterize the difference between the undetermined 3D model and the second 3D model in the 2D image at the target display angle. Therefore, by combining the image differences corresponding to multiple display angles, not only can the differences between the two 3D models in the 2D image be directly expressed, but also, to a certain extent, the differences in the model contour structure can be expressed. Furthermore, the computer device can adjust the model parameters of the initial generated model based on the data differences between the undetermined model and the data constructed from the second model, and the image differences corresponding to multiple display angles, to obtain the generated model. This allows the model construction data determined based on the generated model to construct a 3D model that is closer to the required 3D model, further improving the model training accuracy.

[0147] Next, we will provide a detailed introduction to the process of building data based on point cloud feature generation models.

[0148] In one possible implementation, the first model construction data may include the relative position information of multiple spatial points in three-dimensional space. The relative position information of the target spatial point is used to identify the relative positional relationship between the target spatial point and the surface of the first three-dimensional model. The target spatial point can be any one of the multiple points. Thus, by using the relative position information of the multiple spatial points, it is possible to determine which spatial points are located on the surface of the first three-dimensional model, which are located inside the first three-dimensional model, and which are located outside the first three-dimensional model, thereby enabling the construction of the first three-dimensional model in three-dimensional space. For example, the first model construction data can be SDF data. In SDF data, the absolute value of the value represents the distance between the spatial point and the nearest surface point. A positive value indicates that the spatial point is located outside the first three-dimensional model, a negative value indicates that the spatial point is located inside the first three-dimensional model, and a value of 0 indicates that the spatial point is located on the surface of the first three-dimensional model.

[0149] Since three-dimensional space is composed of three dimensions, in order to accurately analyze the relative positional information of spatial points, computer equipment can analyze the relative positional relationship between spatial points and the model from each of the three dimensions based on point cloud features. When generating the first model construction data based on the first point cloud features, the computer equipment can first determine the relative positional features corresponding to the three dimensions of three-dimensional space based on the first point cloud features. The relative positional features corresponding to the target dimension are used to identify the relative positional relationship between spatial points in three-dimensional space and the surface of the first three-dimensional model in the target dimension. The target dimension can be any one of the three dimensions. For example... Figure 5 As shown, when expressing three-dimensional space using the xyz coordinate system, the three dimensions can be the xy plane dimension, the xz plane dimension, and the yz plane dimension.

[0150] The processing equipment can accurately analyze the relative positional relationships between multiple spatial points and the model surface from three dimensions based on the relative positional features corresponding to each of the three dimensions. This allows for the accurate determination of the relative positional information of multiple spatial points, avoiding the problem of poor accuracy in relative positional relationship analysis caused by analysis from a single dimension, and further improving the accuracy of 3D reconstruction.

[0151] Specifically, in one possible implementation, when determining the relative position information of multiple spatial points based on the relative position features corresponding to the three dimensions, the computer device can determine the position information of the target spatial point in the target dimension. This position information is used to measure the position of the target spatial point in the target dimension. For example, in the xyz coordinate system, the position information corresponding to the xy plane dimension is the (x, y) coordinate of the target spatial point.

[0152] Since the relative position information corresponding to the target dimension can characterize the relative positional relationship between a spatial point and the model surface from the target dimension, and this positional information can characterize the position of the target spatial point in the target dimension, the computer device can determine the relative positional relationship between the spatial point at the position identified by the positional information and the model surface in the target dimension based on the relative positional features and positional information corresponding to the target dimension. In other words, the sub-positional features corresponding to the target spatial point in the target dimension can be determined. The sub-positional features are used to characterize the relative positional relationship between the target spatial point and the model surface of the first three-dimensional model in the target dimension.

[0153] Therefore, based on the sub-positional features corresponding to the target spatial point in the three dimensions, the computer device can combine the three dimensions of the three-dimensional space to comprehensively analyze the relative positional relationship between the target spatial point and the model surface of the first three-dimensional model, so as to determine the relative positional information corresponding to the target spatial point, and enable the relative positional information to accurately identify the relative positional relationship between the target spatial point and the model surface in the three-dimensional space.

[0154] As mentioned above, some objects may possess unique textures, such as the markings on animal fur, the color and material of clothing, etc. When constructing a 3D model corresponding to an object, the ability of the 3D model to have a texture similar to the object is a key factor in evaluating the quality of the 3D model construction. Therefore, this application can also support the restoration of the object texture of the object to be reconstructed during image-based 3D reconstruction.

[0155] In one possible implementation, texture information corresponding to surface points can be collected simultaneously when collecting point cloud data. Multiple first surface points have their own corresponding texture information, which is used to identify the texture of the object's surface corresponding to the first surface point, such as color, material, smoothness, roughness, and lighting information.

[0156] The first point cloud data reveals the positional distribution of multiple first surface points in three-dimensional space. Therefore, the computer device can extract sample texture distribution features based on the first point cloud data and the texture information corresponding to each of the multiple first surface points. These sample texture distribution features characterize the distribution of textures on the surface of the first object in three-dimensional space—that is, which textures on the first object should correspond to each position in three-dimensional space. Since these sample texture distribution features are generated based on actual surface point position information and texture information, they represent accurate texture distribution features corresponding to the first object. The second model part can be used to generate first model construction data based on the first point cloud features and sample texture distribution features. This ensures that the first three-dimensional model constructed based on this first model construction data not only matches the first object in contour structure but also closely approximates the first object in texture, achieving a more comprehensive and accurate three-dimensional reconstruction of the first object.

[0157] Similarly, the computer device needs to enable the initial first model part to learn how to extract relatively accurate texture distribution features based on images. When executing step S303, the computer device can execute step S3031 (not shown in the figure), where step S3031 is a possible implementation of step S303, including:

[0158] S3031: Through the initial first model part, extract the first undetermined point cloud features and undetermined texture distribution features based on the sample image.

[0159] The initial first model part can analyze the contour structure of the first object represented by the sample image, extract the first undetermined point cloud features, and analyze the texture distribution to extract undetermined texture distribution features. Since the sample texture distribution features are the accurate texture features corresponding to the first object, the closer the undetermined texture distribution features are to the sample texture distribution features, the more accurate the initial first model part's analysis of the texture distribution of the first object based on the sample image is. Therefore, when executing step S304, the computer device can execute step S3041 (not shown in the figure). Step S3041 is a possible implementation of step S304, including:

[0160] S3041: Based on the feature differences between the first undetermined point cloud features and the first point cloud features, and the feature differences between the undetermined texture distribution features and the sample texture distribution features, adjust the model parameters corresponding to the initial first model part to obtain the first model part.

[0161] The feature differences between point cloud features characterize the accuracy of the initial first model in extracting point cloud features from images, while the feature differences between texture distribution features characterize the accuracy of the initial first model in extracting texture distribution features from images. By combining these two differences to adjust model parameters, the feature differences between the first undetermined point cloud features and the undetermined texture distribution features output by the initial first model can be reduced. This allows the initial first model to learn how to accurately analyze the contour structure and texture distribution of objects in images, thereby extracting accurate point cloud features and texture distribution features from images. These are then used as input to the second model to generate accurate model construction data. Therefore, the 3D reconstruction model obtained in this way can simultaneously achieve accurate contour structure and texture distribution during 3D reconstruction, maximizing the simulation of the object to be reconstructed.

[0162] Next, we will explain in detail how to generate model building data based on texture distribution features.

[0163] In one possible implementation, the first model construction data may include relative position information and texture information corresponding to multiple spatial points in three-dimensional space. The relative position information corresponding to the target spatial point is used to identify the relative positional relationship between the target spatial point and the model surface of the first three-dimensional model. The texture information corresponding to the target spatial point is used to characterize the texture representation of the target spatial point in three-dimensional space, such as the color, material, lighting, etc. exhibited by the target spatial point. The target spatial point is any one of the multiple spatial points.

[0164] When generating the first model construction data based on the first point cloud features and sample texture distribution features, similar to the above, the relative position information of multiple spatial points can be determined based on the first point cloud features. The parameter generation method here has been introduced above and will not be repeated here.

[0165] Furthermore, similar to positional distribution features, computer equipment can determine the texture distribution features corresponding to the three dimensions of three-dimensional space based on the sample texture distribution features. The texture distribution features corresponding to the target dimension are used to characterize the distribution of the texture on the surface of the first object in the target dimension, where the target dimension can be any one of the three dimensions. Therefore, by combining the texture distribution features corresponding to the three dimensions, the texture representation of each spatial point in three-dimensional space can be accurately analyzed. Furthermore, based on the texture distribution features corresponding to the three dimensions, the texture information corresponding to multiple spatial points can be determined. Through the texture information corresponding to multiple spatial points, the texture distribution of the first object in three-dimensional space can be effectively reconstructed.

[0166] Specifically, in one possible implementation, when determining the texture information corresponding to multiple spatial points based on the texture distribution features corresponding to the three dimensions, the computer device can first determine the position information of the target spatial point in the target dimension. Since the texture distribution features corresponding to the target dimension can characterize the texture distribution of multiple spatial points in the target dimension, and this position information can identify the position of the target spatial point in the target dimension, the sub-texture information corresponding to the target spatial point in the target dimension can be determined based on the texture distribution features and position information. The sub-texture information is used to characterize the texture representation of the target spatial point in the target dimension, that is, to characterize the texture representation observed when viewing the target spatial point from the target dimension. For example, in the xyz coordinate system, the sub-texture information corresponding to the target spatial point in the xy plane dimension can characterize the texture representation that can be observed when viewing the target spatial point from the xy plane dimension.

[0167] Therefore, computer devices can combine the texture representation of a target spatial point in each of the three dimensions based on the sub-texture information corresponding to that point, and ultimately determine the texture information corresponding to the target spatial point. This texture information can accurately represent the texture representation of the target spatial point in three-dimensional space. For example, the target spatial point may appear in three different colors when viewed in three dimensions, and the final color represented by the target spatial point in three-dimensional space can be a combination of these three colors.

[0168] Understandably, features have diverse formats; for example, different features may have different feature sizes and number of channels. When analyzing feature differences, if two features have different formats, the differences between them will be affected by these format differences, failing to express only the differences in the feature values ​​themselves. The accuracy of feature extraction primarily lies in the accuracy of the extracted feature values, regardless of the feature format. Therefore, eliminating differences in feature formats when analyzing feature differences allows for a more effective expression of the model's feature extraction accuracy, thus enabling more effective and accurate adjustment of model parameters based on feature differences.

[0169] Based on this, in one possible implementation, the model parameters corresponding to the initial first model part include feature extraction parameters and feature transformation parameters. The feature extraction parameters are used to extract features from the image, and the feature transformation parameters are used to convert the extracted features into a unified feature format for point cloud features. When executing step S303, the computer device can execute steps S3032-S3033 (not shown in the figure). Steps S3032-S3033 are one possible implementation of step S303, including:

[0170] S3032: Extract initial features from the sample image using feature extraction parameters.

[0171] The initial features are extracted directly from the sample images and are used to characterize the contour structure of the first object analyzed by the initial first model. The initial features extracted in this way may have a different feature format than the first point cloud features, thus directly relying on the initial features and the first point cloud features cannot effectively characterize the accuracy of the features extracted by the initial first model.

[0172] S3033: By using feature transformation parameters, the feature format of the initial features is transformed to obtain the first undetermined point cloud features.

[0173] The first undetermined point cloud feature has the same feature format as the first point cloud feature, such as the same feature size and number of channels. Through feature transformation parameters, the initial features can be converted into a first undetermined point cloud feature with the same feature format as the first point cloud feature, while retaining the feature information used to characterize the location portion of the initial features. This allows the feature differences between the first undetermined point cloud feature and the first point cloud feature to accurately represent the accuracy of the initial first model's analysis of the first object's contour structure based on sample images.

[0174] When executing step S304, the computer device may execute step S3042 (not shown in the figure). Step S3042 is a possible implementation of step S304, including:

[0175] S3042: Based on the feature differences between the first undetermined point cloud features and the first point cloud features, adjust the feature extraction parameters and feature transformation parameters to obtain the first model part.

[0176] By adjusting the feature extraction parameters through this difference, the parameters can be used to accurately extract features from the image that characterize the contour structure of the object. Adjusting the feature transformation parameters through this difference allows for the preservation and highlighting of features that significantly contribute to the contour structure when transforming the feature format. This results in the transformed point cloud features accurately representing the positional distribution of surface points of the object in three-dimensional space. Furthermore, this method unifies the feature format of the point cloud features input to the second model, reducing the impact of feature format on the generation of model construction data and contributing to the generation of more accurate model construction data in the second model.

[0177] The above content mainly introduces the data processing methods involved in 3D reconstruction from the perspective of model training. Next, we will introduce several data processing methods when using 3D reconstruction models in practice.

[0178] See Figure 6 , Figure 6 A flowchart illustrating a three-dimensional reconstruction method provided in this application embodiment is shown. In this embodiment, the computer device can be any type of computer device with data processing capabilities. The method includes:

[0179] S601: Obtain the image to be reconstructed. The image to be reconstructed is used to display the object to be reconstructed.

[0180] The image to be reconstructed can be any image used to display the object to be reconstructed. The object to be reconstructed can be any three-dimensional object, such as a three-dimensional game object in a game, a three-dimensional animated character in an animation, etc.

[0181] S602: Using a 3D reconstruction model, generate target model construction data corresponding to the object to be reconstructed based on the image to be reconstructed.

[0182] The target model construction data is used to construct a three-dimensional model corresponding to the object to be reconstructed. The three-dimensional reconstruction model includes a first model part and a second model part. The first model part is used to extract target point cloud features from the image to be reconstructed. The target point cloud features are used to characterize the positional distribution of multiple surface points on the surface of the object to be reconstructed in three-dimensional space. The second model part is used to generate target model construction data based on the target point cloud features.

[0183] The first model part can be trained in the following way:

[0184] The computer device can acquire first point cloud data and sample images corresponding to a first object. The first point cloud data is used to identify the positions of multiple first surface points located on the surface of the first object in three-dimensional space, and the sample images are used to display the first object. Then, the computer device can extract first point cloud features based on the first point cloud data. The first point cloud features are used to characterize the positional distribution of multiple first surface points in three-dimensional space. The second model part is used to generate first model construction data based on the first point cloud features. The first model construction data is used to construct a first three-dimensional model corresponding to the first object.

[0185] The initial first model part extracts the first undetermined point cloud features from the sample image. The feature difference between the first undetermined point cloud features and the first point cloud features can characterize the accuracy of the initial first model part in extracting point cloud features. Based on this difference, the model parameters corresponding to the initial first model part can be adjusted to obtain the first model part.

[0186] The model training method described above has been detailed and will not be repeated here. Since the difference in point cloud features, representing the overall contour structure in 3D space, is used as the model loss during training, the 3D reconstruction model trained in this way can accurately analyze the overall contour structure of the object to be reconstructed in 3D space during practical applications. This ensures that the reconstructed target 3D model is consistent with the overall contour structure of the object to be reconstructed, avoiding the problem of inaccurate contour structures due to limitations in image display angles. This achieves a more accurate construction of the 3D model without increasing the complexity of the information required for 3D reconstruction.

[0187] As can be seen from the above, in 3D reconstruction, in addition to accurately restoring the contour structure of the object to be reconstructed, it is also possible to accurately restore the texture of the object. In one possible implementation, the first model part is used to extract target point cloud features and target texture distribution features from the image to be reconstructed. The target texture distribution features are used to characterize the distribution of the texture on the surface of the object to be reconstructed in 3D space.

[0188] The second model part can be used to generate target model construction data based on the target point cloud features and target texture distribution features. This allows the target model construction data to express the contour structure and texture distribution of the object to be reconstructed in three-dimensional space. The target three-dimensional model constructed based on this target model construction data can have a contour structure similar to the object to be reconstructed, and a surface texture similar to the object to be reconstructed, thereby further improving the accuracy and comprehensiveness of three-dimensional reconstruction.

[0189] In order to enable the first model part to extract texture distribution features, during the model training process, when extracting the first undetermined point cloud features based on the sample image using the initial first model part, the computer device can extract the first undetermined point cloud features and undetermined texture distribution features based on the sample image using the initial first model part.

[0190] Based on the feature differences between the first undetermined point cloud features and the first point cloud features, and the feature differences between the undetermined texture distribution features and the sample texture distribution features, the model parameters corresponding to the initial first model part are adjusted. This allows the initial first model part to learn how to extract accurate texture distribution features and point cloud features based on the image, thus obtaining the first model part. The sample texture distribution features are used to characterize the distribution of the texture on the surface of the first object in three-dimensional space, that is, the accurate texture distribution of the first object. This training process has been described in detail above and will not be repeated here.

[0191] In addition, to further improve the efficiency and flexibility of 3D reconstruction, computer equipment can also support a variety of automatic acquisition functions for images to be reconstructed, so as to simplify the preliminary preparation work for 3D reconstruction.

[0192] For example, in one possible implementation, when executing step S601, the computer device may execute steps S6011-S6012 (not shown in the figure), where steps S6011-S6012 are a possible implementation of step S601, including:

[0193] S6011: Obtain the initial image and first text information.

[0194] The initial image is used to display multiple objects, including the object to be reconstructed. The initial text information describes the object to be reconstructed. For example... Figure 7 As shown, the initial image may include multiple objects such as a cup, light bulb, table lamp, and bedside lamp, with the cup being the object to be reconstructed. The first text information is "cup with handle," used to describe the object to be reconstructed.

[0195] S6012: Determine the image to be reconstructed from the initial image based on the first text information.

[0196] Computer equipment can identify the object to be reconstructed from the initial image based on the description of the object in the first text information, thereby determining the image to be reconstructed. This image to be reconstructed is the portion of the initial image used to display the image to be reconstructed, eliminating other objects in the initial image and thus avoiding interference from other objects during the 3D reconstruction process of the object to be reconstructed. Figure 7 As shown, by using the description of the cup in the first text information, the computer device can determine the image to be reconstructed from the initial image, which only includes the cup.

[0197] In this way, computer equipment can directly identify the image to be reconstructed that highlights the object to be reconstructed from a complex multi-object image, without requiring the initiator of the 3D reconstruction to perform image processing. This further simplifies the preliminary preparation work required for 3D reconstruction and improves the reconstruction efficiency. For example, in a game scene, a game image may contain multiple game objects. To perform 3D reconstruction on a specific game object, the initiator of the 3D reconstruction can provide text information describing the game object. The computer equipment can then automatically generate the corresponding image to be reconstructed based on this text information.

[0198] In another possible implementation, the computer device can also directly generate the required image to be reconstructed based on text information, without requiring the initiator of the 3D reconstruction to provide any images, further simplifying the preliminary preparation work required for 3D reconstruction. When executing step S601, the computer device can execute steps S6013-S6014 (not shown in the figure). Steps S6013-S6014 are a possible implementation of step S601, including:

[0199] S6013: Obtain the second text information.

[0200] The second text information is used to describe the object to be reconstructed, such as... Figure 8 As shown, the second text information can be "a cup with a handle".

[0201] S6014: Generate the image to be reconstructed based on the second text information.

[0202] Therefore, by analyzing the second text information, the computer device can determine the object to be reconstructed in 3D, and can automatically generate an image to be reconstructed to display the object, such as... Figure 8 As shown, based on the second text information, the computer device can determine that the image to be reconstructed is a cup with a handle, and thus can automatically generate an image to be reconstructed to display the cup. In this way, the initiator does not need to provide an image to display the object to be reconstructed; only text information describing the object needs to be provided to complete the 3D reconstruction. This further simplifies the information required for 3D reconstruction and improves the convenience and efficiency of 3D reconstruction.

[0203] To facilitate understanding of the technical solution provided in this application, the three-dimensional reconstruction method provided in this application will be introduced next in conjunction with a practical application scenario.

[0204] See Figure 9 , Figure 9This application provides a flowchart of a 3D reconstruction method in a practical application scenario. The practical application scenario can be a scenario for constructing 3D game objects. The image to be reconstructed is an image showing the object to be reconstructed in a real-world scene. Through 3D reconstruction, a 3D model corresponding to the object to be reconstructed in the real-world scene can be constructed, which can then be applied to game design as a model of the game object. The method includes a model training part (steps S901-S905) and a model application part (S906-S910), specifically including:

[0205] S901: Obtain the second point cloud data and the second model construction data corresponding to the second object.

[0206] S902: The training yields a generative model that includes a second model part and a third model part.

[0207] The specific training process can be described as above. (See also...) Figure 10 , Figure 10 The model architecture used in this practical application scenario is demonstrated. The first model part includes an image encoder, which is used to extract point cloud features and texture distribution features from the image to be reconstructed. It can be any image encoder, such as a Contrastive Language-Image Pre-training (CLIP) feature extraction model or a self-supervised model (Distillation in Noise, Dino), etc. The extraction process is shown in the following formula:

[0208] f_image = Image_encoder(I)

[0209] Where f_image represents the extracted initial features, and I represents the image to be reconstructed.

[0210] The first model part can perform format conversion on the initial features to obtain point cloud features with the same size and number of channels as the point cloud features, as shown in the following formula:

[0211] f_trans=Transformer(f_image)

[0212] f_trans represents the transformed point cloud features, and Transformer is the neural network used for feature transformation.

[0213] The third model section includes a point cloud encoder (such as various point cloud deep networks) for extracting point cloud features and texture distribution features. The point cloud feature extraction method is shown in the following formula:

[0214] f_point = Encoder_3d(point)

[0215] Where f_point is the point cloud feature, Encoder_3d is the point cloud encoder, and point is the point cloud data.

[0216] The second model part includes a section for determining the features corresponding to each of the three dimensions (such as sub-location features and texture distribution features corresponding to each of the three dimensions). The extracted features can be plane features corresponding to each of the three dimensions. This part can be implemented using multi-layer deep learning convolutional layers or neural networks. The goal is to transform the input three-dimensional features into single-dimensional features. The extraction methods for the features corresponding to each of the three dimensions are shown below:

[0217] f_pf = Plane_feature(f_point)

[0218] Where f_pf represents the feature corresponding to a single dimension, and Plane_feature represents the extraction method for planar features. Therefore, in the initial training of the first model, the loss function can be expressed as follows:

[0219] Loss l1 =||f_trans-f_point||

[0220] The second model also includes a symbolic distance field network for generating relative position information based on features and a texture information generation network for generating texture information based on features. The relative position information and texture information corresponding to each spatial point constitute the model construction data.

[0221] S903: Obtain the sample image, first point cloud data and texture information corresponding to multiple first surface points corresponding to the first object.

[0222] S904: Based on the first point cloud data and the texture information corresponding to multiple first surface points, extract the first point cloud features and sample texture distribution features.

[0223] The computer device can be pre-trained in a similar manner to the pre-trained third model part to obtain a model part for accurately extracting texture distribution features, and then the corresponding texture distribution features can be extracted through this model part.

[0224] S905: The first model part is obtained by training based on the first point cloud features and sample texture distribution features.

[0225] S906: Obtain initial image and text information.

[0226] This text information is used to describe the object to be reconstructed in the initial image.

[0227] S907: Generate the image to be reconstructed based on the initial image and text information.

[0228] S908: Generates target model construction data based on the image to be reconstructed using a 3D reconstruction model.

[0229] By using a 3D reconstruction model, the relative position information and texture information of multiple spatial points in 3D space can be generated, thereby constructing a target 3D model corresponding to the object to be reconstructed.

[0230] S909: Construct a target 3D model using target model construction data.

[0231] S910: Renders the target 3D model in the game scene.

[0232] After constructing the target 3D model, it can be rendered directly in the game scene, allowing it to be directly applied within the game environment.

[0233] As can be seen from the above, this application has significant technical effects in the following aspects:

[0234] 1. This application uses contour structure loss in three-dimensional space to enable the model to learn how to extract the overall contour structure features of the object to be reconstructed based on the image, thereby reconstructing a three-dimensional model whose overall contour structure is more consistent with the object to be reconstructed, solving the problems of incomplete model structure and low accuracy caused by two-dimensional image loss.

[0235] 2. This application can simultaneously restore the contour structure and surface texture during 3D reconstruction, achieving a more comprehensive 3D reconstruction.

[0236] 3. This application does not require relevant personnel to perform complex image processing to generate the image to be reconstructed. Only text information describing the object to be reconstructed needs to be provided to automatically generate the image to be reconstructed.

[0237] 4. This application can combine model loss in three-dimensional space and two-dimensional image to more accurately adjust model parameters, making the model knowledge learned by the model more effective.

[0238] Based on the three-dimensional reconstruction method for the model training side provided in the above embodiments, this application also provides a three-dimensional reconstruction apparatus, see [link to previous document]. Figure 11 , Figure 11 This application provides a structural block diagram of a three-dimensional reconstruction device 1100, which includes a first acquisition unit 1101, a first extraction unit 1102, a second extraction unit 1103, and a first adjustment unit 1104.

[0239] The first acquisition unit 1101 is used to acquire first point cloud data and sample image corresponding to the first object. The first point cloud data is used to identify the positions of multiple first surface points located on the surface of the first object in three-dimensional space, and the sample image is used to display the first object.

[0240] The first extraction unit 1102 is used to extract first point cloud features based on the first point cloud data. The first point cloud features are used to characterize the positional distribution of the plurality of first surface points in three-dimensional space. The second model part is used to generate first model construction data based on the first point cloud features. The first model construction data is used to construct the first three-dimensional model corresponding to the first object.

[0241] The second extraction unit 1103 is used to extract first undetermined point cloud features based on the sample image through the initial first model part;

[0242] The first adjustment unit 1104 is used to adjust the model parameters corresponding to the initial first model part according to the feature difference between the first point cloud feature to be determined and the first point cloud feature, to obtain the first model part. The first model part and the second model part are used to constitute a three-dimensional reconstruction model. The three-dimensional reconstruction model is used to generate target model construction data according to the image to be reconstructed. The image to be reconstructed is used to display the object to be reconstructed. The target model construction data is used to construct the three-dimensional model corresponding to the object to be reconstructed. The first model part is used to extract target point cloud features according to the image to be reconstructed. The target point cloud features are used to characterize the positional distribution of multiple surface points located on the surface of the object to be reconstructed in three-dimensional space. The second model part is used to generate the target model construction data according to the target point cloud features.

[0243] In one possible implementation, the device further includes a second acquisition unit, a first generation unit, and a second adjustment unit:

[0244] The second acquisition unit is used to acquire the second point cloud data and the second model construction data corresponding to the second object. The second model construction data is used to construct the second three-dimensional model corresponding to the second object. The second point cloud data is used to identify the positions of multiple second surface points located on the surface of the second object in three-dimensional space.

[0245] The first generation unit is used to generate pending model construction data based on the second point cloud data through an initial generation model. The initial generation model includes an initial third model part and an initial second model part. The initial third model part is used to extract second pending point cloud features based on the second point cloud data, and the initial second model part is used to generate the pending model construction data based on the second pending point cloud features.

[0246] The second adjustment unit is used to adjust the model parameters corresponding to the initial generated model based on the data difference between the data to be constructed and the data to be constructed of the second model, so as to obtain a generated model. The generated model consists of a third model part and a second model part. The first extraction unit 1102 is specifically used for:

[0247] The third model part extracts the first point cloud features based on the first point cloud data.

[0248] In one possible implementation, the apparatus further includes a third acquisition unit, a construction unit, and a determination unit:

[0249] The third acquisition unit is used to acquire a target display image corresponding to the second three-dimensional model, the target display image corresponds to a target display angle, and the target display image is used to display the second three-dimensional model from the target display angle;

[0250] The construction unit is used to construct the undetermined three-dimensional model based on the undetermined model construction data.

[0251] The determining unit is used to determine a pending display image corresponding to the target display angle based on the target display angle and the pending 3D model. The pending display image is used to display the pending 3D model from the target display angle.

[0252] The second adjustment unit is specifically used for:

[0253] Based on the data differences between the pending model construction data and the second model construction data, and the image differences between the pending display image and the target display image, the model parameters corresponding to the initial generated model are adjusted to obtain the generated model.

[0254] In one possible implementation, the third acquisition unit is specifically used for:

[0255] Acquire multiple display images corresponding to the second 3D model. The multiple display images correspond to multiple display angles, and different display images correspond to different display angles. The target display image is any one of the multiple display images.

[0256] The second adjustment unit is specifically used for:

[0257] Based on the image difference between the pending display image corresponding to the target display angle and the target display image, determine the image difference corresponding to the target display angle;

[0258] Based on the data differences between the pending model construction data and the second model construction data, and the image differences corresponding to the multiple display angles, the model parameters corresponding to the initial generated model are adjusted to obtain the generated model.

[0259] In one possible implementation, the first model construction data includes relative position information corresponding to multiple spatial points in three-dimensional space. The relative position information corresponding to the target spatial point is used to identify the relative positional relationship between the target spatial point and the model surface of the first three-dimensional model. The target spatial point is any one of the multiple points. Generating the first model construction data based on the first point cloud features includes:

[0260] Based on the first point cloud features, the relative position features corresponding to the three dimensions of the three-dimensional space are determined respectively. The relative position features corresponding to the target dimension are used to identify the relative positional relationship between the spatial points in the three-dimensional space and the model surface of the first three-dimensional model under the target dimension. The target dimension is any one of the three dimensions.

[0261] Based on the relative position features corresponding to the three dimensions, the relative position information corresponding to the multiple spatial points is determined.

[0262] In one possible implementation, determining the relative position information corresponding to the plurality of spatial points based on the relative position features corresponding to the three dimensions includes:

[0263] Determine the position information of the target spatial point corresponding to the target dimension;

[0264] Based on the relative position features corresponding to the target dimension and the position information, the sub-position features corresponding to the target spatial point in the target dimension are determined. The sub-position features are used to characterize the relative positional relationship between the target spatial point and the model surface of the first three-dimensional model in the target dimension.

[0265] Based on the sub-positional features corresponding to the target spatial point in the three dimensions, the relative position information corresponding to the target spatial point is determined.

[0266] In one possible implementation, the plurality of first surface points have corresponding texture information, and the device further includes a third extraction unit:

[0267] The third extraction unit is used to extract sample texture distribution features based on the first point cloud data and the texture information corresponding to the plurality of first surface points respectively. The sample texture distribution features are used to characterize the distribution of the texture of the first object surface in three-dimensional space. The second model part is used to generate the first model construction data based on the first point cloud features and the sample texture distribution features.

[0268] The second extraction unit 1103 is specifically used for:

[0269] Through the initial first model part, first undetermined point cloud features and undetermined texture distribution features are extracted from the sample image;

[0270] The first adjustment unit 1104 is specifically used for:

[0271] Based on the feature differences between the first undetermined point cloud features and the first point cloud features, and the feature differences between the undetermined texture distribution features and the sample texture distribution features, the model parameters corresponding to the initial first model part are adjusted to obtain the first model part.

[0272] In one possible implementation, the first model construction data includes relative position information and texture information corresponding to multiple spatial points in three-dimensional space. The relative position information corresponding to the target spatial point is used to identify the relative positional relationship between the target spatial point and the model surface of the first three-dimensional model. The texture information corresponding to the target spatial point is used to characterize the texture representation of the target spatial point in three-dimensional space. The target spatial point is any one of the multiple spatial points. The step of generating the first model construction data based on the first point cloud features and the sample texture distribution features includes:

[0273] Based on the first point cloud features, determine the relative position information corresponding to the plurality of spatial points.

[0274] Based on the sample texture distribution features, the texture distribution features corresponding to the three dimensions of the three-dimensional space are determined respectively. The texture distribution features corresponding to the target dimension are used to characterize the distribution of the texture of the first object surface under the target dimension. The target dimension is any one of the three dimensions.

[0275] Based on the texture distribution features corresponding to the three dimensions, the texture information corresponding to the multiple spatial points is determined.

[0276] In one possible implementation, determining the texture information corresponding to the plurality of spatial points based on the texture distribution features corresponding to the three dimensions includes:

[0277] Determine the position information of the target spatial point corresponding to the target dimension;

[0278] Based on the texture distribution features corresponding to the target dimension and the position information, the sub-texture information corresponding to the target spatial point in the target dimension is determined, and the sub-texture information is used to characterize the texture representation of the target spatial point in the target dimension.

[0279] Based on the sub-texture information corresponding to the target spatial point in each of the three dimensions, the texture information corresponding to the target spatial point is determined.

[0280] In one possible implementation, the model parameters corresponding to the initial first model portion include feature extraction parameters and feature transformation parameters, and the second extraction unit 1103 is specifically used for:

[0281] Initial features are extracted from the sample image using the feature extraction parameters.

[0282] The feature format of the initial feature is transformed using the feature transformation parameters to obtain the first undetermined point cloud feature, which has the same feature format as the first point cloud feature.

[0283] The first adjustment unit 1104 is specifically used for:

[0284] Based on the feature differences between the first undetermined point cloud features and the first point cloud features, the feature extraction parameters and the feature transformation parameters are adjusted to obtain the first model part.

[0285] Based on the three-dimensional reconstruction method for the model application side provided in the above embodiments, this application also provides a three-dimensional reconstruction device, see [link to relevant documentation]. Figure 12 , Figure 12 This is a structural block diagram of a three-dimensional reconstruction device provided in an embodiment of this application. The device 1200 includes a fourth acquisition unit 1201 and a second generation unit 1202.

[0286] The fourth acquisition unit 1201 is used to acquire the image to be reconstructed, and the image to be reconstructed is used to display the object to be reconstructed.

[0287] The second generation unit 1202 is used to generate target model construction data corresponding to the object to be reconstructed based on the image to be reconstructed using a 3D reconstruction model. The target model construction data is used to construct a 3D model corresponding to the object to be reconstructed. The 3D reconstruction model includes a first model part and a second model part. The first model part is used to extract target point cloud features based on the image to be reconstructed. The target point cloud features are used to characterize the positional distribution of multiple surface points located on the surface of the object to be reconstructed in 3D space. The second model part is used to generate the target model construction data based on the target point cloud features. The first model part is trained in the following way:

[0288] Acquire first point cloud data and sample image corresponding to the first object. The first point cloud data is used to identify the positions of multiple first surface points located on the surface of the first object in three-dimensional space, and the sample image is used to display the first object.

[0289] First point cloud features are extracted based on the first point cloud data. The first point cloud features are used to characterize the positional distribution of the plurality of first surface points in three-dimensional space. The second model part is used to generate first model construction data based on the first point cloud features. The first model construction data is used to construct the first three-dimensional model corresponding to the first object.

[0290] The first undetermined point cloud features are extracted from the sample image using the initial first model part;

[0291] Based on the feature differences between the first undetermined point cloud features and the first point cloud features, the model parameters corresponding to the initial first model part are adjusted to obtain the first model part.

[0292] In one possible implementation, the first model part is used to extract target point cloud features and target texture distribution features based on the image to be reconstructed, wherein the target texture distribution features are used to characterize the distribution of the texture of the surface of the object to be reconstructed in three-dimensional space;

[0293] The second model part is used to generate the target model construction data based on the target point cloud features and the target texture distribution features;

[0294] The step of extracting first undetermined point cloud features from the sample image based on the initial first model portion includes:

[0295] Through the initial first model part, first undetermined point cloud features and undetermined texture distribution features are extracted from the sample image;

[0296] The step of adjusting the model parameters corresponding to the initial first model part based on the feature difference between the first undetermined point cloud feature and the first point cloud feature to obtain the first model part includes:

[0297] Based on the feature differences between the first undetermined point cloud feature and the first point cloud feature, and the feature differences between the undetermined texture distribution feature and the sample texture distribution feature, the model parameters corresponding to the initial first model part are adjusted to obtain the first model part. The sample texture distribution feature is used to characterize the distribution of the texture of the first object surface in three-dimensional space.

[0298] In one possible implementation, the fourth acquisition unit 1201 is specifically used for:

[0299] Acquire an initial image and first text information, wherein the initial image is used to display multiple objects, including the object to be reconstructed, and the first text information is used to describe the object to be reconstructed;

[0300] Based on the first text information, the image to be reconstructed is determined from the initial image, wherein the image to be reconstructed is the portion of the initial image used to display the image to be reconstructed.

[0301] In one possible implementation, the fourth acquisition unit 1201 is specifically used for:

[0302] Obtain second text information, which describes the object to be reconstructed;

[0303] The image to be reconstructed is generated based on the second text information.

[0304] This application also provides a computer device; please refer to [link to relevant documentation]. Figure 13 As shown, the computer device can be a terminal device; for example, a mobile phone can be used as a terminal device.

[0305] Figure 13 This diagram illustrates a partial structural representation of a mobile phone related to the terminal device provided in this embodiment. (Reference) Figure 13 The mobile phone includes components such as a radio frequency (RF) circuit 710, a memory 720, an input unit 730, a display unit 740, a sensor 750, an audio circuit 760, a wireless Fidelity (WiFi) module 770, a processor 780, and a power supply 790. Those skilled in the art will understand that... Figure 13 The mobile phone structure shown does not constitute a limitation on the mobile phone and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0306] The following is combined Figure 13 A detailed introduction to each component of a mobile phone:

[0307] RF circuit 710 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink information from the base station and processes it with processor 780; additionally, it transmits uplink data to the base station. Typically, RF circuit 710 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier (LNA), and a duplexer. Furthermore, RF circuit 710 can also communicate wirelessly with networks and other devices. The aforementioned wireless communication can use any communication standard or protocol, including but not limited to Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, and Short Messaging Service (SMS).

[0308] The memory 720 can be used to store software programs and modules. The processor 780 executes various mobile phone functions and data processing by running the software programs and modules stored in the memory 720. The memory 720 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory 720 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0309] The input unit 730 can be used to receive input numerical or character information, and to generate key signal inputs related to user settings and function control of the mobile phone. Specifically, the input unit 730 may include a touch panel 731 and other input devices 732. The touch panel 731, also known as a touch screen, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel 731), and drive the corresponding connected devices according to a pre-set program. Optionally, the touch panel 731 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends it to the processor 780, and can also receive and execute commands sent by the processor 780. In addition, the touch panel 731 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 731, the input unit 730 may also include other input devices 732. Specifically, other input devices 732 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.

[0310] The display unit 740 can be used to display information input by the user or information provided to the user, as well as various menus of the mobile phone. The display unit 740 may include a display panel 741, which may optionally be configured as a Liquid Crystal Display (LCD), Organic Light-Emitting Diode (OLED), or similar display panel. Further, a touch panel 731 may cover the display panel 741. When the touch panel 731 detects a touch operation on or near it, it transmits the information to the processor 780 to determine the type of touch event. Subsequently, the processor 780 provides corresponding visual output on the display panel 741 based on the type of touch event. Although in Figure 13 In this embodiment, the touch panel 731 and the display panel 741 are two separate components to realize the input and output functions of the mobile phone. However, in some embodiments, the touch panel 731 and the display panel 741 can be integrated to realize the input and output functions of the mobile phone.

[0311] The mobile phone may also include at least one sensor 750, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display panel 741 according to the ambient light level, and the proximity sensor can turn off the display panel 741 and / or backlight when the phone is moved to the ear. As a type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity and can be used for applications that recognize the phone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition-related functions (such as pedometer, taps), etc. Other sensors that may be configured in the mobile phone, such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.

[0312] Audio circuit 760, speaker 761, and microphone 762 provide an audio interface between the user and the mobile phone. Audio circuit 760 converts received audio data into electrical signals and transmits them to speaker 761, where speaker 761 converts them into sound signals for output. On the other hand, microphone 762 converts collected sound signals into electrical signals, which are received by audio circuit 760, converted into audio data, and then processed by processor 780 before being transmitted via RF circuit 710 to, for example, another mobile phone, or the audio data can be output to memory 720 for further processing.

[0313] WiFi is a short-range wireless transmission technology. Through the WiFi module 770, mobile phones can help users send and receive emails, browse web pages, and access streaming media, providing users with wireless broadband internet access. Although Figure 13 The WiFi module 770 is shown, but it is understood that it is not an essential component of a mobile phone and can be omitted as needed without changing the essence of the invention.

[0314] The processor 780 is the control center of the mobile phone, connecting various parts of the phone through various interfaces and lines. It executes software programs and / or modules stored in the memory 720, and calls data stored in the memory 720 to perform various functions and process data, thereby performing overall detection of the phone. Optionally, the processor 780 may include one or more processing units; preferably, the processor 780 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into the processor 780.

[0315] The mobile phone also includes a power supply 790 (such as a battery) that supplies power to various components. Preferably, the power supply can be logically connected to the processor 780 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system.

[0316] Although not shown, mobile phones may also include a camera, Bluetooth module, etc., which will not be described in detail here.

[0317] In this embodiment, the processor 780 included in the terminal device also has the following functions:

[0318] Acquire first point cloud data and sample image corresponding to the first object. The first point cloud data is used to identify the positions of multiple first surface points located on the surface of the first object in three-dimensional space, and the sample image is used to display the first object.

[0319] First point cloud features are extracted based on the first point cloud data. The first point cloud features are used to characterize the positional distribution of the plurality of first surface points in three-dimensional space. The second model part is used to generate first model construction data based on the first point cloud features. The first model construction data is used to construct the first three-dimensional model corresponding to the first object.

[0320] The first undetermined point cloud features are extracted from the sample image using the initial first model part;

[0321] Based on the feature differences between the first point cloud features to be determined and the first point cloud features, the model parameters corresponding to the initial first model part are adjusted to obtain the first model part. The first model part and the second model part are used to constitute a three-dimensional reconstruction model. The three-dimensional reconstruction model is used to generate target model construction data based on the image to be reconstructed. The image to be reconstructed is used to display the object to be reconstructed. The target model construction data is used to construct the three-dimensional model corresponding to the object to be reconstructed. The first model part is used to extract target point cloud features based on the image to be reconstructed. The target point cloud features are used to characterize the positional distribution of multiple surface points located on the surface of the object to be reconstructed in three-dimensional space. The second model part is used to generate the target model construction data based on the target point cloud features.

[0322] In this embodiment, the processor 780 included in the terminal device also has the following functions:

[0323] Acquire an image to be reconstructed, which is used to display the object to be reconstructed;

[0324] Using a 3D reconstruction model, target model construction data corresponding to the object to be reconstructed is generated based on the image to be reconstructed. This target model construction data is used to construct a 3D model corresponding to the object to be reconstructed. The 3D reconstruction model includes a first model part and a second model part. The first model part is used to extract target point cloud features from the image to be reconstructed. These target point cloud features characterize the positional distribution of multiple surface points on the surface of the object to be reconstructed in 3D space. The second model part is used to generate the target model construction data based on the target point cloud features. The first model part is trained in the following way:

[0325] Acquire first point cloud data and sample image corresponding to the first object. The first point cloud data is used to identify the positions of multiple first surface points located on the surface of the first object in three-dimensional space, and the sample image is used to display the first object.

[0326] First point cloud features are extracted based on the first point cloud data. The first point cloud features are used to characterize the positional distribution of the plurality of first surface points in three-dimensional space. The second model part is used to generate first model construction data based on the first point cloud features. The first model construction data is used to construct the first three-dimensional model corresponding to the first object.

[0327] The first undetermined point cloud features are extracted from the sample image using the initial first model part;

[0328] Based on the feature differences between the first undetermined point cloud features and the first point cloud features, the model parameters corresponding to the initial first model part are adjusted to obtain the first model part.

[0329] This application also provides a server; please refer to [link / reference]. Figure 14 As shown, Figure 14 This is a structural diagram of a server 800 provided in an embodiment of this application. The server 800 can vary significantly due to different configurations or performance. It may include one or more Central Processing Units (CPUs) 822 (e.g., one or more processors) and a memory 832, and one or more storage media 830 (e.g., one or more mass storage devices) for storing application programs 842 or data 844. The memory 832 and storage media 830 can be temporary or persistent storage. The program stored in the storage media 830 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the server. Furthermore, the CPU 822 may be configured to communicate with the storage media 830 and execute the series of instruction operations in the storage media 830 on the server 800.

[0330] Server 800 may also include one or more power supplies 826, one or more wired or wireless network interfaces 850, one or more input / output interfaces 858, and / or one or more operating systems 841, such as Windows Server. TM Mac OS X TM Unix TM Linux TM FreeBSD TM etc.

[0331] The steps performed by the server in the above embodiments can be based on Figure 14 The server structure shown.

[0332] This application also provides a computer-readable storage medium for storing a computer program that executes any one of the three-dimensional reconstruction methods described in the foregoing embodiments.

[0333] This application also provides a computer program product including a computer program, which, when run on a computer device, causes the computer device to perform any of the three-dimensional reconstruction methods described in the above embodiments.

[0334] It is understood that in the specific embodiments of this application, data related to user information is involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0335] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium can be at least one of the following media: read-only memory (ROM), RAM, magnetic disk, or optical disk, etc., and other media capable of storing program code.

[0336] It should be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the device and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The device and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the solution in this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0337] The above description is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A three-dimensional reconstruction method, characterized in that, The method includes: Acquire first point cloud data and sample image corresponding to the first object. The first point cloud data is used to identify the positions of multiple first surface points located on the surface of the first object in three-dimensional space, and the sample image is used to display the first object. First point cloud features are extracted based on the first point cloud data. The first point cloud features are used to characterize the positional distribution of the plurality of first surface points in three-dimensional space. The second model part is used to generate first model construction data based on the first point cloud features. The first model construction data is used to construct the first three-dimensional model corresponding to the first object. The first undetermined point cloud features are extracted from the sample image using the initial first model part; Based on the feature differences between the first point cloud features to be determined and the first point cloud features, the model parameters corresponding to the initial first model part are adjusted to obtain the first model part. The first model part and the second model part are used to constitute a three-dimensional reconstruction model. The three-dimensional reconstruction model is used to generate target model construction data based on the image to be reconstructed. The image to be reconstructed is used to display the object to be reconstructed. The target model construction data is used to construct the three-dimensional model corresponding to the object to be reconstructed. The first model part is used to extract target point cloud features based on the image to be reconstructed. The target point cloud features are used to characterize the positional distribution of multiple surface points located on the surface of the object to be reconstructed in three-dimensional space. The second model part is used to generate the target model construction data based on the target point cloud features.

2. The method according to claim 1, characterized in that, The method further includes: Acquire the second point cloud data and the second model construction data corresponding to the second object. The second model construction data is used to construct the second three-dimensional model corresponding to the second object. The second point cloud data is used to identify the positions of multiple second surface points located on the surface of the second object in three-dimensional space. The initial generation model generates pending model construction data based on the second point cloud data. The initial generation model includes an initial third model part and an initial second model part. The initial third model part is used to extract second pending point cloud features based on the second point cloud data. The initial second model part is used to generate the pending model construction data based on the second pending point cloud features. Based on the data differences between the undetermined model construction data and the second model construction data, the model parameters corresponding to the initial generated model are adjusted to obtain a generated model. The generated model consists of a third model part and a second model part. The step of extracting the first point cloud features based on the first point cloud data includes: The third model part extracts the first point cloud features based on the first point cloud data.

3. The method according to claim 2, characterized in that, The method further includes: Obtain a target display image corresponding to the second three-dimensional model, the target display image corresponds to a target display angle, and the target display image is used to display the second three-dimensional model from the target display angle; Based on the data for constructing the undetermined model, construct the undetermined 3D model; Based on the target display angle and the undetermined 3D model, a undetermined display image corresponding to the target display angle is determined, and the undetermined display image is used to display the undetermined 3D model from the target display angle; The step of adjusting the model parameters corresponding to the initial generated model based on the data difference between the data to be constructed in the undetermined model and the data to be constructed in the second model, to obtain the generated model, includes: Based on the data differences between the pending model construction data and the second model construction data, and the image differences between the pending display image and the target display image, the model parameters corresponding to the initial generated model are adjusted to obtain the generated model.

4. The method according to claim 3, characterized in that, The step of obtaining the target display image corresponding to the second 3D model includes: Acquire multiple display images corresponding to the second 3D model. The multiple display images correspond to multiple display angles, and different display images correspond to different display angles. The target display image is any one of the multiple display images. The step of adjusting the model parameters corresponding to the initial generated model based on the data difference between the data constructed by the undetermined model and the data constructed by the second model, and the image difference between the undetermined display image and the target display image, to obtain the generated model, includes: Based on the image difference between the pending display image corresponding to the target display angle and the target display image, determine the image difference corresponding to the target display angle; Based on the data differences between the pending model construction data and the second model construction data, and the image differences corresponding to the multiple display angles, the model parameters corresponding to the initial generated model are adjusted to obtain the generated model.

5. The method according to claim 1, characterized in that, The first model construction data includes relative position information corresponding to multiple spatial points in three-dimensional space. The relative position information corresponding to the target spatial point is used to identify the relative positional relationship between the target spatial point and the model surface of the first three-dimensional model. The target spatial point is any one of the multiple points. The step of generating the first model construction data based on the first point cloud features includes: Based on the first point cloud features, the relative position features corresponding to the three dimensions of the three-dimensional space are determined respectively. The relative position features corresponding to the target dimension are used to identify the relative positional relationship between the spatial points in the three-dimensional space and the model surface of the first three-dimensional model under the target dimension. The target dimension is any one of the three dimensions. Based on the relative position features corresponding to the three dimensions, the relative position information corresponding to the multiple spatial points is determined.

6. The method according to claim 5, characterized in that, The step of determining the relative position information corresponding to the plurality of spatial points based on the relative position features corresponding to the three dimensions includes: Determine the position information of the target spatial point corresponding to the target dimension; Based on the relative position features corresponding to the target dimension and the position information, the sub-position features corresponding to the target spatial point in the target dimension are determined. The sub-position features are used to characterize the relative positional relationship between the target spatial point and the model surface of the first three-dimensional model in the target dimension. Based on the sub-positional features corresponding to the target spatial point in the three dimensions, the relative position information corresponding to the target spatial point is determined.

7. The method according to claim 1, characterized in that, The plurality of first surface points have corresponding texture information, and the method further includes: Based on the first point cloud data and the texture information corresponding to the plurality of first surface points, sample texture distribution features are extracted. The sample texture distribution features are used to characterize the distribution of the texture of the first object surface in three-dimensional space. The second model part is used to generate the first model construction data based on the first point cloud features and the sample texture distribution features. The step of extracting first undetermined point cloud features from the sample image through the initial first model part includes: Through the initial first model part, first undetermined point cloud features and undetermined texture distribution features are extracted from the sample image; The step of adjusting the model parameters corresponding to the initial first model part based on the feature difference between the first undetermined point cloud feature and the first point cloud feature to obtain the first model part includes: Based on the feature differences between the first undetermined point cloud features and the first point cloud features, and the feature differences between the undetermined texture distribution features and the sample texture distribution features, the model parameters corresponding to the initial first model part are adjusted to obtain the first model part.

8. The method according to claim 7, characterized in that, The first model construction data includes relative position information and texture information corresponding to multiple spatial points in three-dimensional space. The relative position information corresponding to the target spatial point is used to identify the relative positional relationship between the target spatial point and the model surface of the first three-dimensional model. The texture information corresponding to the target spatial point is used to characterize the texture representation of the target spatial point in three-dimensional space. The target spatial point is any one of the multiple spatial points. Generating the first model construction data based on the first point cloud features and the sample texture distribution features includes: Based on the first point cloud features, determine the relative position information corresponding to the plurality of spatial points. Based on the sample texture distribution features, the texture distribution features corresponding to the three dimensions of the three-dimensional space are determined respectively. The texture distribution features corresponding to the target dimension are used to characterize the distribution of the texture of the first object surface under the target dimension. The target dimension is any one of the three dimensions. Based on the texture distribution features corresponding to the three dimensions, the texture information corresponding to the multiple spatial points is determined.

9. The method according to claim 8, characterized in that, The step of determining the texture information corresponding to the plurality of spatial points based on the texture distribution features corresponding to the three dimensions includes: Determine the position information of the target spatial point corresponding to the target dimension; Based on the texture distribution features corresponding to the target dimension and the position information, the sub-texture information corresponding to the target spatial point in the target dimension is determined, and the sub-texture information is used to characterize the texture representation of the target spatial point in the target dimension. Based on the sub-texture information corresponding to the target spatial point in each of the three dimensions, the texture information corresponding to the target spatial point is determined.

10. The method according to claim 1, characterized in that, The model parameters corresponding to the initial first model part include feature extraction parameters and feature transformation parameters. The step of extracting the first undetermined point cloud features from the sample image using the initial first model part includes: Initial features are extracted from the sample image using the feature extraction parameters. The feature format of the initial feature is transformed using the feature transformation parameters to obtain the first undetermined point cloud feature, which has the same feature format as the first point cloud feature. The step of adjusting the model parameters corresponding to the initial first model part based on the feature difference between the first undetermined point cloud feature and the first point cloud feature to obtain the first model part includes: Based on the feature differences between the first undetermined point cloud features and the first point cloud features, the feature extraction parameters and the feature transformation parameters are adjusted to obtain the first model part.

11. A three-dimensional reconstruction method, characterized in that, The method includes: Acquire an image to be reconstructed, which is used to display the object to be reconstructed; Using a 3D reconstruction model, target model construction data corresponding to the object to be reconstructed is generated based on the image to be reconstructed. This target model construction data is used to construct a 3D model corresponding to the object to be reconstructed. The 3D reconstruction model includes a first model part and a second model part. The first model part is used to extract target point cloud features from the image to be reconstructed. These target point cloud features characterize the positional distribution of multiple surface points on the surface of the object to be reconstructed in 3D space. The second model part is used to generate the target model construction data based on the target point cloud features. The first model part is trained in the following way: Acquire first point cloud data and sample image corresponding to the first object. The first point cloud data is used to identify the positions of multiple first surface points located on the surface of the first object in three-dimensional space, and the sample image is used to display the first object. First point cloud features are extracted based on the first point cloud data. The first point cloud features are used to characterize the positional distribution of the plurality of first surface points in three-dimensional space. The second model part is used to generate first model construction data based on the first point cloud features. The first model construction data is used to construct the first three-dimensional model corresponding to the first object. The first undetermined point cloud features are extracted from the sample image using the initial first model part; Based on the feature differences between the first undetermined point cloud features and the first point cloud features, the model parameters corresponding to the initial first model part are adjusted to obtain the first model part.

12. The method according to claim 11, characterized in that, The first model part is used to extract target point cloud features and target texture distribution features based on the image to be reconstructed. The target texture distribution features are used to characterize the distribution of the texture of the surface of the object to be reconstructed in three-dimensional space. The second model part is used to generate the target model construction data based on the target point cloud features and the target texture distribution features; The step of extracting first undetermined point cloud features from the sample image based on the initial first model portion includes: Through the initial first model part, first undetermined point cloud features and undetermined texture distribution features are extracted from the sample image; The step of adjusting the model parameters corresponding to the initial first model part based on the feature difference between the first undetermined point cloud feature and the first point cloud feature to obtain the first model part includes: Based on the feature differences between the first undetermined point cloud feature and the first point cloud feature, and the feature differences between the undetermined texture distribution feature and the sample texture distribution feature, the model parameters corresponding to the initial first model part are adjusted to obtain the first model part. The sample texture distribution feature is used to characterize the distribution of the texture of the first object surface in three-dimensional space.

13. The method according to claim 11, characterized in that, The acquisition of the image to be reconstructed includes: Acquire an initial image and first text information, wherein the initial image is used to display multiple objects, including the object to be reconstructed, and the first text information is used to describe the object to be reconstructed; Based on the first text information, the image to be reconstructed is determined from the initial image, wherein the image to be reconstructed is the portion of the initial image used to display the image to be reconstructed.

14. The method according to claim 11, characterized in that, The acquisition of the image to be reconstructed includes: Obtain second text information, which describes the object to be reconstructed; The image to be reconstructed is generated based on the second text information.

15. A three-dimensional reconstruction device, characterized in that, The device includes a first acquisition unit, a first extraction unit, a second extraction unit, and a first adjustment unit: The first acquisition unit is used to acquire first point cloud data and sample image corresponding to the first object. The first point cloud data is used to identify the positions of multiple first surface points located on the surface of the first object in three-dimensional space, and the sample image is used to display the first object. The first extraction unit is used to extract first point cloud features based on the first point cloud data. The first point cloud features are used to characterize the positional distribution of the plurality of first surface points in three-dimensional space. The second model part is used to generate first model construction data based on the first point cloud features. The first model construction data is used to construct a first three-dimensional model corresponding to the first object. The second extraction unit is used to extract first undetermined point cloud features from the sample image based on the initial first model part; The first adjustment unit is used to adjust the model parameters corresponding to the initial first model part according to the feature difference between the first point cloud feature to be determined and the first point cloud feature, to obtain the first model part. The first model part and the second model part are used to constitute a three-dimensional reconstruction model. The three-dimensional reconstruction model is used to generate target model construction data according to the image to be reconstructed. The image to be reconstructed is used to display the object to be reconstructed. The target model construction data is used to construct the three-dimensional model corresponding to the object to be reconstructed. The first model part is used to extract target point cloud features according to the image to be reconstructed. The target point cloud features are used to characterize the positional distribution of multiple surface points located on the surface of the object to be reconstructed in three-dimensional space. The second model part is used to generate the target model construction data according to the target point cloud features.

16. A three-dimensional reconstruction device, characterized in that, The device includes a fourth acquisition unit and a second generation unit: The fourth acquisition unit is used to acquire the image to be reconstructed, and the image to be reconstructed is used to display the object to be reconstructed; The second generation unit is used to generate target model construction data corresponding to the object to be reconstructed based on the image to be reconstructed using a 3D reconstruction model. The target model construction data is used to construct a 3D model corresponding to the object to be reconstructed. The 3D reconstruction model includes a first model part and a second model part. The first model part is used to extract target point cloud features based on the image to be reconstructed. The target point cloud features are used to characterize the positional distribution of multiple surface points located on the surface of the object to be reconstructed in 3D space. The second model part is used to generate the target model construction data based on the target point cloud features. The first model part is trained in the following way: Acquire first point cloud data and sample image corresponding to the first object. The first point cloud data is used to identify the positions of multiple first surface points located on the surface of the first object in three-dimensional space, and the sample image is used to display the first object. First point cloud features are extracted based on the first point cloud data. The first point cloud features are used to characterize the positional distribution of the plurality of first surface points in three-dimensional space. The second model part is used to generate first model construction data based on the first point cloud features. The first model construction data is used to construct the first three-dimensional model corresponding to the first object. The first undetermined point cloud features are extracted from the sample image using the initial first model part; Based on the feature differences between the first undetermined point cloud features and the first point cloud features, the model parameters corresponding to the initial first model part are adjusted to obtain the first model part.

17. A computer device, characterized in that, The computer device includes a processor and memory: The memory is used to store computer programs and to transfer the computer programs to the processor; The processor is configured to execute the three-dimensional reconstruction method according to any one of claims 1-10, or to execute the three-dimensional reconstruction method according to any one of claims 11-14, according to instructions in the computer program.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program for performing the three-dimensional reconstruction method according to any one of claims 1-10, or for performing the three-dimensional reconstruction method according to any one of claims 11-14.

19. A computer program product comprising a computer program, which, when run on a computer device, causes the computer device to perform the three-dimensional reconstruction method according to any one of claims 1-10, or to perform the three-dimensional reconstruction method according to any one of claims 11-14.

Images