Data processing method, three-dimensional reconstruction method and its device, equipment and storage medium
By using the initial surface distance extraction network and the combination of normal vector and included angle in three-dimensional model reconstruction, correcting and adjusting the volume density parameters, the problem of insufficient reconstruction accuracy in the implicit surface reconstruction algorithm is solved, and higher 3-dimensional model surface data is achieved Accuracy and detail integrity.
Patent Information
- Application Number
- CN202310230724.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-03-03
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2043-03-03
AI Technical Summary
Implicit surface reconstruction algorithm based on neural rendering. During the three-dimensional model reconstruction process, when observing rays passing through the sampling point of the target object at a high angle, it may cause the intersection of the observation rays and the surface to deviate from the real intersection point, resulting in insufficient reconstruction accuracy and insufficient model details. whole.
By obtaining the bounding box of the target object, multiple sampling points are obtained, and the preset initial surface distance extraction network is used to extract the sampling points parameters, and the volume density conversion is performed based on the normal vector of the sampling point and the angle between the observation rays, and the deviation adjustment is performed. Symbol distance parameters, train the target surface distance extraction network to improve reconstruction accuracy.
The reconstruction accuracy of the surface data of the three-dimensional model is improved, the distortion of the model surface is avoided, the integrity of the model details is enhanced, and the rendering effect of the three-dimensional model is more realistic.
Smart Images

Figure CN116310215B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer vision technology, and in particular to a data processing method, a three-dimensional reconstruction method, and a device, equipment, and storage medium thereof. Background Art
[0002] Multi-view 3D reconstruction technology can collect 2D images from multiple viewpoints and reconstruct a 3D model. 3D reconstruction technology plays an important role in 3D games, virtual reality, augmented reality and other fields.
[0003] In order to avoid large holes on the surface of the reconstructed 3D model and ensure the complete continuity of its surface, the application of implicit surface reconstruction algorithms based on neural rendering is increasing. At present, in the implicit surface reconstruction algorithm based on neural rendering, sampling points can be obtained by sampling within the manually annotated object boundary box, and the signed distance function (SDF) of each sampling point can be predicted. The SDF value of each sampling point can be directly converted to volume density, and then the implicit surface reconstruction algorithm can be trained based on the volume density value of each sampling point, so that a complete smooth surface can be reconstructed without mask supervision.
[0004] However, direct volume density conversion based on the SDF value of each sampling point may cause the intersection of the observation ray and the surface to deviate from the actual surface intersection for the sampling points where the observation ray passes through the target object at a large angle. Therefore, the direct volume density conversion method has a bias, which will make the training of the implicit surface reconstruction algorithm inaccurate, easily limit the reconstruction accuracy of the three-dimensional surface data, cause distortion in the three-dimensional surface data, and make the model details incomplete. Summary of the invention
[0005] In view of this, an embodiment of the present application provides a data processing method, a three-dimensional reconstruction method and its apparatus, device and storage medium to improve the reconstruction accuracy of the three-dimensional model and the completeness of the model details.
[0006] In a first aspect, an embodiment of the present application provides a data processing method, including:
[0007] Get the bounding box of the target object;
[0008] Sampling the three-dimensional space within the bounding box to obtain a plurality of sampling points;
[0009] Using a preset initial surface distance extraction network, extracting parameters of the plurality of sampling points to obtain parameter extraction results of the plurality of sampling points, wherein the parameter extraction result of each sampling point includes: a first signed distance parameter and a normal vector of each sampling point;
[0010] Determine the angle between the normal vector of each sampling point and the observation ray of each sampling point;
[0011] According to the first signed distance parameters corresponding to the multiple sampling points and the angles corresponding to the multiple sampling points, volume density conversion is performed to obtain volume density parameters of the multiple sampling points, and the volume density parameters are used to train the initial surface distance extraction network to obtain a target surface distance extraction network.
[0012] In a second aspect, an embodiment of the present application further provides a three-dimensional reconstruction method, the method comprising:
[0013] Get the bounding box of the target object;
[0014] Inputting a plurality of sampling points within the bounding box into a trained surface distance extraction network to obtain a target symbol distance parameter; wherein the surface extraction network is a target surface distance extraction network obtained by using the data processing method described in any one of the first aspects above;
[0015] According to the target symbol distance parameter, a preset surface extraction algorithm is used to perform three-dimensional reconstruction on the target object to obtain three-dimensional surface data of the target object.
[0016] In a third aspect, an embodiment of the present application further provides a data processing device, including:
[0017] A first acquisition module is used to acquire a bounding box of a target object;
[0018] A sampling module, used for sampling the three-dimensional space within the bounding box to obtain a plurality of sampling points;
[0019] A parameter extraction module, used to use a preset initial surface distance extraction network to perform parameter extraction on multiple sampling points to obtain parameter extraction results of the multiple sampling points, wherein the parameter extraction result of each sampling point includes: a first signed distance parameter and a normal vector of each sampling point;
[0020] A determination module, used to determine the angle between the normal vector of each sampling point and the observation ray of each sampling point;
[0021] A conversion module is used to perform volume density conversion according to the first signed distance parameters of the multiple sampling points and the angles corresponding to the multiple sampling points to obtain volume density parameters of the multiple sampling points, and the volume density parameters are used to train the initial surface distance extraction network to obtain a target surface distance extraction network.
[0022] In a fourth aspect, the present application also provides a three-dimensional reconstruction device, including:
[0023] A second acquisition module is used to acquire a bounding box of a target object;
[0024] An input module, used to input a plurality of sampling points in the bounding box into a trained surface distance extraction network to obtain a target symbol distance parameter; wherein the surface extraction network is a target surface distance extraction network obtained by using the data processing method described in any one of the first aspects above;
[0025] The reconstruction module is used to perform three-dimensional reconstruction on the target object according to the target symbol distance parameter by using a preset surface extraction algorithm to obtain three-dimensional surface data of the target object.
[0026] In the fifth aspect, an embodiment of the present application also provides a computer device, comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the computer device is running, the processor communicates with the storage medium through the bus, and the processor executes the machine-readable instructions to perform the steps of the data processing method described in any one of the first aspect or the three-dimensional reconstruction method described in the second aspect.
[0027] In a sixth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the data processing method described in any one of the first aspect or the three-dimensional reconstruction method described in the second aspect are executed.
[0028] The data processing method, three-dimensional reconstruction method and its apparatus, device and storage medium provided in the embodiments of the present application can obtain a bounding box of the target object, sample the three-dimensional space within the bounding box, obtain multiple sampling points, and use a preset initial surface distance extraction network to extract parameters of the multiple sampling points to obtain first signed distance parameters and normal vectors of the multiple sampling points, and further determine the angle between the normal vector of each sampling point and the observation ray of each sampling point, and then perform volume density conversion according to the first signed distance parameters of the multiple sampling points and the angles corresponding to the multiple sampling points to obtain volume density parameters of the multiple sampling points, which are used to train the initial surface distance extraction network to obtain the target surface distance extraction network. Because in the method provided in the embodiment of the present application, in the process of performing volume density conversion on the first signed distance parameters of multiple sampling points, the volume density conversion can be performed according to the normal vectors of the multiple sampling points and the angle between the normal vector of each sampling point and the observation ray of each sampling point. That is to say, in the process of volume density conversion, the correction adjustment of the first signed distance parameter of each sampling point according to the corresponding angle of the sampling point is added, so as to avoid the error of the signed distance parameter of the sampling point caused by the angle of the observation ray, realize the unbiased conversion of the volume density parameter, and make the volume density conversion of the sampling point as close as possible to the real surface of the object, thereby improving the training accuracy of the target surface distance extraction network, and then improving the reconstruction accuracy of the surface data of the three-dimensional model, avoiding the distortion of the model surface, and improving the completeness of the model details.
[0029] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.
[0031] Figure 1 A flowchart of a data processing method provided in an embodiment of the present application;
[0032] Figure 2 A flow chart of a method for volume density conversion in a data processing method provided in an embodiment of the present application;
[0033] Figure 3A This is a model rendering corresponding to the volume density parameters obtained by traditional volume density conversion;
[0034] Figure 3BA model rendering effect diagram corresponding to the volume density parameters obtained by the volume density conversion provided by the embodiment of the present application;
[0035] Figure 4 A method flow chart of a training surface distance extraction network for a data processing method provided in an embodiment of the present application;
[0036] Figure 5 A flow chart of a method for estimating a bounding box of a target object in a data processing method provided in an embodiment of the present application;
[0037] Figure 6 A flow chart of a method for estimating a three-dimensional point cloud and a bounding box in a data processing method provided in an embodiment of the present application;
[0038] Figure 7 A flow chart of a method for determining a target camera position at each viewing angle in a data processing method provided in an embodiment of the present application;
[0039] Figure 8 A flow chart of a method for determining a bounding box of a target object in a data processing method provided in an embodiment of the present application;
[0040] Fig. 9 A flowchart of a three-dimensional reconstruction method provided in an embodiment of the present application;
[0041] Fig.10 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application;
[0042] Fig.11 A schematic diagram of the structure of a three-dimensional reconstruction device provided in an embodiment of the present application;
[0043] Fig.12 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0044] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.
[0045] The three-dimensional reconstruction method in one embodiment of the present disclosure can be applied to any application scenario involving a virtual three-dimensional model, such as a three-dimensional game scene, a virtual scene based on virtual reality, a virtual scene based on augmented reality, etc. In the specific execution process, the three-dimensional reconstruction method can be run on a local terminal device or a server. When the three-dimensional reconstruction method is run on a server, the method can be implemented and executed based on a cloud interaction system, wherein the cloud interaction system includes a server and a client device.
[0046] In order to facilitate a better understanding of the three-dimensional reconstruction method provided in the embodiments of the present application, the technical terms involved in the following embodiments of the present application are first explained as follows.
[0047] Technical term explanation:
[0048] SDF: Signed Distance Function (SDF), which is an implicit expression of the surface of a 3D model, can determine the distance from a 3D point to the surface of an object in a finite area in 3D space and define the sign of the distance at the same time. For example, if the point is inside the object, it is positive; if the point is outside the object, it is negative; if the point is on the surface of the object, it is 0.
[0049] MLP: Multilayer Perceptron (MLP), a type of neural network architecture.
[0050] BBox: Bounding Box (BBox), refers to the minimum bounding box surrounding an object in three-dimensional (3D) space.
[0051] SFM: Structure From Motion (SFM) is a technique for estimating three-dimensional structure from a series of two-dimensional images containing visual motion information. SFM can be used to obtain camera parameters and sparse point clouds for each perspective.
[0052] Logistic distribution: Logistic distribution, also known as probability distribution.
[0053] Marching Cubes: Surface extraction algorithm, also known as surface rendering algorithm, is an algorithm used to extract isosurfaces in volume data. The core concept is to determine which edges of voxels intersect with isosurfaces by comparing vertices with user-specified thresholds after setting the 3D space and vertex scalar values, create triangular patches, and connect all cube faces on the isosurface boundary to obtain a surface.
[0054] In order to clearly describe the three-dimensional reconstruction method provided in the embodiment of the present application, the data processing method provided in the embodiment of the present application is first explained and illustrated by way of examples in combination with multiple drawings and multiple embodiments as follows, so as to clearly illustrate the data processing of the network model used in the three-dimensional reconstruction method during the training process.
[0055] Figure 1 A flowchart of a data processing method provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the data processing method may include:
[0056] S101: Obtain a bounding box of a target object.
[0057] In one possible implementation, the bounding box of the target object may be automatically estimated based on a plurality of pre-acquired two-dimensional images of the target object at a plurality of viewing angles, where the plurality of two-dimensional images may be, for example, a plurality of two-dimensional images obtained by pre-shooting the target object in a real environment at a plurality of viewing angles with a preset camera. In another possible implementation, the bounding box information of the target object may also be obtained based on the bounding box information input by the user.
[0058] S102: Sampling the three-dimensional space within the bounding box to obtain a plurality of sampling points.
[0059] In a possible implementation example, a three-dimensional point cloud of the target object may be first obtained, and based on the bounding box of the target object obtained above, the three-dimensional space within the bounding box in the three-dimensional point cloud of the target object may be sampled to obtain multiple sampling points, and the multiple sampling points are all three-dimensional sampling points.
[0060] In a specific implementation scenario, the three-dimensional point cloud of the target object may include: the three-dimensional point cloud of the target object at multiple viewing angles. In this way, according to the information of the bounding box, the three-dimensional space within the bounding box in the three-dimensional point cloud at each viewing angle may be sampled to obtain the sampling points corresponding to each viewing angle. Then, for multiple sampling points, it may include: sampling points corresponding to multiple viewing angles. Among them, the three-dimensional point cloud at each viewing angle may be obtained based on the two-dimensional image at each viewing angle, for example. For example, for the three-dimensional point cloud at each viewing angle, according to the information of the bounding box and the preset sampling resolution, the three-dimensional space in the bounding box in the three-dimensional point cloud at each viewing angle may be sampled to obtain the sampling points corresponding to each viewing angle. For example, 512 copies may be sampled on the xyz axis in the three-dimensional space of the bounding box, so that 512*512*512 sampling points may be sampled.
[0061] That is to say, for the multiple sampling points obtained by sampling, each sampling point has information of the corresponding viewing angle.
[0062] S103, using a preset initial surface distance extraction network to perform parameter extraction on multiple sampling points to obtain parameter extraction results of the multiple sampling points, wherein the parameter extraction result of each sampling point includes: a first signed distance parameter and a normal vector of each sampling point.
[0063] For example, the initial surface distance extraction network can be a neural network with an MLP structure, which can also be called an SDF network or an implicit surface SDF neural network. The first signed distance parameter of each sampling point can be used to represent the SDF value from the sampling point to the surface of the object, and the normal vector of each sampling point can be used to represent the normal vector between each sampling point and the surface of the target object.
[0064] S104: Determine the angle between the normal vector of each sampling point and the observation ray of each sampling point.
[0065] Among them, for each sampling point, the observation ray of each sampling point is a ray between the camera position of the viewing angle corresponding to each sampling point and each sampling point. The camera position of each sampling point corresponding to the viewing angle can be the initial camera position at the viewing angle obtained by estimating the pose of multiple two-dimensional images.
[0066] S105. Perform volume density conversion according to the first symbol distance parameters of the multiple sampling points and the angles corresponding to the multiple sampling points to obtain volume density parameters of the multiple sampling points. The volume density parameters are used to train an initial surface distance extraction network to obtain a target surface distance extraction network.
[0067] For example, the first signed distance parameters of the multiple sampling points can be weighted according to the angles corresponding to the multiple sampling points to achieve error correction of the signed distance parameters of each sampling point, thereby avoiding errors in the signed distance parameters of the sampling points caused by the angle of the observation ray. Then, the volume density conversion is performed according to the second signed distance parameters of the multiple sampling points after the weighted processing, so that the unbiased conversion of the volume density parameters of the multiple sampling points can be achieved, and the volume density parameters of the multiple sampling points can be obtained.
[0068] After obtaining the volume density parameters of the multiple sampling points, the initial surface distance extraction network can be trained based on the volume density parameters of the multiple sampling points to obtain the target surface distance extraction network.
[0069] The data processing method provided in this embodiment obtains a bounding box of the target object, samples the three-dimensional space within the bounding box, obtains multiple sampling points, and uses a preset initial surface distance extraction network to extract parameters of the multiple sampling points to obtain first signed distance parameters and normal vectors of the multiple sampling points. The angle between the normal vector of each sampling point and the observation ray of each sampling point is determined, and then volume density conversion is performed according to the first signed distance parameters of the multiple sampling points and the angles corresponding to the multiple sampling points to obtain volume density parameters of the multiple sampling points, which are used to train the initial surface distance extraction network to obtain the target surface distance extraction network. The method provided in the embodiment of the present application can perform volume density conversion according to the normal vectors of the multiple sampling points and the angle between the normal vector of each sampling point and the observation ray of each sampling point during the volume density conversion of the first signed distance parameters of the multiple sampling points. That is, in the process of volume density conversion, the correction adjustment of the first signed distance parameter of each sampling point according to the corresponding angle of the sampling point is added, so as to avoid the error of the signed distance parameter of the sampling point caused by the angle of the observation ray, realize the unbiased conversion of the volume density parameter, and make the volume density conversion of the sampling point as close as possible to the real surface of the object, thereby improving the training accuracy of the target surface distance extraction network, and then improving the reconstruction accuracy of the surface data of the three-dimensional model, avoiding the distortion of the model surface, and improving the completeness of the model details.
[0070] The specific implementation of volume density conversion in the data processing method provided in the above embodiment is explained below in conjunction with the accompanying drawings. Figure 2 This is a flow chart of a method for volume density conversion in a data processing method provided in an embodiment of the present application. Figure 2 As shown above, performing volume density conversion according to the first symbol distance parameters of the plurality of sampling points and the angles corresponding to the plurality of sampling points to obtain the volume density parameters of the plurality of sampling points may include:
[0071] S201. Determine a weight coefficient of each sampling point according to the cosine value of the angle.
[0072] For example, the reciprocal of the absolute value of the cosine value may be determined as the weight coefficient of each sampling point.
[0073] S202: Weight the first signed distance parameter of each sampling point according to the weight coefficient of each sampling point to obtain the second signed distance parameter of each sampling point.
[0074] S203: Perform volume density conversion according to the second signed distance parameter of each sampling point to obtain a volume density parameter of each sampling point.
[0075] In the method provided in this embodiment, the volume density parameter of each sampling point can be obtained by performing volume density conversion based on the second signed distance parameter of each sampling point using the probability distribution function, also known as the Logistic cumulative distribution function, or the volume density conversion formula shown below.
[0076]
[0077] Where σ(sdf) is the volume density parameter of each sampling point, β is a preset learning parameter, which is any value greater than 0; cosθ is the cosine value of the angle as described above. In this example, the SDF value is negated and divided by the absolute value of the cosine value to obtain That is, the weighted second symbol distance parameter of each sampling point, correspondingly, where, That is the weight coefficient for each sampling point.
[0078] In a specific implementation example, the weight coefficient of each sampling point can be, for example, based on the first signed distance parameters of the multiple sampling points, to determine the volume rendering weights of the multiple sampling points, and under the preset surface vertex volume rendering constraints, the parameter curve composed of the volume rendering weights of the multiple sampling points is derived, so as to obtain the expression formula of the weight coefficient of each sampling point through the cosine value of the angle. Among them, the preset surface vertex volume rendering constraints may include, for example: when the angle is maximum, the volume rendering weight corresponding to the sampling point is 0, and when the angle is minimum, the volume rendering weight corresponding to the sampling point is a maximum value. Since the expression formula of the weight coefficient of each sampling point through the cosine value of the angle is obtained under the preset surface vertex volume rendering constraints, the weight coefficient of each sampling point obtained based on the cosine value of the angle corresponding to each sampling point can realize accurate calibration and adjustment of the first signed distance parameters of each sampling point.
[0079] Then, in the method provided in this embodiment, an unbiased volume density conversion is performed based on the second signed distance parameter of each sampling point after calibration to obtain the volume density parameter of each sampling point. This can reduce the error influence on the signed distance parameter of each sampling point due to the large angle between the normal vector at the intersection of the observation ray and the surface, thereby realizing the unbiased conversion of the volume density parameters.
[0080] The following is a rendering effect diagram of a model rendering using the volume density parameters of each sampling point obtained by the unbiased conversion of the volume density parameters provided by this embodiment and a rendering effect diagram of a three-dimensional reconstruction of the same object using the volume density parameters obtained by directly performing volume density conversion in a traditional solution, in conjunction with the accompanying drawings. Figure 3A This is the model rendering effect corresponding to the volume density parameters obtained using traditional volume density conversion. Figure 3B This is a model rendering effect diagram corresponding to the volume density parameters obtained by using the volume density conversion provided in the embodiment of the present application.
[0081] The rendering provided in this embodiment uses the "clock" as the target object for model rendering. When the angle between the normal vector of each sampling point at the "long pointer of the clock" and the corresponding observation ray is too large, such as Figure 3A As shown in , if the traditional method of directly converting volume density to obtain volume density parameters for 3D reconstruction is used, the “long pointer of the clock” in the 3D model will be distorted, or even blurred and invisible. However, in the same case, when the angle between the normal vector of each sampling point at the “long pointer of the clock” and the corresponding observation ray is too large, such as Figure 3B As shown, if the method of unbiased volume density conversion based on the normal vector of each sampling point and the angle of the corresponding observation ray provided in the above embodiment of the present application is adopted to obtain volume density parameters for three-dimensional reconstruction, the "long hand of the clock" in the three-dimensional model will not be distorted or even blurred and invisible compared to the direct volume density conversion in the traditional scheme. On the contrary, the detailed features in the model are more complete and the rendering effect of the model is more realistic.
[0082] On the basis of the data processing method described in any of the above embodiments, after obtaining the volume density parameters of multiple sampling points, the surface distance extraction network can be trained based on the volume density parameters of the multiple sampling points. The training process of the surface distance extraction network is illustrated below with reference to the accompanying drawings. Figure 4 A flow chart of a method for training a surface distance extraction network for a data processing method provided in an embodiment of the present application. Figure 4 As shown, the method also includes:
[0083] S401 . Perform volume rendering according to volume density parameters of a plurality of sampling points and rendering colors of the plurality of sampling points along corresponding observation rays to obtain rendering colors of a plurality of observation rays.
[0084] In a possible implementation, volume rendering may be performed according to the volume density parameter of each sampling point, the rendering color of each sampling point along the corresponding observation ray, and a preset sampling spacing to obtain the rendering color of an observation ray. The preset sampling spacing is the spacing between two adjacent sampling points.
[0085] S402, training an initial surface distance extraction network according to the rendering colors of the plurality of observation rays and the true value colors of the pixel positions corresponding to the plurality of observation rays in the two-dimensional image of the target object to obtain a target surface distance extraction network.
[0086] Optionally, during the process of performing the initial surface distance extraction network, the model loss function value can be calculated according to the rendering colors of multiple sample observation rays and the true value colors of the pixel positions corresponding to the multiple observation rays in the two-dimensional image of the target object, and the parameters of the initial surface distance extraction network can be adjusted according to the model loss function value and the sampling gradient descent method, and the model training can be performed according to the surface distance extraction network after the parameter adjustment until the preset stop iteration condition is met to obtain the target surface distance extraction network. The stop iteration condition can be, for example, reaching a preset maximum number of iterations, or the model loss function value reaches a preset condition.
[0087] In a possible implementation example, the rendering colors of the multiple sampling points along the corresponding observation rays involved in S401 above can be obtained in advance using a preset initial color network. That is, before performing volume rendering according to the volume density parameters of the multiple sampling points and the rendering colors of the multiple sampling points along the corresponding observation rays in S401 above to obtain the rendering colors of the multiple observation rays, the method may further include:
[0088] According to the information of the multiple sampling points and the observation rays of the multiple sampling points, a preset initial color network is used to perform color rendering on the multiple sampling points to obtain the rendering colors of the multiple sampling points along the corresponding observation rays.
[0089] For example, the initial color network may also be a neural network using an MLP structure, which may also be called a Color network. Using the initial color network, the multiple sampling points in the three-dimensional space may be rendered in color according to the input multiple sampling points and the information of the observation ray of each sampling point, and the rendering colors of the multiple sampling points along the corresponding observation ray may be obtained.
[0090] In a possible implementation example, when the rendering colors of multiple sampling points along the corresponding observation rays are obtained by color rendering using a preset initial color network, in the process of training the initial surface distance extraction network, the parameters of the initial color network can be adjusted according to the model loss function value obtained by the above calculation, so that in the process of training the initial surface distance extraction network, the initial color network can also be trained according to the model loss function value. Therefore, in the process of re-training the model according to the initial surface distance extraction network after the parameter adjustment, the model can be trained according to the initial surface distance extraction network after the parameter adjustment and the initial color network after the parameter adjustment until the preset stop iteration condition is met, and the target surface distance extraction network and the target color network can be obtained. The target color network can be used to reconstruct the target object in three dimensions using the target surface distance extraction network to obtain the three-dimensional surface data of the target object, and the rendering color of each surface vertex on the target object can also be obtained.
[0091] Optionally, when the parameters of the initial surface distance extraction network are adjusted based on the model loss function value, the preset learning parameter β in the above-mentioned volume density conversion formula can also be adjusted. In this way, when the target surface distance extraction network and the target color network are obtained through training, the target learning parameter can also be obtained. The target learning parameter can be used in the three-dimensional reconstruction process to perform unbiased volume density conversion on the target symbol distance parameter of each sampling point obtained by using the target surface distance extraction network using the volume density conversion formula with the target learning parameter to obtain the target volume density parameter of each sampling point, and then color rendering can be performed according to the target volume density parameter of each sampling point in combination with the target color network, thereby obtaining the rendering color of each surface vertex on the target object.
[0092] The method provided in this embodiment can perform volume rendering according to the volume density parameters of the multiple sampling points and the rendering colors of the multiple sampling points along the corresponding observation rays when the volume density of the multiple sampling points is obtained to obtain the rendering colors of the multiple observation rays, and train the initial surface distance extraction network according to the rendering colors of the multiple observation rays and the true value colors of the pixel positions corresponding to the multiple observation rays in the two-dimensional image of the target object to obtain the target surface distance extraction network. Since the volume density parameters of the multiple sampling points are volume density parameters obtained by performing unbiased volume density conversion based on the first signed distance parameters of the multiple sampling points and the angles corresponding to the multiple sampling points, the initial surface distance extraction network is trained based on the volume density parameters of the multiple sampling points, so that the trained target surface distance extraction network can be made more accurate, and the distance sign parameters of the extracted sampling points can be made more accurate and more precise, thereby ensuring the reconstruction accuracy of the three-dimensional model.
[0093] In the above Figure 1-Figure 2 Based on the data processing method shown in any one of the embodiments, the embodiments of the present application can also provide a possible implementation method for automatically estimating the bounding box of the target object. Figure 5 This is a flow chart of a method for estimating a bounding box of a target object in a data processing method provided in an embodiment of the present application. Figure 5 As shown, obtaining the bounding box of the target object in S101 as shown above may include:
[0094] S501: Acquire multiple two-dimensional images of a target object at multiple viewing angles.
[0095] S502: Determine information of a three-dimensional point cloud and a bounding box of a target object under multiple viewing angles according to the multiple two-dimensional images.
[0096] In a possible implementation, a preset pose estimation algorithm may be used to perform joint pose estimation on multiple two-dimensional images to obtain three-dimensional point clouds under multiple perspectives and camera parameters under multiple perspectives, wherein the three-dimensional point cloud under each perspective may be, for example, a sparse point cloud. The preset pose estimation algorithm may be, for example, an SFM network.
[0097] S503: construct a bounding box of the target object according to the bounding box information.
[0098] When the information of the bounding box of the target object is obtained by adopting the above S502, a three-dimensional bounding box can be constructed based on the information of the bounding box to obtain the bounding box of the target object. Accordingly, the bounding box of the target object can be, for example, a three-dimensional object bounding box.
[0099] In the method provided in the embodiment of the present application, three-dimensional point clouds under multiple perspectives can be determined based on multiple two-dimensional images of the target object under multiple perspectives, and automatic estimation of bounding box information can be achieved, so that the bounding box of the target object can be obtained by constructing the bounding box, rather than using the information of the manually estimated bounding box. Therefore, compared with the manually estimated bounding box, it can achieve a compact estimation of the object bounding box, thereby making the sampling in the three-dimensional space more accurate and improving the sampling accuracy of the sampling points, thereby improving the training accuracy of the target surface distance extraction network and the three-dimensional reconstruction accuracy of the target object, and thus the completeness of the model details.
[0100] Secondly, since in the method provided in this embodiment, the bounding box of the target object is automatically calculated and estimated rather than manually estimated, this method also reduces the application complexity in the network training and three-dimensional reconstruction process.
[0101] With respect to the estimation of the three-dimensional point cloud and bounding box information in the data processing method provided in the above embodiment, this embodiment also provides some possible implementation methods. Figure 6 This is a flow chart of a method for estimating a three-dimensional point cloud and a bounding box in a data processing method provided in an embodiment of the present application. Figure 6 As shown, in the above embodiment, in S502, determining the information of the bounding box of the three-dimensional point cloud and the target object under multiple viewing angles according to the multiple two-dimensional images may include:
[0102] S601: Determine initial camera parameters and three-dimensional point clouds under multiple viewing angles according to multiple two-dimensional images.
[0103] For example, a preset SFM network can be used to perform joint pose estimation on multiple two-dimensional images to obtain initial camera parameters and three-dimensional point clouds at multiple viewing angles. The initial camera parameters at each viewing angle may include: camera pose information at that viewing angle, which may at least include: initial camera position.
[0104] S602: Determine the target camera position at each viewing angle according to the initial camera position and the three-dimensional point cloud at each viewing angle.
[0105] In a possible implementation, the initial camera parameters at each viewing angle may be adjusted by translation according to the three-dimensional point cloud at each viewing angle to obtain the target camera parameters at each viewing angle, so as to translate the camera position at each viewing angle closer to the target object.
[0106] S603: Determine information of a bounding box of the target object according to the target camera positions under multiple viewing angles.
[0107] In actual application scenarios, when the target camera positions under multiple viewing angles are obtained, the bounding box of the target object can be estimated according to the target camera positions under multiple viewing angles to obtain the bounding box information of the target object.
[0108] In the method provided in this embodiment, when the initial camera positions and three-dimensional point clouds under multiple viewing angles are obtained by estimating multiple two-dimensional images, the initial camera position under each viewing angle can be translated according to the three-dimensional point cloud of that viewing angle to obtain the target camera position under each viewing angle, and then the object border can be estimated based on the target camera positions under multiple viewing angles to obtain the information of the bounding box of the target object, thereby realizing automatic estimation of the bounding box of the target object and ensuring that the estimated bounding box of the target object is more compact.
[0109] For the translation update of the target camera position, the present application embodiment also provides some possible implementations, which are described below in conjunction with the accompanying drawings. The initial camera parameters shown above may also include at least: a camera rotation matrix. Figure 7 A flow chart of a method for determining the target camera position at each viewing angle in the data processing method provided in an embodiment of the present application, such as Figure 7As shown, in S602 shown above, determining the target camera position at each viewing angle according to the initial camera position and the three-dimensional point cloud at each viewing angle may include:
[0110] S701: Determine the minimum depth value between the three-dimensional point cloud and the initial camera position at each viewing angle.
[0111] In a possible implementation, first, the depth value between each 3D point cloud at each viewing angle and the initial camera position at the viewing angle may be calculated respectively, and then the minimum depth value may be determined from the depth values between each 3D point cloud and the initial camera position.
[0112] S702: Determine the main sight direction at each viewing angle according to the initial camera position and the camera rotation matrix at each viewing angle.
[0113] In a possible implementation, the rotation vector of each coordinate axis in the camera coordinate system at each viewing angle can be determined based on the initial camera position and the camera rotation matrix at each viewing angle, and the rotation vector of the z-axis (camera optical axis) can be determined as the main line of sight direction at each viewing angle.
[0114] S703 : Shift the initial camera position at each viewing angle by the minimum depth value along the main sight line direction of each viewing angle to obtain the target camera position of each viewing angle.
[0115] When the minimum depth value and the main sight line direction at each viewing angle are determined, the initial camera position at each viewing angle may be translated along the main sight line direction by the minimum depth value according to the minimum depth value to obtain the target camera position at each viewing angle.
[0116] The method provided in this embodiment can translate the initial camera position at each viewing angle along the main line of sight direction at each viewing angle based on the minimum depth value between the three-dimensional point cloud at each viewing angle and the initial camera position at the viewing angle, so as to obtain the target camera position at each viewing angle, and the target camera position at each viewing angle after the translation can be made as close to the surface of the target object as possible. Then, based on the target camera position at each viewing angle after the translation, the bounding box is estimated, which can ensure that the estimated bounding box of the target object is closer to the surface of the object, and its bounding box is more compact, thereby improving the estimation accuracy of the bounding box of the target object.
[0117] In another embodiment, the following embodiment of the present application also provides a possible implementation method for automatic estimation of a bounding box. Figure 8 A flow chart of a method for determining a bounding box of a target object in a data processing method provided in an embodiment of the present application, such as Figure 8 As shown, in S603 described in the above embodiment, the information of determining the bounding box of the target object according to the target camera positions under multiple viewing angles may include:
[0118] S801: Intersect the main sight lines under multiple viewing angles to determine the center position of the target object.
[0119] In a possible implementation, the main sight lines under multiple viewing angles may be intersected, and the center point may be estimated based on the intersection result to obtain the center position of the target object.
[0120] S802: Determine multiple distance parameters according to the target camera position and the center position under multiple viewing angles.
[0121] Among them, multiple distance parameters are used to represent the distance between the target camera position and the center position under multiple viewing angles.
[0122] S803: Determine a radius parameter of the target object according to multiple distance parameters.
[0123] The information of the bounding box includes: a center position and a radius parameter. In a possible implementation, the radius parameter of the target object may be determined according to a maximum distance parameter among a plurality of distance parameters.
[0124] In the method provided in this embodiment, the center position of the target object can be determined by intersecting the main line of sight directions under multiple perspectives, and the radius parameter of the target object can be determined based on multiple distance parameters between the target camera position and the center position under multiple perspectives, thereby realizing automatic estimation of the bounding box information of the target object, ensuring that the estimated bounding box of the target object is as close to the surface of the object as possible, making the estimated bounding box more compact.
[0125] In the case where the data processing method provided in any of the above embodiments obtains a target surface distance extraction network, the embodiment of the present application may also provide an implementation example of three-dimensional reconstruction of the target object based on the target surface distance extraction network. The device body for executing the three-dimensional reconstruction method may be the same hardware body as the device body of the above-mentioned data processing method, or may be a different hardware body, and the embodiment of the present application is not limited to this. Before executing the three-dimensional reconstruction method, the pre-trained model parameters are first imported, and the three-dimensional reconstruction method provided by the embodiment of the present application can be executed in combination with the model. Among them, the pre-trained model parameters may at least include: the model parameters of the target surface distance extraction network mentioned above.
[0126] The specific implementation process of three-dimensional reconstruction is explained and illustrated in detail below with reference to the accompanying drawings. Fig. 9 A flowchart of a three-dimensional reconstruction method provided in an embodiment of the present application, such as Fig. 9 As shown, the method may include:
[0127] S901: Obtain a bounding box of a target object.
[0128] In this embodiment, the specific implementation of obtaining the bounding box of the target object can be implemented by using the method in the above embodiment. Figure 5-Figure 8 The method provided in any embodiment estimates based on multiple two-dimensional images under multiple perspectives to obtain the bounding box of the target object. The bounding box of the target object can also be stored during the training process of the target surface distance extraction network. Here, the bounding box of the target object obtained during the network training process can be directly obtained.
[0129] S902: Input multiple sampling points in the bounding box into a trained surface distance extraction network to obtain a target symbol distance parameter.
[0130] The surface extraction network is a target surface distance extraction network obtained by using the data processing method provided in any embodiment.
[0131] In an implementation example of this embodiment, the multiple sampling points within the bounding box can be obtained in a similar manner to the above-mentioned S102, or the multiple sampling points extracted based on the bounding box of the target object can be stored during the training process of the target surface distance extraction network. Here, the multiple sampling points within the bounding box obtained during the network training process can be directly obtained.
[0132] S903 . According to the target symbol distance parameter, a preset surface extraction algorithm is used to perform three-dimensional reconstruction on the target object to obtain three-dimensional surface data of the target object.
[0133] In a possible implementation, for example, a preset surface extraction algorithm may be used to perform isosurface extraction based on target signed distance parameters of multiple sampling points, and a three-dimensional point with a signed distance parameter of 0 may be obtained as the three-dimensional surface data of the target object. The surface extraction algorithm may be, for example, the Marching cubes algorithm described above. In a specific implementation example, a preset surface extraction algorithm may be used to perform three-dimensional reconstruction of the target object based on target signed distance parameters of multiple sampling points and a preset voxel resolution, and extract the three-dimensional surface data of the target object.
[0134] Here, the obtained three-dimensional surface data is actually a three-dimensional surface without color information, which may include: position information of each surface vertex, which can be used to render and display a colorless three-dimensional virtual model of the target object. It should be noted that after the three-dimensional surface data is obtained, the three-dimensional surface data can be stored, and according to actual business needs, the three-dimensional model data can be opened through model rendering software, so as to realize the rendering and display of the three-dimensional virtual model corresponding to the target object.
[0135] The target surface distance extraction network used in the three-dimensional reconstruction method provided in the embodiment of the present application has a better training accuracy of the model because the volume density parameters of each sampling point used in the network training process are obtained by unbiased conversion based on the signed distance parameters of each sampling point and the angle corresponding to each sampling point. Therefore, the reconstruction accuracy of the surface data of the three-dimensional model obtained by three-dimensional reconstruction using the target surface distance extraction network is also higher, which effectively avoids distortion of the model surface and improves the completeness of the model details.
[0136] In the case of obtaining the three-dimensional surface data of the target object by performing three-dimensional reconstruction using the target surface distance extraction network provided in the present embodiment, the present application embodiment may also provide a possible implementation example of color rendering of the three-dimensional surface data to determine the rendering color of each surface vertex. It should be noted that, in the case of obtaining the rendering color of each surface vertex, a colored three-dimensional virtual model of the target object can be rendered and displayed based on the three-dimensional surface data and the rendering color of each surface vertex.
[0137] In the specific application implementation process, when the target surface distance extraction network is used to obtain the target signed distance parameters of multiple sampling points, volume density conversion can be performed according to the target signed distance parameters of each sampling point and the angle between the normal vector of each sampling point and the observation ray to obtain the target volume density parameters of each sampling point; volume rendering is performed according to the target volume density parameters of the multiple sampling points and the rendering colors of the multiple sampling points along the corresponding observation rays to obtain the target rendering colors of the multiple observation rays, and then the rendering colors of each surface vertex on the target object can be obtained according to the target rendering colors of the multiple observation rays and the three-dimensional surface data.
[0138] Among them, for the rendering colors of multiple sampling points along corresponding observation rays, for example: the target color network trained in the above embodiment can be used to render the multiple sampling points in the three-dimensional space according to the information of the input multiple sampling points and the observation rays of each sampling point, so as to obtain the rendering colors of the multiple sampling points along the corresponding observation rays.
[0139] The method provided in this embodiment can realize the rendering color of each surface vertices of the target object when the three-dimensional surface data of the target object is obtained by three-dimensional reconstruction. It not only realizes the reconstruction of the three-dimensional model of the target object, but also realizes the rendering of the surface color of the target object. The final three-dimensional reconstruction data can reflect the vertex information of the object surface and the color information of each vertex on the object surface, thereby improving the model reconstruction accuracy and information completeness.
[0140] The following is a description of the data processing device, three-dimensional reconstruction device, equipment and storage medium provided by the present application for execution. The specific implementation process and technical effects are described above and will not be repeated below.
[0141] Fig.10 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application is shown in FIG. Fig.10 As shown, the data processing device 1000 may include:
[0142] The first acquisition module 1001 is used to acquire a bounding box of a target object.
[0143] The sampling module 1002 is used to sample the three-dimensional space within the boundary box to obtain multiple sampling points.
[0144] The parameter extraction module 1003 is used to use a preset initial surface distance extraction network to perform parameter extraction on multiple sampling points to obtain parameter extraction results of the multiple sampling points. The parameter extraction result of each sampling point includes: a first signed distance parameter and a normal vector of each sampling point.
[0145] The determination module 1004 is used to determine the angle between the normal vector of each sampling point and the observation ray of each sampling point.
[0146] The conversion module 1005 is used to perform volume density conversion according to the first signed distance parameters of the multiple sampling points and the angles corresponding to the multiple sampling points to obtain volume density parameters of the multiple sampling points. The volume density parameters are used to train the initial surface distance extraction network to obtain the target surface distance extraction network.
[0147] The data processing device provided in this embodiment can obtain a bounding box of the target object, sample the three-dimensional space within the bounding box, obtain multiple sampling points, and use a preset initial surface distance extraction network to extract parameters of the multiple sampling points to obtain first signed distance parameters and normal vectors of the multiple sampling points. It also determines the angle between the normal vector of each sampling point and the observation ray of each sampling point, and then performs volume density conversion according to the first signed distance parameters of the multiple sampling points and the angles corresponding to the multiple sampling points to obtain volume density parameters of the multiple sampling points, which are used to train the initial surface distance extraction network to obtain the target surface distance extraction network. The data processing device provided in the embodiment of the present application can perform volume density conversion according to the normal vectors of the multiple sampling points and the angle between the normal vector of each sampling point and the observation ray of each sampling point during the process of volume density conversion of the first signed distance parameters of the multiple sampling points. That is, in the process of volume density conversion, the correction adjustment of the first signed distance parameter of each sampling point according to the corresponding angle of the sampling point is added, so as to avoid the error of the signed distance parameter of the sampling point caused by the angle of the observation ray, realize the unbiased conversion of the volume density parameter, and make the volume density conversion of the sampling point as close as possible to the real surface of the object, thereby improving the training accuracy of the target surface distance extraction network, and then improving the reconstruction accuracy of the surface data of the three-dimensional model, avoiding the distortion of the model surface, and improving the completeness of the model details.
[0148] In a feasible implementation scheme, the conversion module 1005 is specifically used to determine the weight coefficient of each sampling point according to the cosine value of the angle; according to the weight coefficient of each sampling point, the first signed distance parameter of each sampling point is weighted to obtain the second signed distance parameter of each sampling point;
[0149] According to the second signed distance parameter of each sampling point, volume density conversion is performed to obtain the volume density parameter of each sampling point.
[0150] In a feasible implementation manner, the conversion module 1005 is specifically configured to determine the reciprocal of the absolute value of the cosine value as the weight coefficient of each sampling point.
[0151] In a feasible implementation scheme, the data processing device 1000 further includes:
[0152] A rendering module, configured to perform volume rendering according to the volume density parameters of the plurality of sampling points and the rendering colors of the plurality of sampling points along corresponding observation rays, so as to obtain rendering colors of the plurality of observation rays;
[0153] The training module is used to train the initial surface distance extraction network according to the rendering colors of multiple observation rays and the true value colors of the pixel positions corresponding to the multiple observation rays in the two-dimensional image of the target object to obtain the target surface distance extraction network.
[0154] In a feasible implementation scheme, the training module is specifically used to calculate the model loss function value based on the rendering colors of multiple observation rays and the true value colors of the pixel positions corresponding to the multiple observation rays in the two-dimensional image of the target object; according to the model loss function value, the initial surface distance extraction network is adjusted by sampling the gradient descent method, and the model is re-trained based on the initial surface distance extraction network after the parameter adjustment until the preset stop iteration condition is met to obtain the target surface distance extraction network.
[0155] In a feasible implementation scheme, the first acquisition module 1001 is specifically used to acquire multiple two-dimensional images of the target object at multiple perspectives; determine the information of the three-dimensional point cloud and the bounding box of the target object at multiple perspectives based on the multiple two-dimensional images; and construct the bounding box of the target object based on the bounding box information.
[0156] In a feasible implementation scheme, the first acquisition module 1001 is specifically used to determine initial camera parameters and three-dimensional point clouds at multiple perspectives based on multiple two-dimensional images, wherein the initial camera parameters include: an initial camera position; determining a target camera position at each perspective based on the initial camera position and three-dimensional point cloud at each perspective; and determining information on a bounding box of a target object based on the target camera position at multiple perspectives.
[0157] In a feasible implementation scheme, the initial camera parameters also include: a camera rotation matrix, a first acquisition module 1001, which is specifically used to determine the minimum depth value between the three-dimensional point cloud and the initial camera position at each viewing angle; according to the initial camera position and the camera rotation matrix at each viewing angle, the main line of sight direction at each viewing angle is determined; the initial camera position at each viewing angle is translated by the minimum depth value along the main line of sight direction of each viewing angle to obtain the target camera position at each viewing angle.
[0158] In a feasible implementation scheme, the first acquisition module 1001 is specifically used to intersect the main line of sight directions under multiple perspectives to determine the center position of the target object; determine multiple distance parameters based on the target camera position and the center position under multiple perspectives, wherein the multiple distance parameters are respectively used to represent the distance between the target camera position and the center position under multiple perspectives; determine the radius parameter of the target object from the multiple distance parameters; the bounding box information includes: the center position and the radius parameter.
[0159] In a feasible implementation manner, the first acquisition module 1001 is specifically configured to determine a radius parameter of the target object according to a maximum distance parameter among a plurality of distance parameters.
[0160] In a feasible implementation scheme, the volume rendering module is also used to perform color rendering on the multiple sampling points based on the information of the multiple sampling points and the observation rays of the multiple sampling points using a preset initial color network to obtain the rendering colors of the multiple sampling points along the corresponding observation rays.
[0161] Fig.11 A schematic diagram of the structure of a three-dimensional reconstruction device provided in an embodiment of the present application is shown in FIG. Fig.11 As shown, the 3D reconstruction device 1100 may include:
[0162] The second acquisition module 1101 is used to acquire a bounding box of the target object.
[0163] An input module 1102 is used to input a plurality of sampling points in the bounding box into a trained surface distance extraction network to obtain a target symbol distance parameter; wherein the surface extraction network is a target surface distance extraction network obtained by using the data processing method described in any of the above embodiments;
[0164] The reconstruction module 1103 is used to perform three-dimensional reconstruction on the target object according to the target symbol distance parameter by using a preset surface extraction algorithm to obtain three-dimensional surface data of the target object.
[0165] The target surface distance extraction network used in the three-dimensional reconstruction process of the three-dimensional reconstruction device provided in the embodiment of the present application has better training accuracy of the model because the volume density parameters of each sampling point used in the network training process are obtained by unbiased conversion based on the signed distance parameters of each sampling point and the angle corresponding to each sampling point. Therefore, the reconstruction accuracy of the surface data of the three-dimensional model obtained by three-dimensional reconstruction using the target surface distance extraction network is also higher, which effectively avoids the distortion of the model surface and improves the completeness of the model details.
[0166] Fig.12 A schematic diagram of the structure of a computer device provided in an embodiment of the present application is shown in FIG. Fig.12 As shown, the computer device includes: a processor 1201, a storage medium 1202 and a bus 1203, the storage medium 1202 stores machine-readable instructions executable by the processor 1201, when the computer device runs a data processing method as in the embodiment, the processor 1201 communicates with the storage medium 1202 through the bus 1203, the processor 1201 executes the machine-readable instructions, and the processor 1201 performs the preamble of the method item to perform the following steps:
[0167] Get the bounding box of the target object;
[0168] Sampling the three-dimensional space within the bounding box to obtain multiple sampling points;
[0169] Using a preset initial surface distance extraction network, extract parameters of multiple sampling points to obtain parameter extraction results of the multiple sampling points, wherein the parameter extraction result of each sampling point includes: a first signed distance parameter and a normal vector of each sampling point;
[0170] Determine the angle between the normal vector of each sampling point and the observation ray of each sampling point;
[0171] According to the first signed distance parameters of the multiple sampling points and the angles corresponding to the multiple sampling points, volume density conversion is performed to obtain volume density parameters of the multiple sampling points. The volume density parameters are used to train the initial surface distance extraction network to obtain the target surface distance extraction network.
[0172] In a feasible implementation manner, when the processor 1201 performs volume density conversion according to the first signed distance parameters of the plurality of sampling points and the angles corresponding to the plurality of sampling points to obtain the volume density parameters of the plurality of sampling points, it is specifically configured to:
[0173] According to the cosine value of the angle, the weight coefficient of each sampling point is determined;
[0174] According to the weight coefficient of each sampling point, the first signed distance parameter of each sampling point is weighted to obtain the second signed distance parameter of each sampling point;
[0175] According to the second signed distance parameter of each sampling point, volume density conversion is performed to obtain the volume density parameter of each sampling point.
[0176] In a feasible implementation manner, when the processor 1201 determines the weight coefficient of each sampling point according to the cosine value of the angle, it is specifically used to:
[0177] The reciprocal of the absolute value of the cosine value is determined as the weight coefficient of each sampling point.
[0178] In a feasible implementation manner, after obtaining the volume density parameters of the plurality of sampling points, the processor 1201 is further configured to:
[0179] Performing volume rendering according to volume density parameters of multiple sampling points and rendering colors of multiple sampling points along corresponding observation rays to obtain rendering colors of multiple observation rays;
[0180] According to the rendering colors of the multiple observation rays and the true value colors of the pixel positions corresponding to the multiple observation rays in the two-dimensional image of the target object, the initial surface distance extraction network is trained to obtain the target surface distance extraction network.
[0181] In a feasible implementation manner, the processor 1201, when executing training of the initial surface distance extraction network according to the rendering colors of the plurality of observation rays and the true value colors of the pixel positions corresponding to the plurality of observation rays in the two-dimensional image of the target object to obtain the target surface distance extraction network, is specifically used to:
[0182] Calculate the model loss function value according to the rendering colors of the multiple observation rays and the true value colors of the pixel positions corresponding to the multiple observation rays in the two-dimensional image of the target object;
[0183] According to the model loss function value, the sampling gradient descent method is used to adjust the parameters of the initial surface distance extraction network, and the model is retrained based on the initial surface distance extraction network after parameter adjustment until the preset stop iteration condition is met to obtain the target surface distance extraction network.
[0184] In a feasible implementation, when executing the acquisition of the bounding box of the target object, the processor 1201 is specifically configured to:
[0185] Acquire multiple two-dimensional images of the target object at multiple viewing angles;
[0186] Determine information of a three-dimensional point cloud and a bounding box of a target object under multiple viewing angles according to the multiple two-dimensional images;
[0187] Construct the bounding box of the target object based on the bounding box information.
[0188] In a feasible implementation, when the processor 1201 determines the information of the bounding box of the three-dimensional point cloud and the target object under multiple perspectives according to the multiple two-dimensional images, it is specifically used to:
[0189] Determine initial camera parameters and three-dimensional point clouds under multiple viewing angles according to the multiple two-dimensional images, wherein the initial camera parameters include: an initial camera position;
[0190] According to the initial camera position and the 3D point cloud at each viewing angle, the target camera position at each viewing angle is determined;
[0191] According to the target camera positions under multiple viewing angles, the information of the bounding box of the target object is determined.
[0192] In a feasible implementation, the initial camera parameters further include: a camera rotation matrix; when the processor 1201 determines the target camera position at each viewing angle according to the initial camera position and the three-dimensional point cloud at each viewing angle, it is specifically used to:
[0193] Determine the minimum depth value between the 3D point cloud and the initial camera position at each viewing angle;
[0194] According to the initial camera position and camera rotation matrix at each viewing angle, determine the main line of sight direction at each viewing angle;
[0195] The initial camera position at each viewing angle is translated by the minimum depth value along the main line of sight of each viewing angle to obtain the target camera position of each viewing angle.
[0196] In a feasible implementation, when the processor 1201 determines the information of the bounding box of the target object according to the target camera position under multiple viewing angles, it is specifically used to:
[0197] Intersecting the main sight lines under the multiple viewing angles to determine the center position of the target object;
[0198] Determine a plurality of distance parameters according to the target camera position and the center position under the plurality of viewing angles, wherein the plurality of distance parameters are used to represent the distance between the target camera position and the center position under the plurality of viewing angles respectively;
[0199] The radius parameter of the target object is determined from the multiple distance parameters; the information of the bounding box includes: a center position and a radius parameter.
[0200] In a feasible implementation manner, when the processor 1201 determines the radius parameter of the target object from a plurality of distance parameters, it is specifically configured to:
[0201] A radius parameter of the target object is determined according to a maximum distance parameter among the multiple distance parameters.
[0202] In a feasible implementation manner, before performing volume rendering according to the volume density parameters of the plurality of sampling points and the rendering colors of the plurality of sampling points along the corresponding observation rays to obtain the rendering colors of the plurality of observation rays, the processor 1201 is further configured to:
[0203] According to the information of the multiple sampling points and the observation rays of the multiple sampling points, a preset initial color network is used to perform color rendering on the multiple sampling points to obtain the rendering colors of the multiple sampling points along the corresponding observation rays.
[0204] Through the above method, by obtaining the bounding box of the target object, sampling the three-dimensional space within the bounding box, obtaining multiple sampling points, and using a preset initial surface distance extraction network to extract parameters of the multiple sampling points, obtaining first signed distance parameters and normal vectors of the multiple sampling points, and determining the angle between the normal vector of each sampling point and the observation ray of each sampling point, and then performing volume density conversion according to the first signed distance parameters of the multiple sampling points and the angles corresponding to the multiple sampling points, obtaining volume density parameters of the multiple sampling points, which are used to train the initial surface distance extraction network to obtain the target surface distance extraction network. The data processing device provided in the embodiment of the present application can perform volume density conversion according to the normal vectors of the multiple sampling points and the angle between the normal vector of each sampling point and the observation ray of each sampling point during the process of volume density conversion of the first signed distance parameters of the multiple sampling points. That is, in the process of volume density conversion, the correction adjustment of the first signed distance parameter of each sampling point according to the corresponding angle of the sampling point is added, so as to avoid the error of the signed distance parameter of the sampling point caused by the angle of the observation ray, realize the unbiased conversion of the volume density parameter, and make the volume density conversion of the sampling point as close as possible to the real surface of the object, thereby improving the training accuracy of the target surface distance extraction network, and then improving the reconstruction accuracy of the surface data of the three-dimensional model, avoiding the distortion of the model surface, and improving the completeness of the model details.
[0205] When the computer device runs the three-dimensional reconstruction method provided in the above embodiment, the processor 1201 communicates with the storage medium 1202 via the bus 1203, and the processor 1201 executes the machine-readable instructions, the preamble of the method item of the processor 1201, to perform the following steps:
[0206] Get the bounding box of the target object;
[0207] Inputting a plurality of sampling points within the bounding box into a trained surface distance extraction network to obtain a target symbol distance parameter; wherein the surface extraction network is a target surface distance extraction network obtained by using the data processing method described in any of the above embodiments;
[0208] According to the target symbol distance parameter, a preset surface extraction algorithm is used to perform three-dimensional reconstruction of the target object to obtain the three-dimensional surface data of the target object.
[0209] In the above manner, since the target surface distance extraction network is used in the three-dimensional reconstruction process, and since the volume density parameters of each sampling point used in the network training process are obtained by unbiased transformation based on the signed distance parameters of each sampling point and the angle corresponding to each sampling point, the training accuracy of the model is better. Therefore, the reconstruction accuracy of the surface data of the three-dimensional model obtained by three-dimensional reconstruction using the target surface distance extraction network is also higher, which effectively avoids the distortion of the model surface and improves the completeness of the model details.
[0210] The embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. The computer program is executed when a processor is running, and the processor performs the following steps:
[0211] Get the bounding box of the target object;
[0212] Sampling the three-dimensional space within the bounding box to obtain multiple sampling points;
[0213] Using a preset initial surface distance extraction network, extract parameters of multiple sampling points to obtain parameter extraction results of the multiple sampling points, wherein the parameter extraction result of each sampling point includes: a first signed distance parameter and a normal vector of each sampling point;
[0214] Determine the angle between the normal vector of each sampling point and the observation ray of each sampling point;
[0215] According to the first signed distance parameters of the multiple sampling points and the angles corresponding to the multiple sampling points, volume density conversion is performed to obtain volume density parameters of the multiple sampling points. The volume density parameters are used to train the initial surface distance extraction network to obtain the target surface distance extraction network.
[0216] In a feasible implementation manner, when the processor performs volume density conversion according to the first signed distance parameters of the plurality of sampling points and the angles corresponding to the plurality of sampling points to obtain the volume density parameters of the plurality of sampling points, it is specifically used to:
[0217] According to the cosine value of the angle, the weight coefficient of each sampling point is determined;
[0218] According to the weight coefficient of each sampling point, the first signed distance parameter of each sampling point is weighted to obtain the second signed distance parameter of each sampling point;
[0219] According to the second signed distance parameter of each sampling point, volume density conversion is performed to obtain the volume density parameter of each sampling point.
[0220] In a feasible implementation manner, when the processor determines the weight coefficient of each sampling point according to the cosine value of the angle, it is specifically used to:
[0221] The reciprocal of the absolute value of the cosine value is determined as the weight coefficient of each sampling point.
[0222] In a feasible implementation manner, after obtaining the volume density parameters of the plurality of sampling points, the processor is further configured to:
[0223] Performing volume rendering according to volume density parameters of multiple sampling points and rendering colors of multiple sampling points along corresponding observation rays to obtain rendering colors of multiple observation rays;
[0224] According to the rendering colors of the multiple observation rays and the true value colors of the pixel positions corresponding to the multiple observation rays in the two-dimensional image of the target object, the initial surface distance extraction network is trained to obtain the target surface distance extraction network.
[0225] In a feasible implementation, the processor, when executing training of the initial surface distance extraction network according to the rendering colors of the plurality of observation rays and the true value colors of the pixel positions corresponding to the plurality of observation rays in the two-dimensional image of the target object to obtain the target surface distance extraction network, is specifically used to:
[0226] Calculate the model loss function value according to the rendering colors of the multiple observation rays and the true value colors of the pixel positions corresponding to the multiple observation rays in the two-dimensional image of the target object;
[0227] According to the model loss function value, the sampling gradient descent method is used to adjust the parameters of the initial surface distance extraction network, and the model is retrained based on the initial surface distance extraction network after parameter adjustment until the preset stop iteration condition is met to obtain the target surface distance extraction network.
[0228] In a feasible implementation, when executing the acquisition of the bounding box of the target object, the processor is specifically used to:
[0229] Acquire multiple two-dimensional images of the target object at multiple viewing angles;
[0230] Determine information of a three-dimensional point cloud and a bounding box of a target object under multiple viewing angles according to the multiple two-dimensional images;
[0231] Construct the bounding box of the target object based on the bounding box information.
[0232] In a feasible implementation, when the processor determines the information of the bounding box of the three-dimensional point cloud and the target object under multiple perspectives according to the multiple two-dimensional images, it is specifically used to:
[0233] Determine initial camera parameters and three-dimensional point clouds under multiple viewing angles according to the multiple two-dimensional images, wherein the initial camera parameters include: an initial camera position;
[0234] According to the initial camera position and the 3D point cloud at each viewing angle, the target camera position at each viewing angle is determined;
[0235] According to the target camera positions under multiple viewing angles, the information of the bounding box of the target object is determined.
[0236] In a feasible implementation, the initial camera parameters further include: a camera rotation matrix; when the processor determines the target camera position at each viewing angle according to the initial camera position and the three-dimensional point cloud at each viewing angle, the processor is specifically used to:
[0237] Determine the minimum depth value between the 3D point cloud and the initial camera position at each viewing angle;
[0238] Determine the main sight direction at each perspective based on the initial camera position and camera rotation matrix at each perspective;
[0239] The initial camera position at each viewing angle is translated by the minimum depth value along the main line of sight of each viewing angle to obtain the target camera position of each viewing angle.
[0240] In a feasible implementation, when the processor determines the information of the bounding box of the target object according to the target camera position under multiple viewing angles, it is specifically used to:
[0241] Intersecting the main sight lines under the multiple viewing angles to determine the center position of the target object;
[0242] Determine a plurality of distance parameters according to the target camera position and the center position under the plurality of viewing angles, wherein the plurality of distance parameters are used to represent the distance between the target camera position and the center position under the plurality of viewing angles respectively;
[0243] The radius parameter of the target object is determined from the multiple distance parameters; the information of the bounding box includes: a center position and a radius parameter.
[0244] In a feasible implementation manner, when the processor determines the radius parameter of the target object from a plurality of distance parameters, the processor is specifically configured to:
[0245] A radius parameter of the target object is determined according to a maximum distance parameter among the multiple distance parameters.
[0246] In a feasible implementation manner, before the processor performs volume rendering according to the volume density parameters of the plurality of sampling points and the rendering colors of the plurality of sampling points along the corresponding observation rays to obtain the rendering colors of the plurality of observation rays, it is further configured to:
[0247] According to the information of the multiple sampling points and the observation rays of the multiple sampling points, a preset initial color network is used to perform color rendering on the multiple sampling points to obtain the rendering colors of the multiple sampling points along the corresponding observation rays.
[0248] Through the above method, by obtaining the bounding box of the target object, sampling the three-dimensional space within the bounding box, obtaining multiple sampling points, and using a preset initial surface distance extraction network to extract parameters of the multiple sampling points, obtaining first signed distance parameters and normal vectors of the multiple sampling points, and determining the angle between the normal vector of each sampling point and the observation ray of each sampling point, and then performing volume density conversion according to the first signed distance parameters of the multiple sampling points and the angles corresponding to the multiple sampling points, obtaining volume density parameters of the multiple sampling points, which are used to train the initial surface distance extraction network to obtain the target surface distance extraction network. The data processing device provided in the embodiment of the present application can perform volume density conversion according to the normal vectors of the multiple sampling points and the angle between the normal vector of each sampling point and the observation ray of each sampling point during the process of volume density conversion of the first signed distance parameters of the multiple sampling points. That is, in the process of volume density conversion, the correction adjustment of the first signed distance parameter of each sampling point according to the corresponding angle of the sampling point is added, so as to avoid the error of the signed distance parameter of the sampling point caused by the angle of the observation ray, realize the unbiased conversion of the volume density parameter, and make the volume density conversion of the sampling point as close as possible to the real surface of the object, thereby improving the training accuracy of the target surface distance extraction network, and then improving the reconstruction accuracy of the surface data of the three-dimensional model, avoiding the distortion of the model surface, and improving the completeness of the model details.
[0249] The embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. The computer program is executed when a processor is running, and the processor performs the following steps:
[0250] Get the bounding box of the target object;
[0251] Inputting a plurality of sampling points within the bounding box into a trained surface distance extraction network to obtain a target symbol distance parameter; wherein the surface extraction network is a target surface distance extraction network obtained by using the data processing method described in any of the above embodiments;
[0252] According to the target symbol distance parameter, a preset surface extraction algorithm is used to perform three-dimensional reconstruction of the target object to obtain the three-dimensional surface data of the target object.
[0253] In the above manner, since the target surface distance extraction network is used in the three-dimensional reconstruction process, and since the volume density parameters of each sampling point used in the network training process are obtained by unbiased transformation based on the signed distance parameters of each sampling point and the angle corresponding to each sampling point, the training accuracy of the model is better. Therefore, the reconstruction accuracy of the surface data of the three-dimensional model obtained by three-dimensional reconstruction using the target surface distance extraction network is also higher, which effectively avoids the distortion of the model surface and improves the completeness of the model details.
[0254] In the embodiment of the present application, the computer program can also execute other machine-readable instructions when run by the processor to execute other methods described in the embodiment. For the specific execution method steps and principles, please refer to the description of the embodiment, which will not be repeated here.
[0255] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.
[0256] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0257] In addition, each functional unit in the embodiments provided in the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0258] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0259] It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description and are not to be understood as indicating or implying relative importance.
[0260] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application. They should all be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.
Claims
1. A data processing method, characterized in that: include: Get the bounding box of the target object; Sampling the three-dimensional space within the bounding box to obtain a plurality of sampling points; Using a preset initial surface distance extraction network, extracting parameters of the plurality of sampling points to obtain parameter extraction results of the plurality of sampling points, wherein the parameter extraction result of each sampling point includes: a first signed distance parameter and a normal vector of each sampling point; Determine the angle between the normal vector of each sampling point and the observation ray of each sampling point; Performing volume density conversion according to the first signed distance parameters of the plurality of sampling points and the angles corresponding to the plurality of sampling points to obtain volume density parameters of the plurality of sampling points, wherein the volume density parameters are used to train the initial surface distance extraction network to obtain a target surface distance extraction network; The step of performing volume density conversion according to the first signed distance parameters of the plurality of sampling points and the angles corresponding to the plurality of sampling points to obtain the volume density parameters of the plurality of sampling points includes: Weighting the first signed distance parameters of the plurality of sampling points respectively according to the included angles corresponding to the plurality of sampling points to obtain second signed distance parameters of the plurality of sampling points; Volume density conversion is performed according to the second signed distance parameters of the plurality of sampling points to obtain volume density parameters of the plurality of sampling points.
2. The method according to claim 1, characterized in that The step of weighting the first signed distance parameters of the plurality of sampling points respectively according to the angles corresponding to the plurality of sampling points to obtain the second signed distance parameters of the plurality of sampling points includes: Determining a weight coefficient of each sampling point according to the cosine value of the angle; The first signed distance parameter of each sampling point is weighted according to the weight coefficient of each sampling point to obtain the second signed distance parameter of each sampling point.
3. The method according to claim 2, characterized in that Determining the weight coefficient of each sampling point according to the cosine value of the angle includes: The reciprocal of the absolute value of the cosine value is determined as the weight coefficient of each sampling point.
4. The method according to claim 1, characterized in that: After obtaining the volume density parameters of the plurality of sampling points, the method further includes: Performing volume rendering according to the volume density parameters of the plurality of sampling points and the rendering colors of the plurality of sampling points along corresponding observation rays to obtain rendering colors of the plurality of observation rays; The initial surface distance extraction network is trained according to the rendering colors of the multiple observation rays and the true value colors of the pixel positions corresponding to the multiple observation rays in the two-dimensional image of the target object to obtain the target surface distance extraction network.
5. The method according to claim 4, characterized in that The initial surface distance extraction network is trained according to the rendering colors of the multiple observation rays and the true value colors of the pixel positions corresponding to the multiple observation rays in the two-dimensional image of the target object to obtain the target surface distance extraction network, including: Calculating a model loss function value according to the rendering colors of the plurality of observation rays and the true value colors of the pixel positions corresponding to the plurality of observation rays in the two-dimensional image of the target object; According to the model loss function value, the sampling gradient descent method is used to adjust the parameters of the initial surface distance extraction network, and the model is retrained according to the initial surface distance extraction network after the parameter adjustment until the preset stop iteration condition is met to obtain the target surface distance extraction network.
6. The method according to claim 1, characterized in that The step of obtaining a bounding box of a target object includes: Acquire multiple two-dimensional images of the target object at multiple viewing angles; Determine, based on the multiple two-dimensional images, information of the three-dimensional point clouds and the bounding box of the target object under the multiple viewing angles; A bounding box of the target object is constructed according to the information of the bounding box.
7. The method according to claim 6, characterized in that The step of determining, based on the multiple two-dimensional images, information about the three-dimensional point clouds and the bounding box of the target object under the multiple viewing angles includes: Determine initial camera parameters and three-dimensional point clouds under the multiple viewing angles according to the multiple two-dimensional images, wherein the initial camera parameters include: an initial camera position; Determine the target camera position at each viewing angle according to the initial camera position and the three-dimensional point cloud at each viewing angle; According to the target camera positions under the multiple viewing angles, information about the bounding box of the target object is determined.
8. The method according to claim 7, characterized in that The initial camera parameters also include: a camera rotation matrix; the target camera position at each viewing angle is determined according to the initial camera position and the three-dimensional point cloud at each viewing angle, including: Determine the minimum depth value between the three-dimensional point cloud at each viewing angle and the initial camera position; Determine the main sight direction at each viewing angle according to the initial camera position and the camera rotation matrix at each viewing angle; The initial camera position at each viewing angle is translated by the minimum depth value along the main line of sight direction of each viewing angle to obtain the target camera position of each viewing angle.
9. The method according to claim 7, characterized in that: The determining, according to the target camera positions under the multiple viewing angles, information of the bounding box of the target object comprises: Intersecting the main sight lines under the multiple viewing angles to determine the center position of the target object; Determine a plurality of distance parameters according to the target camera positions under the plurality of viewing angles and the center position, wherein the plurality of distance parameters are respectively used to represent the distances between the target camera positions under the plurality of viewing angles and the center position; The radius parameter of the target object is determined from the multiple distance parameters; the information of the bounding box includes: the center position and the radius parameter.
10. The method according to claim 9, characterized in that Determining the radius parameter of the target object from the multiple distance parameters includes: A radius parameter of the target object is determined according to a maximum distance parameter among the multiple distance parameters.
11. The method according to claim 4, characterized in that Before performing volume rendering according to the volume density parameters of the plurality of sampling points and the rendering colors of the plurality of sampling points along corresponding observation rays to obtain the rendering colors of the plurality of observation rays, the method comprises: According to the multiple sampling points and the information of the observation rays of the multiple sampling points, a preset initial color network is used to perform color rendering on the multiple sampling points to obtain the rendering colors of the multiple sampling points along the corresponding observation rays.
12. A three-dimensional reconstruction method, characterized in that: The method comprises: Get the bounding box of the target object; Inputting a plurality of sampling points within the bounding box into a trained surface distance extraction network to obtain a target symbol distance parameter; wherein the surface distance extraction network is a target surface distance extraction network obtained by using the data processing method according to any one of claims 1 to 11; According to the target symbol distance parameter, a preset surface extraction algorithm is used to perform three-dimensional reconstruction on the target object to obtain three-dimensional surface data of the target object.
13. A data processing device, characterized in that: include: A first acquisition module is used to acquire a bounding box of a target object; A sampling module, used for sampling the three-dimensional space within the bounding box to obtain a plurality of sampling points; A parameter extraction module, used to use a preset initial surface distance extraction network to perform parameter extraction on multiple sampling points to obtain parameter extraction results of the multiple sampling points, wherein the parameter extraction result of each sampling point includes: a first signed distance parameter and a normal vector of each sampling point; A determination module, used to determine the angle between the normal vector of each sampling point and the observation ray of each sampling point; A conversion module, configured to perform volume density conversion according to the first signed distance parameters of the plurality of sampling points and the angles corresponding to the plurality of sampling points, to obtain volume density parameters of the plurality of sampling points, wherein the volume density parameters are used to train the initial surface distance extraction network to obtain a target surface distance extraction network; The conversion module is specifically configured to weight the first signed distance parameters of the plurality of sampling points according to the angles corresponding to the plurality of sampling points to obtain the second signed distance parameters of the plurality of sampling points; and perform volume density conversion according to the second signed distance parameters of the plurality of sampling points to obtain the volume density parameters of the plurality of sampling points.
14. A three-dimensional reconstruction device, characterized in that: include: A second acquisition module is used to acquire a bounding box of a target object; An input module, used to input a plurality of sampling points in the bounding box into a trained surface distance extraction network to obtain a target symbol distance parameter; wherein the surface distance extraction network is a target surface distance extraction network obtained by the data processing method according to any one of claims 1 to 11; The reconstruction module is used to perform three-dimensional reconstruction on the target object according to the target symbol distance parameter by using a preset surface extraction algorithm to obtain three-dimensional surface data of the target object.
15. A computer device, characterized in that: include: A processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the computer device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the data processing method according to any one of claims 1 to 11 or the three-dimensional reconstruction method according to claim 12.
16. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the data processing method according to any one of claims 1 to 11 or the three-dimensional reconstruction method according to claim 12 are executed.
