Three-dimensional image reconstruction method using depth information

Abstract

The invention relates to the technical field of image processing, in particular to a three-dimensional image reconstruction method utilizing depth information, which comprises the following steps of: acquiring the depth information of a target scene, generating a depth map based on the depth information of the target scene, and acquiring the value of each pixel point in the depth map corresponding to the depth value of the pixel point from a camera in a real scene; in the target scene depth information acquisition stage, a target scene space model is acquired according to a target scene boundary coordinate, effective three-dimensional scene model modeling service is finely brought to a depth map through a 3D Gaussian Splitting scene rapid modeling technology, and in the method execution stage, the depth information of the target scene is acquired through the 3D Gaussian Splitting scene rapid modeling technology. The depth information sampling point density is designed by considering the complexity degree of the target scene space, the modeling precision of the three-dimensional scene model is effectively improved, and the precision of the three-dimensional scene model constructed by the method is improved more comprehensively by carrying out denoising processing on the depth map in advance and smoothing processing after the three-dimensional scene model is constructed.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a three-dimensional image reconstruction method using depth information. Background Art

[0002] 3D scene reconstruction is the process of converting 2D images or point cloud data into 3D models. Data is collected through cameras, lidar, etc., and point clouds are constructed through feature extraction, matching, and depth calculation. Then, a mesh model is generated using triangulation, surface reconstruction, and other technologies. Finally, texture mapping is used to give the model a realistic appearance. It is widely used in games, architecture, and other fields.

[0003] The Chinese invention patent with application number 201510979644.4 discloses an image reconstruction method using a three-dimensional image information acquisition device. The three-dimensional image information acquisition device includes a mirror, a first area is provided in the middle of the mirror, a first camera is provided in the middle of the first area, a first light source is provided on the mirror, the number of the first light sources is more than four, and they are distributed on the outer edge of the first area, and the first camera is used to collect the light signal emitted by the first light source; the first light source and the first camera are both connected to a control unit, and the control unit is used to control each first light source to switch on and off at a predetermined time and control the first camera to work within a corresponding time; a second camera is also provided in the middle of the first area, and more than one second light source is provided on the outer edge of the first area, and the second camera is used to collect the light signal emitted by the second light source; the first light source is an infrared LED lamp, and the first camera is an infrared camera; the second light source is a white light LED lamp, and the second camera is a visible light camera. The image reconstruction method includes: calibrating the inherent parameters of the first camera, and estimating the direction of the main optical axis of the first light source, so as to obtain the coordinates of the first light source; according to the inherent parameters of the first light source, establishing a light emitting coordinate model.

[0004] This application aims to solve the problem that "existing photometric 3D reconstruction methods all use the assumption of point light source or parallel light source, that is, they do not consider the luminous characteristics of the light source itself. When the object is far away from the light source or the light source itself conforms to the characteristics of parallel light or point light source, the existing methods can achieve better 3D reconstruction effects. However, this has high requirements on the light source itself, and in order to obtain parallel light, the light source itself needs to be larger in size, such as using an LED array device; it cannot achieve close-range 3D image reconstruction, the overall size of the device is large, the production cost is high, and it is not convenient to use."

[0005] However, in the above-mentioned 3D image reconstruction scenario, there is no refined method for 3D image reconstruction through 3D Gaussian Splatting scene rapid modeling technology.

[0006] Therefore, a 3D image reconstruction method using depth information is proposed. Summary of the invention

[0007] In view of the above-mentioned shortcomings of the prior art, the present invention provides a three-dimensional image reconstruction method using depth information, which solves the technical problems raised in the above-mentioned background technology.

[0008] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0009] A three-dimensional image reconstruction method using depth information, comprising:

[0010] Collect the depth information of the target scene, generate a depth map based on the depth information of the target scene, and obtain the value of each pixel in the depth map corresponding to its depth value from the camera in the real scene; in the target scene depth information collection stage, obtain the target scene space model according to the target scene boundary coordinates, and decide the target scene depth information collection point based on the complexity of the target scene space model. The collection perspective of the target scene depth information in the collection stage is customized by the user end and is applied once at each collection point;

[0011] The complexity of the target scene space model is expressed as:

[0012]

[0013] Where: C is the complexity of the target scene space model; N faces N is the number of faces actually contained in the target scene space model; ref is the number of reference surfaces; N edges N is the number of edges actually possessed by the target scene space model; ref-edges is the number of reference edges; N patches is the total number of patches after discretization; k i is the average curvature of the i-th patch after discretization of the target scene space model surface; S ref-curvature is the sum of the curvature changes of the reference model surface; N high N is the number of features presented by the target scene space model at high resolution; low N is the number of features that the target scene spatial model can distinguish at low resolution; ref-detail is the difference in the number of features of the reference model under the same high and low resolution comparison, ω 1 ,ω 2 ,ω 3 ,ω 4 is the weight, ω 1 ,ω 2 ,ω 3 ,ω 4All are positive numbers and their sum is 1;

[0014] Acquire a depth map, perform denoising on the depth map, and transform the depth map from the camera coordinate system to the world coordinate system according to the internal and external parameters of the camera; acquire the depth map after the coordinate system transformation, and transform it into three-dimensional point cloud data; associate an initial three-dimensional Gaussian distribution with each point in the three-dimensional point cloud data, and determine the mean vector, covariance matrix, and amplitude parameter of each Gaussian distribution; identify the range and influence of each Gaussian distribution in space to determine the weight and spatial range of each three-dimensional Gaussian distribution; perform splatting operations on each three-dimensional Gaussian distribution according to its weight and spatial range to generate a visual three-dimensional scene model, further render the three-dimensional scene model, and synchronously perform smoothing on the rendered three-dimensional scene model to finally output the reconstructed three-dimensional scene model.

[0015] Furthermore, the target scene depth information includes: distance information between the object and the camera, spatial position coordinates of the object, the geometric shape of the object's outline, and the terrain size of the scene. The target scene depth information is collected by any group of a binocular vision depth camera, a structured light depth camera, a ToF camera, and a light field camera. The depth map generated based on the target scene depth information is output by the camera used for depth information collection, and the value of each pixel in the depth map corresponds to its depth value from the camera in the real scene and is obtained synchronously with the camera used for depth information collection.

[0016] Furthermore, the reference model is a rectangular model or a cube model customized by the user, the feature number of the target scene space model is the number of concave and convex corners on the surface of the target scene space model, the ratio of the complexity C of the target scene space model customized by the user and the number of target scene depth information collection points is customized by the user, the product of the complexity C of the target scene space model and the ratio is the number of target scene depth information collection points, and the corresponding number of target scene depth information collection points are evenly distributed in the target scene space model, so that the adjacent spacing between each target scene depth information collection point is equal.

[0017] Furthermore, the denoising logic of the depth map is expressed as:

[0018]

[0019] Where: D filtered (i, j) is the depth value of the pixel with coordinates (i, j) in the depth map after Gaussian filtering; k and l are the sizes of the filter window; G(m, n) is the Gaussian function value at the corresponding coordinate (m, n); D(i+m, j+n) is the original depth value of the neighborhood pixel of the coordinate (i, j) defined by the filter window;

[0020] in, σ is the standard deviation of the Gaussian function.

[0021] Furthermore, the camera internal and external parameters include focal length, optical center position, distortion coefficient, rotation matrix, and translation vector.

[0022] Furthermore, the operation of converting the depth map into 3D point cloud data is:

[0023] Take each pixel in the depth map as a processing target;

[0024]

[0025] Where: (X w , Y w , Z w ) is the three-dimensional coordinate of the pixel in the world coordinate system; (X c , Y c , Z c ) is the three-dimensional coordinate of the pixel in the camera coordinate system; R and T are the rotation matrix and translation matrix respectively;

[0026] Among them, after the three-dimensional coordinates of each pixel in the world coordinate system are obtained, the PointCloud class in the point cloud library is used to store the three-dimensional coordinates to obtain three-dimensional point cloud data.

[0027] Furthermore, the initial three-dimensional Gaussian distribution of the point association in the three-dimensional point cloud data is:

[0028] The points in the 3D point cloud data include P w =(x w ,y w ,z w ), current point P w The coordinates of are directly used as the mean vector μ of the three-dimensional Gaussian distribution associated with it w =(x w ,y w ,z w );

[0029] The covariance matrix is ​​expressed as:

[0030] Among them, σ x , σ y , σ z Represents the standard deviation in the direction of the corresponding coordinate axis;

[0031] The amplitude parameter is expressed as: a w =ν×r w , ν is the scaling factor set by the user based on experience; r w For point P w The ratio of the point density in the neighborhood to the average point density of the entire point cloud.

[0032] Furthermore, when the range of the Gaussian distribution in space is identified, the equal probability contour of the three-dimensional Gaussian distribution is calculated according to the covariance matrix, and the calculation result is used to represent the range of the effective area of ​​the three-dimensional Gaussian distribution in space;

[0033] When identifying the influence of the three-dimensional Gaussian distribution in space, the weight of the three-dimensional Gaussian distribution in rendering or fusion is set according to the amplitude parameter, and the setting result is used to represent the influence of the three-dimensional Gaussian distribution in space;

[0034] in,

[0035] Where: w is the weight of the three-dimensional Gaussian distribution during rendering or fusion; A w is the amplitude parameter of the wth three-dimensional Gaussian distribution; G is the total number of three-dimensional Gaussian distributions.

[0036] Furthermore, when rendering the three-dimensional scene model, the color corresponding to each position in the target scene is applied for rendering, and when smoothing is performed on the rendered three-dimensional scene model, smoothing is performed based on any one of the Laplace smoothing algorithm, the Taubin smoothing algorithm, and the mean curvature smoothing algorithm.

[0037] Furthermore, in the three-dimensional scene model output stage, the average distance error and root mean square error between the three-dimensional scene model and the real scene are obtained, an error tolerance range is set, and the obtained result is compared with the error tolerance range. When the obtained result meets the error tolerance range, the output three-dimensional scene model is determined to be valid. When the obtained result does not meet the error tolerance range, the output three-dimensional scene model is determined to be invalid, and the reconstruction operation of the three-dimensional scene model corresponding to the target scene is re-executed;

[0038] Among them, during the reconstruction process of the three-dimensional scene model corresponding to the target scene, the number of target scene depth information collection points used is greater than the number of target scene depth information collection points used in the previous three-dimensional scene model reconstruction operation.

[0039] Compared with the known public technology, the technical solution provided by the present invention has the following beneficial effects:

[0040] The present invention provides a 3D image reconstruction method using depth information. During the execution of the method, the method provides an effective 3D scene modeling service for the depth map through the 3D Gaussian Splatting scene rapid modeling technology. During the execution stage of the method, the density of depth information sampling points is designed considering the complexity of the target scene space, which effectively improves the modeling accuracy of the 3D scene model. The depth map is subjected to prior denoising processing, and the 3D scene model is subjected to smoothing processing after construction, which more comprehensively improves the accuracy of the 3D scene model constructed by the method. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0042] Figure 1 A flowchart of a three-dimensional image reconstruction method using depth information is shown. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] The present invention will be further described below in conjunction with the embodiments.

[0045] Embodiment 1:

[0046] A three-dimensional image reconstruction method using depth information in this embodiment is as follows: Figure 1 As shown, including:

[0047] Collect the depth information of the target scene, generate a depth map based on the depth information of the target scene, and obtain the value of each pixel in the depth map corresponding to its depth value from the camera in the real scene;

[0048] In the target scene depth information collection stage, the target scene space model is obtained according to the target scene boundary coordinates, and the target scene depth information collection points are determined based on the complexity of the target scene space model. The collection perspective of the target scene depth information in the collection stage is customized by the user end and is applied once at each collection point;

[0049] The complexity of the target scene space model is expressed as:

[0050]

[0051] Where: C is the complexity of the target scene space model; N faces N is the number of faces actually contained in the target scene space model; ref is the number of reference surfaces; N edges N is the number of edges actually possessed by the target scene space model; ref-edges is the number of reference edges; N patches is the total number of patches after discretization; k i is the average curvature of the i-th patch after discretization of the target scene space model surface; S ref-curvature is the sum of the curvature changes of the reference model surface; N high N is the number of features presented by the target scene space model at high resolution; low N is the number of features that the target scene spatial model can distinguish at low resolution; ref-detail is the difference in the number of features of the reference model under the same high and low resolution comparison, ω 1 ,ω 2 ,ω 3 ,ω 4 is the weight, ω 1 ,ω 2 ,ω 3 ,ω 4 All are positive numbers and their sum is 1;

[0052] Wherein, the reference model is a cuboid model or a cube model customized by the user end, the feature number of the target scene space model is the number of concave and convex corners on the surface of the target scene space model, the ratio of the complexity C of the target scene space model customized by the user end to the number of target scene depth information collection points is customized by the user end, the product of the complexity C of the target scene space model and the ratio is the number of target scene depth information collection points, and the corresponding number of target scene depth information collection points are evenly distributed in the target scene space model, so that the adjacent spacings of each target scene depth information collection point are equal;

[0053] The complexity of the target scene spatial model is calculated through the above logical formula, and then used as a reference to design the density of the collection points of the target scene depth information, ensuring that the construction of the target scene three-dimensional model has complete data support.

[0054] Get the depth map, perform denoising on the depth map, and convert the depth map from the camera coordinate system to the world coordinate system based on the internal and external parameters of the camera;

[0055] The denoising logic of the depth map is expressed as:

[0056]

[0057] Where: D filtered (i, j) is the depth value of the pixel with coordinates (i, j) in the depth map after Gaussian filtering; k and l are the sizes of the filter window; G(m, n) is the Gaussian function value at the corresponding coordinate (m, n); D(i+m, j+n) is the original depth value of the neighborhood pixel of the coordinate (i, j) defined by the filter window;

[0058] in, σ is the standard deviation of the Gaussian function;

[0059] Through the above logical formula, the depth map is denoised, thereby improving the accuracy of the final construction of the three-dimensional scene model.

[0060] Obtain the depth map after coordinate system conversion and convert it into three-dimensional point cloud data;

[0061] The operation of converting the depth map into 3D point cloud data is:

[0062] Take each pixel in the depth map as a processing target;

[0063]

[0064] Where: (X w , Y w , Z w ) is the three-dimensional coordinate of the pixel in the world coordinate system; (X c , Y c , Z c ) is the three-dimensional coordinate of the pixel in the camera coordinate system; R and T are the rotation matrix and translation matrix respectively;

[0065] Among them, after the three-dimensional coordinates of each pixel in the world coordinate system are obtained, the PointCloud class in the point cloud library is used to store each three-dimensional coordinate to obtain three-dimensional point cloud data;

[0066] For each point in the three-dimensional point cloud data, an initial three-dimensional Gaussian distribution is associated with it, and the mean vector, covariance matrix and amplitude parameter of each Gaussian distribution are determined;

[0067] The initial three-dimensional Gaussian distribution of the point association in the three-dimensional point cloud data is:

[0068] The points in the 3D point cloud data include P w =(x w ,y w ,z w ), current point P w The coordinates of are directly used as the mean vector μ of the three-dimensional Gaussian distribution associated with itw =(x w ,y w ,z w );

[0069] The covariance matrix is ​​expressed as: Among them, σ x , σ y , σ z Represents the standard deviation in the direction of the corresponding coordinate axis;

[0070] The amplitude parameter is expressed as: a w =ν×r w , ν is the scaling factor set by the user based on experience; r w For point P w The ratio of the point density in the neighborhood to the average point density of the entire point cloud;

[0071] Identify the range and influence of each three-dimensional Gaussian distribution in space;

[0072] When identifying the range of the three-dimensional Gaussian distribution in space, the equal probability contour of the three-dimensional Gaussian distribution is calculated according to the covariance matrix, and the calculation result is used to represent the range of the action area of ​​the three-dimensional Gaussian distribution in space;

[0073] When identifying the influence of the three-dimensional Gaussian distribution in space, the weight of the three-dimensional Gaussian distribution in rendering or fusion is set according to the amplitude parameter, and the setting result is used to represent the influence of the three-dimensional Gaussian distribution in space;

[0074] in,

[0075] Where: w is the weight of the three-dimensional Gaussian distribution during rendering or fusion; A w is the amplitude parameter of the wth three-dimensional Gaussian distribution; G is the total number of three-dimensional Gaussian distributions;

[0076] Each three-dimensional Gaussian distribution is splatted according to its weight and spatial range to generate a visual three-dimensional scene model, which is then rendered. The rendered three-dimensional scene model is then smoothed and finally a reconstructed three-dimensional scene model is output.

[0077] In this embodiment, a new three-dimensional image reconstruction method is provided through the method in the above embodiment, and compared with the existing technology, the accuracy of this method is controllable, and the reconstruction process is relatively stable and reliable.

[0078] Embodiment 2:

[0079] In terms of specific implementation, based on Example 1, this example refers to Figure 1The following is a further detailed description of a three-dimensional image reconstruction method using depth information in Example 1:

[0080] The target scene depth information includes: the distance information between the object and the camera, the spatial position coordinates of the object, the geometric shape of the object's outline, and the terrain size of the scene. The target scene depth information is collected by any group of binocular vision depth cameras, structured light depth cameras, ToF cameras, and light field cameras. The depth map generated based on the target scene depth information is output by the camera used for depth information collection. The value of each pixel in the depth map corresponds to its depth value from the camera in the real scene and is obtained synchronously with the camera used for depth information collection.

[0081] Through the above settings, further execution data support is provided for the execution of the method in the above embodiment 1, ensuring the stable execution of the method in embodiment 1.

[0082] like Figure 1 As shown, the internal and external parameters of the camera include focal length, optical center position, distortion coefficient, rotation matrix, and translation vector.

[0083] Through the above settings, the contents of the internal and external parameters of the camera are further limited.

[0084] like Figure 1 As shown, when rendering the three-dimensional scene model, the color corresponding to each position in the target scene is applied for rendering, and when smoothing the rendered three-dimensional scene model, smoothing is performed based on any one of the Laplace smoothing algorithm, the Taubin smoothing algorithm, and the mean curvature smoothing algorithm;

[0085] In the 3D scene model output stage, the average distance error and root mean square error between the 3D scene model and the real scene are obtained, the error tolerance range is set, and the obtained result is compared with the error tolerance range. When the obtained result meets the error tolerance range, the output 3D scene model is determined to be valid. When the obtained result does not meet the error tolerance range, the output 3D scene model is determined to be invalid, and the reconstruction operation of the 3D scene model corresponding to the target scene is re-executed;

[0086] Among them, during the reconstruction process of the three-dimensional scene model corresponding to the target scene, the number of target scene depth information collection points used is greater than the number of target scene depth information collection points used in the previous three-dimensional scene model reconstruction operation.

[0087] Through the above formula, the three-dimensional scene model output by the method in Example 1 is further optimized, which effectively improves the quality of the three-dimensional scene model constructed by the method in Example 1.

[0088] In summary, during the execution of the method in the above embodiment, the 3D Gaussian Splatting scene rapid modeling technology is used to refine the depth map to provide an effective three-dimensional scene model modeling service. During the method execution phase, the density of depth information sampling points is designed considering the complexity of the target scene space, which effectively improves the modeling accuracy of the three-dimensional scene model. The depth map is denoised in advance, and the three-dimensional scene model is smoothed after construction, which more comprehensively improves the accuracy of constructing the three-dimensional scene model by this method.

[0089] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A three-dimensional image reconstruction method using depth information, characterized in that: include: Collect the depth information of the target scene, generate a depth map based on the depth information of the target scene, and obtain the value of each pixel in the depth map corresponding to its depth value from the camera in the real scene; Get the depth map, perform denoising on the depth map, and convert the depth map from the camera coordinate system to the world coordinate system based on the internal and external parameters of the camera; Obtain the depth map after coordinate system conversion and convert it into three-dimensional point cloud data; For each point in the three-dimensional point cloud data, an initial three-dimensional Gaussian distribution is associated with it, and the mean vector, covariance matrix and amplitude parameters of each three-dimensional Gaussian distribution are determined; Identify the range and influence of each three-dimensional Gaussian distribution in space to determine the weight and spatial range of each three-dimensional Gaussian distribution; Each three-dimensional Gaussian distribution is splatted according to its weight and spatial range to generate a visual three-dimensional scene model, which is then rendered. The rendered three-dimensional scene model is then smoothed synchronously to finally output a reconstructed three-dimensional scene model.

2. The three-dimensional image reconstruction method using depth information according to claim 1, characterized in that: The target scene depth information includes: distance information between the object and the camera, spatial position coordinates of the object, geometric shape of the object outline, and topographic size of the scene. The target scene depth information is collected by any one of a binocular vision depth camera, a structured light depth camera, a ToF camera, and a light field camera. The depth map generated based on the target scene depth information is output by the camera used for depth information collection. The value of each pixel in the depth map corresponds to its depth value from the camera in the real scene and is obtained synchronously with the camera used for depth information collection.

3. The three-dimensional image reconstruction method using depth information according to claim 1, characterized in that: In the target scene depth information acquisition stage, the target scene space model is obtained according to the target scene boundary coordinates, and the target scene depth information acquisition points are determined based on the complexity of the target scene space model. The acquisition perspective of the target scene depth information in the acquisition stage is customized by the user end and is applied once at each acquisition point; The complexity of the target scene space model is expressed as: Where: C is the complexity of the target scene space model; N faces N is the number of faces actually contained in the target scene space model; ref is the number of reference surfaces; N edges N is the number of edges actually possessed by the target scene space model; ref-edges is the number of reference edges; N patches is the total number of patches after discretization; k i is the average curvature of the i-th patch after discretization of the target scene space model surface; S ref-curvature is the sum of the curvature changes of the reference model surface; N high N is the number of features presented by the target scene space model at high resolution; low N is the number of features that the target scene spatial model can distinguish at low resolution; ref-detail is the difference in the number of features of the reference model under the same high and low resolution comparison, ω1, ω2, ω3, ω4 are weights, ω1, ω2, ω3, ω4 are all positive numbers and their sum is 1; Among them, the reference model is a rectangular model or a cube model customized by the user end, the feature number of the target scene space model is the number of concave and convex corners on the surface of the target scene space model, the ratio of the complexity C of the target scene space model customized by the user end to the number of target scene depth information collection points is customized by the user end, the product of the complexity C of the target scene space model and the ratio is the number of target scene depth information collection points, and the corresponding number of target scene depth information collection points are evenly distributed in the target scene space model, so that the adjacent spacing between each target scene depth information collection point is equal.

4. The three-dimensional image reconstruction method using depth information according to claim 1, characterized in that: The denoising logic of the depth map is expressed as: Where: D filtered (i, j) is the depth value of the pixel with coordinates (i, j) in the depth map after Gaussian filtering; k and l are the sizes of the filter window; G(m, n) is the Gaussian function value at the corresponding coordinate (m, n); D(i+m, j+n) is the original depth value of the neighborhood pixel of the coordinate (i, j) defined by the filter window; in, σ is the standard deviation of the Gaussian function.

5. The three-dimensional image reconstruction method using depth information according to claim 1, characterized in that: The camera internal and external parameters include focal length, optical center position, distortion coefficient, rotation matrix, and translation vector.

6. The three-dimensional image reconstruction method using depth information according to claim 1, characterized in that: The operation of converting the depth map into 3D point cloud data is: Take each pixel in the depth map as a processing target; Where: (X w , Y w , Z w ) is the three-dimensional coordinate of the pixel in the world coordinate system; (X c , Y c , Z c ) is the three-dimensional coordinate of the pixel in the camera coordinate system; R and T are the rotation matrix and translation matrix respectively; Among them, after the three-dimensional coordinates of each pixel in the world coordinate system are obtained, the PointCloud class in the point cloud library is used to store the three-dimensional coordinates to obtain three-dimensional point cloud data.

7. The three-dimensional image reconstruction method using depth information according to claim 1, characterized in that: The initial three-dimensional Gaussian distribution of the point association in the three-dimensional point cloud data is: The points in the 3D point cloud data include P w =(x w ,y w ,z w ), current point P w The coordinates of are directly used as the mean vector μ of the three-dimensional Gaussian distribution associated with it w =(x w ,y w ,z w ); The covariance matrix is ​​expressed as: Among them, σ x , σ y , σ z Represents the standard deviation in the direction of the corresponding coordinate axis; The amplitude parameter is expressed as: a w =ν×r w , ν is the scaling factor set by the user based on experience; r w For point P w The ratio of the point density in the neighborhood to the average point density of the entire point cloud.

8. The three-dimensional image reconstruction method using depth information according to claim 1, characterized in that: When identifying the range of the three-dimensional Gaussian distribution in space, the equal probability contour of the three-dimensional Gaussian distribution is calculated according to the covariance matrix, and the calculation result is used to represent the range of the action area of ​​the three-dimensional Gaussian distribution in space; When identifying the influence of the three-dimensional Gaussian distribution in space, the weight of the three-dimensional Gaussian distribution in rendering or fusion is set according to the amplitude parameter, and the setting result is used to represent the influence of the three-dimensional Gaussian distribution in space; in, Where: w is the weight of the three-dimensional Gaussian distribution during rendering or fusion; A w is the amplitude parameter of the wth three-dimensional Gaussian distribution; G is the total number of three-dimensional Gaussian distributions.

9. The three-dimensional image reconstruction method using depth information according to claim 1, characterized in that: When rendering the three-dimensional scene model, the color corresponding to each position in the target scene is used for rendering. When smoothing the rendered three-dimensional scene model, smoothing is performed based on any one of the Laplace smoothing algorithm, the Taubin smoothing algorithm, and the mean curvature smoothing algorithm.

10. The three-dimensional image reconstruction method using depth information according to claim 1, characterized in that: In the three-dimensional scene model output stage, the average distance error and root mean square error between the three-dimensional scene model and the real scene are obtained, an error tolerance range is set, and the obtained result is compared with the error tolerance range. When the obtained result meets the error tolerance range, the output three-dimensional scene model is determined to be valid. When the obtained result does not meet the error tolerance range, the output three-dimensional scene model is determined to be invalid, and the reconstruction operation of the three-dimensional scene model corresponding to the target scene is re-executed; Among them, during the reconstruction process of the three-dimensional scene model corresponding to the target scene, the number of target scene depth information collection points used is greater than the number of target scene depth information collection points used in the previous three-dimensional scene model reconstruction operation.

Images