Three-dimensional scene reconstruction method and system based on source view angle geometric constraint, computer equipment and medium

Abstract

The invention relates to a three-dimensional scene reconstruction method and system based on source view angle geometric constraint, computer equipment and a medium, and the method comprises the following steps: 1, constructing a three-dimensional scene geometric expression based on a source view angle, and achieving the standardization of the three-dimensional scene geometric expression; step 2, converting sampling points in the three-dimensional space according to the contribution degree of each source view angle to obtain depth distribution of the source view angle; and step 3, applying a loss function on the depth distribution of the source view angle obtained in the step 2, and realizing three-dimensional scene reconstruction based on the source view angle geometric constraint. The three-dimensional scene geometric expression constructed by the method better conforms to the geometric shape of a real scene, the generation effect of the three-dimensional scene reconstruction method based on the forward-pushing network on an unknown target view angle is obviously improved, and the consistency of the three-dimensional scene reconstruction method and a source view angle is improved.

Description

Technical Field

[0001] The present invention belongs to the field of computer three-dimensional scene reconstruction, and in particular relates to a three-dimensional scene reconstruction method, system, computer equipment and medium based on source viewpoint geometry constraints. Background Art

[0002] Existing scene reconstruction methods based on network optimization require separate model training for any new scene. However, scene reconstruction methods based on push-forward networks are different from those based on network optimization. After training, only network inference is required on any new scene to achieve scene reconstruction. Existing 3D scene reconstruction technology based on push-forward networks can achieve high-quality 3D scene reconstruction. However, due to the lack of supervision on how to identify the source perspective and the physical visual relationship between 3D sampling points in space during the training process, the 3D sampling point features cannot be constructed with reference to the correct source perspective, resulting in a large number of geometrically unreasonable appearances in the reconstructed 3D scene expressions.

[0003] Existing methods solve the above problems from two technical schools: First, by designing an adaptive perspective fusion module, the module can learn to select the correct source perspective through data training. This method simply fits the reconstructed target perspective result and lacks supervision of the source perspective, so that this type of perspective fusion module cannot well distinguish whether each source perspective can well observe each 3D sampling point; second, predict the visibility of each 3D sampling point with respect to the source perspective through a priori assumptions. This type of method has special requirements for the network structure, emphasizing a specific model structure design scheme, rather than a general method that can act on a class of networks.

[0004] Based on this, there is an urgent need for a three-dimensional scene reconstruction method, system, computer equipment and medium based on source perspective geometric constraints. By normalizing the geometric expression of the three-dimensional scene based on the forward network, each source perspective only contributes to the three-dimensional expression of the sampling points near the surface of the object it observes. This method constructs a geometric expression of the three-dimensional scene that is more consistent with the geometric shape of the real scene. Summary of the invention

[0005] In order to solve the above problems, the present invention proposes a three-dimensional scene reconstruction method, system, computer equipment and medium based on source perspective geometry constraints.

[0006] The present invention relates to a three-dimensional scene reconstruction method based on source viewpoint geometry constraints, comprising the following steps:

[0007] Step 1: construct a 3D scene geometric expression based on the source perspective to achieve the standardization of the 3D scene geometric expression;

[0008] Step 2: transform the sampling points in the three-dimensional space according to the contribution degree of each source perspective to obtain the depth distribution of the source perspective;

[0009] Step 3: Apply a loss function to the depth distribution of the source perspective obtained in step 2 to achieve three-dimensional scene reconstruction based on the geometric constraints of the source perspective.

[0010] Furthermore, in step 1, the method for constructing the geometric expression of the three-dimensional scene based on the source perspective is as follows:

[0011] A 3D sampling point p is mapped to N source view images The visibility is expressed as a weight vector w(p)=[w1(p),…,w N (p)] T ∈R 1×N , the weight vector is obtained by the weighted source view feature fusion module;

[0012] Among them, H, W represent the resolution of digital image I, R a×b represents the vector domain space of a×b dimensional real vectors, R represents the real number domain, i represents the image number, w i (p) represents the visibility of 3D sampling point p to source view i, N represents the number of source view images, 1≤i≤N;

[0013] A total of S 3D points are sampled for the scene to obtain the matrix W describing the geometric expression of the 3D scene. v =[w(p1),…,w(p S )]∈R N×S ;

[0014] Among them, w(p j )∈W v represents the jth sampling point p j The visibility weight vector, 1≤j≤S, S represents the number of sampled 3D points;

[0015] W v The contents are reorganized into a sequence and converted into a depth distribution W on each source perspective pixel d ∈R (H×W×N)×L ; Normalize these depth distributions to normalize the geometric expression of the three-dimensional scene constructed by the neural network;

[0016] Where L is the number of samples collected for a discrete depth distribution.

[0017] Furthermore, in step 2, the method for converting the visibility of the sampling points in the three-dimensional space is as follows:

[0018] Through bucket sorting and bucket exchange based on depth partitioning, W v Convert to W d, while ensuring W d The samples in each row are ordered according to the spatial position and the row length is constant; for each source view image I i Only K source pixel rays with the highest upsampling frequency are retained for supervision; finally, W v Convert to

[0019] Among them, N bin is a hyperparameter representing the bucket resolution in bucket sorting, K represents the number of source pixel rays, N represents the number of source view images, and R a×b represents the vector field space of a×b-dimensional real vectors, R represents the real field, W d W v A subset of rows.

[0020] Furthermore, in step 3, the method of applying the loss function to the depth distribution of the source perspective is as follows:

[0021] definition is a source pixel ray r of source view i s =o i +td s The sampling points on

[0022] Among them, 1≤i≤N, o i represents the viewpoint of source view i, d s Indicates the direction in which the viewpoint points to the pixel, t∈[a,b) is the line coordinate on the ray, a and b are r s The starting point and end point of the line coordinate interval that needs to be paid attention to, m is the number of sampling points, and N is the number of source view images;

[0023] The discrete depth distribution of source pixels is defined as:

[0024] Q(r s ):q(p1),q(p2),…,q(p m ) (1)

[0025] Among them, q(p k ) represents r s The surface hit falls on p k The probability of being at d Get from

[0026] Q(r s ) is normalized as a probability density function, that is, Q(r s ) is non-zero and sums to 1; ideally, when m→∞ is approximately continuous, Q(r s ) is a pulse distribution, and its pulse appears at r sAt the surface of the impact; in this case, for Q(r s ) is normalized by the unsupervised learning objective function proposed by Mip-NeRF 360;

[0027]

[0028] Among them, q u is q(p u ), t u For r s The line coordinates of the u-th sampling point on satisfy a=t0<… <t m =b,[t u-1 ,t u ) indicates that p u The interval represented by ; Assume that all samples are uniformly sampled, that is, t u =a+(ba)u / m;

[0029] Let the interval length l = t u -t u-1 =(ba) / m, Formula 2 is converted into the following form:

[0030]

[0031] Among them, ∑q u =1 is a constant; the optimal value of formula (3) is l / 3, which is obtained when and only when the last two terms are both 0, that is, Q(r s ) at most one q u Non-zero; since the sampling points will not fall exactly on the real surface, the optimal distribution of formula (3) will introduce a deviation between the real scene surface and the scene surface understood by the model. This convergence comes from the second term of formula (3). Removing the second term of formula (3) yields:

[0032]

[0033] Its optimal value is obtained when and only when there are at most two non-zero samples and the two samples are adjacent; in formula (4), α is a weight constant, corresponding to α=2 in formula (3);

[0034] By scaling the entire equation three times, formula (4) is transformed into:

[0035]

[0036] According to formula (4), as long as α>0, its optimal value point property will be maintained; taking α=1 to remove the last term of formula (5) for simplification, the result is For W d Normalization of Q(r s) is normalized to q u =q u / (∈+∑q u ), where ∈ is a very small positive real constant; by The optimization of Q(r s ) is optimized to 0.

[0037] The present invention also relates to a system for a three-dimensional scene reconstruction method based on source perspective geometric constraints, wherein the system comprises a computer module for running the three-dimensional scene reconstruction method based on source perspective geometric constraints.

[0038] The present invention also relates to a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned three-dimensional scene reconstruction method based on source perspective geometry constraints when executing the computer program.

[0039] The present invention also relates to a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned three-dimensional scene reconstruction method based on source perspective geometry constraints are implemented.

[0040] Beneficial Effects

[0041] The present invention proposes a three-dimensional scene reconstruction method, system, computer equipment and medium based on source perspective geometric constraints. The three-dimensional scene geometric expression constructed by this method is more consistent with the geometric shape of the real scene, and significantly improves the generation effect of the three-dimensional scene reconstruction method based on the forward network for unknown target perspectives, and improves its consistency with the source perspective. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a schematic diagram of the technical framework of the present invention.

[0043] Figure 2a It is a schematic diagram of unconstrained depth distribution under discrete depth distribution in the present invention.

[0044] Figure 2b It is a schematic diagram of depth distribution constrained by the existing normalization function under the condition of discrete depth distribution according to the present invention.

[0045] Figure 2c It is a schematic diagram of depth distribution constrained by the invented normalization function in the case of discrete depth distribution according to the present invention.

[0046] Figure 3a This is a comparison chart of geometric confidence before and after enhancement of the baseline method of the present invention.

[0047] Figure 3b The geometric confidence graphs of the baseline method of the present invention on different data sets before enhancement.

[0048] Figure 3c The geometric confidence graphs on different data sets after the baseline method of the present invention is enhanced.

[0049] Figure 4 This is a comparison chart of the rendering results before and after enhancement using the baseline method of the present invention. DETAILED DESCRIPTION

[0050] The following combination Figures 1 to 4 This embodiment will be described in detail.

[0051] In order to enhance the ability of existing push-forward network-based 3D scene reconstruction methods to discern the physical visual relationship between 3D sampling points and source perspectives, the present invention proposes an unsupervised method based on source perspective geometric constraints to constrain the depth distribution of each source perspective digital image pixel to be similar to the pulse distribution, so as to normalize the geometric representation of the 3D scene constructed by the push-forward network. The visibility of each source perspective to the 3D sampling point is described by using the weights in the weighted source perspective feature fusion module, and these weights are converted into discrete depth distributions on each source perspective pixel ray. By constraining the source perspective pixel depth distribution to a shape close to the pulse, the normalization of the geometric representation of the 3D scene constructed by the push-forward network is achieved, so that each source perspective only contributes to the 3D representation of the sampling points near the surface of the object it observes.

[0052] The present invention relates to a three-dimensional scene reconstruction method based on source viewpoint geometric constraints, comprising the following steps:

[0053] Step 1: Construct a three-dimensional scene geometric expression based on the source perspective to achieve the standardization of the three-dimensional scene geometric expression.

[0054] Figure 1 It is a schematic diagram of the technical framework of the present invention, such as Figure 1 As shown, in order to construct the geometric expression of the three-dimensional scene, the present invention first transforms a three-dimensional sampling point p to N source view images. The visibility is expressed as a weight vector w(p)=[w1(p),…,w N (p)] T ∈R 1×N , the weight vector is obtained by the weighted source view feature fusion module. Where H, W represents the resolution of the digital image I, R a×b represents the vector domain space of a×b dimensional real vectors, R represents the real number domain, i represents the image number, w i (p) represents the visibility of 3D sampling point p to source view i, N represents the number of source view images, 1≤i≤N.

[0055] By sampling S 3D points in total, we can get the matrix W describing the geometric expression of the 3D scene. v =[w(p1),…,w(p S )]∈R N×S Among them, w(p j )∈W v represents the jth sampling point p j The visibility weight vector, 1≤j≤S. v The contents are reorganized into a sequence and converted into a depth distribution W on each source perspective pixel d ∈R (H×W×N)×L After that, these depth distributions can be normalized, so that the geometric expression of the three-dimensional scene constructed by the IBR neural network is more standardized, where L is the number of samples collected for a discrete depth distribution.

[0056] In a network forward process, by simultaneously supervising the visual information obtained from the target perspective rendering and the geometric information from the source perspective, the network is supervised at different perspectives to enhance the network's ability to distinguish the physical visibility between the source perspective and the 3D sampling points.

[0057] Step 2: transform the sampling points in the three-dimensional space according to the contribution degree of each source perspective to obtain the depth distribution of the source perspective.

[0058] Step 2 is responsible for solving the problem of inconsistent number of 3D sampling points obtained based on target perspective pixel rays on each source perspective pixel, so as to make the subsequent calculation process friendly to the pytorch framework, and convert the visibility of the 3D sampling points with respect to the source perspective into depth distribution on each source perspective ray for use in step 3.

[0059] From W v To W d The difficulty of the conversion lies in the fact that the number of sampling points in the three-dimensional space on different source perspective pixel rays is different, that is, the sampling frequencies of different source pixel rays are different. The above problem causes L to be non-constant, that is, W d The length of each row is not constant, which makes it difficult for the GPU to d At the same time, how to make the sampling point information in three-dimensional space orderly according to the spatial position on each source pixel ray is also a major difficulty. In order to solve this problem, the present invention proposes a special conversion algorithm, which converts W by bucket sorting and bucket exchange based on depth division. v Convert to W d , while ensuring W d The samples in each row are ordered according to the spatial position and have a constant length. Considering that it is difficult to describe a depth distribution with too sparse sampling, the present invention performs the following steps for each source view image I iOnly K source pixel rays with the highest sampling frequency are retained for supervision. Finally, through the conversion algorithm proposed by the present invention, W v Successfully converted to GPU-friendly Among them, N bin is a hyperparameter representing the bucket resolution in bucket sorting, K represents the number of source pixel rays, N represents the number of source view images, and R a×b represents the vector field space of a×b-dimensional real vectors, R represents the real field, W d W v A subset of rows.

[0060] The present invention uses W d As a mathematical representation of the geometric expression of the 3D scene to be supervised, and using the normalized constraint W on the discrete depth distribution d , thus reflecting the multi-perspective three-dimensional context in the supervision process.

[0061] Step 3: Apply a loss function to the depth distribution of the source perspective obtained in step 2 to achieve three-dimensional scene reconstruction based on the geometric constraints of the source perspective.

[0062] The loss function in step three aims to solve the discrete error problem between discrete sampling points and the continuous real three-dimensional space, and alleviate the deviation between the discrete three-dimensional scene expression understood by the network during the learning process and the real continuous physical world.

[0063] definition is a source pixel ray r of source view angle i (1≤i≤N) s =o i +td s The sampling points on the i represents the viewpoint of source view i, d s Indicates the direction in which the viewpoint points to the pixel, t∈[a,b) is the line coordinate on the ray, a and b are r s The starting point and end point of the line coordinate interval that needs to be paid attention to, m is the number of sampling points, and N is the number of source view images;

[0064] The discrete depth distribution of source pixels is defined as:

[0065] Q(r s ):q(p1),q(p2),…,q(p m ) (6)

[0066] Among them, q(p k ) represents r s The surface hit falls on p k The probability at which d Q(r s) is normalized as a probability density function, that is, Q(r s ) are all non-zero and sum to 1.

[0067] Ideally, when m→∞ is approximately continuous, Q(r s ) should be a pulse distribution, and its pulse should appear exactly at r s In this case, for Q(r s ) can be simply implemented by the unsupervised learning objective function proposed by Mip-NeRF 360,

[0068]

[0069] Among them, q u is q(p u ), t u For r s The line coordinates of the u-th sampling point on satisfy a=t0<… <t m =b,[t u-1 ,t u ) indicates that p u Represents the interval. Since L dist The core purpose of constraining the pulse to appear near a certain sampling point, without affecting this purpose, assuming that all samples are uniformly sampled, that is, t u =a+(ba)u / m.

[0070] Although Q(t s ) can be simply normalized in the ideal case of m→∞. In reality, due to the limitation of computing resources, m is much smaller than ∞. In this case, formula (7) will produce discretization errors. Specifically, in order to obtain the discrete form from the continuous form of formula (7), Mip-NeRF 360 assumes that each interval [s u-1 ,s u ) within q i Formula (7) is obtained under the condition of constant, but this condition is not valid when the sample discreteness is high. The discretization error of formula (7) comes from here. Let the interval length l = s u -s u-1 =(ba) / m, formula (7) can be converted into the following form,

[0071]

[0072] where ∑q u =1 is a constant. The optimal value of formula (8) is l / 3, which can be obtained when and only when the last two terms are both 0, that is, Q(r s ) at most one qu Non-zero. Figure 2b As shown, since the sampling points almost never fall exactly on the real surface, the optimal distribution of formula (8) will introduce a deviation between the real scene surface and the scene surface understood by the model. Further analyzing formula (8), the present invention finds that this property of converging to only a single non-zero point originates from its second term. The third term of formula (8) allows Q(r s ) has at most two non-zero points, and retaining this term can help Q(r s ) represents the surface of the real scene in the form of continuous regression of adjacent samples, thereby alleviating the geometric deviation problem of the optimization distribution caused by the discretization error of formula (7), as shown in Figure 2c As shown. Therefore, removing the second term of formula (8), the present invention obtains:

[0073]

[0074] Its optimal value is obtained when and only when there are at most two non-zero samples and the two samples are adjacent. In formula (9), α is a weight constant, which corresponds to α=2 in formula (8). By magnifying the entire equation three times, formula (9) can be converted to:

[0075]

[0076] According to formula (9), it is not difficult to find that as long as α>0, its optimal value point property will be maintained. So we take α=1 and remove the last term of formula (10) to simplify it. For W d standardization.

[0077] Because in practice, it may be s All sampling points on the s The real surface hit by Q(r s ) is normalized to q u =q u / (∈+∑q u ), where ∈ is a very small positive real constant. By normalizing in this way, Q(r s ) can be optimized to 0. This adjustment can be done with The optimization is implemented adaptively.

[0078] Figure 2 is a schematic diagram of the differences between different constraint strategies for a discrete depth distribution. The red color is the true depth distribution, and the blue color is the depth distribution learned by the network. Each distribution is represented by both a stem-and-leaf plot (discrete distribution) and a shaded line plot (potential continuous distribution).

[0079] Figure 2aWithout constrained depth distribution, the depth learned by the network may be multi-peaked and irregular.

[0080] Figure 2b The depth distribution constrained by the existing normalization function may deviate from the optimal distribution and the true distribution.

[0081] Figure 2c The deep distribution constrained by the normalization function proposed by the present invention. The true distribution exists in the solution space of the optimal distribution of network optimization.

[0082] The present invention also relates to a system for a three-dimensional scene reconstruction method based on source perspective geometric constraints, wherein the system comprises a computer module for running the three-dimensional scene reconstruction method based on source perspective geometric constraints.

[0083] The present invention also relates to a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned three-dimensional scene reconstruction method based on source perspective geometry constraints when executing the computer program.

[0084] The present invention also relates to a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned three-dimensional scene reconstruction method based on source perspective geometry constraints are implemented.

[0085] Example

[0086] The method proposed in the present invention is verified on the baseline method GNT.

[0087] Step 1: Construct a three-dimensional scene geometric expression based on the source perspective to achieve the standardization of the three-dimensional scene geometric expression.

[0088] The GNT model uses transformers to aggregate information from different known perspectives to obtain the features of three-dimensional sampling points. The attention matrix in the transformer is the visibility information of three-dimensional sampling points from different perspectives simulated by the GNT model. The GNT attention matrix can be directly used as the W mentioned in this invention. v .

[0089] Step 2: transform the sampling points in the three-dimensional space according to the contribution degree of each source perspective to obtain the depth distribution of the source perspective.

[0090] As described in the specific implementation manner, W v Convert to W d .

[0091] Step 3: Apply a loss function to the depth distribution of the source perspective obtained in step 2 to achieve three-dimensional scene reconstruction based on the geometric constraints of the source perspective.

[0092] For W d Each line of As a supervision function, the constraint is 10 -3 The weight is added to the supervision function term originally used by the GNT model.

[0093] Experimental results display

[0094] 1. Objective experimental results

[0095] Table 1 shows the results of training on datasets such as Google Scanned Object, RealEstate10K, Spaces, IBRCollected, and testing on LLFF and NeRF Synthetic datasets. The present invention uses GNT as the baseline method, adds the source perspective geometry loss proposed in this patent to it for retraining, and obtains the baseline method enhanced by the present invention, denoted as GNT+SVG. In particular, the experiment compares an improved work on GNT, GNT-MOVE. This work uses a hybrid expert model to transform the network structure of GNT, and adds perspective consistency loss to optimize the consistency of rendering results from different perspectives. It can be seen that on the NeRF Synthetic dataset, GNT+SVG significantly surpasses GNT and GNT-MOVE, and on the LLFF dataset it also significantly surpasses GNT, and is very close to the results of GNT-MOVE. This shows that the technology of the present invention does effectively improve the quality of the target perspective rendered by the IBR neural network.

[0096] Table 1 Test results on NeRF Synthetic dataset and LLFF dataset

[0097]

[0098] On the other hand, the loss proposed in this patent can be used as a numerical indicator to measure the confidence of the IBR model in estimating the visual relationship between the source view and the three-dimensional points in space. The changes during GNT and GNT+SVG training are presented as curves. Figure 3a As shown in the figure, GNT+SVG has a significant advantage over GNT in estimating the confidence of visibility. Figure 3b , Figure 3c The results of GNT and GNT+SVG on different datasets during training are shown in Figure 2. curve.

[0099] Figure 3a The figure shows the geometric confidence comparison of the network reasoning process of the baseline method before and after the enhancement of the present invention. The blue one is before the enhancement and the red one is after the enhancement. The downward direction of the curve indicates the higher the geometric confidence of the reasoning process. Figure 3b The geometric confidence of the baseline method before being enhanced by the present invention is shown in the reasoning process on the mixed training data set. On different data sets, the geometric confidence of the baseline method before enhancement is different. Figure 3c The geometric confidence of the reasoning process of the baseline method after being enhanced by the present invention on a mixed training data set is shown. On different data sets, the geometric confidence of the reasoning process of the enhanced baseline method remains highly consistent, which shows that the present invention has unified the learning process of the baseline method for different data sets to a certain extent.

[0100] By observing the results, the experiment found that the original GNT's ability to understand the geometric representation of 3D scenes varies on different data sets, while GNT+SVG trained by source view geometry constraints has very well unified the network's capabilities on different data sets. This shows that the technology proposed in this patent can well unify the differences that may exist between different data sets, allowing the IBR neural network to better learn the general knowledge of 3D scene understanding.

[0101] 2. Subjective rendering results

[0102] Experiment in Figure 4 Some typical rendering results are shown in . Figure 4 The figure shows the comparison of the rendering results obtained by the baseline method before and after enhancement by this patent. The left half shows the results on the NeRF Synthetic dataset, and the right half shows the results on the LLFF dataset. The first row shows the baseline results, the second row shows the baseline results enhanced by this invention, and the third row shows the real image. The image quality rendered by the enhanced baseline is significantly better than that before enhancement.

[0103] It can be observed that the images rendered by GNT have obvious defects such as artifacts and hollowing out. GNT+SVG trained by the method proposed in this patent can fix these problems well.

[0104] The above contents of the present invention are only preferred embodiments of the present invention and are not intended to limit the implementation scheme of the present invention. A person skilled in the art can easily make corresponding changes or modifications based on the main concept and spirit of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope required by the claims.

Claims

1. A three-dimensional scene reconstruction method based on source viewpoint geometry constraints, characterized in that: The steps include: Step 1: construct a 3D scene geometric expression based on the source perspective to achieve the standardization of the 3D scene geometric expression; Step 2: transform the sampling points in the three-dimensional space according to the contribution degree of each source perspective to obtain the depth distribution of the source perspective; Step 3: Apply a loss function to the depth distribution of the source perspective obtained in step 2 to achieve three-dimensional scene reconstruction based on the geometric constraints of the source perspective.

2. The three-dimensional scene reconstruction method based on source perspective geometric constraints according to claim 1, characterized in that: In step 1, the method for constructing the geometric expression of the 3D scene based on the source perspective is as follows: A 3D sampling point p is mapped to N source view images The visibility is expressed as a weight vector w(p)=[w1(p),...,w N (p)] T ∈R 1×N , the weight vector is obtained by the weighted source view feature fusion module; Among them, H, W represent the resolution of digital image, R a×b represents the vector domain space of a×b dimensional real vectors, R represents the real number domain, i represents the image number, w i (p) represents the visibility of 3D sampling point p to source view i, N represents the number of source view images, 1≤i≤N; A total of S 3D points are sampled for the scene to obtain the matrix W describing the geometric expression of the 3D scene. v =[w(p1),…,w(p S )]∈R N×S ; Among them, w(p j )∈W v represents the jth sampling point p j The visibility weight vector, 1≤j≤S, S represents the number of three-dimensional points sampled; W v The content is reorganized into a depth distribution on each source perspective pixel W d ∈R (H×W×N)×L ; Normalize these depth distributions to normalize the geometric expression of the three-dimensional scene constructed by the neural network; Where L is the number of samples collected for a discrete depth distribution.

3. The three-dimensional scene reconstruction method based on source perspective geometric constraints according to claim 1, characterized in that: In step 2, the method for converting the visibility of the sampling points in the three-dimensional space is as follows: Through bucket sorting and bucket exchange based on depth partitioning, W v Convert to W d , while ensuring W d The samples in each row are ordered according to the spatial position and the row length is constant; for each source view image I i Only K source pixel rays with the highest upsampling frequency are retained for supervision; finally, W v Convert to Among them, N bin is a hyperparameter representing the bucket resolution in bucket sorting, K represents the number of source pixel rays, N represents the number of source view images, and R a×b represents the vector field space of a×b-dimensional real vectors, R represents the real field, W d W v A subset of rows.

4. The three-dimensional scene reconstruction method based on source viewpoint geometry constraint according to claim 1, characterized in that: In step 3, the method of applying the loss function to the depth distribution of the source view is as follows: definition is a source pixel ray r of source view i s =o i +td s The sampling points on Among them, 1≤i≤N, o i represents the viewpoint of source view i, d s Indicates the direction in which the viewpoint points to the pixel, t∈[a,b) is the line coordinate on the ray, a and b are r s The starting point and end point of the line coordinate interval that needs to be paid attention to, m is the number of sampling points, and N is the number of source view images; The discrete depth distribution of source pixels is defined as: Q(r s ):q(p1),q(p2),…,q(p m ) (1) Among them, q(p k ) represents r s The surface hit falls on p k The probability of being at d Get from Q(r s ) is normalized as a probability density function, that is, Q(r s ) is non-zero and sums to 1; ideally, when m→∞ is approximately continuous, Q(r s ) is a pulse distribution, and its pulse appears at r s At the surface of the impact; in this case, for Q(r s ) is normalized by the unsupervised learning objective function proposed by Mip-NeRF 360; Among them, q u is q(p u ), t u For r s The line coordinates of the u-th sampling point on satisfy a=t0<…<t m =b,[t u-1 , t u ) indicates that p u The interval represented by ; Assume that all samples are uniformly sampled, that is, t u =a+(ba)u / m; Let the interval length l = t u -t u-1 =(ba) / m, Formula 2 is converted into the following form: Among them, ∑q u =1 is a constant; the optimal value of formula (3) is l / 3, which is obtained when and only when the following two terms are both 0, that is, there is at most one q in Q(S). u Non-zero; since the sampling points will not fall exactly on the real surface, the optimal distribution of formula (3) will introduce a deviation between the real scene surface and the scene surface understood by the model. This convergence comes from the second term of formula (3). Removing the second term of formula (3) yields: Its optimal value is obtained when and only when there are at most two non-zero samples and the two samples are adjacent. In formula (4), α is a weight constant, corresponding to α=2 in Formula 3.

5. The three-dimensional scene reconstruction method based on source viewpoint geometry constraint according to claim 4, characterized in that: In step 3, the entire equation is magnified three times, and formula (4) is converted to: According to formula (4), as long as α>0, its optimal value point property will be maintained; taking α=1 to remove the last term of formula (5) for simplification, the result is For W d Normalization of Q(r s ) is normalized to q u =q u / (∈+∑q u ), where ∈ is a very small positive real constant; by The optimization of Q(r s ) is optimized to 0.

6. A system for implementing the three-dimensional scene reconstruction method based on source viewpoint geometry constraints according to any one of claims 1 to 5, characterized in that: The system includes a computer module for running a three-dimensional scene reconstruction method based on source viewpoint geometry constraints.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images