Neural radiation field trusted twin modeling method based on evidence fusion and storage medium

Abstract

The invention discloses a neural radiation field trusted twin modeling method based on evidence fusion and a storage medium. The method comprises the following steps: acquiring multi-view image data; constructing a neural radiation field model based on evidence deep learning, and modeling the volume density, color and uncertainty of spatial points through normal inverse gamma distribution; training the model by adopting a weighted loss function with data uncertainty; in the training process, the sampling density of a high-uncertainty region is increased through a self-adaptive resampling strategy, and the rendering quality is improved; applying the trained model to a defect sample test data set; and generating a three-dimensional reconstruction image and an uncertainty quantization graph through the model, performing credibility evaluation on the defect area, and generating a final detection result in combination with a rendered image and the uncertainty quantization graph. According to the method, a fusion method of evidence deep learning and a neural radiation field is introduced, so that high-precision and high-reliability sample supplement work of product surface defect detection is realized, and the reliability and robustness can be improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method, device and storage medium for credible twin modeling of neural radiation fields based on evidence fusion. Background Art

[0002] With the rapid development of computer vision and deep learning technology, 3D reconstruction technology has been widely used in many fields. Traditional 3D reconstruction methods mostly rely on multi-view stereo vision (MVS) or hardware equipment such as structured light and laser scanning to perform 3D geometric modeling of objects. However, these traditional methods are often unable to efficiently restore high-quality 3D reconstruction when faced with complex surface geometries and objects with rich details, especially when the texture is unclear or the lighting conditions vary greatly, the reconstruction effect is poor.

[0003] In recent years, the emergence of Neural Radiance Field (NeRF) technology has greatly promoted the progress in the field of 3D reconstruction. NeRF uses a multi-layer perceptron (MLP) to infer the details and lighting information of a 3D scene from 2D images from multiple perspectives. By modeling the radiation field of the scene, NeRF is able to generate highly realistic 3D images and shows significant advantages in dealing with complex surfaces and details. However, the NeRF model still faces problems such as high computational overhead, long training time, and dependence on the quality of the input image in complex scenes, and its reconstruction effect may not be satisfactory when faced with high noise or sparse data.

[0004] In the application of deep learning, especially in the field of 3D reconstruction, the uncertainty of the model is usually not fully considered. When the model outputs the prediction results, it may be overconfident about some unknown or difficult to model scenes, resulting in inaccurate reconstruction results. To solve this problem, Deep Evidential Regression (DER) was proposed as a new regression method, which aims to improve the robustness of the model in complex environments by modeling the uncertainty of the prediction results. DER combines the advantages of deep learning and Bayesian reasoning, not only providing accurate predictions of the output results, but also attaching reliability measures, thereby effectively quantifying the uncertainty of the model when facing different data (Regolini, R., et al., "DeepEvidential Regression," NeurIPS, 2019).

[0005] In the application scenarios of NeRF, the deep evidence regression method provides an effective way to deal with the uncertainty problem in the 3D reconstruction process. By introducing the evidence regression model based on NeRF, a reliability measure can be added to the output 3D scene to reduce overconfident predictions and improve the accuracy of reconstruction, especially when the input data is incomplete or noisy. Recent studies have shown that the combination of NeRF and DER can significantly improve the quality of 3D reconstruction, especially in surface defect detection and complex object modeling tasks, showing better performance (Zhang, J., et al., "Multivariate Deep Evidential Regression," ICML, 2020).

[0006] In addition to deep evidence regression, traditional uncertainty modeling methods, such as MC-Dropout (Gal, Y., "Uncertainty in Deep Learning," Ph.D. dissertation, University of Cambridge, 2016) and deep ensembles (Lakshminarayanan, B., et al., "Simple and Scalable PredictiveUncertainty Estimation Using Deep Ensembles," Advances in Neural InformationProcessing Systems, 2017), have also improved the robustness of 3D reconstruction to a certain extent. However, these methods are usually accompanied by high computational costs and complex implementation processes, especially the need for multiple sampling or training of multiple models. In contrast, 3D reconstruction technology combined with deep evidence regression can not only effectively reduce these computational overheads, but also provide more accurate and reliable predictions.

[0007] With the combination of NeRF technology and deep evidence regression methods, these technologies have broader application prospects in industrial inspection, virtual reality, medical imaging and other fields, especially in the tasks of three-dimensional reconstruction, defect detection and high-precision modeling of complex surfaces, which have important practical significance. Summary of the invention

[0008] The present invention proposes a neural radiation field trusted twin modeling method, device and storage medium based on evidence fusion to solve the problem of missing sample quantity and difficulty in quantifying the uncertainty of supplementary samples in surface defect detection of industrial parts based on computer vision.

[0009] To achieve the above object, the present invention adopts the following technical solutions: A neural radiation field trusted twin modeling method based on evidence fusion, which performs the following steps through a computer device: Step 1: Obtain multi-view image data for 3D scene reconstruction, including multi-view images and their corresponding camera internal and external parameters.

[0010] Step 2: Construct a neural radiation field model based on evidence-based deep learning to model the radiation field in three-dimensional space, and model the attributes of spatial points as volume density and color and their uncertainty parameters.

[0011] Step 2.1, neural radiation field input construction. As a deep learning-based 3D scene modeling method, neural radiation field has been widely used in recent years. By learning the radiation field representation of 3D scenes from image data of different viewpoints and camera poses, neural radiation field can efficiently perform 3D reconstruction and more accurately capture the lighting and geometric features in complex scenes. When dealing with complex and high-detail scenes, neural radiation field can provide higher quality image reconstruction. and the corresponding camera pose are input into the neural radiation field model, where the image data is RGB three-channel, and the width and height are and , the input image data will be subdivided into coarse sampling and fine sampling in the input construction stage, which are used as coarse network and fine network input respectively. The coarse network input is ,in is the number of coarse samples, and the fine network input is ,in is the number of fine samples; Step 2.2, Evidence deep learning module construction. We embed the evidence deep learning module in the neural radiation field model to effectively model the prediction uncertainty of each spatial point and improve the stability and robustness of the model in areas with high uncertainty. Introduce the normal inverse gamma distribution (NIG) and assume that the color of the sampling point (correspond , are sampling points on the light) each obeys a mutually independent normal distribution, that is , its mean and variance Controlled by the normal inverse gamma distribution parameters: mean , precision parameter , shape parameter , scale parameter ,Through these parameters, the uncertainty of color prediction is quantified, thereby improving the confidence and stability of the model for predictions in different regions; Step 2.3, neural radiation field network construction. The neural radiation field network consists of two parts with the same structure: a coarse network and a fine network. Each network consists of two multi-layer perceptrons (MLPs). The first MLP is responsible for extracting geometric information and uncertainty parameters from spatial coordinates to generate volume density, internal features, accuracy, shape, and scale parameters; the second MLP uses spatial coordinates and viewing direction, combined with internal features to predict the color mean, capturing color changes related to lighting characteristics and viewing angles.

[0012] Step 3: Propose a loss weighting based on data uncertainty, use the parameters of the normal inverse gamma distribution to calculate the data uncertainty of each sampling point, and weight the loss of each sample in the coarse network.

[0013] Step 3.1, in order to model uncertainty, we simultaneously model the inherent uncertainty of the data and the uncertainty of the model parameters according to the conjugate prior normal inverse gamma distribution in Bayesian statistics. In our model, data uncertainty represents the error in the color prediction of each spatial point due to data noise. For the first pixels and color channels , the data uncertainty and model uncertainty can be calculated as follows As shown:

[0014]

[0015] in represents the scale parameter after volume rendering fusion, Represents the shape parameters after volume rendering fusion, Represents the accuracy parameter after volume rendering fusion. The greater the data uncertainty, the greater the prediction variance of the sampling point, and the model's prediction result for this point is more vague or unreliable; Step 3.2, in our neural radiation field model, in order to reduce the negative impact of areas with large data uncertainty on the coarse network training, a loss weighting strategy is adopted. This strategy adaptively reduces the impact of the model on low uncertainty areas based on the data uncertainty of each pixel color, thereby improving the model's learning effect on high uncertainty areas. The weight of each spatial point is calculated based on the uncertainty of its color prediction and is normalized to avoid over-weighting. This ensures that after the coarse network loss is weighted, the model can more effectively learn areas with small data uncertainty and optimize the training process. The weighted loss of the coarse network is as follows: :

[0016] in , , This is the first batch pixels, color channels The uncertainty weight of The data uncertainty for the color prediction at this sample point.

[0017] Step 4: Propose an adaptive resampling method. During the training process, the sampling density is increased for areas with higher uncertainty based on the model uncertainty estimation of spatial points. For areas with lower model uncertainty, the sampling density is appropriately reduced. This is used as the input of the fine network to improve training efficiency and rendering quality.

[0018] Step 4.1, based on the idea of ​​layered sampling of neural radiation fields, preliminary sampling is performed through the coarse network, and then fine sampling is performed using the fine network. First, the coarse network is used to calculate each sampling point The volume density and weight of the coarse sampling points are , and evenly distributed in the light interval Each sampling point See the formula for the position of :

[0019] Then use a thin network for fine sampling in key areas; Input the sampling points into the coarse network to obtain the volume density and calculate the corresponding weight , which is defined as the formula :

[0020] in Indicates The weight of the sampling points, It is The volume density of the sampling points, It is the total number of sampling points on the ray in coarse sampling. A higher volume density means that the area contributes more to the rendering of the model.

[0021] In the inference phase, the fine network is based on the weights of the coarse sampling points. Determine the position of the additional sampling point in each ray, and perform further sampling based on this. The position of the fine sampling point is determined by the formula Sure:

[0022] in Represents the distance between two adjacent sampling points in coarse sampling; Step 4.2: Generate preliminary volume density distribution and normal inverse gamma parameters based on the coarse network, combined with the model uncertainty described in step 3 , calculate the three-channel average model uncertainty , and introduced into the sampling weight calculation, such as formula :

[0023] in is the adjustment coefficient, which is used to control the impact of uncertainty on the weight. The number of the maximum sampling point of the model uncertainty.

[0024] Based on these weights, more sampling points are dynamically allocated to the high uncertainty area. During the training phase, in the fine network, the positions of the newly added sampling points are as follows: :

[0025] In the training phase, the adaptive resampling strategy concentrates resources on learning difficult areas to improve the ability to capture details; in the inference phase, layered sampling is used to balance efficiency and quality, focusing on refining key areas. This approach not only improves training efficiency and rendering accuracy, but also enhances the ability to reliably model complex surface defects.

[0026] Step 5: Fuse the normal inverse gamma distribution of each sampling point for volume rendering to generate a rendered image and uncertainty quantification map.

[0027] Step 5.1, for multiple sampling points on the light, the color channel of each sampling point corresponds to the parameters of the normal inverse gamma distribution, including mean, precision, shape and scale parameters, and weights related to uncertainty. In order to obtain the final pixel color and uncertainty quantification map, the expected value and uncertainty parameters of the pixel color are calculated using a weighted fusion method. This method weights and sums the parameters of each sampling point according to the weight, and finally obtains the comprehensive pixel color and uncertainty parameters, thereby effectively improving the rendering accuracy and uncertainty quantification capabilities.

[0028] These parameters are calculated by weighted fusion and numbered as The expected pixel color value and uncertainty parameters , as shown in the formula :

[0029]

[0030]

[0031]

[0032] in , Indicates the number of sampling points on the ray, Indicates The volume density of the sampling points, Represents the distance between two adjacent sampling points; Step 5.2: Combine the data uncertainty and model uncertainty definitions in step 3.1 to generate an uncertainty quantification diagram.

[0033] Step 6: Combine the differentiable rendering norm loss and the uncertainty loss function to optimize the global radiation field.

[0034] Step 6.1 introduces a new regression loss function to use the input samples to learn the normal inverse gamma distribution parameter optimization model; for a batch size of For each ray in, calculate each color channel Color Normal Inverse Gamma Negative Log-Likelihood Loss , see formula :

[0035] in, , Indicates the true color of light, Respectively Root ray color channel The mean, precision, shape and scale parameters of is the standard loss coefficient; This loss function minimizes the error between the predicted color mean and the true color by maximizing the log-likelihood of the normal inverse gamma distribution of the color channel, taking into account the confidence of each prediction, and is applied to both coarse and fine network optimization. Combined with the loss weighting algorithm described in step 3.2, the coarse network loss can be rewritten as formula :

[0036] Step 6.2: To prevent the fine network model from over-relying on uncertainty parameters, a regularization loss is designed to ensure that the color estimate takes uncertainty into account. Compared with the estimate without considering the uncertainty The difference between them is as small as possible, see formula :

[0037] in , is the weight coefficient of the normative loss; Step 6.3, in summary, we propose a complex shape trusted twin modeling method for evidence fusion neural radiation field. The overall loss is shown in formula (14):

[0038] On the other hand, the present invention further discloses a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the above method.

[0039] On the other hand, the present invention further discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the above method.

[0040] It can be seen from the above technical scheme that the present invention relates to the technical field of evidence deep learning and three-dimensional reconstruction, and discloses a credible twin modeling method for surface defects of complex-shaped products based on evidence fusion neural radiation field. The method includes: obtaining multi-view image data for three-dimensional scene reconstruction; constructing a neural radiation field model based on evidence deep learning, and modeling the volume density, color and uncertainty of spatial points through normal inverse gamma distribution; training the model using a weighted loss function of data uncertainty; increasing the sampling density of high uncertainty areas through an adaptive resampling strategy during the training process to improve the rendering quality; applying the trained model to a defect sample test data set; generating a three-dimensional reconstructed image and an uncertainty quantification map through the model, conducting a credibility assessment of the defect area, and generating the final detection result by combining the rendered image with the uncertainty quantification map. The present invention effectively solves the problem of poor reconstruction quality caused by noise and data sparsity in traditional three-dimensional reconstruction technology by introducing a fusion method of evidence deep learning and neural radiation field, and provides a new solution to the common problem of missing sample quantity in surface defect detection of industrial products, that is, supplementing samples through high-precision and high-credibility evidence fusion neural radiation field twin reconstruction to improve the reliability and robustness of detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is the overall flow chart of the present invention; Figure 2 This is a structural diagram of a neural radiation field model constructed based on evidence-based deep learning proposed by the present invention; Figure 3 is the adaptive resampling module proposed by the present invention; Figure 4 It is a credible modeling and visualization effect of complex surface defects. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.

[0043] like Figure 1 As shown, For the purpose of formula unification, the following definitions are given for the subscripts and subscripts used in the formulas of this invention. Represents RGB color channel, value is ; Represents the first sampling points, with values ​​ranging from 1 to A natural number, with values ​​ranging from 1 to The natural number of Represents a training batch The A ray of light.

[0044] like Figure 1 As shown, the specific implementation steps of the credible twin modeling method for complex-shaped product surface defects based on evidence fusion neural radiation field of the present invention are as follows: Step 1: Get training data and sample the light coarsely. First, use a multi-view camera to get the RGB image data of the target object, ensure that there is no blind spot in the view coverage, and calibrate the internal and external parameters of the camera (such as focal length, position, and posture) to provide high-quality data for model input. The camera view collects scene images in a diverse layout with a resolution of . The pixel color is taken as the true value, and then the light corresponding to each pixel is sent to perform coarse sampling. The light starts from the center of the camera and the direction is (p is the pixel coordinate, o is the camera position), in the depth range Internal Generation uniform sampling points, Represent the closest and farthest distances respectively, then the depth value of each sampling point can be obtained by the formula gives:

[0045] The sampling point coordinates are calculated as These uniformly distributed coarse sampling points provide the necessary basis for subsequent radiation field modeling, and the sampling distribution is further optimized in combination with uncertainties.

[0046] Step 2, such as Figure 2 As shown, a neural radiation field model of evidence fusion is constructed.

[0047] Specifically, it is divided into the following steps: Step 2.1, model input construction. The model input construction stage is based on pixels and light, combining multi-view image data with camera viewing direction to generate sampling points in three-dimensional space. The coarse network input is ,in is the number of coarse samples; the fine network input is ,in is the number of fine samples; Step 2.2, embedding the evidence deep learning module. The evidence deep learning module is embedded in the neural radiation field model to model the uncertainty of each spatial point, assuming that the color of the sampling point Each follows a mutually independent normal distribution, that is, , its mean and variance Controlled by the normal inverse gamma distribution parameters: mean , precision parameter , shape parameter , scale parameter , then its probability density function can be defined as ; The prediction of the sample point color follows a normal inverse gamma distribution , these parameters are used to quantify the uncertainty of the model, and the output parameters are as follows: Color mean : represents the mean of color prediction; Accuracy parameters : Used to reflect the model's confidence in color prediction; Shape parameters : Indicates the prediction stability of the model for this spatial point; Scale parameter : Used to control the range of color distribution and reflect the discrete degree of predicted distribution; Step 2.3, build the neural radiation field network structure settings. The neural radiation field model consists of a coarse network and a fine network, which are used for global feature extraction and local detail optimization respectively. The structures of the two are exactly the same, both consisting of two multi-layer perceptrons (MLPs). The input and output of the network are constructed as follows: The first MLP is mainly responsible for extracting geometric information and its uncertainty parameters from spatial coordinates to generate volume density , internal hidden features , precision parameters , shape parameters , scale parameters , whose parameters are :

[0048] Color mean Through the second multi-layer perceptron. This MLP uses spatial coordinates and viewing direction , combined with internal implicit features , predict the color mean , capturing the scene's lighting characteristics and view-dependent color variations. Its parameters are :

[0049] in , Represent the NIG estimated means of the red, green, and blue channels, respectively.

[0050] Step 3, use the proposed adaptive resampling method to obtain fine sampling points, such as Figure 3 As shown, train the fine network.

[0051] Step 3.1, stratified sampling. After the sampling points obtained by coarse sampling are input into the coarse network, the key areas can be sampled preliminarily, and then fine sampling can be performed using the fine network.

[0052] Input the sampling points into the coarse network to obtain the volume density and calculate the corresponding weight , which is defined as the formula :

[0053] in Indicates The weight of the sampling points, It is The volume density of the sampling points, It is the total number of sampling points on the ray in coarse sampling. A higher volume density means that the area contributes more to the rendering of the model.

[0054] Step 3.2, adaptive resampling. Adaptively adjust the sampling density to more accurately model the high uncertainty area, and dynamically allocate sampling points according to the model uncertainty. The basis for adaptation is the preliminary volume density distribution of sampling points generated by the coarse network and the model uncertainty, where the model uncertainty can be obtained by the normal inverse gamma parameter: , which is used to calculate the sampling weight, as shown in the formula :

[0055] in is the adjustment coefficient, which is used to control the impact of uncertainty on the weight.

[0056] Based on these weights, more sampling points are dynamically allocated to high uncertainty areas. In the thin network, the locations of the newly added sampling points are as follows: :

[0057] in Indicates the distance between two adjacent sampling points in coarse sampling. During the training phase, the resampling strategy concentrates resources on learning difficult areas to improve the ability to capture details.

[0058] Step 4: Use the volume rendering method fused with normal inverse gamma distribution to synthesize the new view and the uncertainty view.

[0059] Step 4.1, for multiple sampling points along the ray , each sampling point Color channels All correspond to the parameters of the normal inverse gamma (NIG) distribution: mean , accuracy ,shape ,scale , and the weights associated with the uncertainty ; In order to obtain the final pixel color and uncertainty quantization map, we calculate these parameters by weighted fusion and obtain The expected pixel color value and uncertainty parameters , as shown in the formula :

[0060]

[0061]

[0062]

[0063] in , Indicates the number of sampling points on the ray, Indicates The volume density of the sampling points, Indicates the distance between two adjacent sampling points.

[0064] Step 4.2, uncertainty quantification map generation, combined with the data uncertainty and model uncertainty definitions in step 3.1, the pixel data uncertainty can be derived , model uncertainty ,These uncertainty quantification results help to evaluate the prediction confidence of the model in different regions.

[0065] Step 5: Train the model using the proposed loss.

[0066] Step 5.1 introduces a new regression loss function, which uses the input samples to learn the parameters of the normal inverse gamma distribution for a batch size of For each ray in, calculate each color channel Color Normal Inverse Gamma Negative Log-Likelihood Loss , see formula :

[0067] Step 5.2, calculation of coarse network loss weighting. In order to reduce the negative impact of areas with large data uncertainty in the coarse network on the training process, a loss weighting strategy is adopted to adaptively reduce the impact of areas with low model confidence on the training model according to the data uncertainty of each predicted pixel color. For the color of the predicted pixel, the data uncertainty weight can be expressed as: , in order to avoid excessive weight, we use the normalization method to obtain We can get the weighted loss of the coarse network, see formula :

[0068] Step 5.3, Differentiable Rendering Norm Loss, in order to prevent the fine network model from over-relying on uncertainty parameters, a norm loss is designed to ensure that the color estimate takes into account uncertainty Compared with the estimate without considering the uncertainty The difference between is as small as possible, as shown in the formula :

[0069] in , is the weight coefficient of the normative loss; Step 5.4, overall model loss. In summary, we propose a complex shape trusted twin modeling method for evidence fusion neural radiation field. The overall loss is shown in formula (38):

[0070] Step 6, obtain test data and perform coarse sampling on the light corresponding to the pixel color to be predicted. The specific content is the same as step 1.

[0071] Step 7: Use the resampling strategy to perform fine sampling. The coarse sampling points are input into the coarse network to obtain the volume density and calculate the corresponding weights. , which is defined as the formula :

[0072] in Indicates The weight of the sampling points, It is The volume density of the sampling points, It is the total number of sampling points on the ray in coarse sampling. A higher volume density means that the area contributes more to the rendering of the model.

[0073] The fine network is based on the weights of the coarse sampling points Determine the position of the additional sampling point in each ray, and perform further sampling based on this. The position of the fine sampling point is determined by the formula Sure:

[0074] in Represents the distance between two adjacent sampling points in coarse sampling; In step 8, the fine sampling results are input into the trained fine network to obtain the volume density, color and uncertainty information of each point in the space. Through the volume rendering module fused with the normal inverse gamma distribution described in step 4, new views and uncertainty views can be rendered. The model performance can be evaluated by combining the real images of the test data.

[0075] Table 1

[0076] like Figure 4 As shown in Table 1, in order to verify the effectiveness of this scheme in the task of credible modeling of complex surface defects, the present invention constructs an experimental scene based on the llff public dataset (17 training images and 3 test images), and performs model evaluation indicators through model training and output results: The present invention selects PSNR (peak signal-to-noise ratio), SSIM, and LPIPS as measures of the difference between the predicted image and the real image under the condition of 400k-step model full fitting training.

[0077] On the other hand, the present invention further discloses a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the above method.

[0078] On the other hand, the present invention further discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the above method.

[0079] In another embodiment provided in the present application, a computer program product comprising instructions is also provided. When the computer is run on a computer, the computer executes any of the neural radiation field trusted twin modeling methods based on evidence fusion in the above embodiments.

[0080] It is understandable that the system, device and storage medium provided in the embodiments of the present invention correspond to the method provided in the embodiments of the present invention, and the explanation, examples and beneficial effects of the relevant contents can refer to the corresponding parts in the above methods.

[0081] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive Solid State Disk (SSD)), etc.

[0082] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0083] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0084] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A neural radiation field trusted twin modeling method based on evidence fusion, characterized in that: The following steps are included: Step 1, obtaining multi-view image data for three-dimensional scene reconstruction; Step 2: Construct a neural radiation field model and embed the evidence deep learning framework to model the radiation field in three-dimensional space, and model the attributes of spatial points as volume density and color and their uncertainty parameters; Step 3: Weighting the loss based on data uncertainty. The data uncertainty of each sampling point is calculated using the parameters of the normal inverse gamma distribution, and the loss of each sample in the coarse network is weighted. Step 4, adaptive resampling, based on the model uncertainty estimation, proposes an adaptive resampling strategy during the training process, increasing the sampling density in areas with higher model uncertainty and reducing the sampling density in areas with lower model uncertainty; Step 5, a volume rendering method integrating normal inverse gamma distribution is used to fuse uncertainty parameters of all sampling points based on the volume rendering method of neural radiation field to generate a rendered image and an uncertainty quantification map; Step 6: Trusted evidence regression, combining differentiable rendering norm loss and uncertainty loss function to optimize the global radiation field.

2. The method for credible twin modeling of neural radiation fields based on evidence fusion according to claim 1 is characterized in that: Step 2 includes the following steps: Step 2.1, neural radiation field input construction: The input multi-view image data and the corresponding camera pose are input into the neural radiation field model, where the image data is RGB three-channel, and the width and height are and , the input image data is subdivided into coarse sampling and fine sampling during the input construction stage: Coarse sampling: ,in is the number of coarse samples, Represents the three-dimensional coordinates of the coarse sampling points; Fine sampling: ,in is the number of fine samples, Represents the three-dimensional coordinates of the coarse sampling points; Network receiving space coordinates and viewing direction As input; Step 2.2, Evidence deep learning module construction: Embedding an evidence deep learning module in the neural radiance field model is used to model the uncertainty at each spatial point, assuming the color of the sampled point ,correspond , are sampling points on the light, each of which obeys a mutually independent normal distribution, that is, , its mean and variance Controlled by the normal inverse gamma distribution parameters: mean , precision parameter , shape parameter , scale parameter , then its probability density function is defined as ; The fused evidence model will output the following parameters: Color mean : Output from the network, , represents the mean of color prediction; Accuracy parameters : Used to reflect the model's confidence in color prediction, defined as the inverse of accuracy, ,in is the variance of the predicted color; Shape parameters : Indicates the prediction stability of the model for this spatial point; a higher The value indicates that the model has higher stability at this point; Scale parameter : Used to control the range of color distribution, combined with Reflects the degree of dispersion of the predicted distribution; At each sampling point, the color prediction follows a normal inverse gamma distribution , the uncertainty of the model is quantified through these parameters; Step 2.3, neural radiation field network construction: The neural radiation field network structure consists of two parts: a coarse network and a fine network. The structures of the two are exactly the same, both consisting of two multi-layer perceptrons MLP. The input and output of the network are constructed as follows: The first MLP is responsible for extracting geometric information and its uncertainty parameters from spatial coordinates to generate volume density , internal hidden features , precision parameters , shape parameters , scale parameters , whose parameters are : Color mean Through the second multi-layer perceptron; this MLP uses spatial coordinates and viewing direction , combined with internal implicit features , predict the color mean , captures the scene's lighting characteristics and view-dependent color variations; its parameters are : in , Represent the NIG estimated means of the red, green, and blue channels, respectively.

3. The method for credible twin modeling of neural radiation fields based on evidence fusion according to claim 2 is characterized in that: Step 3 includes the following steps: Step 3.1, calculation of data uncertainty. Data uncertainty represents the error caused by data noise in the color prediction of each spatial point. pixels and color channels , calculate data uncertainty and model uncertainty: ,in represents the scale parameter after volume rendering fusion, Represents the shape parameters after volume rendering fusion, Represents the accuracy parameter after volume rendering fusion. The greater the data uncertainty, the greater the prediction variance of the sampling point, and the model's prediction result for this point is more vague or unreliable; Step 3.2, the weighted calculation of the coarse network loss, adopts the loss weighting strategy, and adaptively reduces the impact of the model's lower grasp area on the training model according to the data uncertainty of each predicted pixel color, thereby improving the model's learning effect on the area with less data uncertainty; For predicting the color of a pixel, the data uncertainty weight is expressed as: In order to avoid excessive weight, the normalization method is used to obtain , The size is The training batch pixels, color channels The uncertainty weight of The data uncertainty for the color prediction of this sampling point; Get the weighted loss of the coarse network, see formula : 。 4. The method for credible twin modeling of neural radiation fields based on evidence fusion according to claim 3 is characterized in that: Step 4 includes: Step 4.1, hierarchical sampling, first use the coarse network to perform preliminary sampling to obtain the key area, and then use the fine network for fine sampling; First, use a coarse network to perform preliminary sampling of the light and calculate each sampling point The volume density and weight of , and evenly distributed in the light interval superior, Represents the closest and farthest distances between the sampling point and the optical center coordinates of the light, so each sampling point The position of gives: Input the sampling points into the coarse network to obtain the volume density and calculate the corresponding weight , which is defined as the formula : in Indicates The weight of the sampling points, It is The volume density of the sampling points, It is the total number of sampling points on the ray in coarse sampling. A higher volume density means that the area contributes more to the rendering of the model. The fine network is based on the weights of the coarse sampling points Determine the position of the additional sampling point in each ray, and perform further sampling based on this. The position of the fine sampling point is determined by the formula Sure: in Represents the distance between two adjacent sampling points in coarse sampling; Step 4.2, adaptive resampling, adaptively adjusts the sampling density to more accurately model high uncertainty areas and dynamically allocates sampling points according to model uncertainty; First, a preliminary volume density distribution and normal inverse gamma parameters are generated by the coarse network, combined with the model uncertainty described in step 3 Dynamically adjust the fine sampling points; introduce model uncertainty into the sampling weight calculation, such as formula : in is the adjustment coefficient, which is used to control the impact of uncertainty on the weight. It is the number of the maximum sampling point of model uncertainty; Based on these weights, more sampling points are dynamically allocated to high uncertainty areas; in the thin network, the locations of the newly added sampling points are as follows: : During the training phase, the resampling strategy concentrates resources on learning difficult areas and improves the ability to capture details; during the inference phase, stratified sampling is used to balance efficiency and quality, focusing on refining key areas.

5. The method for credible twin modeling of neural radiation fields based on evidence fusion according to claim 4 is characterized in that: Step 5 includes the following steps: Step 5.1, weighted fusion of sampling points, for multiple sampling points along the ray , each sampling point Color channels All correspond to the parameters of the normal inverse gamma (NIG) distribution: mean , accuracy ,shape ,scale , and the weights associated with the uncertainty ; These parameters are calculated by weighted fusion and numbered as The expected pixel color value and uncertainty parameters , as shown in the formula : in , Indicates the number of sampling points on the ray, Indicates The volume density of the sampling points, Represents the distance between two adjacent sampling points; Step 5.2: Generate uncertainty quantification map. Combine the data uncertainty and model uncertainty definitions in step 3.1 to derive the pixel data uncertainty. , model uncertainty ,These uncertainty quantification results help to evaluate the prediction confidence of the model in different regions.

6. The method for credible twin modeling of neural radiation fields based on evidence fusion according to claim 5 is characterized by: Step 6 includes the following steps: Step 6.1, a new regression loss function is introduced to learn the parameters of the normal inverse gamma distribution using the input samples; For a batch size of For each ray in, calculate each color channel Color Normal Inverse Gamma Negative Log-Likelihood Loss , see formula : in, , Indicates the true color of light, Respectively Root ray color channel The mean, precision, shape and scale parameters of is the standard loss coefficient; This loss function minimizes the error between the predicted color mean and the true color by maximizing the log-likelihood of the normal inverse gamma distribution of the color channel, taking into account the confidence of each prediction, and is applied to both the coarse network and the fine network optimization. Combined with the loss weighting algorithm described in step 3.2, the coarse network loss is rewritten as formula : Step 6.2, a regularization loss is designed to ensure that the color estimate takes into account uncertainty Compared with the estimate without considering the uncertainty The difference between is as small as possible, as shown in the formula : in , is the weight coefficient of the normative loss; Step 6.3, the overall loss of the complex shape credible twin modeling method of the evidence fusion neural radiation field is shown in formula (14): 。 7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 6.

Images