Neural radiation field three-dimensional reconstruction method based on degeneration learning

Abstract

The invention provides a neural radiation field three-dimensional reconstruction method based on degeneration learning, and the method comprises the steps: building a neural radiation field model through data collection; a degeneration learning device is established, a degeneration view synthesized by a neural radiation field model is input, degeneration information between a real view and the degeneration view is solved as supervision, and the degeneration view is input into the degeneration learning device and trained so as to obtain the degeneration information from the degeneration view; dividing the degeneration view and degeneration information obtained by learning into three different scales, inputting the three different scales into a multi-scale network, learning features of different scales, reinforcing key features by adopting a coding-decoding device structure, splicing into a reinforced image of an original scale, applying the reinforced image to a neural radiation field model, and performing three-dimensional scene modeling and volume rendering on a neural radiation field to obtain a neural radiation field model. And reconstructing a three-dimensional scene and synthesizing a new view of each view angle, and inputting the degraded view into the degradation learner and the multi-scale network to obtain an enhanced new view.

Description

Technical Field

[0001] The present invention belongs to the technical fields of computer vision, 3D reconstruction, etc., and particularly relates to a method for 3D reconstruction of neural radiance fields based on degradation learning. Background Art

[0002] As an important problem in the fields of computer vision and graphics, 3D scene reconstruction aims to restore the geometric and appearance information of a 3D scene from a series of 2D images. This problem has wide applications in fields such as virtual reality, augmented reality, and 3D printing. However, since 2D images only contain limited perspective and lighting information, while 3D scenes have infinite complexity and diversity, 3D scene reconstruction has always been a highly challenging task.

[0003] In recent years, deep learning methods have made remarkable progress in the field of 3D scene reconstruction. Among them, neural radiance fields, as a novel and effective method, can recover 3D geometry and appearance information from 2D images, but it still faces some limitations. One of them is that the model often makes errors when generating scene voxels, resulting in the appearance of voxels that do not exist in the actual scene. This error manifests as artifacts similar to cloud effects in the scene, bringing a certain degree of unreality to the reconstruction results. In addition, the neural radiance field model faces challenges in processing object boundaries, resulting in obvious blurring effects at the boundaries, thereby affecting the overall reconstruction accuracy and realism. Summary of the Invention

[0004] To solve the above problems, the present invention proposes a degradation-based method to effectively learn the degradation process of synthesizing new views by the neural radiance field method and the degradation information of images, and introduces a cross-scale aggregation framework to combine training images of different scales, effectively eliminating the image degradation caused by the rendering artifacts of the neural radiance field method, thereby significantly improving the quality of new views.

[0005] The technical solution specifically adopted by the present invention to solve its technical problems is:

[0006] A 3D reconstruction method of neural radiance field based on degradation learning: Through data acquisition, a neural radiance field model is constructed; By establishing a degradation learner, inputting the degraded views synthesized by the neural radiance field model, the degradation information between the real view and the degraded view is obtained as supervision, and the degraded views are input into the degradation learner and trained to obtain the degradation information from the degraded views; The degraded views and the learned degradation information are respectively segmented into three different scales and input into a multi-scale network to learn features of different scales, adopt an encoder-decoder structure to strengthen key features, and synthesize them into an enhanced image of the original scale and apply it to the neural radiance field model for 3D scene modeling and volume rendering of the neural radiance field, reconstructing the 3D scene and synthesizing new views of each perspective, and inputting the degraded views into the degradation learner and the multi-scale network to obtain enhanced new views.

[0007] Further, the specific steps of constructing the neural radiance field model through data acquisition include: Using a handheld camera to capture images of each perspective in a real scene to ensure coverage of different angles and fields of view; According to the camera's movement trajectory and pose information during shooting, calculating the position and orientation of the camera in space; Sorting and preprocessing the calculated camera position and orientation data, removing outliers and incorrect data, and converting the camera coordinate system to the world coordinate system to form the input form of the neural radiance field in one-to-one correspondence. Figure 1 One-to-one correspondence forms the input form of the neural radiance field.

[0008] Further, the specific steps of establishing a degradation learner, inputting the degraded views synthesized by the neural radiance field model, obtaining the degradation information between the real view and the degraded view as supervision, and inputting the degraded views into the degradation learner and training to obtain the degradation information from the degraded views are as follows: Taking the residual between the degraded view I DG synthesized by the neural radiance field and the real view I GT to obtain the degradation D truth caused by the neural radiance field for the real view; Inputting the degraded view I DG synthesized by the neural radiance field into the degradation learner neural network, and after learning by the degradation learner, outputting the predicted degradation information D learned ; Taking the loss between the output predicted degradation information D learned and the real degradation information D truth , and through backpropagation and gradient descent, gradually optimizing the predicted degradation information D learned output by the degradation learner.

[0009] Further, the loss is obtained through the loss function using the output of the Laplace operator Δ to ensure that the learner captures the differences in edges and details between the degraded image and the real image; By continuously iteratively optimizing the loss function, the degradation learner gradually improves its ability to predict degradation information.

[0010] Further, the loss function is expressed as:

[0011] L = L MSE + αL diff

[0012] where the parameter α is used to control the relative importance of the two loss terms, D learned represents the residual information between the ground truth image I GT and the degraded image I DG :

[0013]

[0014] where L MSE represents the MSE loss of the image, L diff represents the difference loss of the image, and ε represents a very small number:

[0015]

[0016] Furthermore, the degraded view and the learned degradation information are respectively segmented into three different scales as inputs to the multi-scale network to learn features at different scales. The encoder-decoder structure is used to enhance the key features and then they are combined into the enhanced image at the original scale. Specifically: The input degraded image I DG passes through the degradation learner to obtain D learned , and I DG is superimposed with D learned to obtain I input containing the degradation information; I input is respectively sliced at the full scale, half scale, and quarter scale and input into the channels of the three scales to learn the image features F full , F half , and F quarter at different scales; The image features F full , F half , and F quarter at the three different scales respectively pass through the encoder-decoder structure to obtain the features F full_new , F half_new , and F quarter_new in new dimensions. Finally, the information obtained at each scale is combined through scale fusion to generate the enhanced image.

[0017] Furthermore, the encoder-decoder structure is specifically:

[0018] The image features F full , F half , and F quarter at the three different scales are respectively input, and after convolution, activation, and pooling operations, they are projected into one-dimensional vectors F full_one , F half_one , and F quarter_one, and use the Sigmoid function to get the importance of each position and the input image feature F full 、F half With F quarter Multiply to get the optimized new feature F full_raw 、F half_raw With F quarter_raw :

[0019] F full_raw =F full ×Sigmoid(F full_one )

[0020] F full_one =Pooling(ReLU(Conv(F full )))

[0021] F half_raw =F half ×Sigmoid(F half_one )

[0022] F half_one =Pooling(ReLU(Conv(F half )))

[0023] F quarter_raw =F quarter ×Sigmoid(F quarter_one )

[0024] F quarter_one =Pooling(ReLU(Conv(F quarter )))

[0025] Where Simoid represents the Sigmoid function, ReLU represents the ReLU activation function, Pooling represents the pooling operation, and Conv represents the convolution operation;

[0026] The new feature F full_raw 、F half_raw With F quarter_raw After downsampling and upsampling operations, the new feature F is obtained full_new 、F half_new With F quarter_new :

[0027] F full_new =DeConv(Conv(F full_raw ))

[0028] F half_new =DeConv(Conv(F half_raw ))

[0029] F quarter_new= DeConv(Conv(F quarter_raw ))

[0030] where DeConv represents the deconvolution operation and Conv represents the convolution operation.

[0031] Furthermore, the three-dimensional scene modeling and volume rendering of the neural radiance field, reconstructing the three-dimensional scene and synthesizing new views of each perspective, and inputting the degraded views into the degradation learning machine and the multi-scale network to obtain enhanced new views are specifically as follows: determining the regions with objects in the scene through the density information and color information generated by the neural radiance field model, generating a three-dimensional volume model Volume of the scene, and mapping the color information on the surface of the three-dimensional model to Volume through volume rendering; outputting the view of the volume-rendered three-dimensional model at a given perspective, reconstructing the specific three-dimensional scene, and generating a new view I of the given new perspective DG ; inputting the new view I DG into the trained degradation learning machine to obtain degradation information D learned , and inputting the new view I DG and the degradation information D learned together into the multi-scale structure to obtain a new view.

[0032] Furthermore, the specific method of the volume rendering is as follows:

[0033] Given the camera position coordinate Position and the viewing direction Direction, obtaining the color COLOR of the object surface at this position and viewing angle through the input sampled point color RGB:

[0034]

[0035] where RGB i is the color of the i-th sampled point along the direction, Weight i is the weight of the color of the i-th sampled point along the direction, Density i represents the density of the i-th sampled point along the direction, Density j represents the density of the j-th sampled point along the direction, Distance i represents the distance between the (i + 1)-th and the i-th sampled points, Distance j represents the distance between the (j + 1)-th and the j-th sampled points, and S represents the upper limit of the sampled points; on the surface of Volume in the scene to be synthesized, rendering the complete three-dimensional model by tracing the camera rays.

[0036] An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the program, the steps of the above-mentioned three-dimensional reconstruction method of the neural radiance field based on degradation learning are implemented.

[0037] A non-transitory computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the method for three-dimensional reconstruction of neural radiation fields based on degenerate learning as described above.

[0038] Compared with the prior art, the present invention and its preferred embodiments have at least the following beneficial effects:

[0039] 1. Compared with the existing neural radiation field method, a post-processing structure of the neural radiation field is constructed while maintaining the inference speed as much as possible, which improves the reconstruction accuracy and new view quality of the method, making it more suitable for real scenes;

[0040] 2. Through the degradation learner (the present invention preferably uses the traditional U-Net network), the degradation that occurs during the synthesis of new views by the neural radiation field is effectively learned, and the degradation (the residual of the real image and the degraded image) can be directly reversely generated from the degraded image;

[0041] 3. Through the multi-scale framework, the multi-scale features of the view can be deeply integrated and combined with the view degradation information obtained by the degradation learner, so that the reconstructed view can eliminate the image degradation caused by rendering artifacts while retaining the original quality, and restore it to a high-quality image that is closer to the real view. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0043] Figure 1 It is a flow chart of a method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to make the features and advantages of this patent more obvious and easy to understand, the following embodiments are specifically described in detail as follows:

[0045] It should be noted that the following detailed descriptions are illustrative and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used in this specification have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0046] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0047] likeFigure 1 As shown in Figure 1 , an embodiment of the present invention provides a three-dimensional reconstruction method for neural radiance fields based on degradation learning, and its implementation process includes the following steps:

[0048] Step S1: Use a handheld camera to capture views of a real scene from various perspectives, calculate the spatial position and orientation of the camera through an algorithm for the view set, preprocess it after organizing it into a dataset, and form the input form of the neural radiance field;

[0049] Step S2: Establish a degradation learner, input the degraded views synthesized by the neural radiance field model, obtain the degradation information between the real view and the degraded view as supervision, input the degraded views into the degradation learner and train it so that the degradation information can be directly obtained from the degraded views;

[0050] Step S3: Divide the degraded views and the learned degradation information into three different scales and input them into a multi-scale network, learn their features at different scales, use an encoder-decoder to further consolidate and strengthen the key features, and finally splice them into an enhanced image of the original scale;

[0051] Step S4: Apply it to the neural radiance field model, the neural radiance field performs three-dimensional scene modeling and volume rendering, reconstructs the three-dimensional scene and synthesizes new views of each perspective, input the degraded views into the degradation learner and the multi-scale network, and finally obtain enhanced new views.

[0052] As a preferred embodiment of the present invention, Step S1 is specifically as follows:

[0053] Step S11: Use a handheld camera to capture images of a real scene from various perspectives, ensuring that different angles and fields of view are covered;

[0054] Step S12: Calculate the position and orientation of the camera in space according to the camera's motion trajectory and pose information during shooting;

[0055] Step S13: Organize and preprocess the calculated camera position and orientation data, remove outliers and incorrect data, convert the camera coordinate system to the world coordinate system, and form the input form of the neural radiance field one-to-one with the view Figure 1 corresponding to it.

[0056] As a preferred embodiment of the present invention, Step S2 is specifically as follows:

[0057] Step S21: Calculate the residual between the degraded view I DG synthesized by the neural radiance field and the real view I GT to obtain the degradation D truth generated by the neural radiance field for this view;

[0058] Step S22: The degraded view I synthesized by the neural radiance fieldDG Input the degradation learner neural network. After being learned by the degradation learner, output the predicted degradation information D learned ;

[0059] Step S23: Take the output predicted degradation information D learned and the true degradation information D truth as the loss. Through backpropagation and gradient descent, gradually optimize the predicted degradation information D output by the degradation learner learned .

[0060] As a preferred embodiment of the present invention, the loss is specifically: Design a loss function that makes full use of the output of the Laplace operator (denoted by Δ) to ensure that the learner can accurately capture the differences in edges and details between the degraded image and the true image. By continuously iteratively optimizing this loss function, our degradation learner can gradually improve its ability to predict degradation information, and thus provide more accurate and reliable guidance for subsequent image restoration or enhancement work. The loss function is specifically expressed as follows:

[0061] L = L MSE +αL diff

[0062] where the parameter α controls the relative importance of the two loss terms and is set to 0.05, D learned represents the residual information between the true image I GT and the degraded image I DG :

[0063]

[0064] where L MSE represents the MSE loss of the image, L diff represents the difference loss of the image, and ε represents a very small number:

[0065]

[0066] As a preferred embodiment of the present invention, step S3 is specifically:

[0067] Step S31: Input the degraded image I DG , obtain D learned through the degradation learner, and superimpose I DG and D learned to obtain I input containing degradation information;

[0068] Step S32: Cut I input into full scale, half scale and quarter scale respectively, input the channels of the three scales, and learn the image features F full 、Fhalf With F quarter ;

[0069] Step S33: The image features F at three different scales full , F half and F quarter are respectively passed through an encoder-decoder structure to obtain features F at a new dimension full_new , F half_new and F quarter_new . Finally, the information obtained at each scale is combined through scale fusion to generate an enhanced image.

[0070] As a preferred embodiment of the present invention, the encoder-decoder structure is:

[0071] The image features F at three different scales full , F half and F quarter are respectively input, and after convolution, activation (using the ReLU activation function), and pooling operations, they are projected into one-dimensional vectors F full_one , F half_one and F quarter_one . The Sigmoid function is used to obtain the importance at each position and multiply it with the input image features F full , F half and F quarter to obtain optimized new features F full_raw , F half_raw and F quarter_raw . It is expressed as:

[0072] F full_raw = F full × Sigmoid(F full_one )

[0073] F full_one = Pooling(ReLU(Conv(F full )))

[0074] F half_raw = F half × Sigmoid(F half_one )

[0075] F half_one = Pooling(ReLU(Conv(F half )))

[0076] F quarter_raw = F quarter × Sigmoid(F quarter_one )

[0077] F quarter_one= Pooling(ReLU(Conv(F quarter )))

[0078] Where Sigmoid represents the Sigmoid function, ReLU represents the ReLU activation function, Pooling represents the pooling operation, and Conv represents the convolution operation.

[0079] The newly obtained feature F full_raw 、F half_raw and F quarter_raw are respectively subjected to downsampling (using convolution) and upsampling (using transposed convolution) operations to obtain new features F full_new 、F half_new and F quarter_new . Specifically expressed as:

[0080] F full_new = DeConv(Conv(F full_raw ))

[0081] F half_new = DeConv(Conv(F half_raw ))

[0082] F quarter_new = DeConv(Conv(F quarter_raw ))

[0083] Where DeConv represents the transposed convolution operation and Conv represents the convolution operation.

[0084] As a preferred embodiment of the present invention, step S4 is specifically as follows:

[0085] Step S41: Determine the regions in the scene where objects exist through the density information and color information generated by the neural radiance field model, generate a three-dimensional volume model Volume of the scene, and map the color information on the surface of the three-dimensional model to Volume through volume rendering;

[0086] Step S42: Given a viewing angle, output the view of the volume-rendered three-dimensional model at this viewing angle, reconstruct a specific three-dimensional scene, and generate a new view I DG .

[0087] Step S43: Input the new view I DG into the trained degradation learner to obtain degradation information D learned , and input the new view I DG and the degradation information D learned together into the multi-scale structure to finally obtain a high-quality new view.

[0088] As a preferred embodiment of the present invention, the specific method of volume rendering is:

[0089] Given the camera position coordinates Position and the viewing direction Direction, the color COLOR of the object surface at this position and viewing angle can be obtained from the input sampled point color RGB. The specific method is as follows:

[0090]

[0091] where RGB i is the color of the i-th sampled point along the direction, Weight i is the weight of the color of the i-th sampled point along the direction, Density i represents the density of the i-th sampled point along the direction, Density j represents the density of the j-th sampled point along the direction, Distance i represents the distance between the (i + 1)-th and the i-th sampled points, Distance j represents the distance between the (j + 1)-th and the j-th sampled points, S represents the upper limit of the sampled points, which is 65536 here. Finally, on the surface of the Volume in the scene to be synthesized, by tracking the camera rays, a complete three-dimensional model can be rendered.

[0092] Based on the same inventive concept, the present invention also provides a computer device, which includes: one or more processors, and a memory for storing one or more computer programs; the program includes program instructions, and the processor is used to execute the program instructions stored in the memory. The processor may be a Central Processing Unit (CPU), or may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, and is used to implement one or more instructions, specifically used to load and execute one or more instructions in the computer storage medium to implement the above method.

[0093] It should be further noted that, based on the same inventive concept, the present invention also provides a computer storage medium, on which a computer program is stored. When the computer program is run by a processor, it executes the above-mentioned method. The storage medium may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electrical, magnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the computer-readable storage medium include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, the computer-readable storage medium may be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device.

[0094] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the present invention should have the ordinary meanings understood by those with ordinary skills in the field to which the present invention belongs. The "first", "second", and similar terms used in the present invention do not denote any order, quantity, or importance, but are only used to distinguish different components. The terms such as "including" or "comprising" mean that the elements or items appearing before this word cover the elements or items listed after this word and their equivalents, without excluding other elements or items. The terms such as "connected" or "coupled" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The terms such as "upper", "lower", "left", "right", etc. are only used to represent relative position relationships. When the absolute position of the object being described changes, the relative position relationship may also change accordingly.

[0095] As described above, these are only the preferred embodiments of the present invention, and the present invention is not limited to other forms. Any person skilled in the relevant art may use the disclosed technical content to make changes or modifications into equivalent embodiments with equivalent changes. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the technical solution content of the present invention still fall within the protection scope of the technical solution of the present invention.

[0096] This patent is not limited to the above-mentioned best implementation modes. Anyone inspired by this patent can derive various other forms of three-dimensional reconstruction methods of neural radiance fields based on degradation learning. All equivalent changes and modifications made within the scope of the patent application of this invention shall fall within the scope covered by this patent.

Claims

1. A method for three-dimensional reconstruction of neural radiation field based on degradation learning, characterized in that: Through data collection, a neural radiation field model is constructed; by establishing a degradation learner, a degraded view synthesized by the neural radiation field model is input, and the degradation information between the real view and the degraded view is obtained as supervision, and the degraded view is input into the degradation learner and trained to obtain the degradation information from the degraded view; The degraded view and the learned degradation information are divided into three different scales and input into the multi-scale network respectively. The features of different scales are learned, and the encoder-decoder structure is used to enhance the key features. The enhanced images are stitched together to form the enhanced images of the original scale and applied to the neural radiation field model. The neural radiation field is subjected to three-dimensional scene modeling and volume rendering. The three-dimensional scene is reconstructed and new views from various perspectives are synthesized. The degraded views are input into the degradation learner and the multi-scale network to obtain enhanced new views.

2. The method for three-dimensional reconstruction of neural radiation field based on degradation learning according to claim 1, characterized in that: The construction of the neural radiation field model through data collection specifically includes: using a handheld camera to capture images of various perspectives in a real scene to ensure coverage of different angles and fields of view; calculating the position and direction of the camera in space based on the camera's motion trajectory and posture information during shooting; organizing and preprocessing the calculated camera position and direction data, removing outliers and erroneous data, converting the camera coordinate system into a world coordinate system, and forming a neural radiation field input form in one-to-one correspondence with the view.

3. The method for three-dimensional reconstruction of neural radiation field based on degradation learning according to claim 1, characterized in that: The method of establishing a degradation learner, inputting a degraded view synthesized by a neural radiation field model, obtaining degradation information between a real view and a degraded view as supervision, inputting the degraded view into the degradation learner and training the degraded view to obtain degradation information from the degraded view is as follows: DG With real view I GT Calculate the residual and obtain the degradation D of the neural radiation field for the real view truth ; The degenerate view I synthesized by the neural radiation field DG Input the degradation learner neural network, after learning by the degradation learner, output the predicted degradation information D learned ; Output the predicted degradation information D learned With the real degradation information D truth As loss, after back propagation and gradient descent, the predicted degradation information D output by the degradation learner is gradually optimized. learned .

4. The method for three-dimensional reconstruction of neural radiation field based on degradation learning according to claim 3, characterized in that: The loss is achieved by using the output of the Laplace operator Δ through a loss function to ensure that the learner captures the differences in edges and details between the degraded image and the real image; by continuously iteratively optimizing the loss function, the degradation learner gradually improves its ability to predict degradation information.

5. The method for three-dimensional reconstruction of neural radiation field based on degradation learning according to claim 4, characterized in that: The loss function is expressed as: L=L MSE +αL diff The parameter α is used to control the relative importance of the two loss terms, D learned Represents the real image I GT With the degraded image I DG Residual information of: Where L MSE represents the MSE loss of the image, L diff Represents the difference loss of the image, and ε represents a very small number:

6. The method for three-dimensional reconstruction of neural radiation field based on degradation learning according to claim 1, characterized in that: The degraded view and the learned degradation information are divided into three different scales and input into the multi-scale network. The features of different scales are learned, and the encoder-decoder is used to enhance the key features and combine them into an enhanced image of the original scale. Specifically: Input degraded image I DG , after the degradation learner, we get D learned , will I DG With D learned The superposition obtains I containing degradation information input ; will I input The image is divided into full scale, half scale and quarter scale respectively, and the channels of three scales are input to learn the image features F at different scales respectively. full 、F half With F quarter ; The three image features F of different scales full 、F half With F quarter The features F of a new dimension are obtained through the encoder-decoder structure respectively full_new 、F half_new With F quarter_new ,Finally, the information obtained at each scale is combined through scale fusion to generate an enhanced image.

7. The method for three-dimensional reconstruction of neural radiation field based on degradation learning according to claim 6, characterized in that: The encoder-decoder structure is specifically: The three image features F of different scales are full 、F half With F quarter Input separately, projected into a one-dimensional vector F after convolution, activation and pooling operations full_one 、F half_one With F quarter_one , and use the Sigmoid function to get the importance of each position and the input image feature F full 、F half With F quarter Multiply to get the optimized new feature F full_raw 、F half_raw With F quarter_raw : F full_raw =F full ×Sigmoid(F full_one ) F full_one =Pooling(ReLU(Conv(F full ))) F half_raw =F half ×Sigmoid(F half_one ) F half_one =Pooling(ReLU(Conv(F half ))) F quarter_raw =F quarter ×Sigmoid(F quarter_one ) F quarter_one =Pooling(ReLU(Conv(F quarter ))) Where Sigmoid represents the Sigmoid function, ReLU represents the ReLU activation function, Pooling represents the pooling operation, and Conv represents the convolution operation; The new feature F full_raw 、F half_raw With F quarter_raw After downsampling and upsampling operations, the new feature F is obtained full_new 、F half_new With F quarter_new : F full_new =DeConv(Conv(F full_raw )) F half_new =DeConv(Conv(F half_raw )) F quarter_new =DeConv(Conv(F quarter_raw )) Where DeConv represents the deconvolution operation, and Conv represents the convolution operation.

8. The method for three-dimensional reconstruction of neural radiation field based on degradation learning according to claim 1, characterized in that: The method performs three-dimensional scene modeling and volume rendering on the neural radiation field, reconstructs the three-dimensional scene and synthesizes new views from various perspectives, and inputs the degraded view into the degraded learner and the multi-scale network to obtain an enhanced new view. Specifically, the density information and color information generated by the neural radiation field model are used to determine the area where objects exist in the scene, generate a three-dimensional volume model Volume of the scene, and map the color information of the surface of the three-dimensional model to the Volume through volume rendering; the view of the three-dimensional model after volume rendering is output at a given perspective, the specific three-dimensional scene is reconstructed, and a new view I given a new perspective is generated. DG ; Set the new view I DG Input the trained degradation learner to obtain the degradation information D learned , and the new view I DG With degradation information D learned Input multi-scale structures together to obtain new views.

9. The method for three-dimensional reconstruction of neural radiation field based on degradation learning according to claim 8, characterized in that: The specific method of volume rendering is: Given the camera position coordinates Position and the viewing direction Direction, the color COLOR of the object surface at the viewing angle at that position is obtained through the input sampling point color RGB: Where RGB i is the color of the i-th sampling point along the direction, Weight i is the weight of the color of the i-th sampling point along the direction, Density i Represents the density of the i-th sampling point along the direction, Density j Represents the density of the jth sampling point along the direction, Distance i Represents the distance between the i+1th and ith sampling points, Distance j Represents the distance between the j+1th and jth sampling points, and S represents the upper limit of the sampling points. On the surface of the Volume in the scene to be synthesized, a complete three-dimensional model is rendered by tracing the camera light.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method for three-dimensional reconstruction of neural radiation field based on degradation learning as described in any one of claims 1 to 9 are implemented.

Images