A method, apparatus, equipment, and storage medium for digital terrain 3D reconstruction based on neural radiation field and bidirectional reflection distribution function.

Abstract

The present application discloses a method, device, equipment and storage medium for digital ground three-dimensional reconstruction based on neural radiation field and bidirectional reflectance distribution function. The present application relates to the field of three-dimensional reconstruction technology. The method includes: inputting the satellite sampling light data corresponding to the target area into a preset neural radiation field model to obtain scene information; constructing a physical imaging model based on the bidirectional reflectance distribution function according to the scene information; optimizing the preset neural radiation field model based on the physical imaging model to obtain a target neural radiation field model; determining the scene elevation information of the target area through the target neural radiation field model, and generating a ground three-dimensional reconstruction model corresponding to the target area. The present application constructs a physical imaging model based on the scene information obtained from satellite image light data, and then optimizes the preset neural radiation field model through the physical imaging model, thereby determining high-precision scene elevation information based on the optimized target neural radiation field model, and improving the reconstruction accuracy of the ground three-dimensional model.

Description

Technical Field

[0001] The present application relates to the field of three-dimensional reconstruction technology, and in particular to a method, device, equipment and storage medium for digital ground three-dimensional reconstruction based on neural radiation field and bidirectional reflectance distribution function. Background Art

[0002] In recent years, with the development of many fields such as virtual reality / augmented reality, digital earth, high-precision maps, urban planning and design, digital ground models and scene three-dimensional models, as important basic data, their reconstruction technology has become more critical in these fields and has shown a diversified trend.

[0003] The current mainstream means of 3D reconstruction include methods based on laser point cloud and optical image. The acquisition equipment based on laser point cloud is usually expensive and has high acquisition cost. In contrast, optical image has the advantages of being more economical, convenient and flexible. The traditional digital ground model reconstruction method based on optical image mainly relies on photogrammetry technology based on stereo parallax. However, this method is prone to failure in areas with poor image texture or occlusion (such as water bodies, building shadows, etc.), resulting in limited reconstruction accuracy of digital ground models, and the model reconstruction accuracy needs to be improved.

[0004] Therefore, how to effectively improve the accuracy of digital ground models for ground three-dimensional reconstruction has become an urgent problem to be solved. Summary of the invention

[0005] The main purpose of this application is to provide a method, device, equipment and storage medium for digital ground three-dimensional reconstruction based on neural radiation field and bidirectional reflectance distribution function, aiming to solve the technical problem of how to effectively improve the accuracy of digital ground model for ground three-dimensional reconstruction.

[0006] To achieve the above objectives, the present application proposes a method for digital ground three-dimensional reconstruction based on neural radiation field and bidirectional reflectance distribution function, the method comprising:

[0007] Inputting the satellite sampling light data corresponding to the target area into a preset neural radiation field model to obtain scene information corresponding to the target area;

[0008] Constructing a physical imaging model based on a bidirectional reflectance distribution function according to the scene information;

[0009] Optimizing the preset neural radiation field model based on the physical imaging model to obtain a target neural radiation field model;

[0010] The scene elevation information of the target area is determined by the target neural radiation field model, and a ground three-dimensional reconstruction model corresponding to the target area is generated.

[0011] In one embodiment, the step of constructing a physical imaging model based on a bidirectional reflectance distribution function according to the scene information includes:

[0012] Determine the initial normal vector corresponding to the scene object in the target area according to the density field information corresponding to the scene information;

[0013] Optimizing the initial normal vector based on the satellite image data corresponding to the target area and the density field information to obtain an optimized normal vector;

[0014] A physical imaging model based on a bidirectional reflectance distribution function is constructed according to the optimized normal vector.

[0015] In one embodiment, the step of optimizing the initial normal vector based on the satellite image data corresponding to the target area and the density field information to obtain the optimized normal vector includes:

[0016] Determine the target ray direction based on the satellite image data corresponding to the target area and the density field information;

[0017] The initial normal vector is optimized in direction according to the target ray direction to obtain an optimized normal vector.

[0018] In one embodiment, the step of constructing a physical imaging model based on a bidirectional reflectance distribution function according to the optimized normal vector includes:

[0019] determining a spherical harmonic basis based on the optimized normal vector;

[0020] A physical imaging model based on a bidirectional reflectance distribution function is constructed according to the albedo corresponding to the scene information, the density field information and the spherical harmonic function basis.

[0021] In one embodiment, the step of constructing a physical imaging model based on a bidirectional reflectance distribution function according to the albedo corresponding to the scene information, the density field information and the spherical harmonic function basis includes:

[0022] Determine a target illumination coefficient corresponding to the target area according to the shadow scalar corresponding to the scene information and the density field information;

[0023] Determine the point reflectivity corresponding to the surface of the scene object in the target area based on the albedo corresponding to the scene information and the density field information;

[0024] The physical imaging model is constructed according to the point reflectivity, the spherical harmonic function basis and the target illumination coefficient.

[0025] In one embodiment, the step of determining the target illumination coefficient corresponding to the target area according to the shadow scalar corresponding to the scene information and the density field information comprises:

[0026] Determine the sampling point weight of the sampling ray based on the density field information;

[0027] Determine the pixel shadow amount at the target pixel corresponding to the sampling ray according to the shadow scalar corresponding to the scene information and the sampling point weight;

[0028] A target illumination coefficient corresponding to the target area is determined based on the pixel shadow amount.

[0029] In one embodiment, the step of optimizing the preset neural radiation field model based on the physical imaging model to obtain a target neural radiation field model further includes:

[0030] Generate a scene composite image corresponding to the target area based on the physical imaging model;

[0031] Generate a scene rendering image corresponding to the target area according to the scene information;

[0032] Determine a preliminary reconstruction loss based on a true image corresponding to the satellite image data, the scene composite image, and the scene rendering image;

[0033] The preset neural radiation field model is iterated according to the preset normal optimization constraint and the preliminary reconstruction loss to obtain a target neural radiation field model.

[0034] In addition, to achieve the above purpose, the present application also proposes a digital ground three-dimensional reconstruction device based on neural radiation field and bidirectional reflectance distribution function, and the digital ground three-dimensional reconstruction device based on neural radiation field and bidirectional reflectance distribution function includes:

[0035] A density field analysis module, used to input the satellite sampling light data corresponding to the target area into a preset neural radiation field model to obtain scene information corresponding to the target area;

[0036] A scene analysis module, used to construct a physical imaging model based on a bidirectional reflectance distribution function according to the scene information;

[0037] An optimization module, used for optimizing the preset neural radiation field model based on the physical imaging model to obtain a target neural radiation field model;

[0038] The three-dimensional reconstruction module is used to determine the scene elevation information of the target area through the target neural radiation field model and generate a ground three-dimensional reconstruction model corresponding to the target area.

[0039] In addition, to achieve the above-mentioned objectives, the present application also proposes a digital ground three-dimensional reconstruction device based on neural radiation field and bidirectional reflectance distribution function. The device includes: a memory, a processor, and a ground three-dimensional reconstruction program stored in the memory and runnable on the processor. The ground three-dimensional reconstruction program is configured to implement the steps of the digital ground three-dimensional reconstruction method based on neural radiation field and bidirectional reflectance distribution function as described above.

[0040] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a storage medium on which a ground three-dimensional reconstruction program is stored. When the ground three-dimensional reconstruction program is executed by a processor, the steps of the digital ground three-dimensional reconstruction method based on neural radiation field and bidirectional reflectance distribution function are implemented as described above.

[0041] The present application provides a method, device, equipment and storage medium for digital ground three-dimensional reconstruction based on neural radiation field and bidirectional reflectance distribution function, the method comprising: inputting satellite sampling light data corresponding to the target area into a preset neural radiation field model to obtain scene information corresponding to the target area; constructing a physical imaging model based on the bidirectional reflectance distribution function according to the scene information; optimizing the preset neural radiation field model based on the physical imaging model to obtain the target neural radiation field model; determining the scene elevation information of the target area through the target neural radiation field model, and generating a ground three-dimensional reconstruction model corresponding to the target area. The present application first obtains multi-view sampling light data corresponding to the satellite image data of the target area, that is, the above-mentioned satellite sampling light data, and then inputs the satellite sampling light data into the preset neural radiation field model to analyze the three-dimensional scene information of the satellite image data of the target area, obtains the scene information corresponding to the target area to optimize the image texture, and then constructs a physical imaging model based on the bidirectional reflectance distribution function based on the scene information, optimizes the preset neural radiation field model based on the physical imaging model, and finally predicts accurate scene information based on the optimized target neural radiation field model, and then determines the high-precision elevation information corresponding to the target area based on the accurate scene information, thereby significantly improving the accuracy of the final generated digital ground three-dimensional reconstruction model. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0043] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0044] Figure 1This is a first flow chart of the first embodiment of the digital ground three-dimensional reconstruction method based on neural radiation field and bidirectional reflectance distribution function of the present application;

[0045] Figure 2 This is a second flow chart of the first embodiment of the digital ground three-dimensional reconstruction method based on neural radiation field and bidirectional reflectance distribution function of the present application;

[0046] Figure 3 This is a first flow chart of the second embodiment of the digital ground three-dimensional reconstruction method based on neural radiation field and bidirectional reflectance distribution function of the present application;

[0047] Figure 4 This is a second flow chart of the second embodiment of the digital ground three-dimensional reconstruction method based on neural radiation field and bidirectional reflectance distribution function of the present application;

[0048] Figure 5 This is a third flow chart of the second embodiment of the digital ground three-dimensional reconstruction method based on neural radiation field and bidirectional reflectance distribution function of the present application;

[0049] Figure 6 This is a schematic diagram of the model improvement effect of the second embodiment of the digital ground three-dimensional reconstruction method based on neural radiation field and bidirectional reflectance distribution function of the present application;

[0050] Figure 7 This is a schematic diagram of the module structure of a digital ground three-dimensional reconstruction device based on neural radiation field and bidirectional reflectance distribution function according to an embodiment of the present application;

[0051] Figure 8 Schematic diagram of the device structure of the hardware operating environment involved in the digital ground three-dimensional reconstruction method based on neural radiation field and bidirectional reflectance distribution function in the embodiment of the present application.

[0052] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0053] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0054] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0055] The main solution of this application is: input the satellite sampling light data corresponding to the target area into the preset neural radiation field model to obtain the scene information corresponding to the target area; construct a physical imaging model based on the bidirectional reflectance distribution function according to the scene information; optimize the preset neural radiation field model based on the physical imaging model to obtain the target neural radiation field model; determine the scene elevation information of the target area through the target neural radiation field model, and generate a ground three-dimensional reconstruction model corresponding to the target area.

[0056] At present, due to the significant discontinuity of building elevation information in urban satellite image processing, especially between high-rise buildings, the generated digital surface model (DSM) is difficult to accurately capture the real terrain changes. This height difference not only causes geometric distortion, but also introduces artifacts and irregular edges, thereby reducing the accuracy of the generated digital surface model.

[0057] To solve the above problems, the present application first obtains multi-perspective sampling light data corresponding to the satellite image data of the target area, that is, the above-mentioned satellite sampling light data, and then inputs the satellite sampling light data into a preset neural radiation field model to analyze the three-dimensional scene information of the satellite image data of the target area, and obtains the scene information corresponding to the target area to optimize the image texture, and then constructs a physical imaging model based on the bidirectional reflectance distribution function based on the scene information, so as to optimize the preset neural radiation field model based on the physical imaging model, and finally predicts accurate scene information based on the optimized target neural radiation field model, and then determines the high-precision elevation information corresponding to the target area based on the accurate scene information, thereby significantly improving the accuracy of the final generated digital ground three-dimensional reconstruction model.

[0058] It should be noted that the execution subject of this embodiment can be a ground three-dimensional reconstruction system, or a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or a digital ground three-dimensional reconstruction device based on neural radiation field and bidirectional reflectance distribution function that can realize the above functions, etc. This embodiment does not specifically limit this. The following takes the digital ground three-dimensional reconstruction device based on neural radiation field and bidirectional reflectance distribution function (referred to as reconstruction device) as an example of the execution subject to illustrate this embodiment and the following embodiments.

[0059] Based on this, the embodiment of the present application provides a digital ground three-dimensional reconstruction method based on neural radiation field and bidirectional reflectance distribution function, referring to Figure 1 , Figure 1 This is a first flow chart of the first embodiment of the digital ground three-dimensional reconstruction method based on neural radiation field and bidirectional reflectance distribution function of the present application.

[0060] In this embodiment, the digital ground three-dimensional reconstruction method based on neural radiation field and bidirectional reflectance distribution function includes steps S10 to S40:

[0061] Step S10, inputting the satellite sampling light data corresponding to the target area into the preset neural radiation field model to obtain the scene information corresponding to the target area;

[0062] It should be understood that the reason why the existing three-dimensional reconstruction methods have low accuracy is that the collected satellite image data has areas with poor image texture or occlusion due to objective reasons such as urban distribution. Therefore, in order to improve the accuracy of the ground three-dimensional model, this embodiment can optimize the generation of digital surface models based on the Neural Radiance Field (NeRF). However, although NeRF performs well in processing complex scenes (such as irregular surfaces, objects of different materials, etc.), due to the particularity of satellite camera models, the large distance between the camera and the scene, and the inconsistency of images taken on different dates, traditional NeRF is not easy to combine with satellite images for large-scale three-dimensional reconstruction of terrain. Therefore, this embodiment constructs the above-mentioned preset neural radiation field model based on Sat-NeRF.

[0063] It is understandable that the above-mentioned satellite sampling light data can be obtained by the reconstruction device after pre-processing the satellite image data of the target area to generate data in an input format that conforms to the preset neural radiation field model. Specifically, the present embodiment can perform image cropping of specific areas on the satellite image data, and at the same time combine the bundle adjustment to optimize the RPC (Rational Polynomial Coefficient) model to adapt to the input data format of the preset neural radiation field model. Among them, the above-mentioned satellite sampling light data may include the coordinates of the sampling point on each sampling light, the direction of the sun, and the index of the image to which the sampling light belongs. In this embodiment, the above-mentioned preset neural radiation field model can learn the implicit representation of the scene through a neural network when inputting the satellite sampling light data, and use the learned scene information to generate high-quality new perspective images and present high-precision texture details, thereby improving the subsequent model accuracy.

[0064] It is easy to understand that the present embodiment can encode the volume information (i.e., radiation field) of the scene in the target area into the above-mentioned preset neural radiation field model. The corresponding input of the above-mentioned preset neural radiation field model is the three-dimensional coordinates and viewing direction of the scene sampling point, and the output is the color and density of the position. Through the information of multi-view images and camera postures, the preset neural radiation field model can interpolate at different points in the scene, and generate high-quality new viewing images through a volume rendering algorithm. Specifically, in the present embodiment, the satellite sampling light data can be processed by a series of fully connected layer models in the preset neural radiation field model to output the volume density, albedo (color), shadow scalar, ambient light (color) and uncertainty coefficient reflecting the probability of transient objects at the sampling point, that is, the scene information corresponding to the above-mentioned target area is obtained.

[0065] Step S20, constructing a physical imaging model based on a bidirectional reflectance distribution function according to the scene information;

[0066] It is easy to understand that the large shadow areas caused by buildings blocking sunlight usually lack texture information, making it difficult for image matching to accurately find pixel correspondences, increasing the difficulty of reconstruction. These shadows cover up the real information on the ground and the subtle structure of the building, resulting in incomplete three-dimensional reconstruction results and accumulated errors. Sat-NeRF cannot handle the reconstruction of shadow areas in the image well, and the reconstruction accuracy of areas with less texture on the edges of buildings and roofs needs to be improved. Therefore, after the sampling point coordinates, the direction of the sun, and the index of the image to which the sampling light belongs on each sampling light are input into the preset neural radiation field model to obtain scene information such as density, albedo color, shadow scalar, ambient light color, and uncertainty coefficient reflecting the probability of transient objects at the sampling point, the physical imaging model based on the above-mentioned bidirectional reflectance distribution function (BRDF) can be constructed based on the above-mentioned series of scene information, and then the preset neural radiation field model can be model-constrained Iterate based on the scene illumination information learned by the physical imaging model to improve the accuracy of the subsequently generated three-dimensional reconstruction model.

[0067] It should be noted that the volume density information of each sampling point in the scene information can constitute a density field. In this embodiment, during the process of constructing a physical imaging model according to the scene information output by a preset neural radiation field model, the reconstruction device can further extract the normal vector information of the scene surface in the target area based on the density field information corresponding to the scene information and optimize it. Thus, through iterative training, the BRDF-based physical imaging model can correctly learn the scene lighting information, thereby effectively improving the accuracy of subsequent determination of the scene elevation information based on the target neural radiation field model.

[0068] Therefore, in a feasible implementation mode, referring to Figure 2 , Figure 2 This is a second flow chart of the first embodiment of the digital ground 3D reconstruction method based on neural radiation field and bidirectional reflectance distribution function of the present application. In this embodiment, step S20 may include steps A1 to A3:

[0069] Step A1, determining an initial normal vector corresponding to a scene object in the target area according to density field information corresponding to the scene information;

[0070] Step A2, optimizing the initial normal vector based on the satellite image data corresponding to the target area and the density field information to obtain an optimized normal vector;

[0071] Step A3: constructing a physical imaging model based on a bidirectional reflectance distribution function according to the optimized normal vector.

[0072] It should be understood that in this embodiment, the reconstruction device can obtain the surface normal vector of the scene object based on the density field gradient corresponding to the volume density field output by the preset neural radiation field model. The density field gradient reflects the rate and direction of density change in space. The direction of the gradient points to the direction of the fastest density change, and the magnitude of the gradient represents the degree of density change. is a scalar field in three-dimensional space. The gradient of the density field can approximate the normal vector of the object surface. Specifically, the above density field gradient It can be expressed as:

[0073]

[0074] Among them, P=(x, y, z) represents the position in space, Represents the gradient vector of the density field at position P, and the unit normal vector at position P, that is, the initial normal vector It can be expressed as:

[0075]

[0076] It is understandable that in the process of extracting the surface normals of scene objects from the density field, the lack of prior information may cause the calculated normals to have a certain degree of randomness, and may even be opposite to the direction of the true normals.

[0077] Therefore, in order to further improve the reliability of the model, in a feasible implementation, in this embodiment, step A2 may include steps A21-A22:

[0078] Step A21, determining the target ray direction based on the satellite image data corresponding to the target area and the density field information;

[0079] Step A22, optimizing the direction of the initial normal vector according to the target ray direction to obtain an optimized normal vector.

[0080] It should be understood that in order to solve the above problem, this embodiment introduces a constraint method, that is, first, the coordinates of the camera origin are determined according to the satellite image data, the coordinates of the surface point are determined according to the density field information corresponding to the scene information, and then the ray direction from the surface point to the camera is calculated by the coordinates of the surface point and the coordinates of the camera origin. , which is the target ray direction mentioned above. Then, the reconstruction device can obtain the normal vector With ray direction The inner product operation is performed to determine whether its value is greater than zero, so as to ensure that the angle between the normal vector and the ray direction is less than 90° and there is no reverse problem. Specifically, if the result of the inner product operation is greater than zero, it means that the angle between the normal vector and the ray direction is less than 90°. At this time, the initial normal vector can be directly used as the optimized normal vector. ; If the result of the inner product operation is less than zero, it means that the angle between the normal vector and the ray direction is greater than 90°, and the initial normal vector needs to be negated to obtain the optimized normal vector .

[0081] It is easy to understand that this embodiment can ensure that the angle between the normal of the surface point observed in the camera's field of view and the line of sight is always maintained within a reasonable range through the above steps, thereby correctly extracting the normal vector information of the surface of the object in the three-dimensional scene and building a physical imaging model based on the correctly extracted optimized normal vector.

[0082] Step S30, optimizing the preset neural radiation field model based on the physical imaging model to obtain a target neural radiation field model;

[0083] Step S40, determining the scene elevation information of the target area through the target neural radiation field model, and generating a ground three-dimensional reconstruction model corresponding to the target area.

[0084] It is easy to understand that the present application can generate a synthetic image through a physical imaging model constructed based on an optimized normal vector, and perform a loss on the synthetic image and the true image corresponding to the satellite image data of the target area, and then add the loss to the training of a preset neural radiation field model to optimize its model parameters to obtain a target neural radiation field model, and finally use the target neural radiation field model to learn and predict the information of the surface points of scene objects in the target area to obtain accurate scene information, and use the optimized scene information to infer the high-precision elevation information corresponding to the target area, and then obtain a high-precision reconstructed digital ground three-dimensional model corresponding to the target area based on the high-precision elevation information.

[0085] Specifically, in this embodiment, high-precision elevation information can be calculated based on the optimized scene information predicted by the target neural radiation field model, that is, the elevation information of each scene surface point in the ground three-dimensional reconstruction model can be obtained by weighted summing the weight of each sampling point on the sampling light in the optimized scene information (required by the density field information) and the depth value of each sampling point.

[0086] In this embodiment, the reconstruction device first obtains the multi-view sampling light data corresponding to the satellite image data, and then inputs the sampling light data into the preset neural radiation field model to analyze the three-dimensional scene information of the satellite image data of the target area, and obtains the scene information corresponding to the target area. Then, this embodiment provides an effective solution to the problem of normal vector extraction in digital model generation, accurately extracts and optimizes the normal vector based on the density field information in the scene information, and constructs a physical imaging model based on the bidirectional reflectance distribution function in combination with the optimized normal vector and the predicted scene information, so as to optimize the preset neural radiation field model based on the physical imaging model, and finally accurately predicts the scene information based on the optimized target neural radiation field model and infers the high-precision elevation information corresponding to the target area based on the optimized scene information, thereby significantly improving the accuracy of the final generated digital ground three-dimensional reconstruction model.

[0087] To summarize, this embodiment can optimize the normal vector so that the physical imaging model based on the bidirectional reflectance distribution function can better learn the scene lighting information, optimize the model parameters of the preset neural radiation field, and thus improve the scene information prediction accuracy of the target neural radiation field model optimized based on the physical imaging model, so that the final target neural radiation field model can accurately predict the scene information of the target area, and then can infer high-precision elevation information based on the optimized scene information, thereby significantly improving the geometric shape accuracy of the final generated digital ground three-dimensional reconstruction model.

[0088] The present embodiment provides a method for digital ground three-dimensional reconstruction based on neural radiation field and bidirectional reflectance distribution function, the method comprising: inputting the satellite sampling light data corresponding to the target area into a preset neural radiation field model to obtain the scene information corresponding to the target area; determining the initial normal vector corresponding to the scene object in the target area according to the density field information corresponding to the scene information; determining the target ray direction based on the satellite image data and density field information corresponding to the target area; optimizing the direction of the initial normal vector according to the target ray direction to obtain an optimized normal vector. Construct a physical imaging model based on the bidirectional reflectance distribution function according to the optimized normal vector. Optimize the preset neural radiation field model based on the physical imaging model to obtain a target neural radiation field model; determine the scene elevation information of the target area through the target neural radiation field model, and generate a ground three-dimensional reconstruction model corresponding to the target area. This embodiment can optimize the normal vector so that the physical imaging model based on the bidirectional reflectance distribution function can better learn the scene lighting information and optimize the model parameters of the preset neural radiation field, thereby improving the scene information prediction accuracy of the target neural radiation field model optimized based on the physical imaging model, so that the final target neural radiation field model can accurately predict the scene information of the target area, and then can infer high-precision elevation information based on the optimized scene information, thereby significantly improving the geometric shape accuracy of the final generated digital ground three-dimensional reconstruction model.

[0089] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction and will not be repeated later.

[0090] Based on the first embodiment, please refer to Figure 3 , Figure 3 This is a first flow chart of the second embodiment of the digital ground 3D reconstruction method based on neural radiation field and bidirectional reflectance distribution function of the present application. In this embodiment, step A3 includes steps B1-B2:

[0091] Step B1, determining a spherical harmonic function basis based on the optimized normal vector;

[0092] Step B2: constructing a physical imaging model based on a bidirectional reflectance distribution function according to the albedo corresponding to the scene information, the density field information and the spherical harmonic function basis.

[0093] It should be noted that the present embodiment may assume that the surface reflection characteristics of scene objects in the target area conform to the Lambertian reflection model (a type of bidirectional reflection distribution function model) in the BRDF empirical model, and the Lambertian reflection model based on spherical harmonics may assume that the reflection intensity of the object surface is uniform in all directions, ignoring the reflection direction of the light. Therefore, the present embodiment may construct a physical imaging model by introducing the Lambertian reflection model based on spherical harmonics to more accurately learn and describe the relationship between the illumination characteristics and geometric shapes in the scene, that is, assuming that the reflection characteristics of the object reflection surface in the target area are only related to the reflectivity of the object surface and the incident direction of the light during the imaging process. This feature enables the constructed physical imaging model to effectively capture the surface illumination effect, thereby generating a more realistic image.

[0094] Specifically, in this embodiment, the image generation process can be regarded as a convolution in a spherical coordinate system through the Lambertian reflection model of spherical harmonics, and the illumination intensity can be expanded into a set of mutually orthogonal spherical harmonic function bases. In the implementation process, this embodiment can use a second-order spherical harmonic function basis to approximate the influence of illumination. Although spherical harmonics can theoretically reach infinite order, low-order spherical harmonics are usually used to simplify the computational complexity while obtaining more accurate results. This low-order representation not only simplifies the computational complexity, but also accurately reflects the influence of ambient light on the reflection of the object surface while ensuring image quality.

[0095] Specifically, the spherical harmonic function basis of the Lambertian body reflection model based on spherical harmonic functions defined based on the optimized normal vector in this embodiment can be expressed as follows:

[0096]

[0097]

[0098]

[0099] in, Represents the unit normal vector corresponding to the i-th object surface point, which corresponds to the above optimized normal vector.

[0100] In a feasible implementation manner, in this embodiment, step B2 may include steps B21 to B23:

[0101] Step B21, determining a target illumination coefficient corresponding to the target area according to the shadow scalar corresponding to the scene information and the density field information;

[0102] Step B22, determining the point reflectivity corresponding to the surface of the scene object in the target area based on the albedo corresponding to the scene information and the density field information;

[0103] Step B23, constructing the physical imaging model according to the point reflectivity, the spherical harmonic function basis and the target illumination coefficient.

[0104] It is easy to understand that this embodiment can use the albedo of the object surface point based on the RGB three-channel (that is, the albedo corresponding to the above scene information), and the above target illumination coefficient, spherical harmonic function basis, to construct a physical imaging model for generating a scene composite image.

[0105] It is understandable that, in this embodiment, the above physical imaging model can be defined as:

[0106]

[0107] in, Represents the image intensity corresponding to the i-th object surface point; Represents the reflectivity of the i-th object surface point; represents the parameters corresponding to the spherical harmonic function basis, that is, the above-mentioned target illumination coefficient; represents the spherical harmonic basis.

[0108] It can be understood that, from the first embodiment, the scene information output by the preset neural radiation field model may include the albedo (color) and shadow scalar at the sampling point. The reflectivity of the object surface point in formula (4) is the above-mentioned point reflectivity, which can be determined based on the object surface point albedo based on the RGB three-channel contained in the scene information predicted by the preset neural radiation field model, that is, the albedo corresponding to the above-mentioned scene information. It may include the values ​​of the three RGB channels, each of which corresponds to the reflection characteristics under the red, green and blue spectra, reflecting the object surface's ability to respond to light of different wavelengths. Accordingly, the image intensity of the object imaging model in this embodiment can be represented by the RGB three-channel representation, and the image intensity of each channel is obtained by comprehensive calculation of the corresponding reflectivity, spherical harmonic function basis and target illumination parameters.

[0109] It is easy to understand that in this embodiment, the illumination coefficient of the physical imaging model can be The illumination coefficient is divided into direct sunlight and ambient illumination coefficient. The reason is that in satellite image data, there are often shadow areas caused by the blocking of sunlight. In these shadow areas, the illumination conditions no longer directly depend on the direct sunlight of the sun, but are affected by the sky light and ambient light. Therefore, in this embodiment, the illumination coefficient of the physical imaging model can be It is divided into direct sunlight illumination coefficient and ambient illumination coefficient, so that corresponding illumination coefficients can be used for model training in different areas. Therefore, this embodiment can determine the specific illumination coefficient corresponding to the scene in the target area, that is, the target illumination coefficient, according to the shadow scalar information and density field information corresponding to the scene information predicted by the preset neural radiation field, so as to build a fine model based on the target illumination coefficient.

[0110] In a feasible implementation manner, in this embodiment, referring to Figure 4 , Figure 4 This is a second flow chart of the second embodiment of the digital ground 3D reconstruction method based on neural radiation field and bidirectional reflectance distribution function of the present application. Step B21 may include steps C1 to C3:

[0111] Step C1, determining the sampling point weight of the sampling ray based on the density field information;

[0112] Step C2, determining the pixel shadow amount at the target pixel corresponding to the sampling ray according to the shadow scalar corresponding to the scene information and the sampling point weight;

[0113] Step C3, determining a target illumination coefficient corresponding to the target area based on the pixel shadow amount.

[0114] It should be understood that in this embodiment, the shadow scalar contained in the target area scene information output by the preset neural radiation field model is the shadow scalar at each sampling point on the sampling ray corresponding to the satellite image data. Therefore, this embodiment can infer the transmittance and opacity of each point in the scene according to the scene density determined in the above density scene information, obtain the above sampling point weight, and then combine the scene shadow scalar and sampling point weight determined in the above satellite sampling light data to calculate the final shadow amount corresponding to each pixel, obtain the pixel shadow amount at the target pixel corresponding to the entire sampling ray, and divide the target area scene into direct sunlight area and indirect sunlight area based on the pixel shadow amount at the target pixel, so as to define the illumination coefficient of different areas and determine the target illumination coefficient corresponding to the target area. Among them, the target pixel is each pixel point in the scene composite image subsequently generated by the physical imaging model.

[0115] Specifically, in this embodiment, the shadow amount S at the corresponding target pixel observed in the direction of the sampling ray r can be expressed as:

[0116]

[0117] in, Represents the sampling point shadow scalar at each sampling point on the sampling ray, and are the transmittance and opacity at each sampling point on each sampling ray, which together constitute the above sampling point weights. and Following the definition in Sat-NeRF:

[0118]

[0119]

[0120] in, Indicates sampling point The density value at Represents the distance between two adjacent sampling points on the ray, Larger values ​​will result in greater opacity. , indicate the point May be on an opaque surface.

[0121] It is easy to understand that in the light and shadow rendering based on the shadow scalar of 0~1, the intensity of the shadow is inversely proportional to the size of the scalar value. When the sampled shadow value is close to 0, the shadow is more obvious, the brightness in the scene is reduced, and the area appears darker; when the sampled shadow value is close to 1, the light intensity is strong, the shadow is weakened or even disappears. Therefore, this embodiment can set the area where the sampled shadow amount is between 0.2 and 1 to use the sunlight illumination coefficient as the target illumination coefficient, and other areas use the ambient illumination coefficient.

[0122] It should be noted that this embodiment can optimize the preset neural radiation field model according to the constructed physical imaging model, so as to solve the problem that the existing Sat-NeRF model cannot handle the reconstruction of shadow areas in the image well, and then output accurate scene information such as the albedo of the object surface, shadow scalar, and ambient light based on the optimized target neural radiation field model, and then infer accurate elevation information based on the precise scene information to generate a ground reconstruction model corresponding to the target area.

[0123] In one possible implementation, refer to Figure 5 , Figure 5 This is a third flow chart of the second embodiment of the digital ground 3D reconstruction method based on neural radiation field and bidirectional reflectance distribution function of the present application. In this embodiment, step S30 may include steps D1 to D4:

[0124] Step D1, generating a scene composite image corresponding to the target area based on the physical imaging model;

[0125] Step D2, generating a scene rendering image corresponding to the target area according to the scene information;

[0126] Step D3, determining a preliminary reconstruction loss based on the true value image corresponding to the satellite image data, the scene composite image and the scene rendering image;

[0127] Step D4, performing model iteration on the preset neural radiation field model according to the preset normal optimization constraint and the preliminary reconstruction loss to obtain a target neural radiation field model.

[0128] It is easy to understand that this embodiment can generate a scene composite image corresponding to each object in the target area based on the image intensity corresponding to the physical imaging model, and then perform light and shadow rendering based on the scene information such as albedo color, shadow scalar, ambient light color output by the preliminary constructed preset neural radiation field model to obtain a scene rendered image.

[0129] Afterwards, the reconstruction device can calculate the error loss based on the scene synthetic image generated by the physical imaging model and the rendered image corresponding to the preset neural radiation field model and the true image corresponding to the satellite image data, thereby providing constraints for the model parameter optimization of the preset neural radiation field model. This process not only enables the preset neural radiation field model to effectively adjust its internal parameters to reduce prediction errors, but also enhances its understanding of the lighting and reflection characteristics in the scene.

[0130] Therefore, this embodiment repeatedly iterates the preset neural radiation field model based on the scene synthetic image generated by the physical imaging model, so that the target neural radiation field model generated after the final iteration can learn the optimal scene information, covering density distribution, elevation changes and other related features. Ultimately, the target neural radiation field model can generate an optimized digital ground model based on the obtained information, which retains the scene details while improving the overall geometric accuracy and visual realism.

[0131] It can be understood that in this embodiment, the above-mentioned preliminary reconstruction loss may include a first loss value calculated based on the difference between the scene rendering image generated according to the scene information output by the preset neural radiation field model and the real image, and a second loss value calculated based on the difference between the scene synthetic image generated according to the physical imaging model and the real image.

[0132] Among them, the first loss value The corresponding loss function can be expressed as follows:

[0133]

[0134] Where N is the number of image pixels rendered by the preset neural radiation field model based on the predicted scene information, The pixel values ​​of the rendered image for the scene, is the pixel value of the corresponding pixel in the true image.

[0135] The second loss value The corresponding loss function can be expressed as follows:

[0136]

[0137] Where N is the number of pixels of the scene synthetic image generated by the physical imaging model, is the pixel value of the scene synthetic image, is the pixel value of the corresponding pixel in the true image.

[0138] Therefore, in this embodiment, by minimizing the loss function corresponding to the first loss value and the second loss value, the model can be effectively optimized to better fit the lighting and reflection characteristics of the real scene.

[0139] In addition, in order to ensure that the normal of the object surface is smooth, this embodiment also introduces a normal smoothness constraint, that is, the above-mentioned preset normal optimization constraint, and is expressed as follows:

[0140]

[0141] in, To calculate the target point The target normal vector is obtained when the density field gradient is The normal vector obtained for the density field gradient for its neighboring points.

[0142] It should be understood that, in this embodiment, the smoothness of the normal can be effectively controlled by minimizing the difference between the normals of the target point and its neighboring points through the above-mentioned preset normal optimization constraint, thereby improving the quality of subsequent three-dimensional reconstruction.

[0143] Therefore, in this embodiment, the first loss value between the scene rendering image originally generated by the preset neural radiation field model and the true value image is combined , the second loss for calculating the difference between the scene synthetic image generated by the physical imaging model and the real image And preset normal optimization constraints To optimize model training, the complete loss function corresponding to the preset neural radiation field model is It can be expressed as:

[0144] in, and is the weight of each sub-item.

[0145] In this embodiment, by introducing a physical imaging model constructed based on a Lambertian body reflectance model of spherical harmonics, the target neural radiation field model generated by the final iteration can more accurately learn and describe the relationship between the illumination characteristics and geometric shapes in the scene. This comprehensive method effectively reduces the reconstruction error of the shadow area in the image and improves the reliability and accuracy of the overall model. At the same time, this embodiment improves the reliability of the model by constructing different illumination coefficients in the physical imaging model; and combines the normal vector smoothing constraint to perform model iteration, further improving the geometric shape accuracy of the reconstructed scene model.

[0146] It is easy to understand that the model accuracy improvement effect of the digital ground 3D reconstruction based on the neural radiation field and the bidirectional reflectance distribution function proposed in this embodiment can be as follows: Figure 6 As shown, Figure 6 This is a schematic diagram of the model improvement effect of the second embodiment of the digital ground three-dimensional reconstruction method based on neural radiation field and bidirectional reflectance distribution function of the present application. Figure 6 In the figure, A represents the reconstructed digital ground 3D model generated based on the original Sat-NeRF, and a1~a3 are the detail areas in the original 3D model; B represents the reconstructed digital ground 3D model based on the target neural radiation field model after the model optimization proposed in this embodiment, and b1~b3 are the detail areas in the optimized 3D model. Figure 6 It is easy to see from the overall model clarity comparison between A and B and the model edge detail comparison between a1~a3 and b1~b3 that, compared with the prior art, the model geometry accuracy in the optimized reconstructed digital ground three-dimensional model of this embodiment is significantly improved.

[0147] To summarize, this embodiment, based on the assumption that the surface reflection characteristics of objects in the target area conform to the Lambertian reflection model in the BRDF empirical model, adopts a deep neural network to implicitly learn the three-dimensional scene information in static satellite images, so that the model can accurately represent the key information such as the volume density, the albedo of the object surface, the shadow scalar and the ambient light in the scene, thereby achieving the effect of significantly improving the geometric shape accuracy of the reconstructed scene model by accurately extracting and optimizing the normal vector, constructing a physical imaging model, and combining the normal vector smoothing constraint to optimize the preset neural radiation field model.

[0148] This embodiment determines the spherical harmonic function basis based on the optimized normal vector; determines the sampling point weight of the sampling ray based on the density field information; determines the pixel shadow amount at the target pixel corresponding to the sampling ray according to the shadow scalar corresponding to the scene information and the sampling point weight; determines the target illumination coefficient corresponding to the target area based on the pixel shadow amount; determines the point reflectivity corresponding to the surface of the scene object in the target area based on the albedo corresponding to the scene information and the density field information; and constructs a physical imaging model based on the point reflectivity, the spherical harmonic function basis and the target illumination coefficient. At the same time, this embodiment generates a scene synthesis image corresponding to the target area based on the physical imaging model; generates a scene rendering image corresponding to the target area based on the scene information; determines the preliminary reconstruction loss based on the true value image, the scene synthesis image and the scene rendering image corresponding to the satellite image data; and iterates the preset neural radiation field model according to the preset normal optimization constraint and the preliminary reconstruction loss to obtain the target neural radiation field model. This embodiment introduces a physical imaging model constructed by a Lambertian body reflection model based on spherical harmonics, so that the target neural radiation field model generated by the final iteration can more accurately learn and describe the relationship between the illumination characteristics and geometric shapes in the scene. This comprehensive method effectively reduces the reconstruction error of the shadow area in the image and improves the reliability and accuracy of the overall model. At the same time, this embodiment improves the reliability of the model by constructing different illumination coefficients in the physical imaging model; and combines the normal vector smoothing constraint to perform model iteration, thereby further improving the geometric shape accuracy of the reconstructed scene model.

[0149] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the digital ground three-dimensional reconstruction method based on neural radiation field and bidirectional reflectance distribution function of the present application. More forms of simple transformations based on this technical concept are all within the scope of protection of the present application.

[0150] This application also provides a digital ground three-dimensional reconstruction device based on neural radiation field and bidirectional reflectance distribution function, please refer to Figure 7 , Figure 7 This is a schematic diagram of the module structure of a digital ground three-dimensional reconstruction device based on a neural radiation field and a bidirectional reflectance distribution function in an embodiment of the present application. In this embodiment, the digital ground three-dimensional reconstruction device based on a neural radiation field and a bidirectional reflectance distribution function includes:

[0151] Density field analysis module 701, used to input satellite sampling light data corresponding to the target area into a preset neural radiation field model to obtain scene information corresponding to the target area;

[0152] A scene analysis module 702, configured to construct a physical imaging model based on a bidirectional reflectance distribution function according to the scene information;

[0153] An optimization module 703 is used to optimize the preset neural radiation field model based on the physical imaging model to obtain a target neural radiation field model;

[0154] The three-dimensional reconstruction module 704 is used to determine the scene elevation information of the target area through the target neural radiation field model, and generate a ground three-dimensional reconstruction model corresponding to the target area.

[0155] As an implementable method, in this embodiment, the scene analysis module 702 is further used to determine the initial normal vector corresponding to the scene object in the target area according to the density field information corresponding to the scene information;

[0156] The scene analysis module 702 is further used to optimize the initial normal vector based on the satellite image data corresponding to the target area and the density field information to obtain an optimized normal vector;

[0157] The scene analysis module 702 is further used to construct a physical imaging model based on the bidirectional reflectance distribution function according to the optimized normal vector.

[0158] As an implementable method, in this embodiment, the scene analysis module 702 is further used to determine the target ray direction based on the satellite image data corresponding to the target area and the density field information;

[0159] The scene analysis module 702 is further used to optimize the direction of the initial normal vector according to the target ray direction to obtain an optimized normal vector.

[0160] As an implementable method, in this embodiment, the scene analysis module 702 is further used to determine the spherical harmonic function basis based on the optimized normal vector;

[0161] The scene analysis module 702 is further used to construct a physical imaging model based on the bidirectional reflectance distribution function according to the albedo corresponding to the scene information, the density field information and the spherical harmonic function basis.

[0162] As an implementable method, in this embodiment, the scene analysis module 702 is further used to determine the target illumination coefficient corresponding to the target area according to the shadow scalar corresponding to the scene information and the density field information;

[0163] The scene analysis module 702 is further used to determine the point reflectivity corresponding to the surface of the scene object in the target area based on the albedo corresponding to the scene information and the density field information;

[0164] The scene analysis module 702 is further used to construct the physical imaging model according to the point reflectivity, the spherical harmonic function basis and the target illumination coefficient.

[0165] As an implementable method, in this embodiment, the scene analysis module 702 is further used to determine the sampling point weight of the sampling ray based on the density field information;

[0166] The scene analysis module 702 is further used to determine the pixel shadow amount at the target pixel corresponding to the sampling ray according to the shadow scalar corresponding to the scene information and the sampling point weight;

[0167] The scene analysis module 702 is further configured to determine a target illumination coefficient corresponding to the target area based on the shadow amount.

[0168] As an implementable method, in this embodiment, the optimization module 703 is further used to generate a scene composite image corresponding to the target area based on the physical imaging model;

[0169] The optimization module 703 is further used to generate a scene rendering image corresponding to the target area according to the scene information;

[0170] The optimization module 703 is further used to determine the preliminary reconstruction loss based on the true value image corresponding to the satellite image data, the scene composite image and the scene rendering image;

[0171] The optimization module 703 is further used to perform model iteration on the preset neural radiation field model according to the preset normal optimization constraint and the preliminary reconstruction loss to obtain a target neural radiation field model.

[0172] The digital ground three-dimensional reconstruction device based on neural radiation field and bidirectional reflectance distribution function provided by the present application adopts the digital ground three-dimensional reconstruction method based on neural radiation field and bidirectional reflectance distribution function in the above-mentioned embodiment, which can solve the technical problem of ground three-dimensional reconstruction. Compared with the prior art, the beneficial effects of the digital ground three-dimensional reconstruction device based on neural radiation field and bidirectional reflectance distribution function provided by the present application are the same as the beneficial effects of the digital ground three-dimensional reconstruction method based on neural radiation field and bidirectional reflectance distribution function provided by the above-mentioned embodiment, and other technical features in the digital ground three-dimensional reconstruction device based on neural radiation field and bidirectional reflectance distribution function are the same as the features disclosed in the above-mentioned embodiment method, which will not be repeated here.

[0173] The present application provides a digital ground three-dimensional reconstruction device based on neural radiation field and bidirectional reflectance distribution function. The digital ground three-dimensional reconstruction device based on neural radiation field and bidirectional reflectance distribution function includes: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the digital ground three-dimensional reconstruction method based on neural radiation field and bidirectional reflectance distribution function in the above-mentioned embodiment one.

[0174] Reference below Figure 8 , which shows a schematic diagram of the structure of a digital ground 3D reconstruction device based on neural radiation field and bidirectional reflectance distribution function suitable for implementing the embodiment of the present application. The digital ground 3D reconstruction device based on neural radiation field and bidirectional reflectance distribution function in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants: personal digital assistants), PADs (Portable Application Description: tablet computers), PMPs (Portable Media Players: portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 8 The digital ground three-dimensional reconstruction device based on neural radiation field and bidirectional reflectance distribution function shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0175] like Figure 8 As shown, the digital ground three-dimensional reconstruction device based on neural radiation field and bidirectional reflectance distribution function may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM: Read Only Memory) 1002 or the program loaded from the storage device 1003 to the random access memory (RAM: Random Access Memory) 1004. Various programs and data required for the operation of the digital ground three-dimensional reconstruction device based on neural radiation field and bidirectional reflectance distribution function are also stored in RAM1004. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. The input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 can allow the digital ground three-dimensional reconstruction device based on neural radiation fields and bidirectional reflectance distribution functions to communicate wirelessly or wired with other devices to exchange data. Although the figure shows a digital ground three-dimensional reconstruction device based on neural radiation fields and bidirectional reflectance distribution functions with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have instead.

[0176] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a ground three-dimensional reconstruction program product, which includes a ground three-dimensional reconstruction program carried on a computer-readable medium, and the ground three-dimensional reconstruction program contains program code for executing the method shown in the flowchart. In such an embodiment, the ground three-dimensional reconstruction program can be downloaded and installed from the network through a communication device, or installed from a storage device 1003, or installed from ROM1002. When the ground three-dimensional reconstruction program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0177] The digital ground three-dimensional reconstruction device based on neural radiation field and bidirectional reflectance distribution function provided by the present application adopts the digital ground three-dimensional reconstruction method based on neural radiation field and bidirectional reflectance distribution function in the above-mentioned embodiment, which can solve the technical problem of ground three-dimensional reconstruction. Compared with the prior art, the beneficial effects of the digital ground three-dimensional reconstruction device based on neural radiation field and bidirectional reflectance distribution function provided by the present application are the same as the beneficial effects of the digital ground three-dimensional reconstruction method based on neural radiation field and bidirectional reflectance distribution function provided by the above-mentioned embodiment, and the other technical features in the digital ground three-dimensional reconstruction device based on neural radiation field and bidirectional reflectance distribution function are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0178] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0179] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0180] The present application provides a storage medium having computer-readable program instructions (i.e., a ground three-dimensional reconstruction program) stored thereon, and the computer-readable program instructions are used to execute the digital ground three-dimensional reconstruction method based on neural radiation field and bidirectional reflectance distribution function in the above-mentioned embodiment.

[0181] The storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0182] The above-mentioned storage medium can be included in the digital ground three-dimensional reconstruction device based on neural radiation field and bidirectional reflectance distribution function; or it can exist independently without being assembled into the digital ground three-dimensional reconstruction device based on neural radiation field and bidirectional reflectance distribution function.

[0183] The above-mentioned storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the digital ground three-dimensional reconstruction device based on neural radiation field and bidirectional reflectance distribution function, the digital ground three-dimensional reconstruction device based on neural radiation field and bidirectional reflectance distribution function can perform high-precision ground three-dimensional reconstruction.

[0184] The ground three-dimensional reconstruction program code for performing the operation of the present application can be written in one or more programming languages ​​or a combination thereof, and the above-mentioned programming languages ​​include object-oriented programming languages, such as python, Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on the remote computer, or completely on the remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0185] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and ground three-dimensional reconstruction program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the function marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the function involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0186] The modules involved in the embodiments of the present application may be implemented by software or hardware, wherein the name of the module does not limit the unit itself in some cases.

[0187] The readable storage medium provided by the present application is a storage medium, which stores computer-readable program instructions (i.e., a ground three-dimensional reconstruction program) for executing the above-mentioned digital ground three-dimensional reconstruction method based on neural radiation field and bidirectional reflectance distribution function, and can solve the technical problem of ground three-dimensional reconstruction. Compared with the prior art, the beneficial effects of the storage medium provided by the present application are the same as the beneficial effects of the digital ground three-dimensional reconstruction method based on neural radiation field and bidirectional reflectance distribution function provided by the above-mentioned embodiment, and will not be repeated here.

[0188] The above are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A method for three-dimensional reconstruction of digital ground based on neural radiation field and bidirectional reflectance distribution function, characterized in that: The method comprises: Inputting the satellite sampling light data corresponding to the target area into a preset neural radiation field model to obtain scene information corresponding to the target area; Constructing a physical imaging model based on a bidirectional reflectance distribution function according to the scene information; Optimizing the preset neural radiation field model based on the physical imaging model to obtain a target neural radiation field model; Determine the scene elevation information of the target area through the target neural radiation field model, and generate a ground three-dimensional reconstruction model corresponding to the target area; The step of constructing a physical imaging model based on a bidirectional reflectance distribution function according to the scene information comprises: Determine the initial normal vector corresponding to the scene object in the target area according to the density field information corresponding to the scene information; Optimizing the initial normal vector based on the satellite image data corresponding to the target area and the density field information to obtain an optimized normal vector; Constructing a physical imaging model based on a bidirectional reflectance distribution function according to the optimized normal vector; The step of constructing a physical imaging model based on a bidirectional reflectance distribution function according to the optimized normal vector comprises: determining a spherical harmonic basis based on the optimized normal vector; Constructing a physical imaging model based on a bidirectional reflectance distribution function according to the albedo corresponding to the scene information, the density field information and the spherical harmonic function basis; The step of constructing a physical imaging model based on a bidirectional reflectance distribution function according to the albedo corresponding to the scene information, the density field information and the spherical harmonic function basis comprises: Determine a target illumination coefficient corresponding to the target area according to the shadow scalar corresponding to the scene information and the density field information; Determine the point reflectivity corresponding to the surface of the scene object in the target area based on the albedo corresponding to the scene information and the density field information; The physical imaging model is constructed according to the point reflectivity, the spherical harmonic function basis and the target illumination coefficient.

2. The method for digital ground three-dimensional reconstruction based on neural radiation field and bidirectional reflectance distribution function according to claim 1, characterized in that: The step of optimizing the initial normal vector based on the satellite image data corresponding to the target area and the density field information to obtain the optimized normal vector comprises: Determine the target ray direction based on the satellite image data corresponding to the target area and the density field information; The initial normal vector is optimized in direction according to the target ray direction to obtain an optimized normal vector.

3. The method for digital ground three-dimensional reconstruction based on neural radiation field and bidirectional reflectance distribution function according to claim 1, characterized in that: The step of determining the target illumination coefficient corresponding to the target area according to the shadow scalar corresponding to the scene information and the density field information comprises: Determine the sampling point weight of the sampling ray based on the density field information; Determine the pixel shadow amount at the target pixel corresponding to the sampling ray according to the shadow scalar corresponding to the scene information and the sampling point weight; A target illumination coefficient corresponding to the target area is determined based on the pixel shadow amount.

4. The method for digital ground three-dimensional reconstruction based on neural radiation field and bidirectional reflectance distribution function according to claim 1, characterized in that: The step of optimizing the preset neural radiation field model based on the physical imaging model to obtain a target neural radiation field model further includes: Generate a scene composite image corresponding to the target area based on the physical imaging model; Generate a scene rendering image corresponding to the target area according to the scene information; Determine a preliminary reconstruction loss based on a true image corresponding to the satellite image data, the scene composite image, and the scene rendering image; The preset neural radiation field model is iterated according to the preset normal optimization constraint and the preliminary reconstruction loss to obtain a target neural radiation field model.

5. A digital ground three-dimensional reconstruction device based on neural radiation field and bidirectional reflectance distribution function, characterized in that: The digital ground three-dimensional reconstruction device based on neural radiation field and bidirectional reflectance distribution function includes: A density field analysis module, used to input the satellite sampling light data corresponding to the target area into a preset neural radiation field model to obtain scene information corresponding to the target area; A scene analysis module, used to construct a physical imaging model based on a bidirectional reflectance distribution function according to the scene information; An optimization module, used for optimizing the preset neural radiation field model based on the physical imaging model to obtain a target neural radiation field model; A three-dimensional reconstruction module, used to determine the scene elevation information of the target area through the target neural radiation field model, and generate a ground three-dimensional reconstruction model corresponding to the target area; The scene analysis module is further used to determine the initial normal vector corresponding to the scene object in the target area according to the density field information corresponding to the scene information; optimize the initial normal vector based on the satellite image data corresponding to the target area and the density field information to obtain an optimized normal vector; and construct a physical imaging model based on the bidirectional reflectance distribution function according to the optimized normal vector; The scene analysis module is further used to determine a spherical harmonic function basis based on the optimized normal vector; and to construct a physical imaging model based on a bidirectional reflectance distribution function according to the albedo corresponding to the scene information, the density field information and the spherical harmonic function basis; The scene analysis module is also used to determine the target illumination coefficient corresponding to the target area based on the shadow scalar corresponding to the scene information and the density field information; determine the point reflectivity corresponding to the surface of the scene object in the target area based on the albedo corresponding to the scene information and the density field information; and construct the physical imaging model based on the point reflectivity, the spherical harmonic function basis and the target illumination coefficient.

6. A digital ground 3D reconstruction device based on neural radiation field and bidirectional reflectance distribution function, characterized in that: The digital ground three-dimensional reconstruction device based on neural radiation field and bidirectional reflectance distribution function includes: a memory, a processor and a ground three-dimensional reconstruction program stored in the memory and executable on the processor, wherein the ground three-dimensional reconstruction program is configured to implement the steps of the digital ground three-dimensional reconstruction method based on neural radiation field and bidirectional reflectance distribution function as described in any one of claims 1 to 4.

7. A storage medium, characterized in that: The storage medium stores a ground three-dimensional reconstruction program, which, when executed by a processor, implements the steps of a digital ground three-dimensional reconstruction method based on a neural radiation field and a bidirectional reflectance distribution function as described in any one of claims 1 to 4.

Images