Efficient mapping method fusing entropy coding and neural symbol distance field

Abstract

The invention discloses an efficient mapping method fusing entropy coding and a neural symbol distance field. A symbol distance field of a scene is predicted by using a multi-layer perceptron network. Parameter quantization and entropy coding methods are introduced, network parameters are quantized into integers, probability distribution of the network parameters is modeled by using a neural network, and then entropy coding is combined, so that the number of model parameters is reduced while the neural network is trained. An entropy loss function is designed, an information entropy of network model parameters is used as a constraint to be embedded into a neural symbol distance field training process, and a unified optimization framework is used for realizing dynamic balance between reconstruction precision and a model compression ratio. According to the method, memory occupation can be remarkably reduced while high-quality scene reconstruction is ensured, and an efficient solution is provided for three-dimensional reconstruction of a large-scale scene.

Description

Technical Field

[0001] The present invention belongs to the field of three-dimensional reconstruction technology under the field of computer vision, and specifically relates to an efficient mapping method integrating entropy coding and neural signed distance field. Background Art

[0002] Three-dimensional scene reconstruction is a key research topic in computer vision and robotics. Its core goal is to efficiently and accurately recover the scene's geometric structure and topological information from sensor data. Current mainstream methods can be divided into two categories based on the scene representation: discretized reconstruction methods based on traditional geometric representations and continuous implicit representation methods based on deep learning.

[0003] Traditional 3D reconstruction methods typically use explicit geometric representations such as point clouds, meshes, or voxels. These methods achieve surface reconstruction through multi-view geometry optimization or dense point cloud registration. Their advantages lie in the interpretability of explicit geometry and mature hardware-accelerated processing pipelines. However, discretized representations have inherent drawbacks: (1) storage complexity increases cubically with scene size, and large-scale scene reconstruction can easily generate terabytes of data; (2) spatial resolution is limited by the preset accuracy, making it difficult to achieve an adaptive balance between local detail enhancement and global sparse storage; and (3) geometric completion of partially occluded or low-texture areas relies on heuristic interpolation algorithms, which can easily lead to surface distortion or holes.

[0004] In recent years, breakthroughs have been made in continuous reconstruction methods based on neural implicit representations. The paradigms represented by Neural Radiance Fields (NeRF) and Neural Signed Distance Function establish continuous mapping functions from three-dimensional coordinates to scene attributes (such as radiance and signed distance values) through multi-layer perceptrons (MLPs). Compared with traditional discrete representations, its advantages are: (1) it can adaptively allocate the storage resources required for modeling to each location in the environment without being limited by resolution; (2) it can reasonably fill in the holes in unobserved areas; (3) it has the characteristics of continuous differentiability, which is easy to optimize; and (4) neural implicit representations are usually more compact than traditional methods and take up less space.

[0005] However, existing neural scene representation methods still require a large number of parameters when modeling larger-scale scenes, which increases storage and transmission overhead. Summary of the Invention

[0006] The purpose of the present invention is to address the shortcomings of the existing technology and propose an efficient mapping method that integrates entropy coding and neural signed distance field, which can perform dense three-dimensional reconstruction of the scene in real time and significantly reduce storage overhead while ensuring the reconstruction quality.

[0007] The object of the present invention is achieved through the following technical solutions:

[0008] According to a first aspect of the present invention, there is provided an efficient mapping method integrating entropy coding and neural signed distance field, comprising the following steps:

[0009] S1, obtains the depth image data stream of the known pose of the scene;

[0010] S2, builds a neural signed distance field network that integrates the entropy coding model, quantizes the network parameters into integer representations, and additionally builds a small neural network to model the probability distribution of the signed distance field network parameters;

[0011] S3, by comparing the information gain of the previous frame, a sparse set of key frames is selected from the depth image data stream, and random sampling is performed based on the key frame set. During the sampling process, the line of sight direction obtained based on the pose is sampled, and the resulting sparse 3D point cloud is used as the training set;

[0012] S4, training the neural signed distance field network: input the spatial coordinates of the point cloud into the neural signed distance field network, output the predicted signed distance value of each spatial point, use the distance to the nearest surface point of the batch of point clouds as the approximate signed distance of each spatial point, and construct the reconstruction loss function;

[0013] Use a small neural network to predict the probability distribution of parameters, calculate the sum of information entropy of network parameters, and construct entropy loss;

[0014] The sum of reconstruction loss and entropy loss is used as the total loss function to perform self-supervised training on the neural signed distance field network;

[0015] S5, input the spatial coordinates into the trained neural signed distance field network to obtain the corresponding SDF value, extract the zero level set of SDF to obtain the scene surface reconstruction result.

[0016] Furthermore, the acquisition of a depth image data stream of a scene with a known posture specifically includes:

[0017] A depth camera is used to collect the depth image sequence of the scene in real time, obtain the timestamp and depth image of each frame, use the SLAM algorithm to get the corresponding pose estimation, align the depth image data stream with the corresponding pose information according to the timestamp, and construct a depth image stream with pose labels.

[0018] Furthermore, the step of filtering out a sparse key frame set from the depth image data stream is specifically as follows:

[0019] Maintain a key frame set and add the first frame as the initial frame to the key frame set. When each frame image is subsequently input, the information gain of the frame compared to the previous key frame is calculated. When the information gain is greater than the preset threshold, the current frame is added to the key frame set.

[0020] The formula for calculating information gain is as follows:

[0021]

[0022] For the depth map D of the current frame, randomly sample s pixels [u, v] and render them to get the predicted depth corresponding to the point The statistically normalized depth error is then less than the threshold t D The proportion of points in the sampling points is P, if the proportion is less than the threshold t P , then add the current frame to the key frame set.

[0023] Furthermore, the sparse three-dimensional point cloud obtained by sampling the key frame set is specifically:

[0024] For each frame, we first randomly select 200 pixels [u, v] on the depth map; then, we calculate the line of sight r = T for each pixel in the world coordinate system based on the camera intrinsic parameter K. WC K -1 [u,b], sample 30 points of different depths along the line of sight of each sampled pixel,

[0025] Among them, 20 points are uniformly sampled in the range [7, D[u, v] + 10], and then the Gaussian distribution is used to obtain the Sample 9 points in the middle and finally sample point d on the depth map i =D[u,v];

[0026] The entire point cloud set obtained by sampling point clouds from the two frames of images at the most recent moment and the three frames of images randomly sampled from the key frame set is used as a batch of the training set.

[0027] Furthermore, the neural signed distance field network of the fusion entropy coding model is specifically:

[0028] The Neural Signed Distance Field network is an MLP network with 6 hidden layers, 256 neural units in each layer, and Softplus as the activation function. The input of the network is the 3D spatial coordinate c = (x, y, z), and the output is the predicted signed distance value scalar. Before the spatial coordinate is input to the hidden layer, it is first mapped to a high-dimensional vector through position encoding.

[0029] During training, the network's weights and bias parameters are quantized to integer values, and the probability distribution of the parameters is modeled. An additional small neural network is used to fit the quantized parameter probability density function.

[0030] Furthermore, the additional small neural network learns a single variable cumulative distribution function and a corresponding probability density function to fit the quantized parameter probability density function.

[0031] Furthermore, the reconstruction loss function includes:

[0032]

[0033] Among them, λ grad and λ eik are the coefficients of the gradient supervision loss function and the Eikonal constraint loss function, respectively. is the dequantization operation;

[0034] is the SDF loss function, specifically:

[0035]

[0036] Among them, λ surf is the coefficient of the near-surface loss function,

[0037] x is the input sampling point cloud, b is the distance threshold, and θ is the weight and bias parameter of the neural signed distance field network;

[0038] is the gradient supervision loss function, specifically:

[0039]

[0040] Where g is the approximate gradient calculated between the sampling point and the nearest surface point;

[0041] is the Eikonal constraint loss function, specifically:

[0042]

[0043] D[u,v] is the depth map of the current frame.

[0044] Furthermore, the entropy loss is specifically the sum of the information entropy of all parameters:

[0045]

[0046] Among them, I(ω)=-log2p(ω) is the parameter information entropy of each layer of the neural network after quantization.

[0047] Furthermore, the total loss function is:

[0048]

[0049] in To reconstruct the loss function, is the entropy loss, the entropy loss coefficient λ entropy Used to achieve a dynamic balance between reconstruction accuracy and compression rate.

[0050] According to another aspect of the specification, an efficient mapping device that integrates entropy coding and neural signed distance field is also provided, comprising a memory and one or more processors, wherein the memory stores executable code, and when the processor executes the executable code, an efficient mapping method that integrates entropy coding and neural signed distance field is implemented.

[0051] The beneficial effects of the present invention are as follows:

[0052] 1. Parameter quantization and entropy coding methods are introduced. By quantizing network parameters into integers and using neural networks to model their probability distribution, and then combining them with entropy coding, end-to-end neural network model compression is achieved.

[0053] 2. The key frame screening and mixed sampling strategy based on information gain is adopted to reduce redundant data sampling, improve training efficiency, and support online real-time reconstruction.

[0054] 3. A new loss function is designed, which embeds the information entropy of the parameters as a constraint in the neural signed distance field training process, and uses a unified optimization framework to achieve a dynamic balance between reconstruction accuracy and model compression rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 A schematic diagram of the overall framework provided by an embodiment of the present invention;

[0056] Figure 2 A schematic diagram of the neural signed distance field network structure used in the method of the present invention provided in an embodiment of the present invention;

[0057] Figure 3 Comparison diagram of the final reconstruction results of the method of the present invention and the uncompressed method in multiple scenarios provided by the embodiment of the present invention;

[0058] Figure 4 A graph showing the reconstruction results and compression ratios of the method of the present invention under different entropy loss coefficients provided in an embodiment of the present invention;

[0059] Figure 5 Schematic diagram of an efficient mapping device that integrates entropy coding and neural signed distance field, provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0061] The embodiment of the present invention provides an efficient mapping method that integrates entropy coding and neural signed distance field. Figure 1 The overall framework flow chart provided for the example of the present invention includes the following steps:

[0062] S1, using a depth camera to obtain a depth image data stream of a known pose of the scene.

[0063] First, an RGB-D depth camera is used to collect the depth image sequence of the scene in real time, and the timestamp and depth image D of each frame are obtained. At the same time, the visual SLAM algorithm ORB-SLAM3 is run synchronously to calculate the pose T corresponding to each depth image frame through feature point matching and pose graph optimization. WC , align the depth image data stream with the corresponding pose information by timestamp and construct a depth image stream with pose labels.

[0064] S2, build an SDF neural signed distance field network integrated with the entropy coding model, quantize the network parameters into integer representation, and additionally design a small neural network to model the probability distribution of the signed distance field network parameters;

[0065] The SDF neural signed distance field network of the fusion entropy coding model is constructed as follows: the structure of the neural signed distance field network is as follows Figure 2 Figure 1 shows an MLP network with 6 hidden layers, 256 neural units per layer, and Softplus as the activation function. The neural network input is a 3D spatial coordinate x = (x, y, z), and the output is a predicted signed distance scalar s. Before entering the hidden layer, the spatial coordinates are first mapped to a high-dimensional vector using position encoding:

[0066] γ(x)=[sin(2 0 Ax),cos(2 0 Ax),...,sin(2 L Ax),cos(2 L Ax)]

[0067] Where L = 5, and the matrix Each row of consists of the unit normal vectors of the icosahedron subdivided twice:

[0068]

[0069] In addition, the result of position encoding is also directly connected to the third hidden layer.

[0070] During training, all weights and bias parameters θ of the network are quantized to integers Scalar quantization is performed on each layer parameter ω∈θ of the neural network The parameters are first affine transformed by two learnable parameters {α, β} and then rounded to the nearest integer:

[0071]

[0072] Then, the dequantization operation is naturally defined as:

[0073]

[0074] Since we quantize each layer of the neural network separately, the affine transformation parameters {α, β} learned by each layer are independent of each other, which allows for more adaptive quantization accuracy.

[0075] In addition, since quantization causes gradients to vanish and backpropagation becomes impossible, we use the StraightThrough Estimator (STE) method to perform quantization during forward propagation and skip the gradient calculation during backpropagation.

[0076] At the same time, we also need to model the probability distribution of the quantized parameters and use an additional small neural network to fit the probability density function of the quantized parameters. Specifically, we need to learn a single variable cumulative distribution function c: And the corresponding probability density function p: The following properties need to be met:

[0077]

[0078] We use a series of composite functions to compose the cumulative distribution function:

[0079]

[0080] Among them, each function f k : In order to satisfy the properties of a single variable, the domain of the function f1 and the function f K The range of the value should be one-dimensional (d1 = r K =1). Therefore, an effective f k The function design is as follows:

[0081] f k (x) = g k (H (k) x+b(k) )1≤k <K

[0082] f K (x) = sigmoid(H (K) x+b (K) )

[0083] Among them, g k (x) = x + a (k) ⊙tanh(x). And the matrix H (k) , vector a (k) and b (k) These are all the parameters that can be learned by this small neural network. k The derivative of a function is defined as:

[0084] f′ k (x) = diagg′ k (H (k) x+b (k) )·H (k) 1≤k <K,with

[0085] g′ k (x) = 1 + a (k) ⊙tanh′(x)and

[0086] f′ K (x) = sigmoid′(H (K) x+b (K) )·H (K)

[0087] In order to ensure the non-negativity of p(x), it is necessary to restrict H (k) All elements are non-negative, and a (k) All elements of are lower bounded by -1, which is achieved by the following reparameterization:

[0088]

[0089] We set K = 4 and r1 = r2 = r3 = 3. This parameter design allows us to fit the probability distribution of the quantized neural network parameters well during training. Furthermore, after training, the probability distribution of the parameter probabilities can be stored in a discrete probability table, and the network weights are no longer needed.

[0090] S3, a sparse set of key frames is screened out from the depth image stream with pose labels, and a sparse 3D point cloud is further sampled from the key frame set as a training set.

[0091] Furthermore, the method for filtering key frames and sampling point clouds is as follows: during initialization, a key frame set is maintained, and the first frame is added to the key frame set as the initial frame. When each frame image is subsequently input, the information gain of the frame compared to the previous key frame is calculated. When the information gain is greater than the preset threshold, the current frame is added to the key frame set. The formula for calculating information gain is as follows:

[0092]

[0093] For the depth map D of the current frame, randomly sample s pixel points [u, v] and input them into the neural network to render the predicted depth corresponding to the point Then the normalized depth error is statistically calculated to be less than the threshold t2 (set to 0.1) of the points in the sampling points. If the ratio is less than the threshold t P (set to 0.65), the current frame is added to the key frame set. In each iteration, a total of 5 frames of depth images are sampled on the point cloud, including the 2 most recent frames and the 3 frames randomly sampled from the key frame set.

[0094] For each frame, we first randomly select 200 pixels [u, v] on the depth map; then, we calculate the sight direction r = T of each pixel in the world coordinate system based on the camera intrinsic parameter K. WC K -1 [u,v], sample 30 points of different depths along the line of sight of each sampling pixel, among which 20 points are uniformly sampled in the range [7,D[u,v]+10], and then sample 30 points from the Gaussian distribution. Sample 9 points in the middle and finally sample point d on the depth map i = D[u,v] (distance units are all cm). All point cloud sets sampled on 5 frames of images are used as a batch of training sets for subsequent training.

[0095] S4, during training, the spatial coordinates of the point cloud are input into the neural signed distance field network, and the predicted signed distance value of each spatial point is output. The SDF neural signed distance field network fits a function f: The input is a 3D spatial coordinate, which is then positionally encoded and mapped to a high-dimensional feature vector. This feature vector is passed through a hidden layer, and the output is a predicted signed distance value for that point, representing the distance between the point and the nearest surface. When the point is inside the surface, the distance is negative, and when it is outside, it is positive.

[0096] For self-supervised training, we use each sample point x to the batch point cloud The distance to the nearest surface point in the space is used as the approximate signed distance for each spatial point Construct the reconstruction loss function.

[0097] First construct the loss function of SDF prediction There are two cases to discuss. When the sampling point is close to the surface point, the loss function is constructed directly using the predicted signed distance value and the L1 norm of the approximate signed distance.

[0098] f is the function fitted by the neural network, x is the input sampling point cloud, θ is the parameter of the neural network, and b is the distance threshold. When the distance between the sampling point and the nearest surface point is greater than the threshold, the sampling point is considered to be in free space. At this time, if the predicted value is positive and less than b, the loss is zero. If the predicted value is greater than b, the loss is a linear function. If the predicted value is negative, it is an exponential function, defined as Therefore, the SDF loss function The definition is as follows:

[0099]

[0100] Second, we can calculate an approximate gradient based on the sampling point and the nearest surface point Construct a loss function that supervises the gradient of the sampling points:

[0101]

[0102] In addition, the modulus of the gradient of the sampling point needs to satisfy the Eikonal equation so that the properties of the signed distance field can be satisfied. Therefore, the Eikonal constraint loss function is constructed:

[0103]

[0104] The weighted sum of the above loss functions can be used to obtain the reconstruction loss function:

[0105]

[0106] Among them, λ surf =5,λ grad =0.02,λ eik =0.25. surf is the coefficient of the near-surface loss function, λ grad is the coefficient of the gradient supervision loss function, λ eik is the coefficient of the eikonal constraint loss function,

[0107] At the same time, we use the probability distribution of the small neural network prediction parameters described in S3 to calculate the sum of the information entropy of the network parameters and then construct the entropy loss function. For the quantized parameter ω of each layer of the neural network, its information entropy is calculated as I(ω) = -log2p(ω). Therefore, the entropy loss function is the sum of the information entropy of all parameters:

[0108]

[0109] The weighted sum of the reconstruction loss and entropy loss function is used as the total loss function. In addition, by adjusting the entropy loss coefficient λ entropy It can achieve a dynamic balance between reconstruction accuracy and compression rate, and perform self-supervised training on the neural signed distance field network. When calculating information entropy, it is necessary to use quantized numbers for calculation. In the forward propagation process of calculating reconstruction loss, it is necessary to perform dequantization on the quantized parameters before calculation:

[0110]

[0111] S5, after training is completed, the space is divided into several voxel grids, and the vertex coordinates are input into the neural signed distance field network to query the corresponding SDF value, and then the marching cubes algorithm is used to extract the zero level set of the SDF

[0112] Then the scene surface reconstruction result is obtained; in addition, according to the probability distribution of the parameters, the model can be entropy coded and compressed using the arithmetic coding rANS (Range Asymmetric numeral systems) algorithm, and then decoded when necessary.

[0113] The following are the experimental details of the embodiment:

[0114] This example uses the ReplicaCAD and ScanNet datasets. Two sequences (apt_2_nav, apt_3_nav) were simulated and generated from the ReplicaCAD dataset, while two long sequences (scene_0030, scene_0031) and two short sequences (scene_0004, scene_0005) were randomly selected from the ScanNet dataset for experimentation. We compared the results with the uncompressed neural signed distance field (iSDF) algorithm, measuring the following metrics on randomly sampled spatial coordinates: SDF error, mesh accuracy, mesh integrity, and model compression rate.

[0115] The SDF error is the absolute error between the SDF value predicted by the neural signed distance field and the true value. The accuracy of mapping surface details can be measured intuitively:

[0116] Mesh accuracy is the average distance between the sample points in the reconstructed mesh and the nearest true point. Used to evaluate how close the reconstructed map is to the overall shape of the real map.

[0117] The mesh completeness measures the average distance between the sample points in the real mesh and the nearest reconstructed mesh point. Used to measure the completeness and filling ability of the reconstructed map.

[0118] The compression ratio is the ratio of the parameter sizes of the uncompressed model to the model after compression in this embodiment.

[0119] First, in λ entropy = 0.001, the method of the present invention was tested with the uncompressed method in multiple scenarios, and the final reconstruction results are compared as shown in the figure below. Figure 3 It can be found that the method of the present invention is effective when only the uncompressed model parameters are used. In this case, the reconstruction effect can still be achieved.

[0120] The comparison of indicators in different scenarios is shown in Tables 1, 2, 3, and 4. It can be found that the method of the present invention is not only better than the iSDF method of the uncompressed model in SDF error, mesh accuracy, and mesh integrity, but also can significantly reduce the required model parameters.

[0121] Table 1 SDF errors of different methods

[0122]

[0123] Table 2 Mesh accuracy of different methods

[0124]

[0125] Table 3 Mesh integrity of different methods

[0126]

[0127] Table 4 Comparison of model parameter size (compression rate) of different methods

[0128]

[0129] To test the algorithm's real-time capabilities, we calculated that each neural network iteration takes an average of 20ms. This includes 2ms for sampling, 2ms for calculating the approximate signed distance, 6ms for forward propagation, and 10ms for backward propagation. This time consumption is acceptable for online, real-time reconstruction.

[0130] In addition, we tested different λ settings in the same scenario apt_3_nav entropyWe use SDF error as a measure of the reconstruction effect and draw the SDF error-compression rate curve as shown in the figure below: Figure 4 As shown. It can be seen that as λ entropy As the entropy loss coefficient λ increases, the SDF error increases accordingly, but the model size also decreases significantly, which shows that we can adjust the entropy loss coefficient λ entropy A dynamic balance between reconstruction accuracy and compression rate can be achieved.

[0131] Corresponding to the aforementioned embodiment of an efficient mapping method integrating entropy coding and neural signed distance field, the present invention also provides an embodiment of an efficient mapping device integrating entropy coding and neural signed distance field.

[0132] See also Figure 5 An embodiment of the present invention provides an efficient mapping device that integrates entropy coding and neural signed distance field, including a memory and one or more processors. The memory stores executable code, and when the processor executes the executable code, it is used to implement an efficient mapping method that integrates entropy coding and neural signed distance field in the above embodiment.

[0133] The embodiment of the efficient mapping device that integrates entropy coding and neural signed distance field provided by the present invention can be applied to any device with data processing capabilities, and the device with data processing capabilities can be a device or apparatus such as a computer. The device embodiment can be implemented through software, or through hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor of any device with data processing capabilities in which it is located reading the corresponding computer program instructions in the non-volatile memory into the memory for execution. From the hardware level, if Figure 5 As shown, this is a hardware structure diagram of an efficient mapping device that integrates entropy coding and neural signed distance field provided by the present invention, in which any device with data processing capabilities is located. Figure 5 In addition to the processor, memory, network interface, and non-volatile memory shown, any device with data processing capabilities in which the apparatus in the embodiment is located may also include other hardware, generally based on the actual functions of the device with data processing capabilities, which will not be described in detail.

[0134] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0135] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments. The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present invention. A person of ordinary skill in the art can understand and implement the present invention without inventive work.

[0136] An embodiment of the present invention further provides a computer-readable storage medium having a program stored thereon. When the program is executed by a processor, an efficient mapping method integrating entropy coding and neural signed distance field in the above embodiment is implemented.

[0137] The computer-readable storage medium may be an internal storage unit of any device with data processing capabilities described in any of the aforementioned embodiments, such as a hard disk or memory. The computer-readable storage medium may also be an external storage device of any device with data processing capabilities, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both an internal storage unit and an external storage device of any device with data processing capabilities. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capabilities, and may also be used to temporarily store data that has been output or is to be output.

[0138] The present invention also provides a computer program product, comprising a computer program, which, when executed by a processor, implements the efficient mapping method that integrates entropy coding and neural signed distance field.

[0139] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the contents disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of the present application and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered merely as exemplary, and the true scope and spirit of the present application are indicated by the claims.

[0140] It should be understood that the above general description and the detailed description that follows are exemplary and explanatory only and do not limit the present application. The present application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes may be made without departing from the scope of the present application. The scope of the present application is limited only by the appended claims.

Claims

1. An efficient mapping method integrating entropy coding and neural signed distance field, characterized in that: The following steps are involved: S1, obtains the depth image data stream of the known pose of the scene; S2, builds a neural signed distance field network that integrates the entropy coding model, quantizes the network parameters into integer representations, and additionally builds a small neural network to model the probability distribution of the signed distance field network parameters; S3, by comparing the information gain of the previous frame, a sparse set of key frames is selected from the depth image data stream, and random sampling is performed based on the key frame set. During the sampling process, the line of sight direction obtained based on the pose is sampled, and the resulting sparse 3D point cloud is used as the training set; S4, training the neural signed distance field network: input the spatial coordinates of the point cloud into the neural signed distance field network, output the predicted signed distance value of each spatial point, use the distance to the nearest surface point of the batch of point clouds as the approximate signed distance of each spatial point, and construct the reconstruction loss function; Use a small neural network to predict the probability distribution of parameters, calculate the sum of information entropy of network parameters, and construct entropy loss; The sum of reconstruction loss and entropy loss is used as the total loss function to perform self-supervised training on the neural signed distance field network; S5, input the spatial coordinates into the trained neural signed distance field network to obtain the corresponding SDF value, extract the zero level set of SDF to obtain the scene surface reconstruction result.

2. The efficient mapping method integrating entropy coding and neural signed distance field according to claim 1, characterized in that: The acquisition of the depth image data stream of the known pose of the scene specifically includes: A depth camera is used to collect the depth image sequence of the scene in real time, obtain the timestamp and depth image of each frame, use the SLAM algorithm to get the corresponding pose estimation, align the depth image data stream with the corresponding pose information according to the timestamp, and construct a depth image stream with pose labels.

3. The efficient mapping method integrating entropy coding and neural signed distance field according to claim 1, characterized in that: The method of screening out a sparse key frame set from the depth image data stream is specifically as follows: Maintain a key frame set and add the first frame as the initial frame to the key frame set. When each frame image is subsequently input, the information gain of the frame compared to the previous key frame is calculated. When the information gain is greater than the preset threshold, the current frame is added to the key frame set. The formula for calculating information gain is as follows: For the depth map D of the current frame, randomly sample s pixel points [u, v] and render them through the neural signed distance field network of the fusion entropy coding model to obtain the predicted depth corresponding to the point The statistically normalized depth error is then less than the threshold t S The proportion of points in the sampling points is P, if the proportion is less than the threshold t P , then add the current frame to the key frame set.

4. The efficient mapping method integrating entropy coding and neural signed distance field according to claim 1, characterized in that: The sparse three-dimensional point cloud obtained by sampling the key frame set is specifically: For each frame, we first randomly select 200 pixels [u, v] on the depth map; then, we calculate the sight direction r = T of each pixel in the world coordinate system based on the camera intrinsic parameter K. WC K -1 [u,v], sample 30 points of different depths along the line of sight of each sampled pixel, Among them, 20 points are uniformly sampled in the range [7, D[u, v] + 10], and then the Gaussian distribution is used to obtain the Sample 9 points in the middle and finally sample point d on the depth map i =D[u,v]; The entire point cloud set obtained by sampling point clouds from the two frames of images at the most recent moment and the three frames of images randomly sampled from the key frame set is used as a batch of the training set.

5. The efficient mapping method integrating entropy coding and neural signed distance field according to claim 1, characterized in that: The neural signed distance field network of the fusion entropy coding model is specifically: The Neural Signed Distance Field network is an MLP network with 6 hidden layers, 256 neural units in each layer, and Softplus as the activation function. The input of the network is the 3D spatial coordinate x = (x, y, z), and the output is the predicted signed distance value scalar. Before the spatial coordinate is input to the hidden layer, it is first mapped to a high-dimensional vector through position encoding. During training, the network's weights and bias parameters are quantized to integer values, and the probability distribution of the parameters is modeled. An additional small neural network is used to fit the quantized parameter probability density function.

6. The efficient mapping method integrating entropy coding and neural signed distance field according to claim 1, characterized in that: The additional small neural network learns a single variable cumulative distribution function and a corresponding probability density function to fit the quantized parameter probability density function.

7. The efficient mapping method integrating entropy coding and neural signed distance field according to claim 1, characterized in that: The reconstruction loss function includes: Among them, λ grad and λ eik are the coefficients of the gradient supervision loss function and the Eikonal constraint loss function, respectively. is the dequantization operation; is the SDF loss function, specifically: Among them, λ surf is the coefficient of the near-surface loss function, x is the input sampling point cloud, b is the distance threshold, and θ is the weight and bias parameter of the neural signed distance field network; is the gradient supervision loss function, specifically: Where g is the approximate gradient calculated between the sampling point and the nearest surface point; is the Eikonal constraint loss function, specifically: D[u,v] is the depth map of the current frame.

8. The efficient mapping method integrating entropy coding and neural signed distance field according to claim 1, characterized in that: The entropy loss is specifically the sum of the information entropy of all parameters: Among them, I(ω)=-log2p(ω) is the parameter information entropy of each layer of the neural network after quantization.

9. The efficient mapping method integrating entropy coding and neural signed distance field according to claim 1, characterized in that: The total loss function is: in To reconstruct the loss function, is the entropy loss, the entropy loss coefficient λ entropy Used to achieve a dynamic balance between reconstruction accuracy and compression rate.

10. An efficient mapping device integrating entropy coding and neural signed distance field, comprising a memory and one or more processors, wherein the memory stores executable code, characterized in that: When the processor executes the executable code, an efficient mapping method integrating entropy coding and neural signed distance field is implemented according to any one of claims 1 to 9.

Images