A digital model point sampling supplement method and a storage medium

Abstract

The application discloses a digital model point sampling supplement method and a storage medium, relates to the field of data processing, and comprises the following steps: obtaining the transparency accumulation values of model points on each projection ray in a three-dimensional reconstruction scene model respectively, and determining the geometric coverage degrees of pixel points corresponding to the projection rays based on the transparency accumulation values; determining a target supplement quantity of the model points based on the geometric complexity of the three-dimensional reconstruction scene model; and performing sampling supplement on the model points of the three-dimensional reconstruction scene model based on the geometric coverage degrees of the pixel points and the target supplement quantity of the model points. The application does not need to rely on a texture gradient signal to trigger sampling, eliminates the sampling blind area of weak texture and a complex occlusion area, and takes into account reconstruction accuracy and model lightweight.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method for sampling and supplementing digital model points and a storage medium. Background Technology

[0002] To meet the demand for high-precision 3D scene models in scenarios such as vehicle environmental perception and simulation testing, existing technologies typically employ a large number of 3D Gaussian ellipsoidal points to fit the geometric structure and texture information of the scene, and gradually improve the accuracy of the 3D reconstructed scene model through iterative optimization training. However, due to the sparsity of the initial data and the insufficient model point density in local areas of the scene model caused by iterative optimization operations, the scene model may exhibit holes or missing details. Therefore, existing technologies usually require sampling and supplementing model points to densify the scene model, thereby compensating for holes or missing details.

[0003] However, existing model point sampling supplementation strategies suffer from low sampling supplementation quality, which seriously affects the accuracy of 3D reconstruction of scene models. Summary of the Invention

[0004] In view of the above problems, this application provides a method and storage medium for sampling and supplementing digital model points to improve the quality of model point sampling and supplementation for 3D reconstructed scene models, thereby improving the accuracy of 3D reconstruction of the scene models. The specific solution is as follows:

[0005] The first aspect of this application provides a method for sampling and supplementing points in a digital model, comprising:

[0006] The cumulative transparency value of model points on each projection ray in the 3D reconstructed scene model is obtained respectively. Based on the cumulative transparency value of the model points on the projection ray, the geometric coverage of the pixel point corresponding to the projection ray is determined. Each projection ray is a ray emitted from the projection center and passes through a pixel point in the 3D reconstructed scene model that corresponds to it.

[0007] Based on the geometric complexity of the 3D reconstructed scene model, the number of additional point targets in the model is determined;

[0008] Based on the geometric coverage of each pixel and the number of model point targets to be supplemented, the model points of the 3D reconstructed scene model are sampled and supplemented.

[0009] The sampling and supplementation method for digital model points provided in the first aspect of this application determines the geometric coverage of each pixel by accumulating the transparency value of model points on the projection ray. It eliminates the need to rely on texture gradient signals to trigger sampling and supplementation; instead, it directly quantifies the geometric fill state based on the accumulated transparency of the viewpoint, thus eliminating sampling blind spots caused by weak gradients in areas with weak textures and gradient distortion in complex occluded areas. This method can accurately screen pixels with geometric holes, blurred edges, and geometric collapse defects. Simultaneously, it dynamically determines the target number of model points to be supplemented based on the geometric complexity of the 3D reconstructed scene model, ensuring that the supplementation scale matches the actual geometric needs of the scene. This avoids the contradiction of excessive addition of points in areas without missing areas or insufficient supplementation in missing areas caused by a fixed supplementation number, balancing reconstruction accuracy and model lightweight design.

[0010] In one possible implementation, based on the geometric coverage of each pixel and the number of model point targets to be supplemented, the 3D reconstructed scene model is sampled and supplemented with model points, including:

[0011] Establish the correspondence between the cumulative transparency value of the model points on the projection ray and the corresponding pixel points of the projection ray;

[0012] Pixels whose cumulative transparency value is less than a preset saturation threshold are identified as valid missing pixels;

[0013] Using the geometric coverage of each pixel as a probability distribution, weighted random sampling is performed on each effective missing pixel, and the three-dimensional spatial coordinates and real texture color values ​​of the sampled effective missing pixels are extracted to generate an initial candidate point set.

[0014] The initial candidate point set is deduplicated and fused to obtain a fused candidate point set;

[0015] Each candidate model point in the fusion candidate point set is loaded into the 3D reconstructed scene model.

[0016] The aforementioned feasible solution filters converged regions by setting a pre-defined saturation threshold, concentrating computational resources on areas with genuine geometric defects, thus improving sampling efficiency. Weighted random sampling using geometric coverage as a probability distribution increases the probability of selecting pixels with higher levels of missing data, prioritizing the repair of severe geometric defects. Simultaneously, it extracts real texture color values, ensuring that newly added model points possess texture information consistent with the real scene from the initialization stage, avoiding rendering artifacts caused by texture distortion. The deduplication and fusion step eliminates spatial redundancy caused by multi-view sampling, prevents model point cloud expansion, and ensures that the final set of candidate model points loaded into the model is concise and conflict-free.

[0017] In one possible implementation, determining the number of supplementary point targets in the model based on the geometric complexity of the 3D reconstructed scene model includes:

[0018] Establish a correspondence between the cumulative transparency value of the model points on the projection ray and the corresponding pixel points of the projection ray, and determine the pixel points whose cumulative transparency value is less than a preset saturation threshold as valid missing pixel points;

[0019] Obtain the adjustment factor characterizing the geometric complexity, obtain the total number of current model points of the 3D reconstructed scene model, and obtain the first total number of pixels and the second total number of effective missing pixels respectively;

[0020] The ratio of the second total to the first total is determined as the pixel missing ratio;

[0021] The product of the adjustment factor, the current total number of model points, and the percentage of missing pixels is determined as the number of model point targets to be supplemented.

[0022] The aforementioned possible solutions use the total number of current model points as the computational base, allowing the supplementary quantity to adaptively increase with the model size. This avoids redundancy caused by excessive supplementation in the early stages of training due to a small number of model points, or insufficient supplementation in the later stages due to a large number of model points. The pixel missing ratio directly quantifies the degree of emptiness in the scene's geometry from the current perspective, constructing a dynamic feedback mechanism based on the real-time rendering state. This ensures that sampled resources match the actual missing state of the scene, achieving on-demand allocation. The adjustment factor, combined with the geometric complexity characteristics of the scene itself, globally corrects the total supplementary quantity, ensuring that simple scenes receive fewer supplementary resources and complex scenes receive more. The calculation method of the product of these three factors transforms the adaptive control strategy into an executable quantization algorithm, controlling computational overhead while ensuring reconstruction accuracy.

[0023] In one possible implementation, obtaining the adjustment factor characterizing the geometric complexity includes:

[0024] Obtain the geometric structural feature parameters of the three-dimensional reconstructed scene model;

[0025] The geometric structural feature parameters are normalized, and based on a preset mapping relationship, the normalized geometric structural feature parameters are mapped to the adjustment factor characterizing the geometric complexity. The preset mapping relationship indicates that the adjustment factor increases as the structural feature parameters increase.

[0026] The aforementioned feasible solution eliminates the influence of different physical dimensions and numerical ranges by normalizing the geometric structure feature parameters. It maps feature values ​​from different sources and scales to a standard dimensionless interval, making the pre-defined mapping relationship independent of the absolute numerical value of a specific scene. This enhances the algorithm's universality and robustness across different scales and types of scenes. The pre-defined mapping relationship establishes a positive correlation between the geometric structure feature parameters and the adjustment factor. The more complex the scene's geometry, the larger the generated adjustment factor, thereby driving the system to calculate more model point targets and achieving an adaptive match between supplementary resources and scene complexity.

[0027] In one possible implementation, the step of deduplicating and fusing the initial candidate point set to obtain a fused candidate point set includes:

[0028] Obtain the three-dimensional spatial coordinates and texture color values ​​of each current model point in the three-dimensional reconstructed scene model;

[0029] Based on the three-dimensional spatial coordinates of the current model point and the three-dimensional spatial coordinates of the effective missing pixel point, calculate the spatial distance between the effective missing pixel point and the current model point;

[0030] Based on the spatial distance, the texture color value, and the real texture color value, the initial candidate point set is deduplicated and fused to obtain the fused candidate point set.

[0031] The aforementioned feasible solution employs a dual-determination mechanism combining spatial distance and texture color for deduplication fusion, overcoming the limitations of single-dimensional deduplication methods in complex scenes. Relying solely on spatial distance can easily lead to the accidental deletion of points that are spatially close but actually belong to different geometric entities, resulting in the loss of foreground details; relying solely on texture color can easily misjudge points with the same color but spatially separated as duplicates, causing the scene to lose its sense of depth. By considering both spatial distance and texture color dimensions simultaneously, the solution effectively eliminates the real redundant data brought about by multi-view sampling while preserving high-frequency details and geometric hierarchy in the scene, achieving a balance between deduplication and detail preservation.

[0032] In one possible implementation, the step of performing deduplication and fusion on the initial candidate point set based on the spatial distance, the texture color value, and the real texture color value to obtain the fused candidate point set includes:

[0033] Add the valid missing pixels in the initial candidate point set whose spatial distance from the current model point is not less than a preset spatial distance threshold to the fusion candidate point set;

[0034] The valid missing pixels in the initial candidate point set whose spatial distance from the current model point is less than a preset spatial distance threshold are marked as suspected duplicate points of the current model point.

[0035] If the actual texture color value of the suspected duplicate point is equal to the texture color value of the current model point, the suspected duplicate point is deleted.

[0036] If the true texture color value of the suspected duplicate point is not equal to the texture color value of the current model point, the true texture color value of the suspected duplicate point is updated to the average of the texture color value and the true texture color value, and the updated suspected duplicate point is added to the fusion candidate point set.

[0037] The aforementioned possible solutions introduce a secondary discrimination mechanism based on color information for suspected duplicate points. When a suspected duplicate point has the same color as a current model point, it is determined to be a duplicate sample of the same geometric surface and deleted, accurately eliminating redundant data. When the colors are inconsistent, a color mean update strategy is adopted, updating the real texture color value of the suspected duplicate point to the mean of the two and then retaining it. In essence, this is a fusion and smoothing of multi-view lighting information, enabling newly added points to present a natural and continuous visual effect from multiple perspectives during subsequent rendering, eliminating splicing marks or lighting abrupt changes caused by single-view sampling, and repairing color jumps while preserving geometric layers when dealing with complex occlusion boundaries.

[0038] In one possible implementation, the sampling supplementation method for the digital model points further includes:

[0039] Assign initial geometric parameters and texture rendering parameters to the candidate model points in the 3D reconstructed scene model that belong to the fusion candidate point set;

[0040] Based on the current optimization gradient information of the 3D reconstructed scene model, as well as the initial geometric parameters and texture rendering parameters of the candidate model points, the 3D reconstructed scene model is iteratively optimized.

[0041] The aforementioned possible solutions assign initial geometric and texture rendering parameters to candidate model points, enabling them to be correctly identified by the rendering engine and participate in subsequent calculations. Iterative optimization is performed based on the current optimization gradient information, allowing candidate model points to directly access the dynamically optimized system that has already run multiple iterations. The scene structure information and convergence trend contained in the existing gradient field guide parameter updates, rather than blindly exploring the parameter space, thus shortening the adaptation time of candidate model points and improving overall training efficiency. Candidate model points and existing model points jointly participate in the backpropagation process, achieving global joint optimization. This eliminates stitching marks or lighting inconsistencies that may arise from staged processing, ensuring the overall consistency of the reconstruction results.

[0042] In one possible implementation, determining the geometric coverage of the pixel points corresponding to the projection ray based on the accumulated transparency value of the model points on the projection ray includes:

[0043] Based on the cumulative transparency value of the model points on the projection ray, calculate the initial geometric coverage of the pixel points corresponding to the projection ray;

[0044] Calculate the sum of the cumulative transparency values ​​of the model points on each of the projection rays;

[0045] The quotient of the initial geometric coverage of the pixel and the sum is determined as the geometric coverage of the pixel.

[0046] The aforementioned possible solution normalizes the initial geometric coverage by dividing it by the total global missing values. This eliminates the differences in the absolute distribution range of the cumulative transparency values ​​across different scene types, camera perspectives, and training stages. It maps the geometric coverage of all pixels to a unified probability distribution space, with a sum of 1, forming a standard probability density function. This allows the sampling mechanism to adaptively adjust the relative weights of each pixel regardless of changes in the overall scene missingness, ensuring that the most severely missing regions always receive the highest sampling priority. This guarantees the consistency and stability of the sampling strategy under different operating conditions.

[0047] One possible implementation also includes:

[0048] If the number of iterations equals the preset iteration triggering period, the total number of optimizations for the 3D reconstructed scene model is incremented by 1.

[0049] If the total number of optimizations is less than the preset total number of optimizations threshold, the step of obtaining the cumulative transparency value of model points on each projection ray in the 3D reconstructed scene model is performed.

[0050] When the total number of optimizations equals the preset total number of optimizations threshold, the 3D reconstructed scene model is output.

[0051] The aforementioned feasible solution defines the minimum training interval between two sampling and supplementation operations by pre-setting the iteration trigger period. This provides the model with sufficient time to adjust the parameters of the old and new points using backpropagation. The next round of missing data detection and supplementation is performed only after the local geometry has stabilized. This avoids the problem of frequent insertion of candidate model points causing drastic fluctuations in the loss function or even gradient explosion or oscillations, thus balancing the stability of model training with the timeliness of geometric repair. Furthermore, the total number of optimizations and a pre-set threshold for the total number of optimizations together constitute the global termination condition for the iterative optimization process, preventing invalid computations and infinite loops caused by getting trapped in local minima or overfitting, thereby improving the timeliness of modeling output.

[0052] A second aspect of this application provides a computer storage medium carrying one or more computer programs that, when executed by an electronic device, enable the electronic device to implement the digital model point sampling supplementation method described in the first aspect or any implementation thereof.

[0053] By employing the aforementioned technical solution, the digital model point sampling and supplementation method and storage medium provided in this application determine the geometric coverage of each pixel in the rendered image of the 3D reconstructed scene model based on the cumulative transparency value of all model points on each projection ray in the 3D reconstructed scene model. Based on the geometric complexity of the 3D reconstructed scene model, the target number of model points to be supplemented is determined. Based on the geometric coverage of each pixel and the target number of model points to be supplemented, the 3D reconstructed scene model is sampled and supplemented. This eliminates the need to rely on texture gradient signals to trigger sampling and supplementation; instead, it uses the geometric coverage directly quantified by the cumulative transparency of the viewpoint as the triggering basis. This eliminates sampling blind spots caused by weak gradients in weak texture areas and gradient distortion in complex occluded areas, accurately selecting pixels that can repair scene holes, blurred edges, and geometric collapse defects. Subsequently, based on the geometric complexity of the 3D reconstructed scene model, the target number of model points to be supplemented is determined, avoiding the contradiction of excessive point addition in areas without missing points and insufficient point supplementation in missing areas, balancing reconstruction accuracy and model lightweighting. Therefore, this application improves the quality of model point sampling and supplementation and the 3D reconstruction accuracy of the scene model. Attached Figure Description

[0054] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0055] Figure 1 A flowchart of a sampling supplementation method for digital model points provided in this application;

[0056] Figure 2 A supplementary flowchart for model point sampling in 3D reconstruction provided in this application;

[0057] Figure 3 This is a schematic diagram of the structure of an electronic device provided in this application. Detailed Implementation

[0058] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is for explaining specific embodiments only and is not intended to limit the scope of this application.

[0059] As will be known to those skilled in the art, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0060] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0061] It should be noted that the digital model point sampling and supplementation method and computer-readable storage medium provided in this application improve the quality of model point sampling and supplementation for 3D reconstructed scene models compared with the prior art, thereby improving the accuracy of 3D reconstruction of scene models. Specifically, the existing sampling and supplementation strategy calculates the gradient changes or rendering residuals of pixels in the rendered image and triggers the insertion of new model points in areas with obvious gradient changes and large rendering errors. However, this leads to the following problems with existing sampling supplementation strategies: For weak texture areas (such as deserts, snowfields, flat mud, etc.), the pixel grayscale changes are gradual or even close to zero, and the gradient signal is extremely weak, which cannot reach the sampling trigger threshold. As a result, the geometric defects in these areas cannot be identified and repaired, forming sampling blind spots. For complex occluded areas (such as tree canopy occlusion edges, ravine shadows, terrain occlusion boundaries, etc.), the pixel gradient information is distorted or covered due to the occlusion relationship, which also cannot effectively trigger sampling, resulting in holes, blurring and geometric collapse of occlusion boundaries. Furthermore, existing sampling supplementation strategies usually use a fixed number of supplementation points, which lacks the ability to adaptively perceive the geometric complexity of the scene and the current missing state of the model. It is easy to have the contradiction of over-supplementing points in areas that do not need to be supplemented, while insufficient supplementation in areas that are truly missing. This causes model redundancy expansion and cannot effectively repair geometric defects. This application determines the geometric coverage of each pixel in the rendered image of the 3D reconstructed scene model by accumulating the transparency of all model points on each projection ray in the 3D reconstructed scene model. Based on the geometric complexity of the 3D reconstructed scene model, it determines the number of model point targets to be supplemented. Based on the geometric coverage of each pixel and the number of model point targets to be supplemented, the 3D reconstructed scene model is sampled and supplemented. Compared with existing technologies, this application does not rely on texture gradient signals to trigger sampling and supplementation. Instead, it uses the geometric coverage directly quantified by the cumulative transparency of the viewpoint as the triggering basis, eliminating sampling blind spots caused by weak gradients in weak texture areas and gradient distortion in complex occluded areas. It accurately selects pixels that can repair scene holes, blurred edges, and geometric collapse defects. Subsequently, based on the geometric complexity of the 3D reconstructed scene model, the number of model point targets to be supplemented is determined, avoiding the contradiction of excessive point addition in areas without missing points and insufficient point supplementation in missing points, balancing reconstruction accuracy and model lightweight. Therefore, this application improves the quality of model point sampling and supplementation and the 3D reconstruction accuracy of the scene model.

[0062] This application provides a method for supplementing sampling points in a digital model, the flowchart of which is shown below. Figure 1 As shown, the sampling supplementation method for the digital model points includes:

[0063] S101. Obtain the cumulative transparency value of the model points on each projection ray in the 3D reconstructed scene model, and determine the geometric coverage of the pixel points corresponding to the projection ray based on the cumulative transparency value of the model points on the projection ray. Each projection ray is a ray emitted from the projection center and passes through a pixel point corresponding to it in the 3D reconstructed scene model.

[0064] It should be noted that in practical applications, the aforementioned 3D reconstructed scene model refers to a digital 3D representation obtained by fitting the geometric structure and texture information of a real physical scene using a large number of model points (such as 3D Gaussian ellipsoidal particles). Each model point carries attribute parameters such as position, shape, opacity, and color, and all model points together constitute an approximate reconstruction of the scene. During the 3D reconstruction rendering process, a ray is emitted from the projection center of the virtual camera to each pixel on the image plane, and this ray passes through the model points in 3D space. The aforementioned cumulative opacity value characterizes the cumulative degree of light obstruction or transmission by all model points along the ray path, reflecting the physical state of the pixel location being filled by geometry from the current viewpoint. The aforementioned geometric coverage is an indicator further derived from the cumulative opacity value, used to quantify the completeness of the geometric structure filling along the ray direction, rather than the texture change rate or color gradient that is the focus of traditional techniques.

[0065] It should be noted that in practical applications, a low cumulative transparency value on a ray indicates sparse or missing geometry in that direction. A higher geometric coverage indicates geometric voids or underexpression in the spatial region represented by that pixel. Conversely, a high cumulative transparency value indicates that the geometry is fully filled, and a lower geometric coverage indicates a lower geometric coverage. This application obtains the cumulative transparency value of model points on each projection ray in the 3D reconstructed scene model through configuration, and determines the geometric coverage of the corresponding pixel based on the cumulative transparency value of the model points on the projection ray. This allows the application to directly perceive the geometric completeness of space, effectively identifying weakly textured regions with smooth textures but missing geometry, as well as complex boundary regions where gradient signals are distorted due to occlusion, thus avoiding the blind spot problem inherent in gradient-driven methods.

[0066] S102. Based on the geometric complexity of the 3D reconstructed scene model, determine the number of additional point targets in the model.

[0067] It should be noted that, in practical applications, the aforementioned geometric complexity is a quantitative representation of the overall or local geometric complexity of the current 3D reconstruction scene model. Different scene types have significantly different requirements for model point density. For example, flat desert or snow scenes have simple geometry and require fewer model points; while jungle, mountain, or urban scenes have large geometric undulations and complex occlusion relationships, requiring more model points for accurate fitting. Using a fixed number of supplementary points often leads to overfitting and model redundancy in simple scenes, or underfitting and loss of detail in complex scenes. Therefore, this application dynamically determines the target number of model point supplements by evaluating geometric complexity. When high geometric complexity is detected, the target number of supplementary points is automatically increased to meet the needs of fine reconstruction; when geometric complexity is low, the number of supplementary points is reduced to maintain model lightweightness. This allows the sampling and supplementation of model points to adapt to various off-road or urban scenes, controlling computational resources and storage overhead while ensuring reconstruction accuracy.

[0068] S103. Based on the geometric coverage of each pixel and the number of model point targets to be supplemented, sample and supplement the model points of the 3D reconstructed scene model.

[0069] It should be noted that geometric coverage provides spatial distribution guidance for sampling, prioritizing sampling in areas with high geometric coverage and severe deficiencies, while the aforementioned target supplementation quantity of model points provides a total constraint on sampling volume. Therefore, by sampling and supplementing model points in the 3D reconstructed scene model based on the geometric coverage of each pixel and the target supplementation quantity of model points, this application achieves targeted and quantitative model point supplementation. For example, geometric coverage can be used as the weight of the sampling probability distribution, and within a defined target supplementation quantity limit, weighted random sampling can be performed on effective missing regions. This approach avoids wasting computational power in converged regions, allowing limited supplementation resources to be concentrated on the geometric defects that need to be repaired. In practical applications, sampling execution, deduplication and fusion, and parameter initialization operations can be implemented in various ways, as long as they follow the principles of transparency coverage guidance and adaptive complexity control, they all fall within the scope of protection of this application.

[0070] This application determines the geometric coverage of each pixel in the rendered image of the 3D reconstructed scene model by accumulating the transparency of all model points on each projection ray in the 3D reconstructed scene model. Based on the geometric complexity of the 3D reconstructed scene model, it determines the number of target model points to supplement. Based on the geometric coverage of each pixel and the number of target model points to supplement, the application samples and supplements model points in the 3D reconstructed scene model. Compared to existing technologies, this application does not rely on texture gradient signals to trigger sampling and supplementation; instead, it uses geometric coverage directly quantified by the cumulative transparency of the viewpoint as the triggering basis. This eliminates sampling blind spots caused by weak gradients in weak texture areas and gradient distortion in complex occluded areas, accurately selecting pixels that can repair scene holes, blurred edges, and geometric collapse defects. Subsequently, based on the geometric complexity of the 3D reconstructed scene model, it determines the number of target model points to supplement, avoiding the contradiction of excessive point addition in areas without missing points and insufficient point supplementation in missing areas, balancing reconstruction accuracy and model lightweighting. Therefore, this application improves the quality of model point sampling and supplementation and the 3D reconstruction accuracy of the scene model.

[0071] In one possible implementation, the 3D reconstructed scene model is supplemented with model points based on the geometric coverage of each pixel and the number of model point targets, including:

[0072] Establish the correspondence between the cumulative transparency values ​​of model points on the projection ray and the corresponding pixels of the projection ray;

[0073] Pixels whose cumulative transparency value is less than a preset saturation threshold are identified as valid missing pixels.

[0074] Using the geometric coverage of each pixel as the probability distribution, weighted random sampling is performed on each effective missing pixel, and the three-dimensional spatial coordinates and real texture color values ​​of the sampled effective missing pixels are extracted to generate an initial candidate point set.

[0075] The initial candidate point set is deduplicated and fused to obtain the fused candidate point set;

[0076] Load each candidate model point from the fusion candidate point set into the 3D reconstruction scene model.

[0077] It should be noted that in practical applications, since the cumulative transparency value is calculated along the ray direction during the rendering process, binding it to specific pixels allows for direct access to and evaluation of the geometric fill state in 3D space using the image plane index, providing a data foundation for pixel-level model point sampling decisions.

[0078] It should be noted that the aforementioned preset saturation threshold is a lower limit of transparency value at which no model point supplementation is required. For example, in a preferred embodiment, the preset saturation threshold can be set to 0.99. When the cumulative transparency value corresponding to a pixel reaches or exceeds 0.99, it indicates that the geometry in that viewing direction is almost completely filled, the rendering result has converged, and no further supplementation is needed; conversely, if it is below this threshold, it is determined that there are geometric defects. This application compares the cumulative transparency value with the preset saturation threshold to identify regions that have been fully reconstructed, thereby concentrating limited computing resources and supplementation quotas on regions that truly have holes or are underexpressed, thus improving sampling efficiency and accuracy, and saving computing power.

[0079] Furthermore, using the geometric coverage of each pixel as the probability distribution, weighted random sampling is performed on each effective missing pixel. A number of target pixels are selected to supplement the model point target, and the 3D spatial coordinates and real texture color values ​​of the sampled effective missing pixels are extracted to generate an initial candidate point set. Here, the 3D spatial coordinates refer to the 3D world coordinates of the sampled effective missing pixel determined by depth rendering along the corresponding projection ray direction, representing the actual position of the pixel in 3D space. The initial candidate point set consists of all newly added candidate model points obtained in this round of sampling. Each candidate model point contains 3D spatial coordinates and real texture color values, serving as input for subsequent deduplication and fusion processing. Using geometric coverage as the sampling probability distribution enables adaptive resource allocation. Pixels with higher geometric coverage (i.e., higher missing degree) have a greater probability of being selected as sampling targets; while pixels with relatively low geometric coverage but still below the saturation threshold have a correspondingly lower sampling probability. This application, by configuring the above weighted random sampling, achieves priority repair of severely missing geometric defects. Meanwhile, by extracting real texture color values ​​during the sampling process directly from ground truth images in the training set, rather than generating them through neighbor interpolation or model prediction, the newly added model points have texture information consistent with the real scene during the initialization phase, avoiding rendering artifacts caused by texture distortion.

[0080] It should be noted that in practical applications, since 3D reconstruction typically involves multiple camera views, the same spatial location may be identified as a missing region and repeatedly sampled in different views. Without processing, directly loading these duplicate points would lead to redundant expansion of the model's point cloud, increasing storage and computational burden, and potentially causing optimization instability due to excessive point density. Therefore, this application configures the initial candidate point set to deduplicate and fuse, identifying and merging spatially overlapping or close candidate model points, eliminating redundancy caused by multi-view sampling, and ensuring that the final set of points loaded into the model is concise and conflict-free.

[0081] It should be noted that in practical applications, this application integrates candidate model points, after screening, sampling, and deduplication, into the existing 3D reconstruction scene model, making them part of the model. These candidate model points will participate in gradient updates and parameter adjustments along with the original model points during subsequent iterative optimization processes, gradually integrating into the overall scene to achieve the repair and detail enhancement of geometrically missing regions.

[0082] In one possible implementation, there can be multiple ways to achieve the above deduplication and fusion, such as filtering based on spatial distance or fusion algorithms that combine color information. As long as redundancy can be eliminated and effective information can be retained, this application does not limit it to a single method.

[0083] In one possible implementation, the number of point targets to be added to the model is determined based on the geometric complexity of the 3D reconstructed scene model, including:

[0084] Establish the correspondence between the cumulative transparency value of the model points on the projection ray and the corresponding pixel points of the projection ray, and determine the pixel points whose cumulative transparency value is less than the preset saturation threshold as valid missing pixel points;

[0085] Obtain the adjustment factor representing geometric complexity, obtain the total number of current model points of the 3D reconstructed scene model, and obtain the first total number of pixels and the second total number of effective missing pixels respectively.

[0086] The ratio of the second total to the first total is determined as the pixel missing ratio;

[0087] The product of the adjustment factor, the current total number of model points, and the proportion of missing pixels is used to determine the number of target points to be added to the model.

[0088] It should be noted that, in practical applications, the aforementioned total number of current model points refers to the total number of model points already included in the 3D reconstructed scene model during this sampling supplement, including model points generated in the initial reconstruction phase and model points added in previous sampling supplements. The aforementioned first total number of pixels refers to the total number of pixels in the rendered image from all camera views, i.e., the count of all pixels involved in rendering detection. The aforementioned second total number of valid missing pixels refers to the total number of pixels in the rendered image from all camera views whose cumulative transparency value is less than a preset saturation threshold, i.e., the total number of pixels determined to have geometrical defects.

[0089] It should be noted that, in practical applications, the number M of additional target points in the above model can be calculated using the following formula:

[0090] In this formula, M represents the number of model points to be added during the current iteration cycle, N is the total number of model points in the 3D reconstruction scene model, f is an adjustment factor representing geometric complexity, and R is the pixel missing ratio. This calculation formula, using the current total number of model points N as the base, reflects the mechanism of the sampling and supplementation strategy adaptively scaling with the model size. During the training process of 3D reconstruction, since the number of model points changes dynamically, using a fixed supplementation number can easily lead to over-supplementation and redundancy in the early stages of training due to a small number of model points, or under-supplementation and stagnation in detail repair in the later stages due to a large number of model points. By using N as the base, the supplementation number can automatically scale with the current size of the model, ensuring that the proportion of newly added points remains within a reasonable range throughout the entire optimization lifecycle, avoiding over-supplementation in the early stages or under-supplementation in the later stages. The pixel missing ratio R constructs a dynamic feedback mechanism based on the real-time rendering state. R directly quantifies the degree of missing geometric structures in the scene from the current perspective. When R is large, it indicates the presence of large areas of geometric voids or underrepresented regions in the scene. In this case, the formula automatically calculates a larger M value, allocating more resources for repair. Conversely, when R is small, it indicates that the scene is already relatively complete, and the number of points to be added decreases accordingly. The adjustment factor f, as a parameter representing geometric complexity, plays a role in balancing reconstruction accuracy and model size. Different off-road or urban scenes have significantly different inherent requirements for point cloud density. Relying solely on the missing proportion R cannot distinguish the priority difference in point addition between simple, flat areas and complex structural areas. Introducing f allows for global correction of the total amount of points to be added based on the geometric characteristics of the scene itself.

[0091] In one possible implementation, an example of calculating the number of model point targets to be supplemented could be as follows: Assuming that within a certain iteration trigger cycle, the total number N of the current model points in the 3D reconstructed scene model is 1 million, and through rendering detection of all camera views, the first total number of pixels is 10 million, of which the second total number of valid missing pixels with a cumulative transparency value below a preset saturation threshold is 1 million, then the pixel missing ratio R is 10% (i.e., 1 million / 10 million). Simultaneously, based on the current scene's geometric complexity assessment, the determined adjustment factor f is 0.3. Substituting these parameters into the above formula, the number of model point targets to be supplemented is calculated as M = 1,000,000 × 0.3 × 10% = 30,000. This means that in this iteration, 30,000 new model points will be generated to repair the current geometric deficiencies. Through this quantitative calculation method, this application transforms the adaptive control strategy into an executable engineering algorithm, ensuring sufficient points to support high-precision reconstruction in complex areas while preventing unnecessary memory usage and computational overhead in simple areas.

[0092] In one possible implementation, the adjustment factor characterizing the geometric complexity is obtained, including:

[0093] Obtain the geometric structural feature parameters of the 3D reconstructed scene model;

[0094] The geometric structural feature parameters are normalized, and based on a preset mapping relationship, the normalized geometric structural feature parameters are mapped to adjustment factors that characterize geometric complexity. The preset mapping relationship indicates that the adjustment factors increase as the structural feature parameters increase.

[0095] It should be noted that, in practical applications, the aforementioned geometric structure feature parameters are physical quantities used to quantify the complexity of the surface geometry of a 3D reconstructed scene model. To comprehensively and accurately characterize the complexity of different off-road or urban scenes, these parameters can encompass various statistical quantities that reflect changes in spatial structure. For example, geometric structure feature parameters may include at least one of curvature variance, depth gradient entropy, or normal consistency. The curvature variance reflects the severity of surface undulation; a larger variance indicates more rugged terrain. Depth gradient entropy characterizes the disorder or uncertainty of depth information in spatial distribution; a higher entropy value means more occlusion relationships, edge details, or noise interference in the scene. Normal consistency describes the degree of convergence of surface orientation within a local area; lower consistency indicates a more fragmented or complex geometric structure in that area. The above parameters are merely illustrative. In practical applications, other features that characterize geometric complexity, such as roughness or fractal dimension, can be selected based on scene characteristics, as long as their numerical changes can establish a monotonic correlation with the difficulty of scene reconstruction.

[0096] It should be noted that in practical applications, different types of geometric feature parameters often have different physical dimensions and numerical ranges. For example, the unit of curvature may be the reciprocal of a meter, while entropy is usually a dimensionless logarithmic value. Directly using the original feature values ​​as adjustment factors can lead to instability and incomparability in the mapping relationship. Therefore, this application normalizes the geometric feature parameters to eliminate the influence of dimensions, mapping feature values ​​from different sources and scales to a standard dimensionless interval (e.g., [0, 1]). Through normalization, the preset mapping relationship no longer depends on the absolute numerical value of a specific scenario, but focuses on the relative position of the feature value within its own distribution, thereby enhancing the universality and robustness of the algorithm for different scales and types of scenarios. Normalization methods can include min-maximum normalization, Z-score standardization, or Sigmoid function mapping, etc., and this application does not impose a specific limitation on these methods.

[0097] It should be noted that the aforementioned pre-defined mapping relationship establishes a positive correlation between geometric feature parameters and adjustment factors. That is, the more complex the scene geometry (the larger the feature values), the larger the generated adjustment factor f, leading to the calculation of more model point targets in subsequent steps. In implementation, the pre-defined mapping relationship can take various forms. For example, a linear function f = k⋅x_norm + b can be used, where x_norm is the normalized feature value, k is the slope, and b is the intercept. Nonlinear functions (such as logarithmic or exponential functions) can also be used to accommodate nonlinear perception of complexity. Alternatively, piecewise functions or lookup tables can be used to assign discrete adjustment factor values ​​to specific feature value intervals to achieve a more refined control strategy.

[0098] In a preferred embodiment, the adjustment factor f is set to a value range of 0.1 to 0.5. This range is based on empirical summaries derived from off-road and urban scene reconstruction experiments, and has a clear physical correspondence and technical effect. Specifically, the value of the adjustment factor f can be further divided into two sub-ranges: when the adjustment factor f is in the sub-range of 0.1 to 0.2, it corresponds to simple scenes with weak textures such as deserts, snowfields, and flat dirt roads. These scenes have simple geometric structures, smooth surfaces, and no complex occlusions, requiring only a small number of model points to fully represent them. A lower f value can avoid generating redundant point clouds in flat areas and prevent the model from over-expanding. When the adjustment factor f is in the sub-range of 0.3 to 0.5, it corresponds to complex occlusion scenes such as jungles, mountains, ravines, and urban building complexes. These scenes have large geometric undulations, intricate occlusion relationships, and rich details, requiring high-density model points to accurately fit geometric boundaries and texture details. A higher f value ensures sufficient sampling resources in these difficult areas, avoiding holes or blurring caused by insufficient point filling. Limiting the adjustment factor f to the range of 0.1 to 0.5 is to overcome the technical shortcomings of fixed or broad parameters. If the adjustment factor f is less than 0.1, even in complex scenes, the number of calculated target additions will be too small, causing the model to fail to capture subtle geometric changes and occlusion boundaries, resulting in underfitting, manifested as geometric collapse and loss of detail in the reconstruction results. Conversely, if the adjustment factor f is greater than 0.5, even in simple scenes, an excessive number of model points will be generated, wasting video memory resources and reducing the rendering frame rate. It may also cause gradient conflicts during the optimization process due to excessively dense points, leading to overfitting artifacts or noise amplification in simple regions. Therefore, the range of 0.1 to 0.5 constitutes a balance between reconstruction accuracy and computational efficiency, enabling this application to maintain performance stability in diverse practical application scenarios.

[0099] In one possible implementation, the initial candidate point set is deduplicated and fused to obtain a fused candidate point set, including:

[0100] Obtain the 3D spatial coordinates and texture color values ​​of each current model point in the 3D reconstructed scene model;

[0101] Based on the three-dimensional spatial coordinates of the current model point and the three-dimensional spatial coordinates of the effective missing pixel point, calculate the spatial distance between the effective missing pixel point and the current model point.

[0102] Based on spatial distance, texture color value, and real texture color value, the initial candidate point set is deduplicated and fused to obtain a fused candidate point set.

[0103] It should be noted that, in practical applications, the aforementioned "current model point" refers to all model points already included in the 3D reconstructed scene model before this sampling and supplementation operation, including model points generated during the initial reconstruction phase and model points added and iteratively optimized in each subsequent sampling and supplementation. The aforementioned texture color value refers to the color attribute stored on the current model point. This color attribute is a rendered color learned by the model itself after multiple rounds of iterative optimization, distinct from the real texture color value extracted from the ground truth image. The aforementioned spatial distance refers to the Euclidean distance between the 3D spatial coordinates of the effective missing pixels and the 3D spatial coordinates of the current model point, used to measure their proximity in 3D space.

[0104] It should be noted that in practical applications, relying solely on spatial distance for deduplication, while eliminating redundant points with overlapping positions, can easily lead to false deletions when dealing with different objects at close range. For example, in a jungle or densely vegetated scene, the leaves in the foreground and the ground in the background may appear extremely close in depth from a specific viewpoint, even almost overlapping on the projection ray, but they are actually completely different geometric entities with significant texture differences. If they are identified as duplicates and deleted based solely on a spatial distance threshold, it will result in the loss of foreground details or the incorrect merging of geometric structures. Conversely, relying solely on texture color for deduplication, while preserving features with obvious color differences, can easily cause geometric confusion when dealing with areas of the same color. For example, on a flat white wall, snow, or desert surface, locations at different depths may have the exact same color value. If they are identified as duplicates simply because of the same color, geometric objects with a hierarchical relationship will be forcibly merged into a single plane, causing the scene to lose its three-dimensionality or even experience geometric collapse. Therefore, this application constructs a more robust deduplication decision space by simultaneously considering both spatial distance and texture color dimensions. Only newly added, valid missing pixels that are spatially close to and visually highly similar to current model points are identified as truly redundant points and processed accordingly. Points that are spatially adjacent but have different colors (such as object edges), or points that are the same color but spatially separated (such as different layers of the same-colored object), are correctly identified and retained. This dual-dimensional fusion strategy of geometry and appearance effectively eliminates redundant data from multi-view sampling, avoids unnecessary expansion of model size, and preserves high-frequency details and geometric hierarchy in the scene.

[0105] In one possible implementation, the initial candidate point set is deduplicated and fused based on spatial distance, texture color value, and real texture color value to obtain a fused candidate point set, including:

[0106] Add valid missing pixels from the initial candidate point set whose spatial distance from the current model point is not less than a preset spatial distance threshold to the fusion candidate point set;

[0107] Valid missing pixels in the initial candidate point set whose spatial distance from the current model point is less than a preset spatial distance threshold are marked as suspected duplicate points of the current model point.

[0108] If the actual texture color value of a suspected duplicate point is equal to the texture color value of the current model point, the suspected duplicate point will be deleted.

[0109] If the true texture color value of a suspected duplicate point is not equal to the texture color value of the current model point, the true texture color value of the suspected duplicate point is updated to the average of the texture color value and the true texture color value, and the updated suspected duplicate point is added to the fusion candidate point set.

[0110] In a preferred embodiment, the preset spatial distance threshold can be linked to the average radius of the Gaussian points in the current model, for example, set to 0.5 to 1.5 times the average radius; or it can be related to the overall bounding box size of the scene, for example, set to one-thousandth of the scene diagonal length. This dynamic or adaptive threshold setting method can ensure that the sensitivity of deduplication is consistent across scenes of different scales, avoiding detail loss due to an excessively large threshold or redundant residue due to an excessively small threshold. When the spatial distance between the effective missing pixel and the current model point is not less than the threshold, the two are determined to be spatially independent, and the missing pixel is directly added as a new point to the fusion candidate point set to supplement the new geometry.

[0111] It should be noted that in practical applications, when the true texture color value of a suspected duplicate point equals the texture color value of the current model point, it indicates that the two not only overlap in spatial location but also have identical appearance attributes, highly suggesting repeated sampling of the same geometric surface. Deleting this suspected duplicate point effectively eliminates redundant data and simplifies the model size. However, the situation is more complex when the true texture color value of a suspected duplicate point does not equal the texture color value of the current model point. This usually means that although the two points are spatially close, they may be due to one of two scenarios: first, color differences at the same location under different camera perspectives caused by changes in lighting conditions; second, two geometric objects at different depths located at object edges or occlusion boundaries, extremely close in projection direction but actually belonging to different entities. Simply deleting them in this case may result in the loss of edge details or incomplete lighting information. To address these color inconsistencies, this application updates the true texture color value of the suspected duplicate point to the average of the current model point's texture color value and the true texture color value, and adds the updated suspected duplicate point to the fusion candidate point set. This is essentially a fusion and smoothing process of multi-view lighting information. By averaging the values, the newly added points inherit color features from different observation directions, enabling them to present a more natural and continuous visual effect from multiple perspectives during subsequent rendering. This effectively eliminates stitching artifacts or lighting abrupt changes caused by single-view sampling. Especially when dealing with complex occlusion boundaries such as tree canopy occlusion and terrain gullies, it can preserve geometric layers while repairing color jumps caused by perspective switching, significantly improving the realism and integrity of the reconstruction results.

[0112] It should be noted that although this application uses the arithmetic mean as an example of color fusion, in other embodiments, a weighted mean can also be used to further improve the fusion quality. For example, the weights can be determined based on the angle between the observation angle and the surface normal, or the confidence level of each viewpoint image, so that the color contribution from the viewpoint directly facing the surface or from a high-confidence viewpoint is greater. This alternative approach also falls within the scope of protection of this application, and its core idea is to solve the attribute conflict problem of spatially adjacent points through the comprehensive utilization of color information. In addition, regarding the determination of color equality, in actual engineering implementation, a certain tolerance range is usually allowed (e.g., the difference between each RGB channel is less than 5), rather than strict numerical equality, to accommodate the influence of image noise and quantization error. Through the above deduplication fusion strategy, this application can preserve the geometric details and texture realism of the scene while ensuring the model's lightweight nature.

[0113] In one possible implementation, the sampling supplementation method for digital model points also includes:

[0114] Assign initial geometric parameters and texture rendering parameters to candidate model points that belong to the fusion candidate point set in the 3D reconstructed scene model;

[0115] Based on the current optimization gradient information of the 3D reconstructed scene model, as well as the initial geometric parameters and texture rendering parameters of the candidate model points, the 3D reconstructed scene model is iteratively optimized.

[0116] It should be noted that in practical applications, the initial geometric parameters mentioned above include 3D spatial position, covariance matrix, and opacity. The 3D spatial position is directly derived from the depth rendering coordinates extracted in the preceding steps, ensuring accurate positioning of candidate model points in geometric space. The covariance matrix is ​​a 3×3 symmetric matrix used to define the ellipsoidal distribution of model points in 3D space. It determines the shape (isotropic sphere or anisotropic ellipsoid), size (spatial extent), and rotational orientation (orientation of the ellipsoid's principal axis) of the model points. During initialization, an inheritance strategy can be used, i.e., copying the covariance matrix of the nearest existing model point in space, so that the candidate model points have a geometric shape similar to their surroundings; or a default strategy can be used, initializing them as small isotropic spheres to avoid artifacts caused by excessively large initial shapes. Opacity is usually initialized to a small non-zero value to prevent abrupt bright spots or occlusions in the rendered image due to excessive opacity when candidate model points are first added, ensuring a smooth visual transition. Texture rendering parameters mainly refer to spherical harmonic coefficients, which are used to describe the color change characteristics of candidate model points under different viewpoints. In this application, the initialization of the aforementioned spherical harmonic coefficients is directly based on the extracted real texture color values. Typically, the true RGB values ​​are assigned to the 0th-order spherical harmonic coefficients, while higher-order coefficients are initialized to zero. This initialization method allows candidate model points to present a basic color consistent with the real scene during the first rendering, avoiding the long convergence process and early color differences caused by learning colors from scratch.

[0117] In one possible implementation, after allocating the initial geometric and texture rendering parameters, the optimizer for the 3D reconstructed scene model can be reset. By resetting the optimizer's state parameters (e.g., the first and second moment buffers of the Adam optimizer) while retaining all currently trained weight parameters, only candidate model points are included in the iterative optimization scope, thus avoiding disrupting the convergence state of existing mature scene parameters. This differentiated processing strategy ensures that the optimization results accumulated by existing model points during long-term training are not overwritten, while providing candidate model points with an independent optimization path to accumulate momentum from scratch, enabling both new and old points to converge efficiently on their respective optimization trajectories.

[0118] It's important to note that in practical applications, candidate model points are not optimized independently in isolated environments but are directly integrated into a dynamic optimization system that has already run thousands of iterations. At this point, the gradient field of the loss function of the entire 3D reconstructed scene model already contains scene structure information and convergence trends. Therefore, utilizing the current optimization gradient information of the 3D reconstructed scene model to guide the parameter updates of candidate model points allows them to converge quickly along the existing optimization trajectory, rather than blindly exploring the parameter space. This mechanism shortens the adaptation time of candidate model points and improves overall training efficiency.

[0119] In one possible implementation, the geometric coverage of the pixels corresponding to the projection ray is determined based on the accumulated transparency value of the model points on the projection ray, including:

[0120] Calculate the initial geometric coverage of the pixel points corresponding to the projection ray based on the cumulative transparency value of the model points on the projection ray.

[0121] Calculate the sum of the cumulative transparency values ​​of the model points on each projection ray;

[0122] The quotient of the initial geometric coverage of a pixel and the sum of the two is used to determine the geometric coverage of the pixel.

[0123] It should be noted that, in practical applications, the geometric coverage of the aforementioned pixels can be calculated using the following formula:

[0124] ,in, It is the geometric coverage of pixel p, 1- It is the cumulative transparency value of the model points on the projection ray, 1- It is the cumulative transparency value corresponding to pixel p. It is the sum of the cumulative transparency values ​​of the model points on each projection ray.

[0125] It should be noted that in practical applications, the above calculation formula is unrelated to parameters such as pixel gradient, texture change rate, or color residual of the rendered image. This mathematical characteristic ensures the adaptability of the method in this application to weak texture regions and complex occlusion regions. In traditional gradient-driven methods, the gradient in weak texture regions such as deserts and snowfields approaches zero, resulting in a zero sampling probability; however, the geometric coverage in this application depends only on the filling state of the geometry (i.e., the cumulative transparency value). Even in regions with completely uniform texture, as long as the geometric structure is not fully filled (i.e., α_acc < 1), the pixel can still obtain a non-zero sampling probability, thus filling the blind spot of traditional methods.

[0126] It's important to note that in practical applications, normalization is introduced to eliminate the incomparability of absolute values ​​and construct a unified probability distribution space. In actual 3D reconstruction, the absolute distribution range of the cumulative transparency value can vary significantly depending on the scene type, camera perspective, and even the training stage of the same scene. For example, in the early stages of training or when facing large areas of voids, the global missing value sum may be very large; while in the later stages of training or when facing minor imperfections, the global missing value sum may be smaller. If the initial geometric coverage (1-α_acc) is directly used as the sampling basis, its numerical fluctuations will not form a stable probability distribution, leading to inconsistent performance of the sampling strategy under different conditions. By normalizing by dividing by the global missing value sum, the geometric coverage of all pixels is mapped to the [0, 1] interval and sums to 1, thus forming a standard probability density function. This allows the sampling mechanism to adaptively adjust the relative weights of each pixel regardless of changes in the overall scene missing value, ensuring that the most severely missing areas always receive the highest sampling priority.

[0127] One possible implementation also includes:

[0128] If the number of iterations equals the preset iteration trigger cycle, increment the total number of optimizations for the 3D reconstructed scene model by 1;

[0129] If the total number of optimizations is less than the preset total number of optimizations threshold, the step of obtaining the cumulative transparency value of model points on each projection ray in the 3D reconstructed scene model is executed.

[0130] When the total number of optimizations equals the preset total number of optimizations threshold, output a 3D reconstructed scene model.

[0131] It should be noted that, in practical applications, the aforementioned preset iteration trigger period defines the minimum time interval between two sampling and supplementation operations. Preferably, the preset iteration trigger period can be set to 5000 iterations. The reason for setting this period is that newly added model points are in a state of non-convergence in the initial loading stage. If candidate model points are frequently inserted, it will cause the loss function to fluctuate drastically, disrupting the optimization trajectory of the original model points, and even causing gradient explosion or oscillation. By setting a fixed trigger period, sufficient interval time is provided for the model to adjust the parameters of the new and old points using backpropagation. After the local geometry tends to stabilize, the next round of missing data detection and supplementation is performed, avoiding the interference of continuous point addition on the convergence process and ensuring the stability of the training process.

[0132] It should be noted that, in practical applications, the total number of optimization iterations and the preset total optimization threshold together constitute the global termination condition of the training process. The total number of optimization iterations records the cumulative number of rounds the model has undergone from complete supplementation to optimization, while the preset total optimization threshold is an upper limit pre-set based on scene size, hardware resources, and accuracy requirements. When the total number of optimization iterations reaches this threshold, it is determined that the model has fully converged or reached the expected quality standard, and the iteration stops, outputting the final 3D reconstructed scene model. This prevents invalid computation and infinite loops caused by getting trapped in local minima or overfitting, ensuring that the algorithm can deliver usable reconstruction results within a finite time.

[0133] In one possible implementation, the complete runtime closed-loop process of the above iterative optimization can be as follows: After the system starts, regular iterative optimization is performed first; whenever the iteration count reaches a preset iteration trigger period (e.g., the 5000th, 10000th, etc.), the system pauses regular optimization and enters the sampling supplementation phase, sequentially executing the entire process of obtaining the transparency accumulation value, calculating the geometric coverage, determining the target supplementation quantity, weighted random sampling, deduplication and fusion, and parameter initialization; after the supplementation is completed, the total number of optimizations is automatically incremented by 1, and the iteration counter is reset. The model continues the next stage of iterative optimization based on the updated point cloud; during this process, the system continuously monitors whether the total number of optimizations has reached the preset total number of optimizations threshold. If it has not reached it, the above "optimization-supplementation" loop is repeated; if it has reached it, training is terminated and the model is output. Although this example uses a fixed period as an example, in other implementations, the preset iteration trigger period can also be dynamically adjustable. For example, in the early stages of training, due to large holes and severe deficiencies in the scene, the triggering cycle can be set to a shorter period (e.g., 3000 times) to quickly fill large areas of geometric defects. In the later stages of training, as the model gradually converges and only minor flaws remain to be repaired, the triggering cycle can be extended (e.g., 8000 times or longer) to reduce unnecessary sampling overhead and focus on fine-tuning the parameters. Timing control works in conjunction with the aforementioned adaptive point-filling strategy: adaptive point-filling solves the spatial distribution problem of the filling location and quantity, while timing control solves the problem of the filling timing. The combination of the two ensures the efficiency and stability of the 3D reconstruction process.

[0134] To facilitate understanding of the sampling and supplementation method for digital model points provided in the embodiments of this application, a possible implementation of this application will be described here:

[0135] like Figure 2 The diagram shown is a flowchart of a model point sampling supplementation process for 3D reconstruction. The specific operation steps are as follows:

[0136] Step S201: Obtain the cumulative transparency value of each model point on each projection ray in the 3D reconstructed scene model. Then trigger step S202.

[0137] Step S202: Based on the cumulative transparency values ​​of model points on the projection rays, calculate the initial geometric coverage of the pixels corresponding to the projection rays, calculate the sum of the cumulative transparency values ​​of model points on each projection ray, and determine the quotient of the initial geometric coverage of the pixel and the sum as the geometric coverage of the pixel. Then, step S203 is triggered.

[0138] Step S203: Establish the correspondence between the cumulative transparency value of the model points on the projection ray and the corresponding pixel points on the projection ray, and determine the pixels with a cumulative transparency value less than a preset saturation threshold as valid missing pixels. Then trigger step S204.

[0139] Step S204: Obtain the geometric structural feature parameters of the 3D reconstructed scene model, normalize the geometric structural feature parameters, and map the normalized geometric structural feature parameters to adjustment factors representing geometric complexity based on a preset mapping relationship. Then, step S205 is triggered.

[0140] Step S205: Obtain the total number of current model points in the 3D reconstructed scene model, and obtain the first total number of pixels and the second total number of effective missing pixels respectively. Determine the ratio of the second total number to the first total number as the pixel missing ratio. Determine the number of model point targets to be supplemented by multiplying the adjustment factor, the current total number of model points, and the pixel missing ratio. Then trigger step S206.

[0141] Step S206: Pixels with a cumulative transparency value less than a preset saturation threshold are identified as valid missing pixels. Using the geometric coverage of each pixel as a probability distribution, weighted random sampling is performed on each valid missing pixel. The 3D spatial coordinates and true texture color values ​​of the sampled valid missing pixels are extracted to generate an initial candidate point set. The initial candidate point set is then deduplicated and fused to obtain a fused candidate point set. Each candidate model point in the fused candidate point set is loaded into the 3D reconstructed scene model. Step S207 is then triggered.

[0142] Step S207 involves assigning initial geometric parameters and texture rendering parameters to candidate model points belonging to the fusion candidate point set in the 3D reconstructed scene model. Based on the current optimization gradient information of the 3D reconstructed scene model, as well as the initial geometric parameters and texture rendering parameters of the candidate model points, the 3D reconstructed scene model is iteratively optimized. This triggers step S208.

[0143] Step S208: Determine whether the number of iterations after this iteration is equal to the preset iteration trigger period. If yes, trigger step S209; otherwise, trigger step S201.

[0144] Step S209: Increment the total number of optimization iterations for the 3D reconstructed scene model by 1. Then trigger step S210.

[0145] Step S210: Determine whether the total number of optimizations is less than a preset total number of optimizations threshold. If yes, trigger step S201; otherwise, trigger step S211.

[0146] Step S211: When the total number of optimizations equals the preset total number of optimizations threshold, output the 3D reconstructed scene model.

[0147] The second aspect of this application provides a computer storage medium carrying one or more computer programs, which, when executed by an electronic device, enable the electronic device to implement the digital model point sampling supplementation method described in the first aspect or any implementation thereof.

[0148] A third aspect of this application also provides an electronic device. (See reference...) Figure 3 The diagram illustrates a structural schematic of an electronic device suitable for implementing the digital model point sampling supplementation method in the embodiments of this application. The electronic device in the embodiments of this application may include, but is not limited to, fixed terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Figure 3 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0149] like Figure 3 As shown, the electronic device may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. When the electronic device is powered on, the RAM 603 also stores various programs and data required for the operation of the electronic device. The processing unit 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0150] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 608 including, for example, memory cards, hard drives, etc.; and communication devices 609. Communication device 609 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 3 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have instead.

[0151] A fourth aspect of this application also provides a computer program product, including computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement any of the digital model point sampling supplementation methods provided in the embodiments of this application.

[0152] The fifth aspect of this application also provides a computer-readable storage medium carrying one or more computer programs that, when executed by an electronic device, enable the electronic device to implement any of the digital model point sampling supplementation methods provided in the embodiments of this application.

[0153] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0154] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0155] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0156] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

Claims

1. A method for sampling and supplementing points in a digital model, characterized in that, include: The cumulative transparency value of model points on each projection ray in the 3D reconstructed scene model is obtained respectively. Based on the cumulative transparency value of the model points on the projection ray, the geometric coverage of the pixel point corresponding to the projection ray is determined. Each projection ray is a ray emitted from the projection center and passes through a pixel point in the 3D reconstructed scene model that corresponds to it. Based on the geometric complexity of the 3D reconstructed scene model, the number of additional point targets in the model is determined; Based on the geometric coverage of each pixel and the number of model point targets to be supplemented, the model points of the 3D reconstructed scene model are sampled and supplemented.

2. The sampling and supplementation method for digital model points according to claim 1, characterized in that, Based on the geometric coverage of each pixel and the number of model point targets supplemented, the 3D reconstructed scene model is sampled and supplemented with model points, including: Establish the correspondence between the cumulative transparency value of the model points on the projection ray and the corresponding pixel points of the projection ray; Pixels whose cumulative transparency value is less than a preset saturation threshold are identified as valid missing pixels; Using the geometric coverage of each pixel as a probability distribution, weighted random sampling is performed on each effective missing pixel, and the three-dimensional spatial coordinates and real texture color values ​​of the sampled effective missing pixels are extracted to generate an initial candidate point set. The initial candidate point set is deduplicated and fused to obtain a fused candidate point set; Each candidate model point in the fusion candidate point set is loaded into the 3D reconstructed scene model.

3. The sampling and supplementation method for digital model points according to claim 1, characterized in that, The determination of the number of supplementary point targets in the model based on the geometric complexity of the 3D reconstructed scene model includes: Establish a correspondence between the cumulative transparency value of the model points on the projection ray and the corresponding pixel points of the projection ray, and determine the pixel points whose cumulative transparency value is less than a preset saturation threshold as valid missing pixel points; Obtain the adjustment factor characterizing the geometric complexity, obtain the total number of current model points of the 3D reconstructed scene model, and obtain the first total number of pixels and the second total number of effective missing pixels respectively; The ratio of the second total to the first total is determined as the pixel missing ratio; The product of the adjustment factor, the current total number of model points, and the percentage of missing pixels is determined as the number of model point targets to be supplemented.

4. The sampling and supplementation method for digital model points according to claim 3, characterized in that, The process of obtaining the adjustment factor characterizing the geometric complexity includes: Obtain the geometric structural feature parameters of the three-dimensional reconstructed scene model; The geometric structural feature parameters are normalized, and based on a preset mapping relationship, the normalized geometric structural feature parameters are mapped to the adjustment factor characterizing the geometric complexity. The preset mapping relationship indicates that the adjustment factor increases as the structural feature parameters increase.

5. The sampling and supplementation method for digital model points according to claim 2, characterized in that, The process of deduplicating and fusing the initial candidate point set to obtain a fused candidate point set includes: Obtain the three-dimensional spatial coordinates and texture color values ​​of each current model point in the three-dimensional reconstructed scene model; Based on the three-dimensional spatial coordinates of the current model point and the three-dimensional spatial coordinates of the effective missing pixel point, calculate the spatial distance between the effective missing pixel point and the current model point; Based on the spatial distance, the texture color value, and the real texture color value, the initial candidate point set is deduplicated and fused to obtain the fused candidate point set.

6. The sampling and supplementation method for digital model points according to claim 5, characterized in that, The process of deduplicating and fusing the initial candidate point set based on the spatial distance, the texture color value, and the real texture color value to obtain the fused candidate point set includes: Add the valid missing pixels in the initial candidate point set whose spatial distance from the current model point is not less than a preset spatial distance threshold to the fusion candidate point set; The valid missing pixels in the initial candidate point set whose spatial distance from the current model point is less than a preset spatial distance threshold are marked as suspected duplicate points of the current model point. If the actual texture color value of the suspected duplicate point is equal to the texture color value of the current model point, the suspected duplicate point is deleted. If the true texture color value of the suspected duplicate point is not equal to the texture color value of the current model point, the true texture color value of the suspected duplicate point is updated to the average of the texture color value and the true texture color value, and the updated suspected duplicate point is added to the fusion candidate point set.

7. The sampling and supplementation method for digital model points according to claim 2, characterized in that, The method for sampling and supplementing digital model points also includes: Assign initial geometric parameters and texture rendering parameters to the candidate model points in the 3D reconstructed scene model that belong to the fusion candidate point set; Based on the current optimization gradient information of the 3D reconstructed scene model, as well as the initial geometric parameters and texture rendering parameters of the candidate model points, the 3D reconstructed scene model is iteratively optimized.

8. The sampling and supplementation method for digital model points according to claim 1, characterized in that, Determining the geometric coverage of the pixel points corresponding to the projection ray based on the cumulative transparency value of the model points on the projection ray includes: Based on the cumulative transparency value of the model points on the projection ray, calculate the initial geometric coverage of the pixel points corresponding to the projection ray; Calculate the sum of the cumulative transparency values ​​of the model points on each of the projection rays; The quotient of the initial geometric coverage of the pixel and the sum is determined as the geometric coverage of the pixel.

9. The sampling and supplementation method for digital model points according to claim 7, characterized in that, Also includes: If the number of iterations equals the preset iteration triggering period, the total number of optimizations for the 3D reconstructed scene model is incremented by 1. If the total number of optimizations is less than the preset total number of optimizations threshold, the step of obtaining the cumulative transparency value of model points on each projection ray in the 3D reconstructed scene model is performed. When the total number of optimizations equals the preset total number of optimizations threshold, the 3D reconstructed scene model is output.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium carries one or more computer programs that, when executed by an electronic device, enable the electronic device to implement the digital model point sampling supplementation method as described in any one of claims 1 to 9.

Images