Image and three-dimensional scene collaborative enhancement method based on adaptive query streaming propagation

Abstract

The invention belongs to the technical field of three-dimensional scene collaborative enhancement, and particularly relates to an image and three-dimensional scene collaborative enhancement method based on self-adaptive query streaming propagation, which comprises the following steps: completing scene voxelization and equipment perception configuration, obtaining an equipment perception utility score through occlusion judgment and attenuation coefficient calculation, screening three-dimensional query based on the score, and constructing a queue. Combining cross-modal feature fusion to optimize a query set, and finally screening an optimal query subset and supplementing the optimal query subset to a queue; configuring an initial query position of geometric denoising pre-training, constructing a multi-task training target, and performing multi-round training to enable sparse three-dimensional query to obtain the capability of autonomously approaching a scene occupied area through semantic Gaussian decoding generation and geometric modeling; and calling a pre-training model to fuse two-dimensional features and three-dimensional query features of the image, decoding to generate a dense semantic Gaussian set, and realizing semantic information alignment and generating a high-precision three-dimensional scene semantic occupation result by optimizing Gaussian and voxel mapping mechanisms and improving calculation efficiency.

Description

Technical Field

[0001] This invention belongs to the field of 3D scene collaborative enhancement technology, and particularly relates to a method for image and 3D scene collaborative enhancement based on adaptive query streaming propagation. Background Technology

[0002] Current 3D scene collaborative enhancement technologies still face numerous challenges in complex environments. Traditional voxelization methods often struggle to balance reconstruction accuracy and computational efficiency, and are prone to perceptual biases in scenes with varying lighting and occlusion. On one hand, existing technologies lack scientific planning in utilizing the perception range of intelligent devices, failing to fully consider the attenuation of perception accuracy due to spatial distance and angular differences, and lacking an effective occlusion judgment mechanism. This results in a lack of targeted 3D query filtering and problems such as redundancy or insufficient coverage in the query queue. On the other hand, cross-modal feature fusion strategies are not perfect, and the information interaction between 2D image features and 3D query features is insufficient, making it difficult to form accurate scene representations and affecting the accuracy of subsequent semantic information alignment and occupancy result generation.

[0003] Meanwhile, traditional reconstruction methods have shortcomings in geometric modeling and dynamic adaptability. The initial query location settings in the pre-training phase lack rationality, are susceptible to noise interference from LiDAR point clouds, and the single-minded design of multi-task training objectives makes it difficult to guide sparse 3D queries to autonomously approach the scene's occupied area. For dynamic scenes, existing technologies lack efficient query queues and semantic Gaussian parameter update mechanisms, failing to respond promptly to changes in target motion within the scene, resulting in insufficient spatiotemporal consistency of the reconstruction results. Furthermore, the Gaussian-voxel mapping process lacks optimization, leading to low computational efficiency and poor synergistic enhancement of semantic and geometric information, making it difficult to meet the demands of autonomous driving, intelligent monitoring, and other fields for high-precision, real-time 3D scene reconstruction. Summary of the Invention

[0004] The purpose of this invention is to address the aforementioned technical problems by providing a method for collaborative enhancement of images and 3D scenes based on adaptive query streaming propagation.

[0005] In view of this, the present invention provides a collaborative enhancement method for three-dimensional scenes based on illumination adaptive planar representation. Step 1: Complete scene voxelization and device perception configuration, obtain device perception utility score through occlusion judgment and attenuation coefficient calculation, filter three-dimensional queries based on the score and build a queue, optimize the query set by combining cross-modal feature fusion, and finally select the optimal query subset to supplement the queue. Step 2: Configure the initial query position for geometric denoising pre-training, construct multi-task training objectives, and through semantic Gaussian decoding and geometric modeling, enable sparse 3D queries to autonomously approach the scene's occupied area through multiple rounds of training. Step 3: Call the pre-trained model to fuse the two-dimensional features of the image and the three-dimensional query features, decode and generate a dense semantic Gaussian set, and achieve semantic information alignment and generate a high-precision three-dimensional scene semantic occupancy result by optimizing the Gaussian and voxel mapping mechanisms and improving computational efficiency. Step 4: Based on the semantic occupancy results and temporal information, set the update cycle and dynamically maintain the query queue, optimize the semantic Gaussian parameters in a targeted manner, and achieve dynamic collaborative enhancement of the image and the 3D scene through accuracy optimization and consistency verification.

[0006] Preferably, the process of completing scene voxelization and device perception configuration involves obtaining a device perception utility score through occlusion detection and attenuation coefficient calculation, filtering 3D queries based on this score and constructing a queue, optimizing the query set through cross-modal feature fusion, and finally selecting the optimal query subset to supplement the queue, including: Edge nodes divide the 3D scene region into a set of 3D voxels of equal size, receive status information sent by participating intelligent devices and complete device registration, and construct a 3D cone-shaped theoretical perception space for each device to determine the range of perceptible voxels. Design an occlusion detection algorithm to determine the visibility of candidate voxels and generate an occlusion identification function; Construct spatial distance attenuation coefficient and angle difference attenuation coefficient, and multiply the two to obtain the perception accuracy attenuation coefficient; traverse the set of voxels that the device can perceive, and calculate the device perception utility score by fusing attenuation coefficient, occlusion mark inverse value, voxel weight and coverage requirement coefficient. Based on the score, 3D queries are filtered, an initial sparse 3D query queue is constructed and historical query information is stored. Cross-modal fusion is performed by combining the 2D features of the current image observation data with the 3D features of the historical queries, the current query set is optimized and updated, a query propagation strategy is designed and a greedy selection algorithm is used to select the optimal query subset to be added to the queue.

[0007] Preferably, the occlusion identification function is defined as: When the spatial connection between the smart device and the candidate voxel intersects with the spatial contours of other devices or static obstacles, the candidate voxel is invisible; otherwise, it is visible. The expression is: ; Where v represents a smart device; g is a candidate voxel.

[0008] Preferably, the expression for the spatial distance attenuation coefficient in step one is: ; in, voxels Optimal perception coefficient; This is the distance attenuation coefficient; The distance between the device and the voxel space; These are the minimum and maximum effective sensing distances of the sensor; The expression for the angle difference attenuation coefficient is: ; in, The angle difference between the direction of equipment movement and the direction of the voxel; This represents the sensor's maximum field of view. The sensing accuracy attenuation coefficient is: .

[0009] Preferably, the formula for calculating the device perceived utility score in step one is: ; in, For the set of voxels that the device can sense; Voxel importance weights; The coefficient for voxel scene coverage requirements.

[0010] Preferably, the initial query position configured for geometric denoising pre-training is used to construct a multi-task training objective. Through semantic Gaussian decoding and geometric modeling, and after multiple rounds of training, the sparse 3D query acquires the ability to autonomously approach the scene's occupied region, including: The method for setting the initial query position in the geometric denoising pre-training stage is determined, and the initial position of the sparse 3D query is anchored on the noisy LiDAR point cloud. A multi-task training objective including denoising loss, depth rendering loss and RGB rendering loss is constructed. The decoding generation of semantic Gaussian and the geometric modeling process are clarified. Through multiple rounds of training, the sparse 3D query is enabled to move autonomously to the scene-occupied area.

[0011] Preferably, the noisy lidar point cloud in step two is obtained by sampling the farthest point of the original lidar point cloud to obtain a fixed number of sampling points, and then adding uniformly distributed noise to each sampling point.

[0012] Preferably, the denoising loss in step two is obtained by calculating the distance error between the optimized query position and the denoised lidar sampling point; The depth rendering loss is obtained by calculating the difference between the depth map rendered by the semantic Gaussian from the current and neighboring views and the depth map obtained by the LiDAR projection; The RGB rendering loss is obtained by calculating the pixel difference between the RGB image rendered from multiple perspectives using semantic Gaussian and the actual acquired RGB image.

[0013] Preferably, the step of calling the pre-trained model to fuse two-dimensional image features and three-dimensional query features, decoding to generate a dense semantic Gaussian set, and by optimizing the Gaussian and voxel mapping mechanisms and improving computational efficiency, achieves semantic information alignment and generates a high-precision three-dimensional scene semantic occupancy result, including: The model, pre-trained with geometric denoising, is invoked to process the current image observation data, extract two-dimensional features, and interact with the optimized sparse three-dimensional query across modalities. The optimized sparse 3D query is decoded into a dense semantic Gaussian set; An opacity-weighted geometric estimation mechanism is introduced to optimize the mapping process from Gaussian to voxel, and an efficient Gaussian voxel projection algorithm is adopted to improve computational efficiency. By semantic decoding, image features are aligned with the semantic information of the 3D scene, generating a high-precision 3D scene semantic occupancy result.

[0014] Preferably, the attributes of the semantic Gaussian in step three include three-dimensional position, rotation parameters, scale parameters, opacity, and semantic category distribution; The efficient Gaussian voxel projection algorithm described in step three is implemented by dividing voxels into three-dimensional blocks of fixed size, and loading nearby semantic Gaussian data together for voxels within the same three-dimensional block. During the reverse computation phase, threads are bound to a single semantic Gaussian to avoid atomic operations on voxel gradients.

[0015] The beneficial effects of this invention are: Through refined design of scene voxelization and device perception configuration, the targeting and effectiveness of 3D query filtering are significantly improved. The introduction of occlusion judgment algorithms and perception accuracy attenuation coefficients accurately eliminates the interference of imperceptible voxels. Combined with the perception utility score calculated by voxel weights and coverage requirement coefficients, it ensures that the filtered queries have both high perception capabilities and uniform scene coverage. Cross-modal feature fusion and greedy selection strategies further optimize the query set, providing high-quality initial data support for subsequent reconstruction. At the same time, the multi-task objective design in the geometric denoising pre-training stage, through the collaborative supervision of denoising, depth rendering, and RGB rendering losses, enables sparse 3D queries to autonomously approach the scene's occupied area. The semantic Gaussian generated by its decoding can accurately fit the local geometric structure, greatly improving the accuracy and completeness of scene geometric modeling.

[0016] In terms of semantic reconstruction and dynamic adaptation, this invention achieves a dual improvement in accuracy and efficiency. The opacity-weighted geometric estimation mechanism optimizes the Gaussian-to-voxel mapping process, effectively avoiding interference from invalid information. Meanwhile, the efficient Gaussian voxel projection algorithm, through 3D block collaborative loading and thread-bound design, significantly reduces computational resource consumption, meeting real-time processing requirements. The semantic decoding process achieves precise alignment between image 2D features and 3D scene semantics, generating semantic occupancy results that contain both complete geometric structures and accurate semantic annotations. Furthermore, the dynamic maintenance of the query queue based on temporal information and the semantic Gaussian parameter optimization strategy can capture the dynamic target motion state in real time. Consistency verification ensures the spatiotemporal consistency of the reconstruction results, enabling the system to stably output high-precision 3D scene data even in complex dynamic scenes, providing reliable technical support for fields such as autonomous driving and intelligent monitoring. Attached Figure Description

[0017] Figure 1 This is a flowchart of the present invention.

[0018] Figure 2 This is a schematic diagram illustrating how the sensing accuracy attenuation coefficient varies with the spatial distance and angular difference between the device and the voxel. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0020] like Figure 1 As shown, a collaborative enhancement method for 3D scenes based on illumination-adaptive planar representation includes the following sequential steps: The first step involves dividing the 3D scene area covered by the edge nodes into a set of 3D voxels of equal size, with each voxel identified by its central 3D coordinates. Intelligent devices participating in 3D scene perception periodically send their own status information to the edge nodes, including their 3D geographical location, sensor parameters, effective sensing distance, field of view, and motion state. The edge nodes complete device registration and establish a status set, while simultaneously constructing a 3D cone-shaped theoretical sensing space for each device based on sensor parameters to determine the range of perceptible voxels.

[0021] Then, an occlusion detection algorithm is designed to determine the visibility of candidate voxels, construct a spatial connection from the device sensor to the voxel center, determine whether this connection intersects with the spatial contours of other devices or static obstacles, and generate an occlusion identification function: ,in For smart devices, Candidate voxels.

[0022] Simultaneously, the sensing accuracy attenuation coefficient is designed by first constructing a spatial distance attenuation coefficient based on the effective sensing range of the device's sensors: ,in voxels Optimal perception coefficient, This is the distance attenuation coefficient. The distance between the device and the voxel space. The minimum and maximum effective sensing distances of the sensor are used to construct the angle difference attenuation coefficient, which is then combined with the sensor's field of view. ,in The angle difference between the direction of equipment movement and the direction of the voxel. The maximum field of view of the sensor is the maximum angle of view. Multiplying the two values ​​yields the final sensing accuracy attenuation coefficient: ;like Figure 2 As shown, the sensing accuracy attenuation coefficient gradually decreases with increasing spatial distance between the device and the voxel. Simultaneously, after introducing the angle difference attenuation constraint, the rate of decrease in the overall sensing accuracy attenuation coefficient further accelerates. This effectively suppresses the impact of long-distance and large-angle voxels on the sensing utility score during the query filtering process, making the filtered 3D query more consistent with the actual sensor sensing characteristics. Finally, the set of voxels that the device can sense is traversed, and the attenuation coefficient, occlusion marker inverse value, voxel weight, and coverage requirement coefficient are fused to calculate the device sensing utility score. ,in For the set of voxels that the device can sense. As the importance weight of voxels, The coefficient represents the voxel scene coverage requirement, and the resulting score is used for subsequent query filtering.

[0023] Next, edge nodes filter 3D queries generated by high-perception-capability devices based on perception utility scores, construct an initial sparse 3D query queue, and store the 3D location, semantic attributes, and timestamps of historical queries. After receiving current image observation data, 2D features are extracted and fused cross-modally with the 3D features of historical queries. The current query set is then optimized and updated through a time-series converter network to ensure that queries carry long-term scene context information. Simultaneously, a query propagation strategy is designed, query opacity values ​​are calculated, and spatial distribution constraints are introduced.

[0024] Finally, based on the opacity value and spatial distribution constraints, a greedy selection algorithm is used to select the optimal query subset. This subset retains queries in high-confidence regions while ensuring uniform scene coverage. It is then added to the sparse 3D query queue for subsequent temporal propagation.

[0025] In one specific embodiment, the 3D scene region covered by the edge nodes is divided into a set of regular 3D voxels with a size of 0.2m × 0.2m × 0.2m, used to describe the spatial structure of roads, buildings, and dynamic targets. The initial sparse 3D query queue of the system is set to 256 query points, each containing 3D spatial location, timestamp, and semantic attribute information. The intelligent devices participating in perception include vehicle-mounted cameras and LiDAR devices, with an effective perception range of 2m to 50m and a maximum field of view of 120°. When calculating the spatial distance attenuation coefficient, the distance attenuation parameter α is set to 0.05, and the optimal voxel perception coefficient is set to 1. The update cycle of the query queue is set to 200ms. Within each update cycle, the system sorts candidate queries according to the device perception utility score and uses a greedy selection strategy to filter the optimal query subset to ensure the uniformity of query distribution and coverage integrity in 3D space.

[0026] The second step involves determining the initial query location setting method for the geometric denoising pre-training stage, anchoring the initial location of the sparse 3D query onto the noisy LiDAR point cloud data. A fixed number of sampling points are obtained by sampling the farthest point from the original LiDAR point cloud, and then uniformly distributed noise is added to each sampling point to obtain a noisy LiDAR point cloud. This noisy point cloud serves as the basis for generating the initial query location, ensuring that the initial query covers key spatial areas of the scene and has reasonable perturbation.

[0027] Then, a multi-task training objective is constructed to achieve query optimization and semantic Gaussian modeling. The first part is the denoising loss, which calculates the distance error between the optimized query position and the denoised LiDAR sampling points to supervise the query's autonomous adjustment of its position to eliminate the influence of initial noise and gradually approach the real scene geometry. The second part is the depth rendering loss, which renders the decoded semantic Gaussian from the current and neighboring views into a depth map, using the depth map obtained from the LiDAR projection as supervision, and calculates the difference between the rendered depth map and the supervised depth map. The third part is the RGB rendering loss, which also renders the semantic Gaussian from multiple views into an RGB image, using the actual acquired RGB image as supervision, to minimize the pixel difference between the rendered image and the real image. The three types of losses work together to guide the query to move towards the scene's occupied area.

[0028] Next, the decoding, generation, and geometric modeling process of semantic Gaussians are clarified. During the pre-training phase, each sparse 3D query decodes and generates multiple semantic Gaussians, which contain attributes such as position, rotation, scale, and opacity. Through supervision by multi-task training objectives, the query continuously optimizes its position, thereby driving the spatial distribution adjustment of the semantic Gaussians it decodes and generates. This allows the semantic Gaussians to accurately fit the local 3D geometric structure, achieving fine modeling of scene details such as road edges and vehicle outlines, forming an effective representation of the scene's geometric structure.

[0029] Finally, through the complete training process of the geometric denoising pre-training stage, the model learns the basic features of the scene's 3D geometry. After multiple rounds of training, the sparse 3D query has the ability to autonomously move to the scene's occupied area, and the semantic Gaussian generated by its decoding can accurately reflect the local 3D geometric attributes, providing strong geometric priors for subsequent image and 3D scene collaborative enhancement tasks. This ensures the integrity and accuracy of the scene structure during subsequent enhancement processes and reduces enhancement bias caused by missing geometric information.

[0030] The third step involves first calling a model pre-trained with geometric denoising to process the image observation data at the current moment. A convolutional neural network is used to extract two-dimensional features from the image, which include texture, color, and semantic contour information of the scene. These extracted two-dimensional features are then input into the model's feature fusion module, where they interact cross-modally with the optimized sparse three-dimensional query. This allows the two-dimensional image features and the three-dimensional query features to complement each other, providing rich multimodal information support for subsequent semantic Gaussian decoding.

[0031] Then, the optimized sparse 3D queries are decoded into a dense set of semantic Gaussians. Each sparse query generates multiple semantic Gaussians with specific attributes, including 3D position, rotation parameters, scale parameters, opacity, and semantic category distribution. During decoding, the model adjusts the attribute parameters of each semantic Gaussian based on the geometric priors learned in the pre-training phase, ensuring that the semantic Gaussians accurately cover the occupied areas in the scene, such as different types of scene elements like roads, vehicles, and buildings, forming a dense representation of the 3D scene.

[0032] Next, an opacity-weighted geometric estimation mechanism is introduced to optimize the Gaussian-to-voxel mapping process. When calculating the voxel occupancy probability, the opacity of the semantic Gaussian is incorporated into the calculation. The occupancy contribution of the Gaussian is weighted by opacity, so that high-opacity semantic Gaussians have a greater impact on the voxel occupancy probability, while low-opacity semantic Gaussians contribute less. This mechanism effectively avoids interference from semantic Gaussians in unoccupied regions on the voxel prediction results, while allowing semantic Gaussians in occupied regions to more accurately reflect the true occupancy state of the voxels, thus improving the accuracy of the mapping process.

[0033] Meanwhile, an efficient Gaussian voxel projection algorithm is employed to improve computational efficiency. During the projection process, voxels are divided into fixed-size 3D blocks, allowing voxels within the same block to collaboratively load nearby semantic Gaussian data, reducing redundant data read operations. In the reverse computation phase, threads are bound to individual semantic Gaussians, avoiding atomic operations on voxel gradients and reducing computational resource consumption. Through this optimization, the forward and reverse computation times for Gaussian-to-voxel projection are significantly shortened, meeting real-time processing requirements.

[0034] Finally, semantic decoding is used to align image features with the semantic information of the 3D scene. The semantic category information contained in the image features is spatially mapped and associated with corresponding semantic Gaussians and voxels, ensuring that the semantic category prediction of each voxel is consistent with the semantic features of the corresponding region in the image. Based on the aligned semantic information and the optimized voxel occupancy probability, a high-precision 3D scene semantic occupancy result is generated. This result includes both the 3D geometric structure of the scene and accurately labels the semantic category of each region, providing reliable data support for subsequent scene analysis and applications.

[0035] The fourth step involves calculating the velocity information corresponding to the current sparse 3D query based on the semantic occupancy results generated through image and 3D scene co-enhancement. For each query, its positional changes across consecutive time frames are analyzed. Combined with the motion characteristics of dynamic targets in the scene, the velocity parameters of the query are estimated using a temporal interpolation algorithm. This velocity information is used to model the motion state of dynamic targets, such as vehicle trajectories and pedestrian paths, ensuring that the model can capture the positional change trends of dynamic elements in the scene in real time, providing a motion basis for subsequent query updates.

[0036] Then, a preset time interval is set as the update cycle for the query queue and semantic Gaussian parameters. The interval length is determined based on the degree of dynamic change in the scene. The update interval is shortened in areas with high traffic volume and frequent target movement, and appropriately extended in areas with a high proportion of static scenes, in order to balance update accuracy and computational resource consumption. When the update cycle is reached, the edge node initiates the update process, first filtering out outdated queries or queries with large location deviations from the query queue and removing them from the queue, while retaining queries with high confidence and accurate location for subsequent optimization.

[0037] Next, the semantic Gaussian parameters are optimized and adjusted based on the latest acquired image observation data. The two-dimensional features of the new image are re-fused with the three-dimensional features of the existing query, and the position, rotation, scale, and opacity attributes of the semantic Gaussian are updated through a cross-modal attention mechanism. For the semantic Gaussian corresponding to dynamic targets, its current position is predicted based on previously calculated velocity information and matched and calibrated with features observed in the new image to correct positional deviations. For the semantic Gaussian corresponding to static scene elements, parameters are adjusted only when image observations show changes in the scene structure, ensuring the stability of the static region representation.

[0038] Simultaneously, the accuracy of 3D scene representation is continuously optimized. By comparing the updated semantic occupancy results with historical results, the difference index between the two is calculated. If the difference exceeds a preset threshold, the number of queries and the semantic Gaussian distribution density are further adjusted. In areas with large differences, the number of queries and semantic Gaussians is increased to improve the ability to represent details, while redundant queries are reduced in areas with small differences to reduce computational costs. In addition, a scene consistency verification mechanism is introduced to ensure that the updated 3D scene is consistent with the image observation in terms of spatial structure and semantic category, avoiding geometric contradictions or semantic conflicts.

[0039] Finally, through the aforementioned dynamic update and optimization process, dynamic collaborative enhancement of images and 3D scenes is achieved. After each update, the model can output more accurate 3D scene semantic occupancy results based on the latest image observation and motion state information. This ensures the accuracy of static scene structure while tracking the motion trajectory of dynamic targets in real time, providing continuous and reliable 3D scene data support for applications such as autonomous driving environmental perception and intelligent monitoring scene analysis.

[0040] The embodiments of this application have been described above with reference to the accompanying drawings. Unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other. This application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A method for collaborative enhancement of a three-dimensional scene based on illumination-adaptive planar representation, characterized in that: The method comprises the following steps: Step one: complete scene voxelization and device perception configuration, obtain device perception utility score through occlusion judgment and attenuation coefficient calculation, filter three-dimensional queries based on the score and build a queue, combine cross-modal feature fusion to optimize the query set, and finally filter the optimal query subset to supplement the queue; Step two: configure the initial query position of geometric denoising pre-training, build a multi-task training target, generate and model through semantic Gaussian decoding, and make sparse three-dimensional queries have the ability to autonomously approach the scene occupancy area through multiple rounds of training; Step three: call the pre-trained model to fuse two-dimensional image features and three-dimensional query features, decode to generate a dense semantic Gaussian set, optimize Gaussians, voxel mapping mechanism and calculation efficiency, realize semantic information alignment and generate high-precision three-dimensional scene semantic occupancy results; Step four: based on the semantic occupancy results and time sequence information, set the update period and dynamically maintain the query queue, optimize the semantic Gaussian parameters, and through precision optimization and consistency verification, achieve dynamic collaborative enhancement of images and three-dimensional scenes.

2. The method of adaptive query streaming based image and 3D scene collaborative enhancement according to claim 1, characterized in that: The method for completing scene voxelization and device perception configuration, obtaining device perception utility score through occlusion judgment and attenuation coefficient calculation, filtering three-dimensional queries based on the score and building a queue, combining cross-modal feature fusion to optimize the query set, and finally filtering the optimal query subset to supplement the queue comprises: The edge node divides the three-dimensional scene area into a three-dimensional voxel set of equal size, receives the state information sent by the intelligent device participating in perception and completes device registration, constructs a three-dimensional cone theoretical perception space for each device to determine the range of perceivable voxels; An occlusion judgment algorithm is designed to determine the visibility of candidate voxels, and an occlusion identification function is generated; The spatial distance attenuation coefficient and the angle difference attenuation coefficient are constructed, and the perception accuracy attenuation coefficient is obtained by multiplying the two; the device perception utility score is calculated by fusing the attenuation coefficient, the inverse value of the occlusion identification, the voxel weight and the coverage demand coefficient; According to the score, three-dimensional queries are filtered, an initial sparse three-dimensional query queue is built and historical query information is stored, cross-modal fusion is performed on the two-dimensional features of the current image observation data and the three-dimensional features of the historical queries, the current query set is optimized and updated, a query propagation strategy is designed and a greedy selection algorithm is used to filter the optimal query subset and add it to the queue.

3. The method of collaborative augmentation of a three-dimensional scene based on adaptive planar representations of illumination according to claim 2, characterized in that: The occlusion identification function is defined as: When the spatial connection line between the intelligent device and the candidate voxel intersects with other devices or static obstacles, the candidate voxel is invisible, otherwise it is visible, and the expression is: ； Wherein, v is the intelligent device; g is the candidate voxel.

4. The method of collaborative augmentation of a three-dimensional scene based on adaptive planar representations of illumination according to claim 2, characterized in that: The expression of the spatial distance attenuation coefficient in step one is: ； wherein for voxels optimal perceptual coefficient; is the distance attenuation coefficient; is the distance between the device and the voxel space; min, max effective sensing distance for the sensor; The expression of the angle difference attenuation coefficient is: ； wherein, is the angle difference between the device motion direction and the voxel direction; for the sensor maximum field of view angle; The perception accuracy attenuation coefficient is: 。 5. The method of collaborative augmentation of a three-dimensional scene based on adaptive planar representations of illumination according to claim 2, characterized in that: The calculation formula of the device perception utility score in step one is: ； wherein, is a set of device perceptible voxels; V is the voxel importance weight; Cover the demand coefficient for the voxel scene.

6. The method of collaborative augmentation of a three-dimensional scene based on adaptive planar representations of illumination according to claim 1, wherein: The method for configuring the initial query position of geometric denoising pre-training, building a multi-task training target, generating and modeling through semantic Gaussian decoding, and making sparse three-dimensional queries have the ability to autonomously approach the scene occupancy area through multiple rounds of training comprises: The initial query position setting mode of the geometric denoising pre-training stage is determined, the sparse three-dimensional query initial position is anchored on the noisy lidar point cloud, a multi-task training target including denoising loss, depth rendering loss and RGB rendering loss is constructed, the decoding generation and geometric modeling process of semantic Gaussian are clarified, and the sparse three-dimensional query is enabled to autonomously move to the scene occupancy area through multiple rounds of training.

7. The method of collaborative augmentation of a three-dimensional scene based on a light-adaptive planar representation according to claim 6, characterized in that: The noisy lidar point cloud in step two is obtained by sampling a fixed number of sampling points from the original lidar point cloud, and then adding uniform distributed noise to each sampling point.

8. The method of collaborative augmentation of a three-dimensional scene based on adaptive planar representations of illumination according to claim 2, characterized in that: The denoising loss in step two is obtained by calculating the distance error between the optimized query position and the denoised lidar sampling point. The depth rendering loss is obtained by calculating the difference between the depth map rendered by the semantic Gaussian from the current and adjacent view and the depth map obtained by projecting the lidar. The RGB rendering loss is obtained by calculating the pixel difference between the RGB image rendered by the semantic Gaussian from multiple views and the actually collected RGB image.

9. The method of collaborative augmentation of a three-dimensional scene based on adaptive planar representations of illumination according to claim 1, characterized in that: The pre-trained model is called to fuse two-dimensional image features and three-dimensional query features, and a dense semantic Gaussian set is generated by decoding, optimizing Gaussian, voxel mapping mechanism and improving calculation efficiency, realizing semantic information alignment and generating high-precision three-dimensional scene semantic occupancy results, including: The model pre-trained by geometric denoising is called to process the current image observation data, extract two-dimensional features and interact with the optimized sparse three-dimensional query across modalities; The optimized sparse three-dimensional query is decoded into a dense semantic Gaussian set; An opacity weighted geometry estimation mechanism is introduced to optimize the mapping process of Gaussian to voxel, and an efficient Gaussian voxel projection algorithm is used to improve the calculation efficiency; Through semantic decoding, the image features and three-dimensional scene semantic information are aligned to generate high-precision three-dimensional scene semantic occupancy results.

10. The method of collaborative augmentation of a three-dimensional scene based on a light-adaptive planar representation according to claim 9, characterized in that: The attributes of the semantic Gaussian in step three include three-dimensional position, rotation parameter, scale parameter, opacity and semantic class distribution; The implementation of the efficient Gaussian voxel projection algorithm in step three is as follows: the voxels are divided into fixed-size three-dimensional blocks, and the voxels in the same three-dimensional block are loaded with nearby semantic Gaussian data; In the reverse calculation stage, the thread is bound to a single semantic Gaussian to avoid atomic operation on voxel gradient.

Images