Real-time collision detection method and system based on three-dimensional gaussian scene reconstruction

Abstract

The application discloses a real-time collision detection method and system based on three-dimensional Gaussian scene reconstruction, relates to three-dimensional Gaussian scene image processing, and has the implementation process as follows: target scene cross-view light difference analysis and definition of three-dimensional scene light sensitivity contained therein are performed, the spatial position of the target scene is calibrated as a light-sensitive area and a light consistency area, light coordination processing of the target scene image containing adaptive coordination transformation is driven for the light-sensitive area, three-dimensional Gaussian scene reconstruction based on 3DGS model parameter initialization and iterative optimization is driven for the light consistency area, after the three-dimensional Gaussian scene reconstruction is completed, the three-dimensional Gaussian ellipsoid set in the three-dimensional Gaussian scene reconstruction model is used to extract continuous triangular meshes through density field superposition and moving cube algorithm as collision detection geometric primitives, and collision detection is performed. By using the application, the collision detection precision can be improved when performing collision detection in the reconstructed three-dimensional Gaussian scene.

Description

Technical Field

[0001] This invention relates to the field of 3D Gaussian scene image processing technology, and in particular to a real-time collision detection method and system based on 3D Gaussian scene reconstruction. Background Technology

[0002] Against the backdrop of the rapid development of generative artificial intelligence and 3D reconstruction technology, building a high-fidelity, interactive 3D (Three-Dimensional) digital content ecosystem has become a core trend in the field of spatial computing. New scene reconstruction technologies, represented by 3D Gaussian Splatting, have greatly reduced the threshold for generating high-quality 3D models due to their explicit scene representation and real-time rendering capabilities, making it possible to quickly build photorealistic virtual environments. However, current 3D scenes reconstructed based on 3DGS (3D Gaussian Splatting) still face limitations in collision detection, which restricts the in-depth application of 3DGS technology in core spatial computing scenarios such as virtual reality and real-time digital human interaction.

[0003] The main implementation process of existing 3DGS collision detection methods is as follows: First, based on a 3D Gaussian splash model, using multi-view RGB images of the target scene and corresponding camera poses as input, and applying SfM (Structure from...) to... Motion (Motion Recovery Structure) generates sparse point clouds and camera poses. Each point is initialized as a 3D Gaussian ellipsoid defined by parameters such as position, covariance matrix, and spherical harmonic coefficients. After iterative optimization using differentiable rasterization and image reconstruction loss, a 3DGS Gaussian representation of the target scene is obtained. Then, algorithms such as moving cubes and convex hull decomposition are used to extract explicit triangular meshes and convex hull data from the 3DGS Gaussian representation of the target scene as collision detection geometric primitives. Simultaneously, the virtual character or robot body is modeled separately as a set of Gaussian ellipsoids or a triangular mesh model as the geometric representation of the agent, and its pose is updated in real time with the kinematic state. Then, based on the above-extracted collision detection geometric primitives of the target scene and the geometric representation of the agent, combined with algorithms such as overlap integral and control obstacle function, the collision risk between the virtual character or robot body and the scene is quantified to obtain real-time collision state feedback such as collision point position, penetration depth, and minimum safe distance.

[0004] Existing 3D scene reconstruction methods have developed technical solutions that fuse multi-sensor data to further improve reconstruction accuracy and robustness. For example, the Chinese invention patent application with publication number CN120088403B discloses a scene 3D reconstruction method and related apparatus based on camera-LiDAR fusion, which includes: first, acquiring the relative pose between the camera and LiDAR, as well as the camera image and LiDAR point cloud sensor data of the scene based on the selected scene; then, performing sensor tracking, map updating, and loop closure detection sequentially based on 3DGS to obtain the scene 3D reconstruction result; wherein, in the sensor tracking stage, pose optimization is performed by minimizing photometric loss and geometric loss; in the map updating stage, LiDAR point clouds that meet preset conditions are added to the Gaussian point cloud map.

[0005] Existing 3D scene reconstruction methods can improve pose estimation accuracy in scenes with varying lighting by introducing LiDAR point clouds as geometric constraints. However, these methods still rely on an optimization framework that minimizes photometric loss, failing to fundamentally address the cross-view lighting differences caused by inconsistent lighting in multi-view images. During 3DGS scene reconstruction and collision detection, multi-view images of the target scene are typically taken at different times and under different lighting conditions. Significant differences in light intensity, color temperature, and shadow distribution may exist between these viewpoints. Because existing techniques often employ an optimization strategy based on minimizing photometric loss, treating all image pixel values ​​as absolute true values, this can lead to conflicting gradient directions for the Gaussian gradients of the same spatial region in the target scene images from different viewpoints. When iteratively optimizing Gaussian parameters based on multi-view images of target scenes where lighting differences are caused by variations in light intensity, color temperature, and shadow distribution, the position and covariance of the Gaussian gradients are sensitive to color gradients, while the gradients for color gradients are more complex. The lighting differences caused by temperature shift and shadow occlusion can prevent the optimization process from converging to a geometrically consistent surface position. This may result in the reconstructed 3D Gaussian scene not having a physically consistent surface representation in space. For example, the center of the Gaussian density distribution of the same wall may be shifted to the left when viewed from the left and to the right when viewed from the right, with a possible shift of several centimeters in the center position. Furthermore, when calculating collision distances based on such geometrically inconsistent 3DGS models, the collision distances obtained when approaching the same collider from different directions may differ significantly because the Gaussian center position of colliders such as walls and tabletops may change with the viewing direction. For example, when approaching the same wall from the left, the collision distance may be underestimated because the Gaussian center is shifted to the right, causing the robot or virtual object to pass through the wall. When approaching from the right, the collision distance may be overestimated because the Gaussian center is shifted to the right, leading to premature collision alarms. Ultimately, this results in insufficient collision detection accuracy when performing collision detection in the reconstructed 3D Gaussian scene. Summary of the Invention

[0006] This invention provides a real-time collision detection method and system based on 3D Gaussian scene reconstruction, which can solve the technical problem of insufficient collision detection accuracy when performing collision detection in the reconstructed 3D Gaussian scene.

[0007] This application provides a real-time collision detection method based on 3D Gaussian scene reconstruction, comprising: 1) performing cross-view illumination difference analysis of the target scene and defining its 3D scene illumination sensitivity; 2) grading the spatial location of the target scene as illumination-sensitive and illumination-consistent regions; 3) driving illumination coordination processing of the target scene image with adaptive coordination transformation for the illumination-sensitive regions; 4) selecting to send an illumination coordination processing failure message or perform 3D Gaussian scene reconstruction based on the illumination coordination processing result after the illumination coordination processing is completed; 5) driving 3D Gaussian scene reconstruction based on parameter initialization and iterative optimization of the 3DGS model for the illumination-consistent regions; 6) after the 3D Gaussian scene reconstruction is completed, extracting continuous triangular meshes based on the 3D Gaussian ellipsoid set in the 3D Gaussian scene reconstruction model using density field superposition and moving cube algorithms, using these meshes as collision detection geometric primitives, and performing collision detection.

[0008] This application provides a real-time collision detection system based on 3D Gaussian scene reconstruction, comprising: an illumination difference analysis module, a 3D scene reconstruction and illumination coordination module, and a collision detection module. The illumination difference analysis module is used to define the spatial location of the target scene as an illumination-sensitive region and an illumination-consistent region by performing cross-view illumination difference analysis of the target scene and defining the illumination sensitivity of the 3D scene. The 3D scene reconstruction and illumination coordination module is used to drive illumination coordination processing of the target scene image with adaptive coordination transformation for the illumination-sensitive region. After the illumination coordination processing of the target scene image is completed, it selects to send an illumination coordination processing failure prompt or perform 3D Gaussian scene reconstruction based on the illumination coordination processing result. For the illumination-consistent region, it drives 3D Gaussian scene reconstruction based on parameter initialization and iterative optimization of the 3DGS model. The collision detection module is used to extract continuous triangular meshes based on the 3D Gaussian ellipsoid set in the 3D Gaussian scene reconstruction model, using density field superposition and moving cube algorithms, as the geometric primitives for collision detection, and perform collision detection after the 3D Gaussian scene reconstruction is completed.

[0009] The beneficial effects achieved by the technical solutions provided in the embodiments of the present invention include at least the following: (1) By performing cross-view illumination difference analysis of the target scene and defining the illumination sensitivity of the three-dimensional scene, the spatial location of the target scene is marked as illumination-sensitive area and illumination-consistent area. This enables precise positioning of the illumination-sensitive area in the target scene that suffers from geometric deviation due to illumination differences. At the same time, by identifying and marking illumination differences at the three-dimensional space level rather than the two-dimensional image level, subsequent processing can directly intervene in the root area of ​​geometric deviation. Compared with the blind processing on the two-dimensional image, the processing is more targeted and efficient, laying the foundation for improving the geometric accuracy of the three-dimensional Gaussian scene reconstruction. For illumination-sensitive areas, the target scene image illumination coordination processing with adaptive coordination transformation is driven. After the target scene image illumination coordination processing is completed, the illumination coordination processing failure prompt or the three-dimensional Gaussian scene reconstruction is selected based on the result of the target scene image illumination coordination processing. The illumination-consistent region-driven 3D Gaussian scene reconstruction, based on parameter initialization and iterative optimization of the 3DGS model, enables the application of differentiated processing strategies to spatial regions under different lighting conditions. This ensures the geometric consistency of illumination-sensitive areas while avoiding redundant processing of areas that already possess illumination consistency, effectively reducing clipping and accidental collisions caused by geometric deviations. After the 3D Gaussian scene reconstruction is completed, continuous triangular meshes are extracted from the 3D Gaussian ellipsoid set in the 3D Gaussian scene reconstruction model using density field superposition and moving cube algorithms. These meshes serve as geometric primitives for collision detection, thus transforming the 3D Gaussian scene model from a renderable visual representation into a collision-detection geometric representation capable of participating in physical interactions. This allows the 3D scene based on Gaussian splashing to be directly applied to spatial computing scenarios requiring real-time spatial overlap judgment, such as virtual human or robot navigation and obstacle avoidance, and virtual reality interaction.

[0010] (2) By acquiring the illumination difference correlation map of the target scene multi-view images, it is possible to quantitatively characterize the comprehensive difference in illumination intensity, color temperature, and shadow distribution between any two views in the target scene, providing a unified and comparable data basis for subsequent three-dimensional scene illumination sensitivity definition. It can comprehensively reflect the comprehensive differences in illumination conditions from different views, making the identification of illumination-sensitive areas have a clear quantitative basis, and improving the accuracy and reliability of subsequent illumination sensitivity definition; for each spatial region in the target scene, three-dimensional scene illumination sensitivity definition is performed, and the local illumination difference correlation value between the local image blocks corresponding to any two target scene RGB images in the effective view set is obtained. If there are any two views in the effective view set, their If the local illumination difference between corresponding local image patches is greater than a preset local illumination difference threshold, it indicates that the current spatial location belongs to a light-sensitive area, and target scene image illumination coordination processing is performed. If the local illumination difference between local image patches corresponding to any two viewpoints in the effective viewpoint set is not greater than the preset local illumination difference threshold, it indicates that the current spatial location belongs to a lighting-consistent area, and 3D Gaussian scene reconstruction is directly performed. This achieves differentiated processing strategies for different regions of the target scene due to differences in illumination conditions, resulting in a significant improvement in the geometric accuracy of the finally reconstructed 3D Gaussian scene model. This provides a reliable geometric benchmark for subsequent collision detection and effectively avoids clipping and accidental collision problems caused by geometric deviations introduced by illumination differences.

[0011] (3) If there are sudden changes in local illumination in multi-view images of the target scene from different perspectives, such as local highlights or local occlusion shadows, relying solely on weighted summation of global statistical features may not accurately depict the actual situation of local illumination differences. Therefore, it is necessary to use an image patch-based illumination difference measurement method to obtain illumination difference correlation values. Each set of target scene RGB images is divided into a preset number of non-overlapping image patches to obtain a first image patch set and a second image patch set. For each image patch, the average brightness, color temperature feature vector, and shadow area ratio of the image patch are extracted to achieve a refined representation of local illumination differences at the spatial resolution level. Compared with the method that relies on the global statistics of the entire target scene RGB image, the image patch-based method is more effective. Block feature extraction can preserve the non-uniform spatial distribution information of illumination differences, reduce the masking effect of global statistical averaging on the actual contribution of local illumination abrupt changes to image blocks, and enable accurate capture of the degree of illumination difference between different viewpoints in these local areas even when facing local highlights or local occlusion shadows in subsequent steps. This effectively improves the accuracy of illumination sensitivity definition in illumination abrupt change areas. The block-level illumination difference value between any image block in the first image block set and any image block in the second image block set is obtained. Using the block-level illumination difference value as the acquisition benchmark, the illumination difference correlation value between the corresponding two target scene RGB images is obtained, thereby achieving effective convergence from local measurement of image blocks to global measurement of the entire target scene RGB image.

[0012] It should be understood that the content described in this section is not intended to identify key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Attached Figure Description

[0013] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a flowchart of the real-time collision detection method based on 3D Gaussian scene reconstruction provided by the present invention; Figure 2 This is an overall judgment flowchart of the real-time collision detection method based on three-dimensional Gaussian scene reconstruction provided by the present invention; Figure 3 This is a logic diagram of the target scene image illumination coordination processing of the real-time collision detection method based on 3D Gaussian scene reconstruction provided by the present invention. Figure 4This invention provides a real-time collision detection method based on 3D Gaussian scene reconstruction, which uses multi-view samples with illumination intensity and color temperature proxy feature maps. Figure 5 This is a diagram showing the evolution of collision states in a three-dimensional scene using the real-time collision detection method based on three-dimensional Gaussian scene reconstruction provided by this invention. Figure 6 This is a diagram illustrating the collision query performance and microscopic mechanism of the real-time collision detection method based on 3D Gaussian scene reconstruction provided by this invention. Figure 7 This is a schematic diagram of the structure of the real-time collision detection system based on three-dimensional Gaussian scene reconstruction provided by the present invention. Detailed Implementation

[0015] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0016] The following is a brief explanation of the relevant technical terms used in this application.

[0017] Example 1, by Figure 1The flowchart of the real-time collision detection method based on 3D Gaussian scene reconstruction shows that during the 3DGS scene reconstruction and collision detection process, the spatial location of the target scene is marked as a lighting-sensitive region and a lighting-consistent region by performing cross-view illumination difference analysis of the target scene and its included 3D scene illumination sensitivity definition. The 3D scene illumination sensitivity definition is used to identify spatial regions in all spatial locations of the target scene that may have cross-view illumination differences due to significant illumination differences between different viewing angles, reducing unnecessary computational overhead and image distortion introduced by indiscriminate processing of multi-view images of the target scene. The cross-view illumination difference analysis of the target scene is used to quantify the degree of difference in illumination conditions from different viewing angles at each spatial location. Illumination coordination processing of the target scene image with adaptive coordination transformation is driven for the illumination-sensitive regions. After the illumination coordination processing of the target scene image is completed, the image is selected for transmission based on the result of the illumination coordination processing. The system prompts for lighting coordination failure or performs 3D Gaussian scene reconstruction. Adaptive coordination transformation is used to reduce redundant gradient interference caused by lighting differences between multi-view images of the target scene. The target scene image lighting coordination processing is used to separate the lighting-related components and geometric-related components of the multi-view images of the target scene corresponding to the lighting-sensitive areas, ensuring that the coordinated target multi-view images have consistent lighting conditions in the lighting-sensitive areas. For the lighting-consistent areas, the system drives the parameter initialization and iterative optimization of 3D Gaussian scene reconstruction based on the 3DGS model. After the 3D Gaussian scene reconstruction is completed, based on the 3D Gaussian ellipsoid set in the 3D Gaussian scene reconstruction model, continuous triangular meshes are extracted as collision detection geometric primitives through density field superposition and moving cube algorithm. This realizes the generation of collision surrogates based on the 3D Gaussian point attributes, and then collision detection is performed to perform real-time spatial overlap measurement between the agent geometric model and the 3D Gaussian scene reconstruction model.

[0018] The real-time collision detection method based on 3D Gaussian scene reconstruction provided in this application can reuse a pre-built 3D scene reconstruction and collision detection knowledge base in practical applications. This knowledge base stores multiple core control parameters preset by experts in spatial computing and 3D vision, such as preset radius, preset viewpoint, preset maximum number of iterations, and preset number of image blocks. Furthermore, this 3D scene reconstruction and collision detection knowledge base also includes historical operational data of 3D scene reconstruction and collision detection that has undergone desensitization and standardization preprocessing. This data covers various scene types, such as indoor office scenes, industrial warehousing scenes, and commercial display scenes, and includes data under different lighting conditions. The system records the acquisition parameters of multi-view images under various lighting conditions, the density field distribution statistics of the 3D Gaussian ellipsoid set, the vertex and face distribution of the triangular mesh extracted by the moving cube algorithm, the time consumption records of spatial index structure construction and collision query, the trajectory planning and collision feedback sequences of the agent in different scenes, and also includes corresponding geometric deviation correction records and collision detection accuracy evaluation feedback information. Specifically, in terms of storage architecture, this 3D scene reconstruction and collision detection knowledge base adopts a hierarchical hybrid storage method: a relational database is used to store structured index data, such as the correlation values ​​of illumination differences in images from different viewpoints, the illumination sensitivity determination results of each spatial location, and the data before illumination coordination processing. The database stores image patch difference metrics, the number and distribution statistics of 3D Gaussian ellipsoids, the number of vertices and faces of triangular meshes, the shortest distance and penetration depth values ​​for collision detection, etc.; while the non-relational database is used to store unstructured data, such as the separation results of illumination and geometric components of local image patches, smoothed and detailed versions of image patch data before and after the coordinated transformation, the original triangular mesh data generated by the moving cube algorithm, collision detection geometric primitive data simplified by the quadratic error metric, and the real-time pose sequence and spatial overlap visualization results of the agent during the collision detection process; spatial computing system operators or algorithm developers can use the data from the 3D scene reconstruction and collision detection process to... The system collects multi-source monitoring data, including the distribution map of light-sensitive areas output by the light difference analysis module, the geometric accuracy evaluation index output by the 3D scene reconstruction and light coordination module, the collision marker and penetration depth feedback sequence output by the collision detection module, and the overall detection success rate and real-time performance feedback of the system. It periodically verifies and dynamically iterates and adjusts various preset parameters stored in the knowledge base, so that the 3D scene reconstruction and collision detection knowledge base can continuously adapt to scene reconstruction needs under different lighting conditions, scene geometric structure differences of different complexities, collision detection requirements of different agent sizes and movement speeds, and diverse real-time interaction and safety obstacle avoidance application scenarios.

[0019] In this embodiment, by accurately locating the spatial regions that may cause geometric deviations due to differences in illumination, the indiscriminate processing of all scene spatial positions is avoided. This significantly reduces the computational overhead of subsequent illumination coordination processing while ensuring recognition accuracy. At the same time, it improves the balance between geometric accuracy and computational efficiency in 3D Gaussian scene reconstruction. It realizes the generation of collision proxy bodies based on the attributes of 3D Gaussian points. Finally, by measuring the spatial overlap relationship between the intelligent body geometric model and the collision detection geometric primitives in real time, high-precision and low-latency real-time collision detection in 3D Gaussian scenes is achieved.

[0020] Depend on Figure 2 The overall judgment flowchart of the real-time collision detection method based on 3D Gaussian scene reconstruction provided shows that: the target scene cross-view illumination difference analysis is initiated, the 3D scene illumination sensitivity definition is performed, the obtained local illumination difference correlation value is used as a benchmark, and it is judged whether the local illumination difference correlation value is greater than the preset local illumination difference threshold. If it is, the current spatial position is determined to be in the illumination sensitive area, and the target scene image illumination coordination processing is performed. Otherwise, the current spatial position is determined to be in the illumination uniform area, the 3D Gaussian scene reconstruction is performed directly, and collision detection is performed based on the obtained 3D Gaussian scene reconstruction model.

[0021] Further, the cross-view illumination difference analysis of the target scene is performed as follows: Step 1, obtain a set of multi-view images of the target scene containing RGB images of the same target scene taken from different spatial positions and directions, as well as the camera poses corresponding to the multi-view images of the target scene; based on the SfM algorithm, obtain the spatial position and orientation of the camera in the spatial coordinate system when taking multi-view images of the target scene, i.e., the camera pose. The camera pose determines the spatial ray direction corresponding to each pixel in each RGB image of the target scene, for example, represented by a 3×3 rotation matrix S and a 3×1 translation vector p, denoted as [S|p]; Step 2, based on the set of multi-view images of the target scene and the corresponding camera poses, derive the illumination difference correlation graph of the multi-view images of the target scene. The specific derivation process is as follows: For the multi-view images of the target scene, extract the corresponding multi-view illumination feature parameters, including the average brightness, color temperature feature vector, and shadow area mask; the average brightness is used to reflect the overall illumination intensity, and the specific formula is as follows: Where u_v represents the average brightness, W represents the width of the brightness channel image, H represents the height of the brightness channel image, and I_v(x,y) represents the brightness value of the pixel at coordinates (x,y) in the brightness channel image, where x represents the horizontal coordinate of the brightness channel image and y represents the vertical coordinate of the brightness channel image. The target scene RGB image is converted from the RGB color space to the HSV color space based on the color space conversion algorithm to obtain a brightness channel image with the same resolution as the input target scene RGB image. Each pixel position (x,y) corresponds to a brightness value.

[0022] Specifically, the process of obtaining the color temperature feature vector is as follows: Obtain the mean values ​​of the R, G, and B channels of the RGB image of the target scene; calculate the ratio of the mean values ​​of the R and G channels and label it as the warm image ratio; calculate the ratio of the mean values ​​of the B and G channels and label it as the cool image ratio; combine the warm image ratio r_RG and the cool image ratio r_BG into a color temperature feature vector (r_RG, r_BG); the color temperature feature vector reflects the overall color bias of the image. A larger warm image ratio indicates a warmer image with a lower color temperature, while a larger cool image ratio indicates a cooler image with a higher color temperature; input the RGB image of the target scene into a pre-trained shadow detection neural network to obtain a shadow probability map. In the shadow probability map, each pixel's value is between 0 and 1, representing the probability that the pixel belongs to a shadow region; perform threshold binarization on the shadow probability map, that is, mark pixels with a shadow probability greater than a preset probability threshold as 1, and vice versa. The element is marked as 0 to obtain the shadow area mask, where the preset probability threshold is represented by the average shadow probability over a historical time period; based on the multi-view illumination feature parameters, the illumination difference correlation value between any two target scene RGB images is calculated as a quantitative measure of the comprehensive difference in illumination conditions between the two; the illumination difference correlation value is used to characterize the comprehensive difference in illumination conditions between two viewpoints; the illumination difference correlation value is obtained by weighted summation of the illumination intensity difference value, color temperature difference value, and shadow distribution difference value; the illumination intensity difference value is obtained by calculating the absolute value of the difference between the mean brightness values ​​of the corresponding two target scene RGB images; the color temperature difference value is obtained by performing Euclidean distance calculation on the color temperature feature vectors of the two target scene RGB images based on the Euclidean distance algorithm; the shadow distribution difference value is obtained by calculating the ratio of the symmetric difference set size to the union set size of the two shadow area masks of the corresponding two target scene RGB images; the specific calculation formula for the illumination difference correlation value is as follows: Where D(i,j) represents the illumination difference correlation value between the i-th target scene RGB image and the j-th target scene RGB image, i and j represent the image index numbers in the target scene multi-view image set, i≠j, and the values ​​of i and j are positive integers between 1 and N, N represents the number of images in the target scene multi-view image set, Dintensity(i,j) represents the illumination intensity difference value between the i-th target scene RGB image and the j-th target scene RGB image, Dcolor(i,j) represents the color temperature difference value between the i-th target scene RGB image and the j-th target scene RGB image, Dshadow(i,j) represents the shadow distribution difference value between the i-th target scene RGB image and the j-th target scene RGB image, and α, β, γ are preset weight coefficients that satisfy α+β+γ=1.

[0023] Traversing all RGB image combinations of the target scene, an illumination difference correlation graph is obtained, with each RGB image of the target scene as a node and the illumination difference correlation value as the edge weight. The illumination difference correlation graph is used to characterize the similarity or difference between any two views in terms of illumination conditions. Based on the illumination difference correlation graph, the illumination sensitivity of each spatial region in the target scene is defined in three dimensions. The specific process is as follows: For any initial spatial position in the target scene, the set of views that can be observed in the multi-view images of the target scene is determined. The specific determination process is as follows: Input the multi-view images of the target scene into the SfM algorithm to obtain a sparse 3D point cloud; input the sparse 3D point cloud into a preset 3DGS model, and take each 3D spatial point in the sparse 3D point cloud as the initial center position of the 3D Gaussian ellipsoid to obtain the initial spatial position set, i.e., the initialized 3D Gaussian ellipsoid set; where the preset 3DGS model is an algorithm framework, which is set in advance by the preset personnel, specifically: First, acquire multi-view images of the target scene and use the SfM algorithm, such as COLMAP (Colored... Multi-View Stereo Mapping (MDR) estimates the camera pose of each image from multiple viewpoints of a target scene and generates a sparse 3D point cloud of the target scene. Next, each 3D point in the sparse 3D point cloud is initialized as a 3D Gaussian ellipsoid, with the spatial coordinates of each 3D point serving as the initial center position of the corresponding 3D Gaussian ellipsoid. Simultaneously, an initial isotropic spherical covariance is preset for each 3D Gaussian ellipsoid, i.e., it is initialized as an isotropic Gaussian sphere. The K-nearest neighbor algorithm is used to calculate the average distance between each 3D point and its neighbors as the initial radius of the Gaussian ellipsoid. Based on this, an initial opacity parameter and an initial spherical harmonic coefficient are assigned to each 3D Gaussian ellipsoid. Subsequently, the 3D Gaussian ellipsoid set is projected onto the camera plane of each input image through differentiable rasterization for rendering, and the difference between the rendered image and the real image is analyzed. As the loss function, gradient descent is used for backpropagation to optimize the parameters of each 3D Gaussian ellipsoid, including center position, covariance matrix, opacity, and spherical harmonic coefficients. During the optimization process, adaptive density control is performed on the 3D Gaussian ellipsoids based on the gradient information from backpropagation. Ellipses that are too large or too small are split and cloned, and ellipsoids with opacity below a preset transparency threshold are deleted. Ultimately, the set of 3D Gaussian ellipsoids can accurately represent the geometric structure and appearance information of the target scene. The preset transparency threshold is represented by the average opacity over a historical time period. The initial spatial position is used to characterize the initial spatial distribution of the 3D Gaussian ellipsoids in the target scene before optimization. Based on the camera poses corresponding to the multi-view images of each target scene, the initial spatial positions are projected onto the RGB image plane of the target scene from each viewpoint using the camera projection matrix to obtain the corresponding spatial position pixel coordinates.If the spatial location pixel coordinates fall within the boundary range of the target scene RGB image from the current viewpoint, i.e., 0≤u≤W and 0≤v≤H, where u and v are the horizontal and vertical coordinates of the spatial location pixel coordinates, and W and H are the width and height of the target scene RGB image from the current viewpoint, it indicates that the spatial location pixel coordinates can be observed in the multi-view image of the target scene from the current viewpoint. The index numbers of all target scene RGB images satisfying the above conditions are assigned to the viewpoint set. The camera projection matrix is ​​composed of a combination of the camera intrinsic parameter matrix and the camera extrinsic parameter matrix, denoted as Pi=Ki[Ri|ti], where... The camera intrinsic parameter matrix Ki contains internal parameters such as focal length and principal point coordinates. The camera extrinsic parameter matrix [Ri|ti] contains external parameters such as rotation matrix and translation vector. Both intrinsic and extrinsic parameters are obtained using the SfM algorithm. The viewpoint set represents the set of viewpoint indices of all target scene RGB images that can observe the initial spatial position. Its elements are the viewpoint indices of the target scene RGB images that can observe the spatial position. For example, the viewpoint set {2,5,8,12} means that the 2nd, 5th, 8th, and 12th target scene RGB images can observe the current spatial position.

[0024] Occlusion visibility determination based on a set of viewpoints involves the following steps: First, obtain the pixel coordinates of the initial spatial position in each target scene's RGB image, and the camera optical center in the camera pose corresponding to each target scene's RGB image. Second, based on the camera pose, generate depth maps for each viewpoint using a multi-view stereo matching algorithm. The depth map is a two-dimensional image with the same resolution as the target scene's RGB image. The value of each pixel in the depth map represents the distance from the corresponding spatial point in the world coordinate system to the camera optical center along the optical axis. Third, for each viewpoint in the set of viewpoints, using the pixel coordinates of the initial spatial position under the current viewpoint as a reference, obtain the depth value of the corresponding pixel position in the viewpoint's depth map, which is then used as the depth value corresponding to the initial spatial position under the current viewpoint. Finally, using the camera pose of the current viewpoint as the starting point and the direction of the spatial ray corresponding to the initial spatial position under the current viewpoint as the direction... The distance from the initial spatial position along the spatial ray direction to the camera's optical center is obtained as the depth reference value of the initial spatial position in the current viewpoint. The depth reference value is compared with the depth value of the corresponding pixel position in the depth map. The depth reference value represents the depth value that the projection point of the initial spatial position should present in the depth map of the current viewpoint under the ideal condition that the initial spatial position is not occluded. If the depth value is less than the depth reference value, it means that there are other object surfaces between the initial spatial position and the camera's optical center that occlude the current initial spatial position, that is, the current initial spatial position is not visible in the current viewpoint, and the target scene RGB image of the current viewpoint is removed from the viewpoint set. If the depth value is not less than the depth reference value, it means that there are no other object surfaces between the initial spatial position and the camera's optical center that occlude the initial spatial position, and the initial spatial position is visible in the current viewpoint, and the current viewpoint is retained. After the above occlusion visibility judgment, the effective viewpoint set of the initial spatial position is obtained.

[0025] Based on local illumination feature parameters, calculate the local illumination difference correlation value between local image patches corresponding to any two target scene RGB images in the effective viewpoint set. The local illumination difference correlation value is obtained by weighted summation of local illumination intensity difference, local color temperature difference, and local shadow distribution difference. The calculation process for local illumination intensity difference, local color temperature difference, and local shadow distribution difference is the same as that for corresponding illumination intensity difference, color temperature difference, and shadow distribution difference, using absolute difference operation, Euclidean distance operation, and symmetric difference and union ratio operation, respectively. However, the former operates on local illumination feature parameters, while the latter operates on multi-view illumination feature parameters. Obtain local image patches corresponding to each target scene RGB image in the effective viewpoint set, extract the local illumination feature parameters of each local image patch, and calculate the local illumination difference value between any two local image patches. A local image patch is represented as an image patch centered on the current spatial location pixel coordinates and within a preset radius. Local illumination feature parameters include local brightness mean, local color temperature feature vector, and local shadow region mask. The acquisition of local illumination feature parameters... The process is the same as the acquisition process of multi-view illumination feature parameters, both including the extraction of local brightness mean, local color temperature feature vector, and local shadow region mask. The difference is that the local illumination feature parameters are extracted using local image patches corresponding to the target scene RGB image as the processing object, rather than the entire target scene RGB image as the processing object. If there are any two views in the effective view set whose corresponding local image patches have a local illumination difference value greater than a preset local illumination difference threshold, it indicates that there is a significant difference in illumination conditions in the local image region corresponding to the current spatial position under different views. The current spatial position is determined to be a light-sensitive region, and target scene image illumination coordination processing is performed. The preset local illumination difference threshold is represented by the average value of the local illumination difference value of the corresponding spatial position in the historical time period. If the local illumination difference value between any two views in the effective view set is not greater than the preset local illumination difference threshold, it indicates that all views that can observe the current spatial position have consistent illumination conditions at the current spatial position. The current spatial position is determined to be a region with consistent illumination, and 3D Gaussian scene reconstruction is performed directly.

[0026] It should be noted that the calibration process of each weight coefficient involved in this application relies on a pre-built weight parameter mapping library. This mapping library is pre-built and stored in the 3D scene reconstruction and collision detection knowledge base by the system designers based on the lighting physics laws in the field of 3D vision reconstruction and the statistical characteristics of a large amount of measured data. It consists of the weight coefficients α, β, and γ in the lighting difference correlation value calculation formula, and the weight coefficient α in the block-level lighting difference value calculation formula. 1 β 1 γ 1and image patch weight coefficient w k The values ​​of these parameters provide a basis for calibration. Specifically, for the construction of the mapping library for the preset weight coefficients α, β, and γ in the correlation values ​​of illumination differences, it is necessary to first collect multiple sets of multi-view image samples of target scenes covering different combinations of illumination conditions, such as combinations of strong and weak light, warm and cool colors, and combinations with and without shadows. The mean brightness, color temperature feature vector, and shadow area mask corresponding to each multi-view image sample set of target scenes are obtained, and the corresponding baseline weights are assigned according to the sensitivity ranking results of each feature dimension in the analysis of the impact of geometric reconstruction accuracy. Secondly, for each combination of illumination conditions, the measured values ​​of the geometric center offset of the three-dimensional Gaussian ellipsoid in the same spatial region under multiple views are obtained, and the correspondence between the difference values ​​of each feature dimension and the geometric center offset is established. Then, a multivariate regression analysis method is used, with the minimization of the geometric center offset as the optimization objective, to calculate the contribution coefficient of each feature dimension, remove invalid data points caused by extreme illumination conditions, such as overexposed or underexposed areas, and screen out the α, β, and γ corresponding to the feature difference intervals with statistical significance.

[0027] For the construction of the mapping library with preset block-level weight coefficients, it is necessary to further refine it based on the calibration results of the aforementioned illumination difference correlation value weight coefficients. Specifically, multiple sets of multi-view image samples of target scenes containing local illumination change regions, such as local highlight regions, local occluded shadow regions, and local light spot regions, are collected. Each image is divided into multiple non-overlapping image blocks. The mean brightness, warm color ratio, cool color ratio, and shadow region proportion are extracted for each image block, and the measured value of the geometric center offset of the three-dimensional Gaussian ellipsoid corresponding to each image block is obtained. Using a local weighted regression method, with the minimization of the local region geometric center offset as the optimization objective, the contribution coefficient of each image block-level feature dimension under the local illumination change condition is calculated. Invalid data blocks that cause feature expression failure due to improper image block size settings are removed, such as excessively large size leading to local feature averaging, and excessively small size leading to unstable feature statistics. The α values ​​corresponding to the feature difference intervals with local statistical significance are then selected. 1 β 1 γ 1 The pre-selected personnel will review and classify the coefficient combinations corresponding to different types of local illumination abrupt changes, so that α 1 β 1 γ 1 The value of α can be dynamically matched according to different types of local illumination abrupt changes. For local highlight areas, α 1 The value of should be greater than β. 1 and γ 1 To enhance the contribution of the average brightness; for local shaded areas, γ 1 The value of should be greater than α. 1 and β 1To enhance the contribution of shadow proportion; for areas with local color temperature abrupt changes, β 1 The value of should be greater than α. 1 and γ 1 To enhance the contribution of color temperature difference.

[0028] The construction of the mapping library for the weight coefficients of image patches requires calibration based on the sum of the gradient magnitudes of each image patch. Specifically, multiple sets of multi-view image samples of target scenes containing different texture richness levels, such as high-texture regions and medium-texture regions, are collected. The sum of the gradient magnitudes of each image patch is calculated separately, and the measured records of the effective gradient contribution of each image patch to the Gaussian position update during the 3D Gaussian scene reconstruction process are obtained, as well as failure records where the reconstruction quality deteriorates due to excessively low gradient magnitudes. A quantile statistical method based on gradient magnitude distribution is adopted, with the optimization objective of maximizing the effective gradient contribution, for different textures. The image blocks corresponding to the level undergo gradient amplitude normalization processing to remove invalid data blocks that cause gradient amplitude distortion due to overexposure or underexposure, such as areas with saturation exceeding a preset threshold or brightness values ​​exceeding the measurable range. The contribution weight of each image block in the overall illumination difference correlation value is determined. Through the construction and dynamic calibration mechanism of the above-mentioned weight parameter mapping library, each weight coefficient can be adaptively matched according to the illumination condition characteristics of different scenes, the type of local illumination change, and the distribution of image texture, ensuring that the calculation of illumination difference correlation value is accurate and reliable in diverse practical application scenarios.

[0029] In this embodiment, the pre-trained shadow detection neural network used is specifically a shadow-aware decomposition Transformer network. This network adopts a Transformer architecture, firstly decomposing the input target scene RGB image into a luminance map and a color map through its luminance channel, and then concatenating them with the original target scene RGB image as input data. This allows the network to pay balanced attention to luminance and color information when calculating the relationship between regions. In optional embodiments, shadow detection network architectures such as Res-CCNet and SwinShadow can also be used. The specific training process of the pre-trained shadow detection neural network is as follows: First, a shadow detection training dataset is obtained. The training dataset contains multiple target scene RGB images and their corresponding shadow region ground truth masks. The shadow region ground truth mask is a binary image with the same resolution as the training image. Pixels with a value of 1 in the ground truth mask indicate that the pixel belongs to the shadow region, and pixels with a value of 0 indicate that the pixel belongs to the non-shadow region. Next, the training image is input into the shadow detection neural network to be trained. After step-by-step feature extraction and fusion by the encoder and decoder, the output is the same as the input training image. The process involves: 1) generating a high-resolution predicted shadow probability map; 2) inputting the predicted shadow probability map and the ground truth mask of the shadow region into a loss function to calculate the difference between the predicted shadow probability map and the ground truth mask, thus obtaining the loss value. The loss function uses the balanced error rate as an evaluation metric, and the difference between the predicted result and the ground truth is calculated using the cross-entropy loss function; 3) using the backpropagation algorithm to calculate the gradient of the learnable parameters in the shadow detection neural network to be trained, including the weight matrix and bias terms in the encoder, the convolution kernel parameters in the decoder, and the projection matrix parameters of the attention mechanism; 4) inputting the gradient into the AdamW optimizer to iteratively update the learnable parameters of the shadow detection neural network to be trained in order to minimize the loss value; 5) repeating the above process of forward propagation, loss calculation, backpropagation, and parameter update until the loss value converges, i.e., the change in the loss value is less than a preset loss threshold in a preset number of iterations, thus obtaining the trained shadow detection neural network, i.e., the pre-trained shadow detection neural network. The preset number of iterations is set in advance by a preset group of people, and the preset loss threshold is represented by the average value of the loss value over a historical time period.

[0030] In this embodiment, by analyzing the cross-view illumination difference of the target scene, the spatial location of the geometric deviation caused by the illumination difference can be accurately located, avoiding redundant processing of the entire target scene RGB image. This fundamentally ensures the accuracy and real-time performance of spatial overlap judgment, providing reliable support for the safe and autonomous operation of intelligent agents in complex environments.

[0031] Depend on Figure 3The provided logic diagram for the target scene image illumination coordination processing of the real-time collision detection method based on 3D Gaussian scene reconstruction shows that: Illumination coordination processing of the target scene image is initiated. Using the set of local image blocks corresponding to the obtained illumination-sensitive region as a benchmark, adaptive separation of illumination-related components and geometric-related components is performed to obtain the illumination-related components and geometric-related components. An adaptive coordination transformation is then performed on the illumination-sensitive region. After the adaptive coordination transformation, pixel-level synthesis is performed to obtain locally corrected image blocks. Based on the locally corrected image blocks, the local illumination difference correlation value is re-obtained. It is determined whether the local illumination difference correlation value is not greater than a preset local illumination difference threshold. If it is, 3D Gaussian scene reconstruction is performed; otherwise, the target scene image illumination coordination processing continues. When the number of processing iterations reaches the preset maximum number of iterations, and the local illumination difference correlation value is still greater than the preset local illumination difference threshold, an illumination coordination processing failure prompt is sent.

[0032] Depend on Figure 4 The provided real-time collision detection method based on 3D Gaussian scene reconstruction, using multi-view sample illumination intensity and color temperature proxy feature maps, shows that: Figure 4 (a) is a scatter plot showing the relationship between brightness and dark pixel ratio. The horizontal axis represents the normalized average brightness, and the vertical axis represents the normalized dark pixel ratio. Each circular dot in the plot represents an RGB image of the target scene from a specific viewpoint. The color intensity of the dot represents the normalized brightness standard deviation; the darker the color, the lower the brightness standard deviation in the image from that viewpoint. The scatter plot shows that the average brightness of the images from different viewpoints is distributed within the normalization range of 0.15 to 0.45, and the brightness standard deviation is distributed within the normalization range of 0.12 to 0.26, indicating that the same target scene... The images of the scene from different perspectives show significant differences in overall illumination intensity and the degree of dispersion of illumination distribution. The horizontal dashed line in the figure corresponds to the normalized dark pixel ratio boundary value matched by the preset illumination difference threshold of this invention, and the vertical dashed line corresponds to the normalized average brightness threshold value. At the same time, there is a clear negative correlation between the dark pixel ratio and the average brightness. The lower the average brightness of the image, the higher its dark pixel ratio. This reflects the uneven distribution of illumination intensity caused by factors such as shooting time, lighting conditions or shadow occlusion between different perspectives, which verifies the necessity of this invention to perform cross-view illumination difference analysis. Figure 4 (b) is a scatter plot of the color temperature proxy feature distribution. The horizontal axis represents the warm color ratio, i.e., the ratio of warm-colored images, and the vertical axis represents the cool color ratio, i.e., the ratio of cool-colored images. Each scatter point in the figure represents a sample image from a certain viewpoint, and the scatter point represents the mean brightness of the image from that viewpoint. Figure 4(b) The background color is a two-dimensional kernel density thermal background. The higher the saturation and the deeper the hue of the background color, the more viewing angle samples are gathered in that color temperature range. That is, a large number of multi-view images are concentrated in this color temperature combination, which easily leads to cross-view character temperature lighting differences. The continuous closed contour lines in the panel are isolines of the lighting difference correlation value. Any warm and cool color ratio combination on the same contour line corresponds to the lighting difference correlation value of the two viewing angles being completely equal. The denser the contour lines, the more significant the fluctuation in the lighting difference correlation value will be caused by a small change in color temperature in that area. This corresponds to areas with a high incidence of sudden changes in local highlights and shadows in the scene. Areas with sparse contour lines represent areas where color temperature changes have a weak impact on lighting differences, corresponding to areas with a high incidence of sudden changes in local highlights and shadows. The image shows a uniformly flat region. The scatter plot of the color temperature proxy feature distribution shows that the ratio of warm colors in images from different viewpoints ranges from 0.9 to 1.6, while the ratio of cool colors ranges from 0.7 to 1.4. The distribution of different viewpoints in the color temperature proxy feature space is relatively dispersed, indicating that there are significant differences in color temperature between images of the same target scene from different viewpoints. Some viewpoints are warmer, while others are cooler, reflecting a coupling relationship between changes in light intensity and color temperature. This verifies the rationality of using the color temperature feature vector as one of the light feature parameters in this invention and provides data support for calculating the correlation value of light difference based on multi-dimensional light feature parameters.

[0033] Further, the specific process of target scene image illumination coordination processing is as follows: Processing step one, obtain the local image blocks in the target scene RGB image corresponding to the multi-view image set of the target scene, with the spatial position pixel coordinates corresponding to the illumination sensitive area as the center and the range as the preset radius, to obtain the local image block set corresponding to the illumination sensitive area; the preset radius is set in advance by a preset person, and in one embodiment, the value of r is 5 to 15 pixels; each local image block in the local image block set contains the appearance information of the same spatial area under different views, but is independently affected by the illumination conditions of each view; Processing step two, adaptively separate the illumination-related components and geometric-related components of the local image blocks, the specific process is as follows: apply an edge-preserving filter operation to the local image blocks to obtain a smooth version of the local image blocks; the edge-preserving filter operation uses guided filtering or bilateral filtering as the filtering operator; the smooth version of the local image block represents the image block that retains the low-frequency information in the original local image block, the low-frequency information includes at least information related to illumination conditions such as illumination intensity changes, color temperature shifts and shadow gradients, while weakening or removing high-frequency detail information such as texture edges and local contrast; then The original local image patch is subtracted pixel-by-pixel from the smoothed version local image patch to obtain the detailed version local image patch. The detailed version local image patch represents an image patch that retains the high-frequency information in the original local image patch. The high-frequency information includes at least information related to geometric structure, such as texture edges and local contrast. The illumination-related component is represented in the form of image data of the smoothed version local image patch. That is, the illumination-related component is a two-dimensional pixel matrix. Each pixel position in the two-dimensional pixel matrix corresponds to an RGB three-dimensional vector. The component values ​​in the three-dimensional vector are used to represent the degree of contribution of the illumination conditions to the appearance of the pixel position under the current viewpoint. For example, suppose that the original local image patch Bi corresponding to a certain illumination-sensitive area contains a local area of ​​a wall with a texture pattern. The wall surface presents a warm overall appearance under a viewpoint with good illumination conditions. After the original local image patch Bi is processed by edge-preserving filtering, the smoothed version local image patch Bismooth is obtained. In this case, the RGB three-dimensional vector values ​​corresponding to each pixel position of the wall local area remain close to the overall brightness and hue level caused by illumination in the original local image patch, while the pixel values ​​on both sides of the texture edge tend to be consistent due to smoothing.For each pixel position (x, y), its corresponding RGB three-dimensional vector (Rismooth(x, y), Gismooth(x, y), Bismooth(x, y)) constitutes the specific numerical representation of the illumination-related components at that pixel position. The RGB three-dimensional vectors of all pixel positions in the smoothed version of the local image patch Bismooth together constitute the illumination-related component representation sequence of that local image patch under the preset viewpoint i. This sequence is a three-dimensional tensor of H×W×3, where H and W are the height and width of the local image patch, respectively, and 3 corresponds to the three RGB color channels. Each value in the quantity quantifies the contribution of lighting conditions to the appearance of a pixel location; the geometrically correlated component is represented by image data of a detailed version of a local image patch. The geometrically correlated component is a two-dimensional pixel matrix, where each pixel location corresponds to an RGB three-dimensional vector. The component values ​​in this three-dimensional vector represent the geometric texture information of the spatial region itself. The preset viewing angle represents the shooting angle direction of the target scene RGB image corresponding to the current local image patch, i.e., the index number of the viewing angle to which this local image patch belongs in the multi-view image set of the target scene, which is preset by the preset personnel.

[0034] Based on the obtained illumination-related components and geometrical correlation components, an adaptive coordination transformation is performed on the illumination-sensitive region. The specific process of the adaptive coordination transformation is as follows: Based on the illumination difference correlation map, the target scene RGB image with the most representative illumination conditions is selected from all viewpoints of the illumination-sensitive region as the illumination reference viewpoint image. The specific selection process is as follows: For each viewpoint in the effective viewpoint set of the illumination-sensitive region, the illumination difference correlation values ​​between the current viewpoint and all other viewpoints in the effective viewpoint set are summed. The viewpoint with the smallest summation result is selected as the illumination reference viewpoint. The viewpoint with the smallest summation of illumination difference correlation values ​​with all other viewpoints indicates that its illumination conditions are closest to the average illumination level of all observed viewpoints, thus possessing the best representativeness. The viewpoint image to be coordinated refers to all target scene RGB images that can be observed in the illumination-sensitive region, excluding the illumination reference viewpoint image. The reference illumination-related component refers to the illumination-related component corresponding to the illumination reference viewpoint image. Based on the least squares method, the reference... Taking the illumination-dependent components as the target, a second-order polynomial is applied to the pixel distribution of the illumination-dependent components of the image to be coordinated to obtain a brightness mapping function. Each pixel value in the illumination-dependent components of the image to be coordinated is then substituted into the brightness mapping function to obtain the illumination-dependent components after adaptive coordination transformation. Simultaneously, the geometric-dependent components of each viewpoint are completely preserved. This is because the geometric-dependent components contain effective gradient information driving Gaussian position updates; preserving them maintains the geometric constraint of each viewpoint on the Gaussian position update, while adjusting only the illumination-dependent components reduces redundant gradient interference caused by illumination differences. Pixel-level synthesis is performed based on the adaptively coordinated transformed illumination-dependent components and their corresponding geometric-dependent components to obtain locally corrected image patches. Pixel-level synthesis means adding the adaptively coordinated transformed illumination-dependent components and their corresponding geometric-dependent components pixel by pixel. For each pixel location, the RGB values ​​of the corresponding pixel location in the smoothed version of the adaptively coordinated local image patch and the detailed version of the local image patch are added to obtain the corrected RGB value of that pixel location.

[0035] Reacquire the local illumination difference correlation value between the local corrected image patch and the local image patch of the reference lighting viewpoint image. If the local illumination difference correlation value is not greater than the preset local illumination difference threshold, it indicates that the local image patch of the current viewpoint has been successfully coordinated to a state consistent with the lighting of the reference viewpoint. Replace the corresponding pixel region in the original target scene multi-view image with the local corrected image patch to obtain the target scene multi-view image after illumination coordination processing, and perform 3D Gaussian scene reconstruction. If the local illumination difference correlation value is still greater than the preset local illumination difference threshold, it indicates that a single coordination transformation has not completely reduced the illumination difference. Continue to perform target scene image illumination coordination processing. If the local illumination difference correlation value is still greater than the preset local illumination difference threshold when the number of iterations reaches the preset maximum number, an illumination coordination processing failure prompt will be sent. The preset maximum number of iterations is set in advance by preset personnel. 3D Gaussian scene reconstruction means inputting multi-view images of the target scene and their corresponding camera poses into a preset 3DGS model for scene reconstruction, resulting in a reconstructed 3D Gaussian scene model, which is used as the scene geometric reference for subsequent collision detection. The 3D Gaussian scene reconstruction model refers to the set of 3D Gaussian ellipsoids of the target scene. Each 3D Gaussian ellipsoid contains parameters such as center position, covariance matrix, opacity, and spherical harmonic coefficients.

[0036] In this embodiment, by coordinating the lighting of the target scene image and reconstructing the 3D Gaussian scene, the redundant gradient interference introduced by the lighting differences across the viewpoint can be reduced without redundancy processing of the entire image. This ensures that the multi-view images of the target scene have consistent lighting conditions in the lighting-sensitive areas, thereby guaranteeing the geometric consistency of the 3D Gaussian scene reconstruction from the source. While ensuring the overall reconstruction efficiency, the geometric accuracy of the 3D Gaussian scene model is improved.

[0037] Depend on Figure 5The provided 3D scene collision state evolution diagram of the real-time collision detection method based on 3D Gaussian scene reconstruction shows that: the collision detection process of this invention, based on the 3D Gaussian scene reconstruction model, extracts continuous isosurface meshes as collision detection geometric primitives through density field superposition and moving cube algorithm, realizing the transformation from Gaussian ellipsoid set to explicit collision geometry. This 3D visualization coordinate system uses the camera world coordinate origin initially calibrated during scene reconstruction as the spatial origin. The negative values ​​of the spatial coordinates originate from the scene space regions to the left of the origin, the negative X-axis direction, behind the origin, the negative Y-axis direction, below the origin, and the negative Z-axis direction in the world coordinate system. The coordinate axes, in meters (m), inherit the camera intrinsic and extrinsic parameter scales obtained from the input image calibration during the 3D Gaussian reconstruction stage, and uniformly use the metric world scale for coordinate calculation. The scattered yellow discrete points in the diagram represent the original 3D Gaussian center point set of the scene, corresponding to... The center position of each Gaussian ellipsoid in the 3D Gaussian scene reconstruction model represents the spatial distribution of the entire environment scene; the central colored continuous trajectory is the motion sampling path of the intelligent agent. According to the collision state discrimination rule of this invention, collision features are calculated point by point and assigned corresponding colors. The figure shows a trajectory that evolves from a safe state to a collision state, passing through the scene sequentially. Green dots on the trajectory represent safe states, yellow dots represent warning states, and red dots represent collision states. This intuitively reflects the dynamic change process of the spatial relationship between the intelligent agent and the scene geometry during the movement. The above observation results intuitively present the complete process of the intelligent agent gradually approaching the scene surface from the safe area, contacting the surface, and finally penetrating the surface. This verifies that the collision detection method of this invention can distinguish the state and continuously monitor the spatial relationship between the intelligent agent and the scene, providing an intuitive state judgment basis for safe trajectory planning and dynamic obstacle avoidance.

[0038] Depend on Figure 6The provided diagram illustrating the collision query performance and micro-mechanism of the real-time collision detection method based on 3D Gaussian scene reconstruction shows that: the horizontal axis represents the batch query task of 75 trajectory points arranged in sampling order in a specific embodiment; the vertical axis represents the time consumed in a single collision query (in milliseconds); the light blue irregular outline in the diagram represents the violin plot distribution area, and the horizontal width of the outline represents the probability density of the sample occurrence under the corresponding time delay value; the larger the width, the more trajectory query samples are in that time delay interval; the red horizontal line in the diagram represents the interquartile range of the time delay data of all query samples, and the orange diamond marks the corresponding P95 quantile position of the time delay; the scattered gray dots represent the original time delay sample value of a single collision query for each trajectory point. The query time of most trajectory points is concentrated between 0.05 milliseconds and 0.10 milliseconds, with a median of 0.086 milliseconds and a P95 quantile of 0.131 milliseconds, indicating that the collision detection of the present invention... The proposed method exhibits stable and efficient query performance during continuous trajectory sampling. Even when facing the most time-consuming 5% of query tasks, the single query time is still far lower than the frame time budget required for typical real-time applications, such as approximately 16.7 milliseconds / frame for 60 FPS. Meanwhile, the violin body is concentrated with a small proportion of long tails, and there is no large-scale drastic dispersion in the latency sampling points. This indicates that the collision query time has good consistency across different trajectory points, and there is no performance degradation due to sudden increases in local scene geometric complexity. These results demonstrate that the collision query method based on spatial index structures, such as BVH bounding volume hierarchical structures, can stably complete a single collision detection calculation within milliseconds. This verifies that the collision detection method of this invention meets real-time requirements in 3D Gaussian scenes and has practical engineering feasibility for deployment in latency-sensitive spatial computing scenarios such as robot navigation and obstacle avoidance, and virtual reality interaction.

[0039] Furthermore, the specific process of collision detection is as follows: Obtain the 3D Gaussian scene reconstruction model and the intelligent agent's geometric model; the intelligent agent is a virtual character or the robot itself; the intelligent agent's geometric model is represented using a triangular mesh model, that is, the intelligent agent's geometric model consists of a set of vertex coordinates in 3D space and a set of triangular facet indices formed by these vertices. Each triangular facet is defined by three vertex indices, and all triangular facets together constitute the surface geometry of the intelligent agent; map the intelligent agent's geometric model to the world coordinate system of the 3D Gaussian scene reconstruction model, and update the spatial position and orientation of the intelligent agent's geometric model in real time according to the intelligent agent's current pose. The specific mapping and updating process is as follows: Obtain the intelligent agent's position in the 3D Gaussian scene reconstruction model... The current pose in the world coordinate system includes the position vector t=(tx,ty,tz) and the orientation rotation matrix R (a 3×3 orthogonal matrix). Based on the rigid body transformation algorithm, each vertex in the agent's geometric model is transformed into the world coordinate system through rigid body transformation. When the agent moves or rotates, its pose (R,t) is updated in real time. The above rigid body transformation is re-executed on all vertices in the agent's geometric model to achieve real-time updates of spatial position and orientation. Each Gaussian ellipsoid in the 3D Gaussian ellipsoid set in the 3D Gaussian scene reconstruction model is superimposed using a Gaussian kernel function, that is, the Gaussian function values ​​of all Gaussian ellipsoids in the target scene at the same spatial position are summed to obtain the 3D density field of the target scene, where the 3D density... The generation of the field relies on preset density field parameters, including a density threshold, voxel resolution, and opacity truncation threshold. The density threshold is used to determine whether a spatial point belongs to the surface of an object. In one embodiment, the density threshold is determined by a pre-selected person after statistical analysis of the opacity distribution of all Gaussian ellipsoids in the target scene. The voxel resolution is used to control the granularity of the voxel mesh in the moving cube algorithm. In one embodiment, the voxel resolution is determined by a pre-selected person based on the scale of the target scene. The opacity truncation threshold is used to filter Gaussian ellipsoids so that they do not participate in the superposition calculation of the density field, and is set in advance by a pre-selected person. The density value of any spatial point in the three-dimensional density field is the weighted superposition of the opacities of all Gaussian ellipsoids near that point. The higher the density value, the greater the probability that the spatial location is occupied by the scene. Based on the moving cube algorithm, the 3D density field is sampled using voxelization to obtain a continuous triangular mesh. The generated triangular mesh is then denoised and simplified to obtain a simplified triangular mesh with fewer vertices but preserved geometric features. The vertices of the triangular mesh are iteratively folded using a quadratic error metric algorithm to obtain optimized collision detection geometric primitives. The denoising and simplification process involves removing triangular faces and isolated vertices with areas smaller than a preset area threshold from the triangular mesh. The preset area threshold is represented by the average area of ​​triangular faces over a historical time period. The simplified triangular mesh after denoising and simplification is used as the collision detection geometric primitive of the 3D Gaussian scene reconstruction model.The collision detection geometric primitives are input into a pre-defined spatial acceleration structure construction algorithm to obtain a spatial index structure for accelerating collision queries.

[0040] In one embodiment, the preset spatial acceleration structure construction algorithm is a Bounding Volume Hierarchy (BVH). The specific construction method is as follows: the triangular facets in the triangular mesh are used as the bottom leaf nodes. A top-down recursive partitioning strategy is adopted to divide spatially adjacent triangular facets into the same subset. For each subset, a bounding box that can tightly wrap all the triangular facets in the subset is constructed. In one embodiment, the bounding box is an axis-aligned bounding box or a directional bounding box. The bounding boxes of all subsets and their inclusion relationships together form a tree index structure, which is used to accelerate collision queries and avoid traversing all triangular facets or all Gaussian ellipsoids.

[0041] Based on the spatial index structure, collision detection queries are performed on the agent's geometric model and collision detection geometric primitives. The specific process of collision detection query is as follows: First, the bounding box of the agent's geometric model in the current pose is input into the spatial index structure for querying, quickly filtering out a set of candidate triangular faces that may collide with the agent; the agent's geometric model is input into the V-HACD (Voxelized Hierarchical Approximate Convex Decomposition) algorithm, which outputs a preset number of approximate convex sub-components. Based on the QuickHull algorithm, convex hulls are constructed for each approximate convex sub-component, resulting in a set of convex hulls of the agent's geometric model. The convex hull set consists of multiple convex hulls, each of which is the smallest convex polyhedron enclosing the corresponding convex sub-component; the triangular faces in the candidate triangular face set are clustered according to spatial proximity. Specifically, the first triangular face in the candidate triangular face set is selected as the seed face, and its adjacent faces are traversed and the included normal angle is obtained. If the included normal angle is less than a preset angle threshold, they are grouped into the same group. Clustering subsets and expanding outwards using the current neighboring face as the new seed face; otherwise, skipping the current neighboring face and continuing to process the next neighboring face, repeating the above process until all triangular faces have been traversed, resulting in multiple clustered subsets. Convex hulls are constructed for the vertex set of each clustered subset based on the QuickHull algorithm, resulting in a scene convex hull set. The scene convex hull set consists of multiple convex hulls, each being the smallest convex polyhedron enclosing the corresponding clustered subset. Each convex hull in the agent's geometric model's convex hull set and each convex hull in the scene convex hull set are then paired and input into GJK (Gilbert Johnson) software. The Keerthi (Gilbert-Johnson-Kelsey) algorithm obtains the shortest Euclidean distance and nearest point pair between each pair of convex hulls. The minimum Euclidean distance among all pairs of convex hulls is taken as the overall shortest distance between them, and the corresponding nearest point pair is taken as the overall nearest point pair. Among them, the V-HACD algorithm is used to decompose the non-convex triangular mesh model into approximately convex sub-components, the QuickHull algorithm is used to calculate the minimum convex hull of a given set of points, the preset angle threshold is represented by the average of the angle between normals over a historical time period, and the preset number of components is set in advance by preset personnel; the GJK algorithm is used to calculate the shortest Euclidean distance between two convex geometries. Its core idea is to iteratively construct a geometry that approximates the origin in the Minkowski difference space. If the geometry can enclose the origin, it indicates that the two geometries overlap, i.e., a collision occurs; otherwise, the distance from the geometry to the origin is the shortest distance between the two geometries. At the same time, this algorithm can also output the pair of points closest to each other on the two geometries.If the overall shortest distance is greater than the first preset safety threshold, it indicates that there is no overlap between the two geometric objects, they are in a separated state, and no collision has occurred. The current state is determined to be safe, and collision detection continues to monitor the spatial relationship between the intelligent agent and the target scene. If the overall shortest distance is greater than the second preset safety threshold but not greater than the first preset safety threshold, the current state of the intelligent agent is determined to be a Level 1 warning state, and a deceleration prompt is sent. If the overall shortest distance is greater than 0 but not greater than the second preset safety threshold, the current state of the intelligent agent is determined to be a Level 2 warning state, and an emergency braking prompt is sent. If the overall shortest distance is equal to 0, it indicates that the two geometric objects are in contact or have penetrated each other. The current state of the intelligent agent is determined to be a collision state, and a collision alarm is sent. The first preset safety threshold is greater than the second preset safety threshold, and the second preset safety threshold is greater than 0. The first preset safety threshold is represented by the average of the overall shortest distance over a historical time period, and the second preset safety threshold is preset by a designated person.

[0042] In this embodiment, collision detection can effectively reduce missed detections and false alarms caused by inaccurate geometric representation. Furthermore, since the geometric primitives on which collision detection is based are directly derived from the density field extraction of the three-dimensional Gaussian ellipsoid set, and the three-dimensional Gaussian scene reconstruction model has eliminated the geometric deviation caused by cross-view illumination differences in the illumination coordination process, it is equivalent to improving the geometric confidence on which collision detection depends from the source. This reduces problems such as clipping, false collisions, and collisions with empty air caused by inaccurate reconstruction geometry, and achieves high-precision, low-latency real-time collision detection. This provides a reliable spatial judgment basis for the safe navigation, dynamic obstacle avoidance, and natural interaction of intelligent agents in complex virtual environments.

[0043] Depend on Figure 7 The provided structural diagram of the real-time collision detection system based on 3D Gaussian scene reconstruction shows that the system includes: an illumination difference analysis module, a 3D scene reconstruction and illumination coordination module, and a collision detection module. The illumination difference analysis module is used to perform cross-view illumination difference analysis of the target scene and its included 3D scene illumination sensitivity definition to mark the spatial location of the target scene as illumination-sensitive areas and illumination-consistent areas. By analyzing the cross-view illumination difference consistency of the target scene, it can accurately locate the areas that may cause geometric deviations due to illumination differences from the perspective of 3D scene spatial location, avoiding indiscriminate processing of all spatial locations.

[0044] The 3D scene reconstruction and lighting coordination module is used to drive lighting coordination processing of target scene images with adaptive coordination transformation for lighting-sensitive areas. After the lighting coordination processing of the target scene image is completed, it can choose to send a lighting coordination processing failure prompt or perform 3D Gaussian scene reconstruction based on the lighting coordination processing result. For areas with consistent lighting, it drives 3D Gaussian scene reconstruction based on parameter initialization and iterative optimization of the 3DGS model. Through 3D scene reconstruction and lighting coordination, differentiated processing strategies can be applied to spatial areas with different lighting conditions. While reducing cross-view gradient interference to ensure geometric accuracy, the standard reconstruction process can be directly used for areas with consistent lighting to maintain computational efficiency, thereby achieving a balance between accuracy and efficiency.

[0045] The collision detection module is used to extract continuous triangular meshes based on the set of three-dimensional Gaussian ellipsoids in the three-dimensional Gaussian scene reconstruction model after the reconstruction is completed. These meshes are then used as geometric primitives for collision detection. Collision detection is performed by converting the three-dimensional Gaussian scene model from a visual rendering representation into a collision detection geometric representation that can directly participate in the physical space overlap judgment, thereby improving the geometric accuracy and reliability of collision detection.

[0046] In this embodiment, through the illumination difference analysis module, the 3D scene reconstruction and illumination coordination module, and the collision detection module, a closed-loop processing system can be achieved, encompassing the entire chain from illumination difference identification and localization to ensuring differentiated geometric accuracy and high-precision collision detection. This systematically solves the problem of insufficient collision detection accuracy caused by cross-view illumination differences in 3D Gaussian scenes. Specifically, the illumination difference analysis module provides the 3D scene reconstruction and illumination coordination module with spatial positioning of illumination-sensitive areas and quantitative measurement of conflict levels, enabling targeted intervention at precise locations in the illumination coordination process. The high-geometric-precision 3D Gaussian scene model output by the 3D scene reconstruction and illumination coordination module provides the collision detection module with an accurate scene geometric benchmark, enabling spatial overlap calculation based on high-fidelity geometric representation. The three modules are sequentially connected and progressively enhanced, jointly ensuring the geometric accuracy and real-time performance of 3D Gaussian scene reconstruction and collision detection.

[0047] Example 2: If there are sudden changes in local illumination in multi-view images of a target scene from different perspectives, such as local highlights or local occlusion shadows, relying solely on weighted summation of global statistical features may not accurately depict the actual situation of local illumination differences. Therefore, it is necessary to use an image block-based illumination difference measurement method to obtain illumination difference correlation values. The specific acquisition process is as follows: Divide each group of target scene RGB images into a preset number of non-overlapping image blocks to obtain a first image block set {Pi1,Pi2,...,PiN} and a second image block set {Pj1,Pj2,...,PjN}, where the preset number of image blocks is set in advance by a preset person; For each image block, extract the average brightness, color temperature feature vector, and shadow area proportion of the image block; The shadow area proportion is represented by the result of the ratio calculation of the number of pixels with a value of 1 in the shadow area mask of the image block to the total number of pixels in the image block; Obtain the block-level illumination difference value between any image block in the first image block set and any image block in the second image block set, using the following formula: Where, d k (i,j) represents the block-level illumination difference between the k-th image patch in the first image patch set and the k-th image patch in the second image patch set, where k represents the index number of the image patch, k=1,2,...,N, and N is the total number of image patches. k and u_v,j k Let r_RG,i be the average brightness between the k-th image patch in the first image patch set and the second image patch set, respectively. k and r_RG,j k Let r_BG,i be the ratio of warm colors between the k-th image patch in the first image patch set and the second image patch set, respectively. k and r_BG,j k The ratios of cool colors in the k-th image patch in the first image patch set and the second image patch set are s_i and s_i, respectively. k and s_j k The proportion of the shadow area of ​​the k-th image patch in the first image patch set and the second image patch set, respectively, α 1 β 1 and γ 1 Preset block-level weight coefficients; Using block-level illumination difference as the benchmark, the illumination difference correlation value between two corresponding target scene RGB images is obtained. The specific formula is as follows: in, w is the correlation value for illumination difference. k is the weight coefficient for the k-th image patch.

[0048] In this embodiment, by obtaining the illumination difference correlation value, spatial non-uniform illumination changes that are difficult to reflect by global statistical features, such as local highlights and local occlusion shadows, can be accurately captured. Based on this, by calculating block-level illumination difference values ​​and combining them with weight coefficients determined based on gradient magnitude for weighted aggregation, the texture-rich regions will play a dominant role in the correlation value, avoiding interference from sparse or flat regions on the measurement results. This significantly improves the characterization accuracy of illumination difference correlation values ​​in local illumination abrupt change scenarios, and ultimately ensures the geometric consistency of 3D Gaussian scene reconstruction.

[0049] The terms "first," "second," etc., in the specification, claims, and drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the term "comprising," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or device that includes a series of steps or units is not limited to the listed steps or modules, but may optionally include steps or modules not listed, or may optionally include other step units inherent to these processes, methods, apparatuses, products, or devices.

[0050] Example 3: To verify the effectiveness of each module of this application, an ablation experiment was designed for comparative evaluation. The experiment set up three control schemes: Scheme A is the complete scheme of this invention; Scheme B is control scheme 1, which removes the illumination difference analysis module based on scheme A, that is, it does not define the three-dimensional scene illumination sensitivity of the spatial location of the target scene, and uniformly performs three-dimensional Gaussian scene reconstruction based on the parameter initialization and iterative optimization of the 3DGS model for all spatial locations; Scheme C is control scheme 2, which retains the illumination difference analysis module based on scheme A but does not perform target scene image illumination coordination processing, that is, after identifying the illumination sensitive area, it directly skips the illumination coordination processing step and directly performs three-dimensional Gaussian scene reconstruction on the area.

[0051] The experiment used a standard 3D scene dataset containing multi-view images. Three schemes were run to obtain corresponding 3D Gaussian scene reconstruction models, and collision detection was performed based on each model. The evaluation metrics included reconstruction geometric accuracy and collision detection accuracy. Reconstruction geometric accuracy was measured by the average offset between the center position of the 3D Gaussian ellipsoid and the real surface position, in centimeters. Collision detection accuracy was measured by precision and recall. The data for each metric and their acquisition methods are shown in Table 1. Table 1 Comparison of Quantitative Results of Ablation Experiments The experimental data above shows that Scheme A outperforms Scheme B and Scheme C in terms of geometric offset, collision detection accuracy, and collision detection recall. Scheme B, lacking an illumination difference resolution module, cannot identify illumination-sensitive areas, resulting in the most severe decrease in global geometric accuracy. Although Scheme C can identify illumination-sensitive areas, it does not perform illumination coordination processing, and its geometric accuracy is improved compared to Scheme B but is still significantly lower than that of Scheme A.

[0052] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. A real-time collision detection method based on 3D Gaussian scene reconstruction, characterized in that, include: By performing cross-view illumination difference analysis of the target scene and defining the illumination sensitivity of the three-dimensional scene, the spatial location of the target scene is marked as illumination-sensitive area and illumination-consistent area. For illumination coordination processing of target scene images with adaptive coordination transformation driven by illumination-sensitive areas, after the illumination coordination processing of target scene images is completed, the system chooses to send an illumination coordination processing failure prompt or perform 3D Gaussian scene reconstruction based on the illumination coordination processing result. 3D Gaussian scene reconstruction driven by illumination-consistent regions and based on 3DGS model parameter initialization and iterative optimization; After the 3D Gaussian scene reconstruction is completed, based on the 3D Gaussian ellipsoid set in the 3D Gaussian scene reconstruction model, continuous triangular meshes are extracted through density field superposition and moving cube algorithm as collision detection geometric primitives, and collision detection is performed.

2. The real-time collision detection method based on 3D Gaussian scene reconstruction according to claim 1, characterized in that, The specific process for analyzing the cross-view illumination differences in the target scene is as follows: Acquire a set of multi-view images of a target scene, which includes RGB images of the same target scene taken from different spatial positions and directions, and the camera poses corresponding to the multi-view images of the target scene; The SfM algorithm is used to obtain the spatial position and orientation of the camera in the spatial coordinate system when the camera captures multi-view images of the target scene. Based on the set of multi-view images of the target scene and the corresponding camera poses, an illumination difference correlation diagram of the multi-view images of the target scene is derived. The specific derivation process is as follows: For multi-view images of the target scene, extract the corresponding multi-view illumination feature parameters, including the average brightness, color temperature feature vector, and shadow area mask. The specific process for obtaining the color temperature feature vector is as follows: Obtain the mean values ​​of the R channel, G channel, and B channel of the RGB image of the target scene, calculate the ratio of the mean values ​​of the R channel to the G channel, and label it as the warm color image ratio; Calculate the ratio of the mean values ​​of the B channel to the G channel, and label it as the ratio of cool colors in the image; The ratios of warm and cool colors in the image are combined to form a color temperature feature vector; The target scene RGB image is input into a pre-trained shadow detection neural network to obtain a shadow probability map; The shadow probability map is subjected to threshold binarization, that is, pixels with shadow probabilities greater than a preset probability threshold are marked as 1, and vice versa, the corresponding pixels are marked as 0, thus obtaining the shadow region mask. Based on the multi-view illumination feature parameters, calculate the illumination difference correlation value between any two target scene RGB images, as a quantitative measure of the degree of comprehensive difference in illumination conditions between the two; The illumination difference correlation value is obtained by weighted summation of illumination intensity difference value, color temperature difference value, and shadow distribution difference value; The light intensity difference value is obtained by calculating the absolute value of the difference between the mean brightness values ​​of the two corresponding target scene RGB images. The Euclidean distance algorithm is used to perform Euclidean distance calculation on the color temperature feature vectors of two target scene RGB images to obtain the color temperature difference value; The shadow distribution difference value is obtained by performing a ratio operation on the symmetric difference and union size of the two shadow region masks of the corresponding two target scene RGB images; Traverse all combinations of RGB images of the target scene to obtain the multi-view image set of the target scene, with each RGB image of the target scene as a node and the illumination difference correlation value as the edge weight; The illumination difference correlation diagram is used to characterize the degree of similarity or difference between any two viewpoints in terms of illumination conditions; Based on the illumination difference correlation graph, the three-dimensional scene illumination sensitivity is defined for each spatial region in the target scene.

3. The real-time collision detection method based on 3D Gaussian scene reconstruction according to claim 2, characterized in that, The specific process for defining the lighting sensitivity of the three-dimensional scene is as follows: For any initial spatial location in the target scene, determine the set of viewpoints in the multi-view image of the target scene that can observe that initial spatial location. The specific determination process is as follows: Input the multi-view images of the target scene into the SfM algorithm to obtain sparse 3D point clouds; Input the sparse 3D point cloud into a preset 3DGS model, and take each 3D spatial point in the sparse 3D point cloud as the initial center position of the 3D Gaussian ellipsoid to obtain the initial spatial position set. Based on the camera poses corresponding to the multi-view images of each target scene, the initial spatial position is projected onto the RGB image plane of the target scene from each view through the camera projection matrix to obtain the corresponding spatial position pixel coordinates. If the spatial location pixel coordinates fall within the boundary range of the target scene RGB image from the current viewpoint, the index numbers of all target scene RGB images that meet the conditions are included in the viewpoint set. The camera projection matrix is ​​composed of a combination of a camera intrinsic parameter matrix and a camera extrinsic parameter matrix. The camera intrinsic parameter matrix contains internal parameters, and the camera extrinsic parameter matrix contains external parameters. The occlusion visibility determination is based on a set of viewpoints, and the specific process is as follows: Obtain the pixel coordinates of the initial spatial position in each target scene RGB image, and the camera optical center in the camera pose corresponding to each target scene RGB image; Based on the camera pose, depth maps for each viewpoint are generated using a multi-view stereo matching algorithm. For each view in the view set, based on the pixel coordinates of the initial spatial position under the current view, the depth value of the corresponding pixel position in the view depth map is obtained as the depth value corresponding to the initial spatial position under the current view. The depth reference value is compared with the depth value of the corresponding pixel position in the depth map; The depth reference value represents the depth value that its projection point should present in the current viewpoint depth map under ideal conditions where the initial spatial position is not obstructed; If the depth value is less than the depth reference value, the target scene RGB image from the current viewpoint will be removed from the viewpoint set; If the depth value is not less than the depth reference value, the current viewpoint is retained; After the above occlusion visibility determination, the effective viewpoint set of the initial spatial position is obtained; Based on the local illumination feature parameters, calculate the local illumination difference correlation value between local image patches corresponding to any two target scene RGB images in the effective viewpoint set; Obtain local image patches corresponding to the RGB images of each target scene in the effective viewpoint set, extract the local illumination feature parameters of each local image patch, and calculate the local illumination difference value between any two local image patches; The local illumination feature parameters include the local brightness mean, the local color temperature feature vector, and the local shadow region mask; If any two views in the effective view set have a local illumination difference value between their corresponding local image blocks that is greater than a preset local illumination difference threshold, the current spatial location is determined to be a light-sensitive area, and target scene image illumination coordination processing is performed. If the local illumination difference between any two local image blocks corresponding to any two viewpoints in the effective viewpoint set is not greater than the preset local illumination difference threshold, the current spatial location is determined to belong to the illumination consistency region, and three-dimensional Gaussian scene reconstruction is performed directly.

4. The real-time collision detection method based on 3D Gaussian scene reconstruction according to claim 3, characterized in that, The specific process of the target scene image illumination coordination processing is as follows: In the target scene RGB image corresponding to the multi-view image set of the target scene, a local image patch is obtained with the spatial position pixel coordinates corresponding to the light-sensitive area as the center and a preset radius as the range. The adaptive separation of illumination-dependent and geometric-dependent components for local image patches is performed as follows: An edge-preserving filter is applied to the local image patch to obtain a smoothed version of the local image patch; The edge-preserving filtering operation uses guided filtering or bilateral filtering as the filtering operator; The original local image patch is subtracted pixel by pixel from the smoothed version of the local image patch to obtain the detailed version of the local image patch; For each pixel location, its corresponding RGB three-dimensional vector constitutes the specific numerical representation of the illumination-related components at that pixel location. The RGB three-dimensional vectors of all pixel locations in the smoothed version of the local image patch together constitute the illumination-related component representation sequence of the local image patch under the preset viewpoint. The geometric correlation component is represented in the form of image data of the local image patch of the detailed version. The geometric correlation component is a two-dimensional pixel matrix, in which each pixel position corresponds to an RGB three-dimensional vector. Based on the obtained illumination-related components and geometric-related components, an adaptive and coordinated transformation is performed on the illumination-sensitive region.

5. The real-time collision detection method based on 3D Gaussian scene reconstruction according to claim 4, characterized in that, The specific process of the adaptive coordination transformation is as follows: Based on the illumination difference correlation map, the target scene RGB image is selected as the illumination reference view image from all viewpoints of the target scene RGB images in the illumination-sensitive area; Based on the least squares method, with the reference illumination-related component as the target, the pixel distribution of the illumination-related component of the image to be coordinated is fitted with a second-order polynomial to obtain the brightness mapping function. By substituting each pixel value in the illumination-related component of the image to be coordinated into the brightness mapping function, the illumination-related component after adaptive coordination transformation is obtained. Pixel-level synthesis is performed on the illumination-related components after adaptive coordination transformation and the corresponding geometric-related components to obtain locally corrected image blocks.

6. The real-time collision detection method based on 3D Gaussian scene reconstruction according to claim 5, characterized in that, The pixel-level synthesis means adding the illumination-related components after adaptive coordination transformation to the corresponding geometric-related components pixel by pixel. For each pixel position, the RGB values ​​of the corresponding pixel positions of the smoothed version local image block after adaptive coordination transformation and the detailed version local image block are added to obtain the corrected RGB value of the pixel position. Reacquire the local illumination difference correlation value between the local image patch and the local image patch of the illumination reference view image. If the local illumination difference correlation value is not greater than the preset local illumination difference threshold, replace the corresponding pixel region in the original target scene multi-view image with the local image patch to obtain the target scene multi-view image after illumination coordination processing, and perform three-dimensional Gaussian scene reconstruction. If the local illumination difference correlation value is still greater than the preset local illumination difference threshold, continue to perform target scene image illumination coordination processing. When the number of processing times reaches the preset maximum number of iterations, if the local illumination difference correlation value is still greater than the preset local illumination difference threshold, then send an illumination coordination processing failure prompt. The three-dimensional Gaussian scene reconstruction means inputting multi-view images of the target scene and their corresponding camera poses into a preset 3DGS model to reconstruct the scene and obtain a three-dimensional Gaussian scene reconstruction model. Collision detection based on a 3D Gaussian scene reconstruction model.

7. The real-time collision detection method based on 3D Gaussian scene reconstruction according to claim 6, characterized in that, The specific process of collision detection is as follows: Obtain a 3D Gaussian scene reconstruction model and an intelligent agent geometric model; The intelligent agent geometric model is represented by a triangular mesh model, that is, the intelligent agent geometric model consists of a set of vertex coordinates in three-dimensional space and a set of triangular patch indices formed by the vertices; The intelligent agent's geometric model is mapped to the world coordinate system of the 3D Gaussian scene reconstruction model, and the spatial position and orientation of the intelligent agent's geometric model are updated in real time according to the current pose of the intelligent agent. The specific mapping and updating process is as follows: Obtain the current pose of the agent in the world coordinate system of the 3D Gaussian scene reconstruction model, including the position vector and orientation rotation matrix. Transform each vertex in the agent's geometric model to the world coordinate system through rigid body transformation. When the agent moves or rotates, its pose is updated in real time. Re-execute the above rigid body transformation on all vertices in the agent's geometric model. The three-dimensional density field of the target scene is obtained by superimposing each Gaussian ellipsoid in the set of three-dimensional Gaussian ellipsoids in the three-dimensional Gaussian scene reconstruction model through a Gaussian kernel function. The three-dimensional density field is sampled using voxelization to obtain a continuous triangular mesh; The generated triangular mesh is denoised and simplified to obtain a simplified triangular mesh. The vertices of the triangular mesh are iteratively folded based on a quadratic error metric algorithm to obtain the optimized collision detection geometric primitives. The denoising and simplification process refers to removing triangular faces and isolated vertices with areas smaller than a preset area threshold from the triangular mesh. The simplified triangular mesh after denoising and simplification is used as the collision detection geometric primitive of the three-dimensional Gaussian scene reconstruction model; The collision detection geometric primitives are input into a preset spatial acceleration structure construction algorithm to obtain a spatial index structure for accelerating collision queries. Based on the spatial index structure, a collision detection query is performed on the intelligent agent geometric model and the collision detection geometric primitive.

8. The real-time collision detection method based on 3D Gaussian scene reconstruction according to claim 7, characterized in that, The specific process of the collision detection query is as follows: The bounding box of the intelligent agent's geometric model in the current pose is input into the spatial index structure for querying, and a set of candidate triangular faces that may collide with the intelligent agent is filtered out. Based on the intelligent agent geometric model, the convex hull set of the intelligent agent geometric model is obtained; Using the candidate triangular facet set as a reference, the scene convex hull set is obtained; Each convex hull in the set of convex hulls of the intelligent agent's geometric model and each convex hull in the set of convex hulls of the scene are input into the GJK algorithm one by one to obtain the shortest Euclidean distance and the nearest point pair between each pair of convex hulls. The minimum value of the distance among all pairs of convex hulls is taken as the overall shortest distance between the two, and the corresponding nearest point pair is taken as the overall nearest point pair. If the overall shortest distance is greater than the first preset safety threshold, the current state is determined to be a safe state, and the collision detection query continues to be executed. If the overall shortest distance is greater than the second preset safety threshold but not greater than the first preset safety threshold, the current state of the agent is determined to be a level one warning state, and a deceleration prompt is sent. If the overall shortest distance is greater than 0 and not greater than the second preset safety threshold, the current state of the agent is determined to be a level 2 warning state, and an emergency braking prompt is sent. If the overall shortest distance is equal to 0, the agent is determined to be in a collision state and a collision alarm is sent.

9. The real-time collision detection method based on 3D Gaussian scene reconstruction according to claim 2, characterized in that, The acquisition of the illumination difference correlation value also includes: Each set of target scene RGB images is divided into a preset number of non-overlapping image blocks to obtain a first image block set and a second image block set. For each image patch, extract the average brightness, color temperature feature vector, and shadow area proportion of that image patch; The proportion of the shadow area is represented by the result of a ratio calculation between the number of pixels with a value of 1 in the shadow area mask of the image block and the total number of pixels in the image block. Obtain the block-level illumination difference value between any image block in the first image block set and any image block in the second image block set; Using block-level illumination difference as the benchmark, obtain the illumination difference correlation value between the corresponding two target scene RGB images.

10. A real-time collision detection system based on 3D Gaussian scene reconstruction, characterized in that, include: The modules include: lighting difference analysis module, 3D scene reconstruction and lighting coordination module, and collision detection module. The illumination difference analysis module is used to define the spatial location of the target scene as an illumination-sensitive region and an illumination-consistent region by performing cross-view illumination difference analysis of the target scene and the three-dimensional scene illumination sensitivity definition it includes. The 3D scene reconstruction and illumination coordination module is used to drive illumination coordination processing of target scene images with adaptive coordination transformation for illumination-sensitive areas. After the illumination coordination processing of the target scene image is completed, it selects to send an illumination coordination processing failure prompt or perform 3D Gaussian scene reconstruction based on the illumination coordination processing result. It also drives 3D Gaussian scene reconstruction based on parameter initialization and iterative optimization of the 3DGS model for illumination-consistent areas. The collision detection module is used to perform collision detection after the 3D Gaussian scene reconstruction is completed, based on the 3D Gaussian ellipsoid set in the 3D Gaussian scene reconstruction model, extract continuous triangular meshes through density field superposition and moving cube algorithm, and use them as collision detection geometric primitives.

Images