A method and system for full lifecycle traceability of 3DGS vehicle data

Abstract

This invention discloses a method and system for full-cycle traceability of 3DGS automotive data, belonging to the field of 3D reconstruction technology. The method reconstructs 3DGS models of the same vehicle at each maintenance stage, dividing the Gaussian volume into a rigid structure subset and a movable part subset, performing two-level registration to achieve accurate alignment of multi-stage models; constructing a voxelized Gaussian field in a unified coordinate system, transforming unstructured Gaussian representations into structured voxel representations, calculating a comprehensive difference index for each voxel, and outputting the difference results after adaptive threshold filtering based on reconstruction confidence and semantic region type, generating a full-cycle traceability digital archive. This invention solves the problems of difficult cross-stage registration of multi-stage 3DGS models and the lack of effective measurement methods for detecting differences in unstructured Gaussian representations, providing reliable technical support for full-cycle traceability of automotive maintenance.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional reconstruction technology, specifically to a method and system for full-lifecycle tracing of 3DGS automotive data. Background Technology

[0002] The automotive repair process involves multiple stages, including vehicle reception, disassembly and inspection, repair, and handover. Accurate recording and comparison of the vehicle's condition before and after repair are crucial for ensuring repair quality and resolving disputes. Traditional repair records primarily rely on two-dimensional photographs and textual descriptions, which struggle to fully reflect the geometric shape and appearance of the vehicle's three-dimensional surface. This results in fragmented information, a lack of unified standards, and difficulties in post-repair traceability.

[0003] In recent years, the emergence of 3D Gaussian Splatting (3DGS) technology has provided a new solution to the aforementioned problems. 3DGS reconstructs images from multiple perspectives, explicitly expressing the geometric and appearance information of a scene as a set of 3D Gaussian volumes. It boasts advantages in both high reconstruction quality and fast rendering speed, making it suitable as a carrier for 3D digital records of vehicles. However, when applying 3DGS to the entire lifecycle of automotive repair traceability, the 3DGS models at each stage are reconstructed independently under different time and workstation conditions, each existing in its own independent local coordinate system. Furthermore, vehicles contain movable parts such as doors and hoods, whose opening and closing states may differ at different stages. The relationships between models at different stages are not simple global rigid body transformations, resulting in a lack of a unified spatial benchmark between multi-stage models, making direct cross-stage comparisons difficult. Even after model alignment is completed, 3DGS models represent scenes using unstructured Gaussian distribution sets. Models at different stages differ in the number of Gaussian volumes and parameter distributions, lacking a natural element-by-element correspondence. Existing difference measurement methods for point clouds or meshes cannot be directly applied. In addition, differences in acquired lighting and fluctuations in reconstruction accuracy introduce noise unrelated to real physical changes. This makes it difficult to establish reliable difference measurement on unstructured Gaussian representations and effectively distinguish between real changes and reconstruction noise, thus affecting the implementation of the 3DGS full-lifecycle traceability solution. Summary of the Invention

[0004] To address the shortcomings of existing technologies, the present invention aims to provide a method and system for full-cycle tracing of 3DGS automotive data. This method achieves precise alignment of multi-stage models by semantically segmenting Gaussian volumes and performing two-level registration based on rigid structure subsets and movable component subsets. Furthermore, it constructs a voxelized Gaussian field to transform unstructured Gaussian representations into structured voxel representations. Finally, it achieves reliable difference detection and full-cycle digital archive generation by combining reconstruction confidence with an adaptive threshold filtering mechanism based on semantic region type.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] A full-lifecycle traceability method for 3DGS automotive data, applied to the automotive repair process, wherein the automotive repair process is divided into at least two stages according to time sequence, the method comprising:

[0007] Multi-view images of the same vehicle at different stages were collected and reconstructed to obtain 3DGS models for each stage;

[0008] The Gaussian bodies in the 3DGS models at each stage are divided into a rigid structure subset and a movable component subset;

[0009] Using the 3DGS model of the first stage as the reference model, and the 3DGS models of the remaining stages as the models to be registered, a two-level registration is performed on each model to be registered. The first-level registration is based on solving the global rigid body transformation based on the rigid structure subsets of the reference model and the models to be registered. The second-level registration is based on solving the local rigid body transformation increments based on the subsets of movable parts of the reference model and the models to be registered, so that each model to be registered is aligned to the coordinate system of the reference model.

[0010] A three-dimensional voxel mesh is established in the coordinate system of the baseline model. The Gaussian volumes in the 3DGS model at each stage are assigned to the corresponding voxels and the features of each voxel are aggregated and calculated to form the voxelized Gaussian field corresponding to each stage.

[0011] For any two stages, calculate the comprehensive difference index for each voxel of the voxelized Gaussian field, output the difference result after adaptive threshold filtering, and generate a full-cycle traceability digital archive.

[0012] Furthermore, the step of acquiring multi-view images of the same vehicle at different stages and reconstructing 3DGS models for each stage includes:

[0013] Sparse reconstruction is performed on multi-view images acquired at each stage to obtain sparse 3D point clouds and camera parameters corresponding to each image.

[0014] The three-dimensional coordinates of each point in the sparse three-dimensional point cloud are used as the initial position mean of the three-dimensional Gaussian body, and the covariance matrix, opacity and spherical harmonic coefficient of each Gaussian body are initialized to obtain the initial 3DGS model.

[0015] The initial 3DGS model is rendered into two-dimensional images from each training perspective using differentiable rendering. The loss between the rendered image and the corresponding acquired image is calculated, and the gradient is backpropagated to update the parameters of each Gaussian body.

[0016] Adaptive density control is performed on the Gaussian volume during the optimization process to obtain 3DGS models at each stage.

[0017] Furthermore, the process of dividing the Gaussian bodies in the 3DGS model at each stage into a rigid structure subset and a movable component subset includes:

[0018] For the multi-view images used for reconstruction at each stage, semantic labels of vehicle components are predicted pixel by pixel using a semantic segmentation model;

[0019] The mean position of each Gaussian body in the 3DGS model at each stage is projected onto each semantically segmented image through the corresponding camera parameters to obtain the semantic label at each projection position. The semantic label of each Gaussian body is determined by multi-view voting.

[0020] Based on the semantic tags, Gaussian bodies are categorized into either the rigid structure subset or the movable component subset. The rigid structure subset includes Gaussian bodies corresponding to load-bearing structural components whose geometry remains unchanged throughout each maintenance stage, while the movable component subset includes Gaussian bodies corresponding to components with independent degrees of freedom of motion.

[0021] Furthermore, the first-level registration solves the global rigid body transformation based on the rigid structure subsets of the reference model and the model to be registered, including:

[0022] Voxel downsampling is performed on Gaussian bodies within the rigid structure subsets of the baseline model and the model to be registered, based on their spatial positions. The average position of the Gaussian body with the highest opacity within each voxel is selected as the feature point, forming the feature point set of the baseline model and the feature point set of the model to be registered.

[0023] Calculate local geometric feature descriptors for each feature point in the two feature point sets, establish the correspondence between the two feature point sets through descriptor matching, and remove outliers;

[0024] Based on the correspondence after removing outliers, the opacity of the Gaussian body to which the corresponding feature point belongs is used as the weight to solve the weighted optimal global rigid body transformation. The coordinate transformation of all Gaussian bodies of the model to be registered is performed according to the global rigid body transformation to generate a coarse registration model.

[0025] Furthermore, the second-level registration solves for the local rigid body transformation increments based on subsets of movable components of the reference model and the model to be registered, including:

[0026] The mean position values ​​of Gaussian bodies within each subset of movable parts in the coarse registration model and the baseline model are extracted as point sets respectively;

[0027] For each movable part, the iterative nearest point algorithm with opacity as the weight is used to iteratively solve the local rigid body transformation increment for the corresponding point set of the two models.

[0028] Based on the local rigid body transformation increments, the Gaussian bodies of the corresponding movable component subsets in the coarse registration model are subjected to coordinate transformations to generate a fine registration model aligned to the reference model coordinate system.

[0029] Furthermore, after generating the fine registration model, the process also includes:

[0030] Extract the mean position values ​​of Gaussian bodies within the rigid structure subsets of the fine registration model and the baseline model respectively, search for the nearest neighbor mean position value in the baseline model for each mean position value in the fine registration model, and establish a point-to-point correspondence between the rigid structure subsets;

[0031] The root mean square value of the distance between all point pairs is calculated as the registration residual. When the registration residual exceeds the registration anomaly threshold, the registration result of the model to be registered is determined to be abnormal and an alarm is triggered.

[0032] Furthermore, the step of assigning Gaussian volumes in the 3DGS model at each stage to corresponding voxels and aggregating and calculating the features of each voxel to form a voxelized Gaussian field corresponding to each stage includes:

[0033] Calculate the joint bounding box of the average position of all Gaussian bodies in the 3DGS model at each stage, and establish a three-dimensional voxel mesh by uniformly dividing along the three coordinate axes with a fixed step size. Then, assign each Gaussian body to the voxel according to its average position.

[0034] The geometric and appearance features of Gaussian volumes within each voxel are aggregated and calculated as the attributes of that voxel. The geometric features include Gaussian volume density, weighted centroid with opacity as the weight, average opacity, and geometric dispersion. The appearance features are the weighted mean of low-order spherical harmonic coefficients with opacity as the weight.

[0035] The reconstruction confidence score of each voxel is calculated based on the Gaussian volume density, average opacity, and geometric dispersion of each voxel as the attribute of that voxel. The higher the Gaussian volume density, the higher the average opacity, and the lower the geometric dispersion, the higher the reconstruction confidence.

[0036] The three-dimensional voxel mesh and the geometric features, appearance features and reconstruction confidence scores of each voxel together constitute the voxelized Gaussian field corresponding to each stage.

[0037] Furthermore, the step of calculating the comprehensive difference index on a voxel-by-voxel basis for any two stages of the voxelized Gaussian field, filtering it with an adaptive threshold, and then outputting the difference result to generate a full-cycle traceability digital archive includes:

[0038] For any two stages, in the voxelized Gaussian field corresponding to each voxel, which is not empty in both stages, calculate the geometric difference index and the appearance difference index on a voxel-by-voxel basis. The geometric difference index is calculated based on the difference between the weighted centroids and the average opacity of the voxels corresponding to the two stages, while the appearance difference index is calculated based on the Euclidean distance between the weighted mean of the low-order spherical harmonic coefficients of the voxels corresponding to the two stages.

[0039] The geometric and appearance difference indices of each voxel are weighted and summed to obtain a comprehensive difference index. After adaptive threshold filtering, voxels with comprehensive difference indices exceeding the threshold are retained as difference voxels. The difference information of each difference voxel is quantitatively output to generate a full-cycle traceability digital archive.

[0040] Furthermore, the adaptive threshold filtering includes:

[0041] For each voxel, the lowest value of the reconstruction confidence score in the two stages is taken as the effective confidence score of that voxel. The difference significance threshold of that voxel is determined based on the effective confidence score. The difference significance threshold is inversely correlated with the effective confidence score.

[0042] The semantic region type of each voxel is determined based on the semantic label of the Gaussian body within each voxel, and the significance threshold of the difference is then corrected.

[0043] A full-lifecycle traceability system for 3DGS vehicle data, used to implement the aforementioned full-lifecycle traceability method for 3DGS vehicle data, the system comprising:

[0044] The multi-stage acquisition and reconstruction module is used to acquire multi-view images of the same vehicle at different stages and reconstruct 3DGS models for each stage.

[0045] The semantic partitioning module is used to divide the Gaussian volume in the 3DGS model at each stage into a subset of rigid structures and a subset of movable parts;

[0046] The two-level registration module is used to perform first-level registration and second-level registration on the 3DGS models of the remaining stages, using the first-stage 3DGS model as the reference model, so that each model to be registered is aligned to the reference model coordinate system.

[0047] The voxelization module is used to create a three-dimensional voxel mesh in the reference model coordinate system, assign the Gaussian volumes of each stage to the corresponding voxels, and aggregate and calculate the features of each voxel to form the voxelized Gaussian field corresponding to each stage.

[0048] The difference detection and archiving module is used to calculate the comprehensive difference index for each voxel of the voxelized Gaussian field corresponding to any two stages, and output the difference results after adaptive threshold filtering to generate a full-cycle traceability digital archive.

[0049] Compared with the prior art, the beneficial effects of the present invention are:

[0050] 1. This invention divides the Gaussian bodies in the 3DGS model at each stage into a rigid structure subset and a movable part subset. First, it solves the global rigid body transformation based on the rigid structure subset, and then solves the local rigid body transformation increment for each movable part subset. This enables vehicle models at different maintenance stages to be accurately aligned in a unified coordinate system, avoiding misjudgment of differences in the opening and closing states of components such as doors and hoods as vehicle damage or maintenance changes, and improving the accuracy and stability of cross-stage 3D data comparison.

[0051] 2. This invention constructs a voxelized Gaussian field in the reference model coordinate system, transforming the unstructured 3DGS model with inconsistent Gaussian volume and distribution into a structured representation with a unified spatial index. It also combines reconstruction confidence and semantic region type for adaptive threshold filtering, which can suppress false alarms caused by occlusion, reflection, weak texture and reconstruction fluctuation while preserving the detection capabilities of geometric and appearance differences, thereby generating reliable full-cycle traceability digital archives. Attached Figure Description

[0052] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0053] Figure 1 This is a schematic diagram illustrating an application scenario of the full-lifecycle traceability system for 3DGS vehicle data provided in an embodiment of the present invention.

[0054] Figure 2 A flowchart illustrating the full-lifecycle traceability method for 3DGS vehicle data provided in this embodiment of the invention;

[0055] Figure 3 This is a schematic diagram illustrating the process of performing two-level registration on each model to be registered, provided in an embodiment of the present invention.

[0056] Figure 4 A schematic diagram illustrating the construction of a voxelized Gaussian field provided in an embodiment of the present invention;

[0057] Figure 5 This is a schematic diagram of the structure of a full-lifecycle traceability system for 3DGS vehicle data provided in an embodiment of the present invention.

[0058] Reference numerals: 100, vehicle; 101, maintenance bay; 102, multi-view image acquisition device; 103, acquisition camera. Detailed Implementation

[0059] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0060] Example 1

[0061] Please see Figure 1 This application provides a method and system for full-cycle traceability of 3DGS automotive data, applicable to scenarios such as vehicle repair, inspection, delivery verification, and repair dispute review, for digital record-keeping of vehicle status. A vehicle 100 is located in a repair bay 101. Multi-view image acquisition devices 102 are arranged around the vehicle 100, and multiple acquisition cameras 103 acquire images from the front, rear, sides, and diagonally above the vehicle, obtaining multi-view image data covering the vehicle's exterior and key component areas. Exemplarily, the repair stage includes the vehicle reception stage, disassembly and inspection stage, repair stage, and delivery stage. Data from each stage is transmitted to a processing terminal, which performs 3DGS model reconstruction, semantic annotation, cross-stage registration, voxelization processing, and difference detection. The digital archive server stores the models, registration parameters, difference results, and acquisition metadata for each stage, forming a full-cycle traceability digital archive.

[0062] In the aforementioned scenarios, the condition of the same vehicle at different repair stages may undergo localized geometric changes and variations in appearance and texture due to disassembly, bodywork repair, painting, or component replacement. Simultaneously, movable components such as doors, hoods, and trunk lids may be in different opening angles or assembly positions at different stages, meaning the vehicle's three-dimensional data at each stage does not satisfy a single rigid body transformation relationship. While traditional two-dimensional photographs, videos, and text records can capture some appearance information, they are insufficient to accurately represent surface dents, repair marks, paint color differences, and component positional changes. Furthermore, it is difficult to establish a unified, quantifiable, and verifiable spatial comparison benchmark between different repair stages.

[0063] 3DGS technology can reconstruct three-dimensional vehicle models based on multi-view images and represent the geometric distribution and appearance information of the vehicle surface as a set of three-dimensional Gaussian volumes. Each Gaussian volume includes at least the position mean, covariance matrix, opacity, and spherical harmonic coefficients. Since 3DGS models at different stages are usually reconstructed independently at different times and under different acquisition conditions, the models are in their own local coordinate systems, and the number and spatial distribution of Gaussian volumes are not consistent at different stages. If the position or appearance parameters of Gaussian volumes are directly compared, the results will be mixed with differences in acquisition pose, coordinate system differences, and reconstruction fluctuations, making it difficult to stably reflect the actual maintenance changes.

[0064] To address the aforementioned issues, this embodiment does not simply subtract the vehicle's 3DGS models from different repair stages after global registration. Instead, based on the vehicle's structural characteristics, the Gaussian bodies in the 3DGS models at each stage are divided into a rigid structure subset and a movable component subset according to semantic attributes. The rigid structure subset may include Gaussian bodies corresponding to components with relatively stable geometric states during the repair stage, such as the roof, pillars, sill beams, and windshield frames. The movable component subset may include Gaussian bodies corresponding to components with independent degrees of freedom of motion, such as doors, hoods, and trunk lids.

[0065] Specifically, the 3DGS model of the first stage is used as the reference model, and the 3DGS models of the remaining stages are used as models to be registered. First, the global rigid body transformation is solved based on the rigid structure subsets of the reference model and the models to be registered to establish a unified spatial reference between the vehicle models of different stages. Then, the local rigid body transformation increment is solved for each subset of movable parts to compensate for differences in the opening, closing, or assembly poses of the parts. Through this two-level registration method, the 3DGS models of each stage can be aligned to the coordinate system of the reference model and the interference caused by normal posture changes of movable parts on cross-stage comparisons can be reduced.

[0066] Furthermore, considering that the number, scale, opacity, and spherical harmonic coefficient distribution of Gaussian volumes in different stages of the 3DGS model may change during the reconstruction process, this embodiment establishes a three-dimensional voxel mesh in the reference model coordinate system after registration. Gaussian volumes at each stage are then allocated to the corresponding voxels according to their positional averages, and the Gaussian volumes within each voxel are aggregated to form a voxelized Gaussian field corresponding to each stage. Voxel features include Gaussian volume density, weighted centroids with opacity as the weight, average opacity, geometric dispersion, and the weighted average of low-order spherical harmonic coefficients. Among these, Gaussian volume density, average opacity, and geometric dispersion are also used to calculate the reconstruction confidence score of the voxel.

[0067] Based on a voxelized Gaussian field, this embodiment uses voxels at a uniform spatial location as comparison units, without requiring a one-to-one correspondence between individual Gaussian voxels at different stages. For any two stages, geometric difference indices and appearance difference indices can be calculated at the same voxel location and weighted to obtain a comprehensive difference index. Subsequently, a significance threshold for the difference is determined based on the lower reconstruction confidence level between the two stages, and the threshold is adjusted in conjunction with the semantic region type to which the voxel belongs. For example, the threshold is lowered for flat panel areas to improve the detection sensitivity of minor dents, scratches, or color differences, while the threshold is increased for geometrically complex areas such as grilles, headlight edges, and wheel arches to reduce false alarms caused by reconstruction noise.

[0068] After adaptive threshold filtering, the retained differential voxels can be further merged into effective differential regions, and the differential location, associated component, geometric offset, appearance change, comparison stage, and visualization results are output. These are stored together with the 3DGS models, acquired metadata, and registration parameters of each stage to form a full-cycle traceability digital archive. Thus, this embodiment can quantitatively verify and archive the three-dimensional state changes of the same vehicle during the stages of vehicle receipt, disassembly and inspection, repair, and delivery.

[0069] It should be noted that the vehicle mentioned is not limited to passenger cars, but may also be a commercial vehicle, a special vehicle, or other vehicle with a rigid body structure and movable external body panels; the repair stage is not limited to the above example, and may be extended to stages such as accident investigation, before disassembly, after sheet metal work, after painting, after final assembly, and delivery review, depending on the actual process.

[0070] Please see Figure 2 The present invention provides an embodiment of a full-lifecycle traceability method for 3DGS automotive data, applied to an automotive repair process, wherein the automotive repair process is divided into at least two stages according to time sequence, and the specific steps of the method are as follows:

[0071] S1: Collect multi-view images of the same vehicle at each stage and reconstruct the 3DGS model for each stage.

[0072] In this embodiment, the stages may include at least two of the following: vehicle reception stage, disassembly and inspection stage, repair in progress stage, repair completion stage, and vehicle delivery stage. Alternatively, depending on the actual repair process, it can be expanded to include stages such as vehicle reception and inspection, disassembly and inspection, sheet metal repair, painting completion, final assembly verification, and delivery confirmation. For routine repair scenarios, only the pre-repair and post-repair stages may be collected. The number and names of stages do not constitute a limitation on the scope of protection of this application; they only need to reflect the different temporal states of the same vehicle during the repair process.

[0073] In each stage, multiple perspective images of the same vehicle are acquired. These images can be obtained using a fixed camera array, mobile imaging equipment, handheld terminals, robotic inspection equipment, or a combination thereof. The preferred acquisition perspectives cover the front, rear, left and right sides, roof edge, side skirts, and areas of interest for repair. For accident-damaged, sheet metal repaired, or repainted areas, additional close-up perspectives can be added to improve the reconstruction detail of those areas. Metadata such as stage identifiers, acquisition time, vehicle identifier, repair work order number, acquisition equipment parameters, and operator identifiers are recorded simultaneously during acquisition for subsequent archiving and association with the 3D model and discrepancy results.

[0074] After obtaining multi-view images at each stage, 3DGS reconstruction is performed independently on the images of each stage to obtain the corresponding 3DGS model. The 3DGS model consists of multiple three-dimensional Gaussian volumes, each of which includes at least the position mean, covariance matrix, opacity, and spherical harmonic coefficients. Through this processing, the spatial morphology and surface appearance of the vehicle at each stage are recorded as a renderable and computable three-dimensional representation, providing a data foundation for subsequent cross-stage registration and difference detection.

[0075] S2: Divide the Gaussian bodies in the 3DGS model at each stage into a subset of rigid structures and a subset of movable parts.

[0076] In this embodiment, the entire vehicle 3DGS model is not treated as a single rigid body. Instead, semantic attribution is determined for Gaussian bodies in the 3DGS model at each stage, taking into account the vehicle's structural characteristics. The rigid structure subset may include Gaussian bodies corresponding to relatively stable load-bearing structures or reference structures during normal maintenance processes, such as the roof, A-pillar, B-pillar, C-pillar, sill beam, windshield frame, and exposed areas of the body frame. The movable parts subset may include Gaussian bodies corresponding to doors, hood, trunk lid, fuel tank cap, and other external coverings with independent degrees of freedom for opening, rotation, or assembly.

[0077] For example, vehicle component-level semantic segmentation can be performed first on the multi-view images used for 3DGS reconstruction. Then, the mean position of each Gaussian body in the 3DGS model is projected onto the corresponding view image, and the vehicle component to which the Gaussian body belongs is determined based on the semantic label at the projection position. For cases where the same Gaussian body obtains different semantic labels in multiple views, multi-view voting, confidence-weighted voting, or prioritizing clear view labels can be used to determine the final semantic label. The semantic label category can be adjusted according to vehicle model, maintenance business, and model training data; it only needs to be able to distinguish between relatively stable vehicle body structures and components with the potential for independent movement.

[0078] The above division provides a structural basis for subsequent two-level registration. If the entire vehicle model is uniformly registered as a rigid body, differences in the opening and closing of components such as doors and hoods can easily be misjudged as spatial differences on the vehicle body surface. By first distinguishing between rigid structures and movable parts, a vehicle-level spatial benchmark can be established using stable structures, while retaining independent correction capabilities for movable parts, thus better reflecting the real structural motion patterns in vehicle maintenance scenarios.

[0079] S3: Using the 3DGS model of the first stage as the reference model and the 3DGS models of the remaining stages as the models to be registered, perform two-level registration on each model to be registered. The first-level registration solves the global rigid body transformation based on the rigid structure subsets of the reference model and the models to be registered. The second-level registration solves the local rigid body transformation increment based on the subsets of movable parts of the reference model and the models to be registered, so that each model to be registered is aligned to the coordinate system of the reference model.

[0080] In this embodiment, the first stage is preferably the vehicle reception stage. Since the models of each stage are usually reconstructed independently at different times, different acquisition locations, and different vehicle postures, the original coordinate systems are not consistent. If they are directly compared, it will be impossible to distinguish between actual maintenance changes and coordinate deviations.

[0081] The first-level registration is based on solving the global rigid body transformation using rigid structural subsets of the reference model and the model to be registered. Specifically, feature points with stable spatial distribution and high reconstruction quality can be extracted from the rigid structural subsets of the two models. For example, the mean position of a Gaussian volume can be selected as the point location, and feature points can be selected by combining Gaussian volume opacity, local geometric distribution, or semantic label confidence. Then, the global rigid body transformation is solved based on the feature points, and this global rigid body transformation is applied to all Gaussian volumes of the model to be registered, so that the model to be registered is transformed as a whole to the coordinate system of the reference model.

[0082] The second-level registration solves for local rigid body transformation increments based on subsets of movable components in the reference model and the model to be registered. After completing the global coarse registration, Gaussian sets of corresponding movable components such as doors, hoods, and trunk lids are extracted, and local rigid body transformation increments are solved between these components to compensate for independent pose differences of the component relative to the main body of the vehicle, such as different door opening angles, different hood heights, or the trunk lid being in a half-open state. Through two-level registration, a unified coordinate reference can be established across stages, and spurious differences caused by normal pose changes of movable components can be reduced, improving the reliability of subsequent difference detection.

[0083] S4: Establish a three-dimensional voxel mesh in the reference model coordinate system, assign the Gaussian volumes in the 3DGS model of each stage to the corresponding voxels, and aggregate and calculate the characteristics of each voxel to form the voxelized Gaussian field corresponding to each stage.

[0084] In this embodiment, after the 3DGS models at each stage are registered, it is still not advisable to directly compare them one by one using individual Gaussian bodies as corresponding elements. 3DGS models at different stages are typically generated independently through optimization, and the number, spatial distribution, covariance matrix, opacity, and spherical harmonic coefficients of Gaussian bodies are not entirely consistent. Even when representing the same vehicle region, there may not necessarily be the same number or index of Gaussian bodies. Therefore, this embodiment establishes a unified three-dimensional voxel mesh in the reference model coordinate system, transforming the unstructured Gaussian body set into a voxelized Gaussian field with a unified spatial index.

[0085] Specifically, the joint bounding box can be determined based on the average position of the Gaussian bodies in the models participating in the comparison, and a voxel mesh can be generated along the three-dimensional direction according to the preset voxel edge length. The voxel edge length can be determined based on vehicle size, detection accuracy, and computational resources; for fine-grained changes such as dents, scratches, and paint color differences, a smaller voxel edge length can be used; for whole-vehicle-level state tracing or coarse-grained quality verification, a larger voxel edge length can be used. The same joint bounding box, voxel edge length, and voxel indexing rules should be used in each stage to ensure that the same voxel index corresponds to the spatial position in the same reference coordinate system.

[0086] After assigning Gaussian volumes to corresponding voxels, feature aggregation is performed on the Gaussian volumes within each voxel. Voxel features include Gaussian volume density, weighted centroids based on opacity, average opacity, geometric dispersion, and the weighted mean of low-order spherical harmonics. Specifically, Gaussian volume density and average opacity reflect the adequacy of the voxel region's reconstruction, the weighted centroids reflect the local geometric location, geometric dispersion reflects the spatial distribution stability of the Gaussian volume, and the weighted mean of low-order spherical harmonics reflects local appearance color or brightness variations. Furthermore, a reconstruction confidence score for the voxel can be calculated based on Gaussian volume density, average opacity, and geometric dispersion for subsequent difference filtering. By voxelizing the Gaussian field, different stages do not rely on a one-to-one correspondence between individual Gaussian volumes; instead, voxels in the same spatial location are used as comparison units.

[0087] S5: Calculate the comprehensive difference index for each voxel of the voxelized Gaussian field corresponding to any two stages, output the difference result after adaptive threshold filtering, and generate a full-cycle traceability digital archive.

[0088] In this embodiment, any two stages can be selected for difference comparison based on traceability requirements. For example, the vehicle receiving stage can be compared with the vehicle delivery stage to confirm changes in the vehicle's state after repair; the disassembly and inspection stage can be compared with the repair stage to track local changes caused by the repair work; adjacent stages can also be compared sequentially to form a chain of changes throughout the vehicle repair process. Since the voxelized Gaussian fields of each stage are under the same reference coordinate system and the same voxel grid, a comprehensive difference index can be calculated at the voxel level.

[0089] The comprehensive difference index includes geometric difference index and appearance difference index. Geometric difference indexes can be determined based on the positional shift of the weighted centroid within the same voxel, changes in average opacity, or changes in Gaussian volume density, reflecting spatial morphological changes such as dents, sheet metal repairs, structural additions or subtractions, or component replacements. Appearance difference indexes can be determined based on changes in the weighted mean of low-order spherical harmonic coefficients, reflecting changes in paint touch-ups, color differences, surface contamination, oxidation, or cleaning conditions. A weighted sum of the geometric and appearance difference indices yields the comprehensive difference index, thus taking into account both spatial and appearance changes on the vehicle surface.

[0090] Considering the varying reconstruction stability across different regions of the vehicle, this embodiment employs adaptive threshold filtering for the comprehensive difference index. Specifically, for the same voxel, the lowest reconstruction confidence score across two stages is taken as the effective confidence score, and a significance threshold is determined based on this effective confidence score. Furthermore, the significance threshold is adjusted according to the semantic region type of the voxel. For example, the threshold is lowered for flat panel areas such as the center of the door and the center of the hood to improve detection sensitivity, while the threshold is raised for geometrically complex areas such as the edges of the headlights, the grille, and the wheel arches to enhance false alarm suppression.

[0091] After adaptive threshold filtering, the retained difference voxels can be merged into difference regions according to spatial connectivity, and the output includes the difference location, the vehicle component to which it belongs, the difference type, geometric offset, appearance change, scope of impact, and corresponding comparison stage. These difference results, along with the 3DGS models from each stage, acquired metadata, registration parameters, and visualization screenshots, are stored together to form a full-cycle traceability digital archive. This digital archive allows for quantitative verification of the actual changes in the vehicle's condition before and after maintenance, and provides replayable and verifiable 3D data evidence in scenarios involving maintenance disputes, delivery acceptance, or quality accountability.

[0092] Preferably, the step of acquiring multi-view images of the same vehicle at different stages and reconstructing 3DGS models for each stage includes:

[0093] S1.1: Perform sparse reconstruction on the multi-view images acquired at each stage to obtain sparse 3D point clouds and camera parameters corresponding to each image.

[0094] S1.2: Using the three-dimensional coordinates of each point in the sparse three-dimensional point cloud as the initial position mean of the three-dimensional Gaussian body, and initializing the covariance matrix, opacity and spherical harmonic coefficients of each Gaussian body to obtain the initial 3DGS model.

[0095] S1.3: The initial 3DGS model is rendered into two-dimensional images from each training perspective using differentiable rendering, the loss between the rendered image and the corresponding acquired image is calculated, and the gradient is backpropagated to update the parameters of each Gaussian body.

[0096] S1.4: Adaptive density control is performed on the Gaussian body during the optimization process to obtain the 3DGS model at each stage.

[0097] In this embodiment, each maintenance stage independently forms an image set, which is bound and stored with stage identifiers and vehicle identifiers. For any stage, the acquired multi-view images can be screened and preprocessed first, removing images with severe motion blur, overexposure, underexposure, or excessive vehicle occlusion, and the remaining images undergo distortion correction, scale normalization, and exposure consistency processing. Subsequently, sparse reconstruction is performed on the preprocessed images. Specifically, feature point extraction, inter-image feature matching, and incremental or global motion recovery structure algorithms can be used to estimate the camera parameters of each image and generate a sparse 3D point cloud. The camera parameters include at least the camera intrinsic matrix, rotation matrix, and translation vector, and may also include distortion parameters; sparse reconstruction can be implemented using COLMAP, OpenMVG, or other 3D reconstruction processes capable of outputting sparse 3D point clouds and camera parameters.

[0098] After obtaining the sparse 3D point cloud, the 3D coordinates of each point in the sparse 3D point cloud are used as the initial values ​​of the position mean of the 3D Gaussian volume. A covariance matrix, opacity, and spherical harmonic coefficients are configured for each 3D Gaussian volume. The covariance matrix describes the scale and orientation of the Gaussian volume in 3D space. Initially, it can be set to an isotropic covariance matrix. Its initial scale can be determined based on the mean or median of the distances between the point and its nearest neighbors to avoid the Gaussian volume being too large, resulting in surface blurring, or too small, resulting in insufficient coverage. The opacity characterizes the cumulative transparency contribution of the Gaussian volume in volume rendering. Initially, the opacity α of each Gaussian volume can be set to 0.1 and iteratively updated during subsequent differentiable rendering optimization. The spherical harmonic coefficients express color information. Initially, the zeroth-order spherical harmonic coefficient can be determined based on the color observations of the 3D point in the visible image, and higher-order spherical harmonic coefficients are initialized to zero or near-zero values. When the same sparse point has multiple color observations, the multi-view color mean, weighted mean, or the observation with the clearest image can be used as the initial color.

[0099] After obtaining the initial 3DGS model, differentiable rendering is performed on the initial 3DGS model from the training perspective using the camera parameters corresponding to each image, resulting in a rendered image consistent with the perspective of the acquired images. During the differentiable rendering process, the 3D Gaussian volume is projected onto the 2D imaging plane based on the positional mean, covariance matrix, opacity, and spherical harmonic coefficients of each volume, and transparency accumulation and color synthesis are performed in depth order. Subsequently, a loss function is calculated between the rendered image and the corresponding acquired image. The loss function can include pixel-level color difference loss and structural similarity loss. For example, L1 loss can be used to constrain pixel color deviation, and structural similarity loss can be used to constrain local texture and edge structure. The gradient is backpropagated based on the loss function, and the positional mean, covariance matrix, opacity, and spherical harmonic coefficients of each Gaussian volume are iteratively updated. When there are small errors in the camera extrinsic parameters, small-scale joint optimization of the camera extrinsic parameters can also be performed, but the consistency of the model coordinate system at the same stage should be avoided.

[0100] During optimization, adaptive density control is performed on the Gaussian volume, with a density control operation performed every 100 iterations. When the average reprojection error of a Gaussian volume in a certain region exceeds 2 pixels over three consecutive statistical periods, or the L2 norm of the position gradient exceeds 1e-4, the corresponding Gaussian volume undergoes proliferation processing: for Gaussian volumes with a maximum eigenvalue greater than 0.05m in the covariance matrix, they are split into two Gaussian volumes with halved scales along their principal axis; for Gaussian volumes with a maximum eigenvalue less than or equal to 0.05m in the covariance matrix, a new Gaussian volume is cloned at a position offset of 0.01m along the gradient direction from its mean position. For Gaussian volumes with an opacity consistently below 0.01 for more than five statistical periods, visible in fewer than three training views, or with a mean position distance from the vehicle's bounding box exceeding 0.5m, pruning is performed to obtain the 3DGS models for each stage. Each stage generates 3DGS models according to the same rules, ensuring that subsequent cross-stage registration and voxelization difference detection are based on 3D data with consistent parameter meanings.

[0101] Preferably, the step of dividing the Gaussian bodies in the 3DGS model at each stage into a rigid structure subset and a movable component subset includes:

[0102] S2.1: For the multi-view images used for reconstruction at each stage, predict the semantic labels of vehicle parts pixel by pixel using a semantic segmentation model.

[0103] S2.2: Project the mean position of each Gaussian body in the 3DGS model at each stage onto the semantically segmented image through the corresponding camera parameters, obtain the semantic label at each projection position, and determine the semantic label of each Gaussian body through multi-view voting.

[0104] S2.3: Based on the semantic tags, classify the Gaussian bodies into the rigid structure subset or the movable component subset, wherein the rigid structure subset includes Gaussian bodies corresponding to load-bearing structural components whose geometry remains unchanged in each maintenance stage, and the movable component subset includes Gaussian bodies corresponding to components with independent degrees of freedom of motion.

[0105] In this embodiment, semantic segmentation is used to establish spatial constraints that conform to the vehicle's structural motion laws for subsequent cross-stage registration. Since the same vehicle may undergo state changes during maintenance, such as doors being open, hood raised, trunk lid open, bumper removed, or partial body panels replaced, directly treating the entire vehicle 3DGS model as a single unit for registration could easily lead to pose differences in movable parts affecting the global registration result. Therefore, this embodiment first performs semantic segmentation on vehicle components at the 2D image level, and then maps the semantic segmentation results to a 3D Gaussian volume, allowing each Gaussian volume to obtain a corresponding semantic label for the vehicle component.

[0106] The semantic segmentation model can employ a vehicle component segmentation model based on convolutional neural networks, Transformer structures, or a combination of both. Its input consists of multi-view images used for 3DGS reconstruction at each stage, and its output is a pixel-level semantic label map consistent with the image resolution. The semantic labels include 21 categories of standard vehicle components, such as the body frame, doors, hood, trunk, wheels, chassis, and bumpers. The specific number of semantic label categories can be adjusted according to the vehicle model and maintenance needs; it only needs to be able to distinguish between stable load-bearing structures and components with the potential for independent movement.

[0107] For example, a vehicle component semantic segmentation model based on the SegFormer-B2 backbone network is used to predict the semantic labels of vehicle components pixel by pixel. This model employs an encoder-decoder structure, where the encoder outputs a feature map at 1 / 8 the size of the input image, and the decoder uses a multilayer perceptron to upsample to the input image resolution. The training dataset contains 120,000 multi-view images covering 56 types of passenger and commercial vehicles, labeled with 21 categories of standard vehicle components, including body frames, doors, hoods, trunks, wheels, chassis, and bumpers. The training process uses a weighted sum of cross-entropy loss and Dice loss as the loss function, with weights of 0.7 and 0.3, respectively. The initial learning rate is set to 1e-4, using a cosine annealing learning rate scheduling strategy, a batch size of 16, and 80,000 training iterations. The input image size is uniformly adjusted to 1024×1024 pixels, and the output is a pixel-level semantic label map with the same size as the input image and the corresponding pixel segmentation confidence map.

[0108] After obtaining the semantic label maps for images from each viewpoint, for any Gaussian volume, its mean position is taken as the representative spatial location. Based on the camera intrinsic and extrinsic parameters obtained during sparse reconstruction or 3DGS training in this stage, this mean position is projected onto the pixel plane of each training image. If the projection point is located within the effective area of ​​the image, and the Gaussian volume is visible or nearly visible relative to the corresponding camera, the semantic label at the projected pixel location is read as the candidate semantic label for that viewpoint of the Gaussian volume. Visibility can be determined based on conditions such as depth consistency, viewpoint angle, and whether the projection position is within the vehicle segmentation mask; observations that are occluded, projected into the background area, or have abnormal depth do not participate in subsequent voting.

[0109] For candidate semantic labels obtained from multiple perspectives for the same Gaussian body, the semantic label of each Gaussian body is determined by multi-view voting. The voting weight is the product of the visibility score and segmentation confidence of the Gaussian body under the corresponding perspective. The visibility score is calculated by the intersection volume of the camera frustum and the covariance of the Gaussian body. When the vote rate of a certain semantic label exceeds 70%, the label is assigned to the corresponding Gaussian body; Gaussian bodies with a vote rate below 70% are marked as low-confidence semantic labels, and their weight is reduced by 50% in subsequent registration and difference detection.

[0110] After determining the semantic labels of the Gaussian bodies, the vehicle components are categorized into rigid structure subsets or movable component subsets based on their structural attributes. The rigid structure subset preferably includes Gaussian bodies corresponding to components that reflect the overall vehicle's posture, such as the roof, A-pillar, B-pillar, C-pillar, windshield frame, sill beams, and fixed side panel areas. The movable component subset is preferably established separately for each component, such as a left front door subset, a right front door subset, an engine hood set, and a trunk lid set. Each subset corresponds to a component with independent opening / closing, rotation, disassembly / reassembly, or assembly posture changes.

[0111] Through the semantic segmentation described above, the 3DGS model at each stage is transformed from a single set of Gaussian volumes into a set of Gaussian volumes with meaning related to vehicle structure. On the one hand, subsequent global registration can mainly rely on stable rigid structure subsets to avoid the opening and closing states of components such as doors and hoods affecting the overall vehicle coordinate reference. On the other hand, subsequent local registration can solve pose compensation for each movable component separately, making it possible to distinguish between normal component movement and actual maintenance changes, thereby improving the reliability of cross-stage difference detection.

[0112] For preferred options, please refer to [link / reference]. Figure 3 The first-level registration solves the global rigid body transformation based on the rigid structure subsets of the reference model and the model to be registered, including:

[0113] S3.1: Perform voxel downsampling on the Gaussian bodies within the rigid structure subsets of the baseline model and the model to be registered, respectively, according to their spatial positions. Select the average position of the Gaussian body with the highest opacity within each voxel as the feature point to form the feature point set of the baseline model and the feature point set of the model to be registered.

[0114] Specifically, since there are a large number of Gaussian volumes in the 3DGS model, and the distribution density of Gaussian volumes is affected by image texture, viewpoint coverage, reflectivity, and the optimization process, directly using all Gaussian volumes for registration would result in a large computational load and could easily cause local high-density regions to bias the registration results. Therefore, this embodiment performs voxel downsampling on the rigid structure subsets of the baseline model and the model to be registered, respectively, so that feature points remain relatively uniformly distributed in stable structural regions such as the roof, pillars, door sill beams, and windshield frames.

[0115] During voxel downsampling, the mean position of each Gaussian voxel within the rigid structure subset of both models is acquired, a joint spatial bounding box is calculated, and a 3D voxel mesh is created using preset voxel edge lengths. The voxel edge lengths can be determined based on vehicle scale, acquisition accuracy, and registration accuracy, for example, by taking a fixed proportion of the vehicle body length. Subsequently, each Gaussian voxel is assigned to its corresponding voxel according to its mean position. For each non-empty voxel, the Gaussian voxel with the highest opacity is selected as the representative Gaussian voxel, and its mean position is used as the feature point. If multiple Gaussian voxels with similar opacities exist within the same voxel, the representative Gaussian voxel can also be determined by combining the scale characteristics of the covariance matrix and the number of times the viewpoint is visible, in order to reduce the impact of occlusion edges and reconstruction noise.

[0116] After the above processing, the baseline model forms a baseline feature point set, and the model to be registered forms a feature point set to be registered. Each feature point, in addition to its three-dimensional position coordinates, can also retain auxiliary information such as the opacity of its associated Gaussian body, the scale characteristics of its covariance matrix, semantic labels, and the voxel number it belongs to. The opacity is used for subsequent weighted rigid body transformation solutions, and the semantic labels are used to ensure that matching preferentially occurs within the same or similar rigid structural regions.

[0117] S3.2: Calculate the local geometric feature descriptor for each feature point in the two feature point sets, establish the correspondence between the two feature point sets through descriptor matching, and remove outliers.

[0118] Specifically, feature points obtained from voxel downsampling only provide spatial location. If a direct correspondence is established based on the distance between points before the two models are aligned, incorrect matching is likely to occur. Therefore, this embodiment calculates local geometric feature descriptors for each feature point and establishes an initial correspondence through the shape similarity of the local neighborhood. Local geometric feature descriptors can employ fast point feature histograms, local descriptors based on normal distribution, or descriptors constructed based on the eigenvalues ​​of the neighborhood covariance matrix, to characterize local geometric shapes such as roof edges, pillar transitions, windshield frame corners, and sill beam bends.

[0119] When calculating local geometric feature descriptors, local neighborhoods can be established through fixed-radius search or K-nearest neighbor search. The fixed radius can be set in relation to the voxel side length, and the number of K-nearest neighbors can be determined based on the feature point density. For any feature point, the local covariance matrix can be calculated based on the spatial distribution of neighborhood points, and eigenvalue decomposition can be performed to obtain geometric quantities reflecting linear structure, planar structure, and degree of randomness. Alternatively, the local normal direction can be estimated, and the normal angles, distance relationships, and directional relationships between neighborhood points can be statistically analyzed to form a histogram-type descriptor.

[0120] When establishing the correspondence, for each feature point in the set of feature points to be registered, a candidate matching point with the smallest descriptor distance is searched in the set of reference feature points. The descriptor distance can be Euclidean distance, chi-square distance, or cosine distance. To reduce false matches, a bidirectional nearest neighbor constraint can be used, along with a semantic consistency constraint, to ensure that the matching points belong to the same rigid structure semantic category or to adjacent structure categories that are allowed to match, thus avoiding cross-matching of symmetrical structures on the left and right sides of the vehicle body or similar edge structures.

[0121] After obtaining the initial correspondence, a random sampling consistency method is used to eliminate outliers. Specifically, the minimum set of point pairs that satisfy the rigid body transformation solution is randomly selected from the initial corresponding point pairs. Candidate rotation matrices and translation vectors are calculated, and the candidate transformation is applied to the feature points to be registered. The number of point pairs whose distance to the corresponding reference feature point after transformation is less than the consistency threshold is counted and taken as the number of interior points of the candidate transformation. After repeating the above process, the candidate transformation with the largest number of interior points is selected. If multiple candidate transformations have the same number of interior points, the candidate transformation with the smallest average residual is selected, and its corresponding interior point set is used as the correspondence after eliminating outliers. The consistency threshold can be related to the voxel side length or adaptively determined based on the statistical value of the initial matching residual. If necessary, a geometric consistency check can also be performed on the interior point pairs to delete matching point pairs with significantly inconsistent relative distance relationships.

[0122] S3.3: Based on the correspondence after removing outliers, the opacity of the Gaussian body to which the corresponding feature point belongs is used as the weight to solve the weighted optimal global rigid body transformation. The coordinate transformation of all Gaussian bodies of the model to be registered is performed according to the global rigid body transformation to generate a coarse registration model.

[0123] Specifically, after obtaining the set of corresponding point pairs after removing outliers, the global rigid body transformation is solved to align the rigid structure subset of the model to be registered with the rigid structure subset of the reference model. The global rigid body transformation includes a rotation matrix and a translation vector, where the rotation matrix describes the attitude change of the model to be registered relative to the reference model, and the translation vector describes the positional offset between the origins of the two models.

[0124] For each pair of corresponding feature points after removing outliers, the weight of the corresponding point pair is determined based on the opacity of the Gaussian body to which the feature point belongs in the model to be registered and the opacity of the Gaussian body to which the corresponding feature point belongs in the baseline model. The weight can be the product or average of the opacities of the two Gaussian bodies, with the product being preferred to reduce the influence of the corresponding point pair on the rigid body transformation solution when the reconstruction contribution of the corresponding region is insufficient at any stage. To avoid individual high-opacity points from excessively dominating the solution, the weights of each pair of corresponding points can be normalized.

[0125] The weighted optimal global rigid body transformation can be solved as follows: Let the corresponding point pairs after removing outliers be... and ,in For the feature points in the model to be registered, For the corresponding feature points in the baseline model, the corresponding weights are... First, calculate the weighted centroids of the feature points to be registered. Weighted centroid of reference feature points Then calculate the decentralized coordinates. , And construct a weighted covariance matrix. ;T represents transpose; for Perform singular value decomposition to obtain ,in It is a left singular matrix. It is a singular value matrix. Let be a right singular matrix; and let the rotation matrix be... If the rotation matrix Determinant Then Recalculate after inverting the last column To exclude mirror reflection transformation; finally, the translation vector is calculated based on the two sets of weighted centroids. Thus, we can obtain The minimum global rigid body transformation is used, and this transformation is applied to unify the coordinates of the model to be registered. This represents the squared distance error of the corresponding point pair. This represents the sum of the weighted squared errors of all corresponding point pairs.

[0126] After obtaining the global rigid body transformation, it is applied to all Gaussian bodies in the model to be registered, thus transforming the entire model to the reference model coordinate system. Specifically, for any Gaussian body in the model to be registered, the mean position of the Gaussian body is transformed using a rotation matrix and a translation vector; for the covariance matrix of the Gaussian body, a similarity transformation is performed using a rotation matrix, so that the spatial scale and orientation of the Gaussian body represented by the covariance matrix rotate synchronously with the model's pose; the opacity parameter, which represents the contribution intensity of the Gaussian body to the rendering result, does not change with the coordinate system transformation; the spherical harmonic coefficient, which is used to characterize the appearance color information of the Gaussian body, is usually kept unchanged. After the above transformations, a coarse registration model is obtained in the reference model coordinate system.

[0127] Preferably, the second-level registration is based on solving the local rigid body transformation increments for each subset of movable parts of the reference model and the model to be registered, including:

[0128] A1: Extract the mean position of the Gaussian body within each subset of movable parts in the coarse registration model and the baseline model respectively as the point set.

[0129] Specifically, after the first-level registration is completed, the main body of the vehicle in the coarse registration model has been transformed into the coordinate system of the reference model. However, movable parts such as doors, hoods, and trunk lids may still have local misalignments with the reference model due to differences in opening angles, support states, or disassembly / removal states. If voxel-based difference detection is performed directly, these local misalignments are easily misidentified as changes in the vehicle surface. Therefore, this embodiment establishes the local point set correspondence between the coarse registration model and the reference model on a per-movable-part basis, and solves the local rigid body transformation increment separately for each movable part.

[0130] In this embodiment, based on the semantic segmentation results, subsets of movable parts with the same semantic category are extracted from both the coarse registration model and the baseline model. For example, the left front door subset in the baseline model corresponds to the left front door subset in the coarse registration model, and the hood set in the baseline model corresponds to the hood set in the coarse registration model. For each subset of movable parts, the mean position of each Gaussian body is extracted as a point set, while auxiliary attributes such as opacity, scale features of the covariance matrix, semantic label confidence, and number of times the viewpoint is visible are retained. The mean position is used for local geometric registration, the opacity is used to construct weights in the iterative nearest point algorithm, and the semantic label confidence is used to remove semantically unstable Gaussian bodies at the part boundaries.

[0131] A2: For each movable part, the iterative nearest point algorithm with opacity as the weight is used to iteratively solve the local rigid body transformation increment for the corresponding point set of the two models.

[0132] Specifically, the differences in movable parts at different maintenance stages are usually manifested as local rigid body movements relative to the main body of the vehicle. For example, a door may rotate around a hinge, the hood may change in height, the trunk lid may be in different open or closed states, or the parts may undergo slight positional changes due to disassembly or assembly. Based on the vehicle reference coordinate system established in the first-level registration, the remaining misalignment of each movable part can be compensated for by the corresponding local rigid body transformation increment.

[0133] For any movable component, the local rigid body transformation increment can be initialized to a state with no rotation and no translation, or the translation amount can be initialized based on the weighted centroid difference between the component point set of the coarse registration model and the component point set of the reference model. Then, the iterative process begins: first, the current local rigid body transformation increment is applied to the component point set in the coarse registration model, and then the nearest neighbor point is searched in the corresponding component point set of the reference model to establish a temporary correspondence.

[0134] When establishing temporary correspondences, spatial distance should not be the sole criterion. To avoid erroneous correspondences caused by component boundaries, local missing parts, or areas of actual maintenance changes, point pairs can be selected by combining distance thresholds, semantic consistency, opacity conditions, and normal angle conditions. The distance threshold for nearest neighbor search is set to twice the voxel side length (preferably 10 cm); semantic consistency ensures that corresponding points belong to the same movable component; the opacity condition excludes low-contribution Gaussian bodies; and the normal angle condition reduces erroneous correspondences with excessively large surface orientation differences. Through these constraints, the iterative nearest point algorithm can primarily rely on reliable point pairs on the stable surface of the same component.

[0135] Furthermore, for the retained corresponding point pairs, weights are constructed using opacity and a weighted rigid body transformation is solved. The weight of the corresponding point pair can be the product of the opacity of the Gaussian body to which the coarse registration component point belongs and the opacity of the Gaussian body to which the reference component point belongs, or the minimum of the two. The product or minimum value is preferred because when the reconstruction contribution of the corresponding region is insufficient in any stage, the influence of the point pair on the local transformation solution should be reduced. To avoid excessive influence of local opacity points on the transformation result, the weights of all point pairs can be normalized. Subsequently, the incremental rotation and translation of this iteration are calculated using weighted least squares: first, the weighted centroids of the current coarse registration point pair set and the reference point pair set are calculated separately; then, a weighted covariance matrix is ​​constructed, and the optimal rotation matrix is ​​solved through singular value decomposition; finally, the translation vector is obtained from the two sets of weighted centroids. If the determinant of the rotation matrix is ​​negative, the sign of the singular value decomposition result is corrected to ensure that the local transformation is a legal rigid body transformation.

[0136] After each iteration, the incremental rotation and translation obtained in this round are combined with the local rigid body transformation increment of the previous round, and the nearest neighbor correspondence is re-established until the convergence condition is met. The convergence condition includes the number of consecutive iterations reaching the maximum number of iterations, such as 50 times.

[0137] Furthermore, for components with clearly defined hinge motion characteristics, such as doors, hoods, and trunk lids, hinge positions can be estimated based on vehicle model templates or semantic regions, and the rotation center or rotation axis direction can be constrained. The vehicle model template contains the standard number of hinges, installation positions, and rotation axis direction parameters for each movable component, stored in the vehicle model database and accessible via the vehicle VIN code. When no vehicle model template is available, a geometric estimation method based on semantic segmentation results is used: the boundary edge between the movable component and the rigid structure of the vehicle body is extracted, the edge line is fitted using the RANSAC algorithm, the hinge rotation axis direction is solved by combining the geometric constraints of the component's opening and closing motion, and the point on the rotation axis that minimizes the component point set registration residual is searched as the rotation center. The constraint is applied by adding a regularization term to the objective function of the weighted iterative nearest point algorithm, ensuring that the deviation between the solved local rigid body transformation rotation axis and the estimated hinge axis direction does not exceed 5°, and the deviation between the rotation center and the estimated hinge center does not exceed 2 cm.

[0138] A3: Based on the local rigid body transformation increments, perform coordinate transformations on the Gaussian bodies of the corresponding movable component subsets in the coarse registration model to generate a fine registration model aligned to the reference model coordinate system.

[0139] Specifically, after obtaining the local rigid body transformation increment for each movable component, this increment is applied only to the corresponding subset of movable components in the coarse registration model, without altering the vehicle body rigid structure region that has already been aligned through the first-level registration. For any Gaussian body belonging to a movable component in the coarse registration model, its position mean is transformed according to the local rotation matrix and local translation vector corresponding to that component; for its covariance matrix, a similarity transformation is performed using the local rotation matrix, so that the spatial direction represented by the covariance matrix is ​​updated synchronously with the component's attitude; opacity is usually kept unchanged; and spherical harmonic coefficients can retain their original values ​​when only low-order appearance statistics are used subsequently.

[0140] In this embodiment, different movable parts use their own local rigid body transformation increments. For example, the local transformation of the left front door only applies to the left front door subset, and the local transformation of the hood only applies to the hood set. Different parts do not share the same local transformation. After completing the local coordinate transformation of all movable parts, the coarse registration model is updated to a fine registration model, making the differences in the normal opening and closing or disassembly postures of the movable parts distinguishable from the changes in the real surface, providing a more reliable spatial alignment basis for subsequent voxelized Gaussian field construction and difference detection.

[0141] Preferably, after generating the fine registration model, the method further includes:

[0142] B1: Extract the mean position values ​​of Gaussian bodies within the rigid structure subsets of the fine registration model and the baseline model respectively, search for the nearest neighbor mean position value in the baseline model for each mean position value in the fine registration model, and establish a point-to-point correspondence between the rigid structure subsets.

[0143] B2: Calculate the root mean square value of the distance between all point pairs as the registration residual. When the registration residual exceeds the registration anomaly threshold, it is determined that the registration result of the model to be registered is abnormal and an alarm is triggered.

[0144] In this embodiment, the registration quality assessment is preferably performed after the fine registration model is generated, to determine whether the model to be registered has been reliably aligned to the reference model coordinate system. Since the second-level registration mainly affects the subset of movable parts, the accuracy of the vehicle spatial reference should still be evaluated based on the rigid structural area of ​​the vehicle body. Therefore, this embodiment does not prioritize using all Gaussian bodies of the entire vehicle to calculate the registration residual, but instead extracts the mean position values ​​of Gaussian bodies within the rigid structural subset from the fine registration model and the reference model for comparison. The rigid structural subset may include the roof, A-pillar, B-pillar, C-pillar, windshield frame, sill beam, and other load-bearing areas of the vehicle body that are usually stable during maintenance, to avoid the normal opening and closing states of movable parts such as doors, hood, and trunk lid affecting the registration quality assessment.

[0145] In practice, the rigid structure evaluation points in the fine registration model are used as query points, and the rigid structure evaluation points in the baseline model are used as the reference point set. For each query point, the spatially nearest point is searched in the reference point set, and the two are established as a point pair. The nearest neighbor search can be implemented using a kd-tree, octree, or voxel hash index. To avoid excessive computational burden caused by too many Gaussian voxels, the two rigid structure subsets can be downsampled with the same voxel side length before establishing the point pair, and the average position of a Gaussian voxel within each voxel can be selected as the evaluation point; alternatively, the rigid structure feature points formed in the first-level registration can be directly used as evaluation points. The above processing does not change the residual calculation principle, but is only used to reduce the computational load and make the distribution of evaluation points more uniform.

[0146] After establishing the point pair correspondence, calculate the Euclidean distance between each pair. Let the set of valid point pairs be { , },in For the rigid structure evaluation points in the fine registration model, For the baseline model and The nearest evaluation point is the distance of that point pair. The registration residual is obtained by averaging the squares of the distances between all points and taking the square root. This registration residual characterizes the overall spatial deviation of the rigid structural region of the finely registered model relative to the reference model. The smaller the value, the higher the alignment degree of the two models in the vehicle stability structural region; the larger the value, the more likely there is an offset, rotation error, or local mismatch in the global coordinate reference.

[0147] Furthermore, the calculated registration residual is compared with the registration anomaly threshold. The registration anomaly threshold can be determined based on maintenance traceability accuracy requirements, acquisition device resolution, 3DGS reconstruction accuracy, and voxel downsampling scale. When rigid structural deformation exists, the registration anomaly threshold can be adjusted appropriately according to the size of the deformation area, but should not exceed 5 cm. In practical implementation, the registration anomaly threshold can be set to 0.5 times the voxel side length (preferably 2.5 cm). When the registration residual exceeds this threshold, a registration anomaly alarm is triggered, prompting a re-acquisition of data. Through this registration quality assessment mechanism, unreliable coordinate alignment data can be intercepted before difference detection, preventing large-area false differences from entering the full-cycle traceability digital archive due to registration failure.

[0148] For preferred options, please refer to [link / reference]. Figure 4 Among them, stage T1-T n This represents the various stages of vehicle maintenance, where n represents the number of stages, and x, y, and z represent the three coordinate axes in three-dimensional space. The Gaussian volumes in the 3DGS model for each stage are assigned to corresponding voxels, and the features of each voxel are aggregated and calculated to form the voxelized Gaussian field corresponding to each stage, including:

[0149] S4.1: Calculate the joint bounding box of the average position of all Gaussian volumes in the 3DGS model at each stage, and establish a three-dimensional voxel mesh by uniformly dividing the mesh along the three coordinate axes with a fixed step size. Distribute each Gaussian volume to its corresponding voxel according to its average position.

[0150] S4.2: Calculate the geometric and appearance features of the Gaussian volume aggregate within each voxel as the attributes of that voxel. The geometric features include Gaussian volume density, weighted centroid with opacity as the weight, average opacity, and geometric dispersion. The appearance features are the weighted mean of low-order spherical harmonic coefficients with opacity as the weight.

[0151] S4.3: Calculate the reconstruction confidence score of each voxel based on its Gaussian volume density, average opacity, and geometric dispersion as an attribute of that voxel. The higher the Gaussian volume density, the higher the average opacity, and the lower the geometric dispersion, the higher the reconstruction confidence score.

[0152] S4.4: The three-dimensional voxel mesh and the geometric features, appearance features and reconstruction confidence scores of each voxel together constitute the voxelized Gaussian field corresponding to each stage.

[0153] Specifically, after cross-stage registration, although the 3DGS models at each stage are now in the same reference coordinate system, the number, positional distribution, covariance matrix, opacity, and spherical harmonic coefficients of Gaussian volumes in different stage models do not have a natural one-to-one correspondence. Directly comparing Gaussian volume indexes or nearest-neighbor Gaussian volumes is easily affected by proliferation, pruning, and local density changes during the 3DGS optimization process. Therefore, this embodiment converts the unstructured Gaussian volume set into a voxelized Gaussian field under a unified voxel index, using voxels at the same spatial location as subsequent difference comparison units.

[0154] When establishing a 3D voxel mesh, a joint bounding box can be calculated based on the average position of all Gaussian bodies in the 3DGS models participating in the comparison at each stage. The joint bounding box can be an axis-aligned bounding box, with a preset margin at the boundary to prevent boundary Gaussian bodies from falling out of the voxel mesh due to numerical errors. Subsequently, voxels are divided along the three coordinate axes with a fixed step size. The voxel side length can be determined based on the target tracing accuracy, vehicle size, and computational resources. For example, a 3D voxel mesh with a side length of 5cm is established, covering the minimum bounding box of the vehicle model and extending outwards by 10cm. Voxel allocation uses a neighborhood contribution weighting method based on the influence range of Gaussian body covariance. The contribution weight of each Gaussian body to its centroid voxel and its 26 adjacent voxels is proportional to the volume percentage of the Gaussian body within that voxel. The volume percentage of a Gaussian body within a voxel is approximated using an axis-aligned bounding box: First, the 3σ axis-aligned bounding box of the Gaussian body is calculated. If this bounding box intersects with a voxel, the ratio of the intersection volume to the total volume of the Gaussian body's bounding box is used as the initial contribution weight of the Gaussian body to that voxel. For each Gaussian body, its initial contribution weights to all 27 neighboring voxels are normalized so that the sum of the weights is 1. When the largest eigenvalue of the Gaussian body's covariance is less than half the voxel's side length, it is simplified to single-voxel allocation, where the Gaussian body is only assigned to the voxel containing its centroid, and the contribution weight is set to 1.

[0155] For each non-empty voxel in each stage, the parameters of the Gaussian bodies within that voxel are aggregated to form voxel attributes. Gaussian body density represents the number of effective Gaussian bodies within the voxel; the weighted centroid is calculated based on the average position of each Gaussian body, weighted by opacity; the average opacity represents the entity contribution of that voxel region in the rendering; and the geometrical dispersion is calculated based on the dispersion of the average position of the Gaussian bodies within the voxel relative to the weighted centroid, reflecting the stability of the spatial distribution of Gaussian bodies in that region. Appearance features can be represented by the opacity-weighted average of the low-order spherical harmonics within the voxel, preferably using zero-order spherical harmonics or a combination of zero-order and first-order spherical harmonics, to characterize local base color, brightness, and low-frequency appearance changes, and to reduce interference from higher-order view-dependent components.

[0156] Furthermore, during the voxel attribute aggregation process, Gaussian volume validity rules can be set. For Gaussian volumes with excessively low opacity, abnormal covariance matrix scaling, excessively low semantic label confidence, or those in obvious outlier positions, their aggregation weight can be reduced or they can be excluded from voxel attribute calculation. For voxels containing multiple semantic labels, the semantic label distribution of Gaussian volumes within the voxel can be statistically analyzed, and the semantic category with the highest weight percentage can be used as the voxel semantic type; if the weight percentages of multiple semantic categories are similar, the voxel can be marked as a semantic boundary voxel for subsequent difference threshold correction.

[0157] After obtaining the geometric and appearance features of the voxels, the reconstruction confidence score is calculated based on Gaussian volume density, average opacity, and geometric dispersion. For example, the formula for calculating the reconstruction confidence score is as follows:

[0158]

[0159] in, To pre-set reliability weights, satisfying The preferred value is , , ; This represents the normalized value of the Gaussian volume density within the voxel. This is the normalized value of the average opacity of all Gaussian volumes within the voxel. is the normalized value of the geometric dispersion of the Gaussian centroid within the voxel. The geometric dispersion is the average distance from all centroids to the voxel center, and the normalization interval is [0,1]. To reconstruct the confidence score.

[0160] After the above processing, each stage generates a voxelized Gaussian field corresponding to the same 3D voxel mesh. The voxelized Gaussian field includes at least voxel indices, voxel spatial locations, geometric features, appearance features, reconstruction confidence scores, and voxel semantic types. Since each stage uses the same voxel mesh and indexing rules, the same voxel index corresponds to the same spatial location in the same reference coordinate system across different stages, thus providing a unified computational basis for analyzing geometric differences, appearance differences, and difference confidence between stages.

[0161] Preferably, the step of calculating the comprehensive difference index on a voxel-by-voxel basis for any two stages of the voxelized Gaussian field, and outputting the difference result after adaptive threshold filtering to generate a full-cycle traceability digital archive includes:

[0162] S5.1: For any two stages, calculate the geometric difference index and appearance difference index for each voxel in the voxelized Gaussian field that are not empty in both stages. The geometric difference index is calculated based on the difference between the weighted centroids and the average opacity of the voxels corresponding to the two stages. The appearance difference index is calculated based on the Euclidean distance between the weighted mean of the low-order spherical harmonic coefficients of the voxels corresponding to the two stages.

[0163] S5.2: The geometric difference index and appearance difference index of each voxel are weighted and summed to obtain the comprehensive difference index. After adaptive threshold filtering, voxels with comprehensive difference index exceeding the threshold are retained as difference voxels. The difference information of each difference voxel is quantitatively output to generate a full-cycle traceability digital archive.

[0164] Specifically, after forming the voxelized Gaussian fields corresponding to each stage, any two stages can be selected for difference comparison based on the traceability purpose. For example, the vehicle receiving stage and the vehicle delivery stage can be selected for overall repair result verification, or the disassembly and inspection stage and the sheet metal repair stage, or the painting completion stage and the delivery stage can be selected for segmented state comparison. Since the voxelized Gaussian fields of each stage use the same reference coordinate system, joint bounding box, and voxel indexing rules, the same voxel index corresponds to the same spatial region in different stages.

[0165] For voxels that are not empty in both stages, their geometric and appearance features are read separately, and differences are calculated under the same voxel index. The geometric difference index can be determined by the changes in local spatial location and local entity contribution, specifically based on the difference in the Euclidean distance between the weighted centroids of the corresponding voxels in the two stages and the average opacity. For example, the formula for calculating the geometric difference index is as follows:

[0166]

[0167] in, These are the weighted centroids of the Gaussian bodies within the voxels corresponding to the two stages, with the weights being the opacity of each Gaussian body. Let Euclidean distance be the two weighted centroids in three-dimensional space. These represent the average opacity of the Gaussian bodies within the voxels corresponding to the two stages. The absolute value of the difference in average opacity between the two voxels; This is a geometric difference index.

[0168] The appearance difference index can be calculated based on the Euclidean distance between the weighted averages of the low-order spherical harmonic coefficients of the corresponding voxels in two stages. If the low-order spherical harmonic coefficients include multiple color channels, the differences of each color channel can be calculated separately and then combined; if only the zero-order spherical harmonic coefficients are used, the appearance difference can be used to characterize changes in base color or brightness. For example, the formula for calculating the appearance difference index is as follows:

[0169]

[0170] in, These are weighted mean vectors of the zeroth to first order spherical harmonic coefficients of all Gaussian volumes within the corresponding voxels for the two stages, with the weights being the opacity of each Gaussian volume. The Euclidean distance between the weighted mean vectors of two spherical harmonic coefficients in a high-dimensional space; This is used as an indicator of appearance differences. Before calculating appearance differences, a color transformation matrix for two stages is calculated using a preset standard color chart area on the vehicle (white area for license plate, black area for tires) as a reference, and the spherical harmonic coefficients of all Gaussian bodies are corrected. The color transformation matrix is ​​a 3×3 linear transformation matrix, solved using the least squares linear regression method: at least 20 pixels are uniformly selected within each standard color chart area, excluding points within a 10-pixel edge and reflective / dark points with brightness values ​​exceeding 240 or below 15; the RGB values ​​of the zero-order spherical harmonic coefficients of the corresponding points in the two stages are extracted, and a system of linear equations is constructed to solve the transformation matrix; after the solution is completed, this matrix is ​​used to transform the RGB channels of the zero-order to first-order spherical harmonic coefficients of all Gaussian bodies to achieve color consistency correction at different stages.

[0171] After obtaining the geometric difference index and the appearance difference index, they are weighted and summed to obtain the comprehensive difference index. The weights can be determined based on the traceability task: for scenarios primarily involving geometric changes such as sheet metal deformation, dents, and component replacement, the weight of geometric differences can be increased; for scenarios primarily involving appearance changes such as paint color difference, touch-up paint marks, and surface contamination, the weight of appearance differences can be increased; in general maintenance traceability scenarios, fixed weights can be used, or the weights can be adjusted based on the vehicle's semantic region. For example, the comprehensive difference index... The calculation formula is as follows:

[0172]

[0173] in , These are the normalized values ​​of the geometric difference index and the appearance difference index, respectively. , For the weighting coefficients, satisfying The default value is , For appearance-sensitive areas (such as the car body paint), adjustments are made to... , For structurally sensitive areas (such as chassis components), adjustments were made to... , .

[0174] Furthermore, after obtaining the comprehensive difference index, it is not directly used as the final difference result. Instead, adaptive threshold filtering is performed on each voxel. Specifically, the reconstructed confidence scores of the same voxel in the two comparison stages are read, and the minimum of the two scores is taken as the effective confidence score of the voxel. At the same time, the semantic region type of the voxel is determined based on the semantic label of the Gaussian body within the voxel. Based on the effective confidence score and the semantic region type, the difference significance threshold corresponding to the voxel is determined, and the comprehensive difference index is compared with the difference significance threshold. If the comprehensive difference index exceeds the corresponding difference significance threshold, the voxel is retained as a difference voxel; if the comprehensive difference index does not exceed the corresponding difference significance threshold, the voxel is filtered out.

[0175] After adaptive threshold filtering, voxels whose comprehensive difference index exceeds the corresponding difference significance threshold are identified as differential voxels. For each differential voxel, its difference information is quantitatively output, including at least the voxel index, voxel spatial location, comparison stage, geometric difference index, appearance difference index, comprehensive difference index, corresponding difference significance threshold, and voxel reconstruction confidence score. By simultaneously recording the comprehensive difference index and difference significance threshold, the calculation basis for identifying the voxel as a differential voxel can be verified during subsequent retrospective analysis.

[0176] Furthermore, the difference information of each voxel is associated and stored with the corresponding stage identifier, vehicle identifier, maintenance work order identifier, 3DGS model identifier, registration transformation parameters, and voxelized Gaussian field identifier to generate a full-cycle traceability digital archive. The full-cycle traceability digital archive is used to record the difference voxel results formed under a unified voxel index between any two maintenance stages, enabling the 3D state changes of the same vehicle in the maintenance process to be queried and verified in the order of stages.

[0177] Preferably, the adaptive threshold filtering includes:

[0178] C1: For each voxel, the lowest value of the reconstruction confidence score in the two stages is taken as the effective confidence score of the voxel. The difference significance threshold of the voxel is determined based on the effective confidence score. The difference significance threshold is inversely correlated with the effective confidence score.

[0179] C2: Determine the semantic region type of each voxel based on the semantic label of the Gaussian body within each voxel, and correct the significance threshold of the difference.

[0180] Specifically, since the 3DGS models at each stage are reconstructed from multi-view images, the reconstruction reliability varies for different vehicle regions. For regions with sufficient view coverage, a sufficient number of effective Gaussian volumes, and a concentrated distribution of Gaussian volumes, the reconstruction confidence score approaches 1; for regions with occlusion, reflection, weak texture, or discrete Gaussian volume distribution, the reconstruction confidence score approaches 0. Therefore, when performing difference filtering on the same voxel in both stages, the lowest reconstruction confidence score of the corresponding voxel in both stages is taken as the effective confidence score, so that the difference determination for that voxel is based on the side with insufficient reconstruction reliability between the two stages.

[0181] For example, suppose the first Stage voxels The reconstruction confidence score is , No. The reconstruction confidence score of stage voxel v is Then the body element Valid confidence level satisfy:

[0182]

[0183] in, , All values ​​are normalized reconstruction confidence scores, ranging from 0 to 1.

[0184] When determining the significance threshold based on the effective confidence level, a hierarchical mapping method can be used. Specifically, the effective confidence level... Divided into three confidence intervals: when When ≥0.75, voxels It belongs to the first confidence interval; when 0.40 ≤ When <0.75, voxels It belongs to the second confidence interval; when When <0.40, voxels This falls within the third confidence interval. The first, second, and third confidence intervals correspond to the first, second, and third threshold coefficients, respectively, with the first threshold coefficient being less than the second threshold coefficient, and the second threshold coefficient being less than the third threshold coefficient. For example, the first threshold coefficient can be 0.8, the second threshold coefficient can be 1.0, and the third threshold coefficient can be 1.3. Let the preset benchmark difference threshold be... If 0, then the significance threshold for the difference is obtained based on the effective confidence level. for:

[0185] when When ≥0.75, =0.8 0;

[0186] When 0.40≤ When <0.75, = 0;

[0187] when When <0.40, =1.3 0.

[0188] Among them, the preset benchmark threshold 0 refers to the basic threshold used to determine whether the comprehensive difference index constitutes a valid difference when not considering effective confidence differences and semantic region type differences. 0 can be determined based on voxel side length, difference index normalization rules, and maintenance traceability accuracy requirements. Within the same traceability task, The value is kept consistent with 0. Through the above hierarchical mapping, a clear correspondence is established between the effective confidence interval and the significance threshold, avoiding the use of uncertain relative expressions.

[0189] Furthermore, the semantic region type of the voxel is determined based on the semantic labels of the Gaussian volumes within the voxel. Specifically, the sum of the opacity weights of the Gaussian volumes corresponding to each semantic label within the voxel is calculated, and the semantic label with the largest weight is taken as the primary semantic label of the voxel. If the weight of the primary semantic label is not less than 0.60, the semantic region type of the voxel is determined based on the primary semantic label; if the weight of the primary semantic label is less than 0.60, the voxel is determined as a semantic boundary region.

[0190] In this embodiment, if the main semantic label of a voxel belongs to the center of the door, the center of the hood, the large surface of the roof, or the large surface of the fender, then the voxel is defined as a flat panel area; if the main semantic label of a voxel belongs to the grille, the edge of the headlights, the wheel arch, the edge of the door seam, or the bumper opening, then the voxel is defined as a geometrically complex area; if the main semantic label of a voxel belongs to the A-pillar, B-pillar, C-pillar, sill beam, or windshield frame, then the voxel is defined as a stable load-bearing structure area; if the weight of the main semantic label is less than 0.60, then the voxel is defined as a semantic boundary area. A significance threshold for differences is applied based on the semantic region type. Corrections are made. For example, the flat panel area corresponds to the first semantic correction coefficient, the stable load-bearing structure area corresponds to the second semantic correction coefficient, the semantic boundary area corresponds to the third semantic correction coefficient, and the geometrically complex area corresponds to the fourth semantic correction coefficient; wherein, the first semantic correction coefficient is less than the second semantic correction coefficient, the second semantic correction coefficient is less than the third semantic correction coefficient, and the third semantic correction coefficient is less than the fourth semantic correction coefficient.

[0191] For example, the first semantic correction coefficient can be 0.85, the second semantic correction coefficient can be 1.00, the third semantic correction coefficient can be 1.10, and the fourth semantic correction coefficient can be 1.20. Let voxels... The semantic correction coefficient is Then the adaptive significance threshold for voxel v for:

[0192]

[0193] in, The threshold for significance of difference is determined based on the effective confidence level. These are the semantic correction coefficients determined based on the semantic region type.

[0194] Ultimately, voxels Comprehensive difference index Adaptive difference significance threshold Comparison. When > When voxel v is identified as a differential voxel; when ≤ At that time, voxels Voxels were identified as having no valid differences. Therefore, adaptive threshold filtering uses both valid confidence level and semantic region type to determine the adaptive significance threshold for each voxel, and uses this threshold as the criterion for whether to retain the comprehensive difference index.

[0195] Example 2

[0196] Please see Figure 5 One embodiment of the present invention provides: a full-lifecycle traceability system for 3DGS automotive data, the system comprising:

[0197] The multi-stage acquisition and reconstruction module is used to acquire multi-view images of the same vehicle at different stages and reconstruct 3DGS models for each stage.

[0198] The semantic partitioning module is used to divide the Gaussian volume in the 3DGS model at each stage into a subset of rigid structures and a subset of movable parts;

[0199] The two-level registration module is used to perform first-level registration and second-level registration on the 3DGS models of the remaining stages, using the first-stage 3DGS model as the reference model, so that each model to be registered is aligned to the reference model coordinate system.

[0200] The voxelization module is used to create a three-dimensional voxel mesh in the reference model coordinate system, assign the Gaussian volumes of each stage to the corresponding voxels, and aggregate and calculate the features of each voxel to form the voxelized Gaussian field corresponding to each stage.

[0201] The difference detection and archiving module is used to calculate the comprehensive difference index for each voxel of the voxelized Gaussian field corresponding to any two stages, and output the difference results after adaptive threshold filtering to generate a full-cycle traceability digital archive.

[0202] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A full-cycle tracing method for 3DGS automobile data, applied to an automobile maintenance process, the automobile maintenance process being divided into at least two stages in time sequence, characterized in that, The method includes: Multi-view images of the same vehicle at different stages were collected and reconstructed to obtain 3DGS models for each stage; The Gaussian bodies in the 3DGS models at each stage are divided into a rigid structure subset and a movable component subset; Using the 3DGS model of the first stage as the reference model, and the 3DGS models of the remaining stages as the models to be registered, a two-level registration is performed on each model to be registered. The first-level registration is based on solving the global rigid body transformation based on the rigid structure subsets of the reference model and the models to be registered. The second-level registration is based on solving the local rigid body transformation increments based on the subsets of movable parts of the reference model and the models to be registered, so that each model to be registered is aligned to the coordinate system of the reference model. A three-dimensional voxel mesh is established in the coordinate system of the baseline model. The Gaussian volumes in the 3DGS model at each stage are assigned to the corresponding voxels and the features of each voxel are aggregated and calculated to form the voxelized Gaussian field corresponding to each stage. For any two stages, calculate the comprehensive difference index for each voxel of the voxelized Gaussian field, output the difference result after adaptive threshold filtering, and generate a full-cycle traceability digital archive.

2. The full-cycle traceability method for 3D GS automotive data according to claim 1, characterized in that, The process of acquiring multi-view images of the same vehicle at different stages and reconstructing 3DGS models for each stage includes: Sparse reconstruction is performed on multi-view images acquired at each stage to obtain sparse 3D point clouds and camera parameters corresponding to each image. The three-dimensional coordinates of each point in the sparse three-dimensional point cloud are used as the initial position mean of the three-dimensional Gaussian body, and the covariance matrix, opacity and spherical harmonic coefficient of each Gaussian body are initialized to obtain the initial 3DGS model. The initial 3DGS model is rendered into two-dimensional images from each training perspective using differentiable rendering. The loss between the rendered image and the corresponding acquired image is calculated, and the gradient is backpropagated to update the parameters of each Gaussian body. Adaptive density control is performed on the Gaussian volume during the optimization process to obtain 3DGS models at each stage.

3. The full-cycle traceability method for 3D GS automotive data according to claim 1, wherein, The process of dividing the Gaussian bodies in the 3DGS model at each stage into a rigid structure subset and a movable component subset includes: For the multi-view images used for reconstruction at each stage, semantic labels of vehicle components are predicted pixel by pixel using a semantic segmentation model; The mean position of each Gaussian body in the 3DGS model at each stage is projected onto each semantically segmented image through the corresponding camera parameters to obtain the semantic label at each projection position. The semantic label of each Gaussian body is determined by multi-view voting. Based on the semantic tags, Gaussian bodies are categorized into either the rigid structure subset or the movable component subset. The rigid structure subset includes Gaussian bodies corresponding to load-bearing structural components whose geometry remains unchanged throughout each maintenance stage, while the movable component subset includes Gaussian bodies corresponding to components with independent degrees of freedom of motion.

4. The full-cycle traceability method for 3D GS automotive data according to claim 1, wherein, The first-level registration solves the global rigid body transformation based on the rigid structure subsets of the reference model and the model to be registered, including: Voxel downsampling is performed on Gaussian bodies within the rigid structure subsets of the baseline model and the model to be registered, based on their spatial positions. The average position of the Gaussian body with the highest opacity within each voxel is selected as the feature point, forming the feature point set of the baseline model and the feature point set of the model to be registered. Calculate local geometric feature descriptors for each feature point in the two feature point sets, establish the correspondence between the two feature point sets through descriptor matching, and remove outliers; Based on the correspondence after removing outliers, the opacity of the Gaussian body to which the corresponding feature point belongs is used as the weight to solve the weighted optimal global rigid body transformation. The coordinate transformation of all Gaussian bodies of the model to be registered is performed according to the global rigid body transformation to generate a coarse registration model.

5. The full-lifecycle traceability method for 3DGS vehicle data according to claim 4, characterized in that, The second-level registration is based on solving for the local rigid body transformation increments of each movable component subset of the reference model and the model to be registered, including: The mean position values ​​of Gaussian bodies within each subset of movable parts in the coarse registration model and the baseline model are extracted as point sets respectively; For each movable part, the iterative nearest point algorithm with opacity as the weight is used to iteratively solve the local rigid body transformation increment for the corresponding point set of the two models. Based on the local rigid body transformation increments, the Gaussian bodies of the corresponding movable component subsets in the coarse registration model are subjected to coordinate transformations to generate a fine registration model aligned to the reference model coordinate system.

6. The full-lifecycle traceability method for 3DGS vehicle data according to claim 5, characterized in that, After generating the fine registration model, the process also includes: Extract the mean position values ​​of Gaussian bodies within the rigid structure subsets of the fine registration model and the baseline model respectively, search for the nearest neighbor mean position value in the baseline model for each mean position value in the fine registration model, and establish a point-to-point correspondence between the rigid structure subsets; The root mean square value of the distance between all point pairs is calculated as the registration residual. When the registration residual exceeds the registration anomaly threshold, the registration result of the model to be registered is determined to be abnormal and an alarm is triggered.

7. The full-lifecycle traceability method for 3DGS vehicle data according to claim 1, characterized in that, The process of assigning Gaussian volumes in the 3DGS model at each stage to corresponding voxels and aggregating and calculating the features of each voxel to form a voxelized Gaussian field corresponding to each stage includes: Calculate the joint bounding box of the average position of all Gaussian bodies in the 3DGS model at each stage, and establish a three-dimensional voxel mesh by uniformly dividing along the three coordinate axes with a fixed step size. Then, assign each Gaussian body to the voxel according to its average position. The geometric and appearance features of Gaussian volumes within each voxel are aggregated and calculated as the attributes of that voxel. The geometric features include Gaussian volume density, weighted centroid with opacity as the weight, average opacity, and geometric dispersion. The appearance features are the weighted mean of low-order spherical harmonic coefficients with opacity as the weight. The reconstruction confidence score of each voxel is calculated based on the Gaussian volume density, average opacity, and geometric dispersion of each voxel as the attribute of that voxel. The higher the Gaussian volume density, the higher the average opacity, and the lower the geometric dispersion, the higher the reconstruction confidence. The three-dimensional voxel mesh and the geometric features, appearance features and reconstruction confidence scores of each voxel together constitute the voxelized Gaussian field corresponding to each stage.

8. The full-lifecycle traceability method for 3DGS vehicle data according to claim 7, characterized in that, The method involves calculating a comprehensive difference index for each voxel of the voxelized Gaussian field corresponding to any two stages, filtering the results using an adaptive threshold, and then outputting the difference results to generate a full-cycle traceability digital archive, including: For any two stages, in the voxelized Gaussian field corresponding to each voxel, which is not empty in both stages, calculate the geometric difference index and the appearance difference index on a voxel-by-voxel basis. The geometric difference index is calculated based on the difference between the weighted centroids and the average opacity of the voxels corresponding to the two stages, while the appearance difference index is calculated based on the Euclidean distance between the weighted mean of the low-order spherical harmonic coefficients of the voxels corresponding to the two stages. The geometric and appearance difference indices of each voxel are weighted and summed to obtain a comprehensive difference index. After adaptive threshold filtering, voxels with comprehensive difference indices exceeding the threshold are retained as difference voxels. The difference information of each difference voxel is quantitatively output to generate a full-cycle traceability digital archive.

9. The full-lifecycle traceability method for 3DGS vehicle data according to claim 8, characterized in that, The adaptive threshold filtering includes: For each voxel, the lowest value of the reconstruction confidence score in the two stages is taken as the effective confidence score of that voxel. The difference significance threshold of that voxel is determined based on the effective confidence score. The difference significance threshold is inversely correlated with the effective confidence score. The semantic region type of each voxel is determined based on the semantic label of the Gaussian body within each voxel, and the significance threshold of the difference is then corrected.

10. A full-lifecycle traceability system for 3DGS vehicle data, used to implement the full-lifecycle traceability method for 3DGS vehicle data as described in any one of claims 1 to 9, characterized in that, The system includes: The multi-stage acquisition and reconstruction module is used to acquire multi-view images of the same vehicle at different stages and reconstruct 3DGS models for each stage. The semantic partitioning module is used to divide the Gaussian volume in the 3DGS model at each stage into a subset of rigid structures and a subset of movable parts; The two-level registration module is used to perform first-level registration and second-level registration on the 3DGS models of the remaining stages, using the first-stage 3DGS model as the reference model, so that each model to be registered is aligned to the reference model coordinate system. The voxelization module is used to create a three-dimensional voxel mesh in the reference model coordinate system, assign the Gaussian volumes of each stage to the corresponding voxels, and aggregate and calculate the features of each voxel to form the voxelized Gaussian field corresponding to each stage. The difference detection and archiving module is used to calculate the comprehensive difference index for each voxel of the voxelized Gaussian field corresponding to any two stages, and output the difference results after adaptive threshold filtering to generate a full-cycle traceability digital archive.

Images