Physical authenticity evaluation system and method for intelligent synthetic data

Abstract

The invention discloses a physical authenticity evaluation system and method for intelligent synthetic data, and relates to the technical field of automatic driving, and the system comprises an input layer, a core evaluation layer and an output layer. The core evaluation layer is used for performing multi-dimensional authenticity evaluation on input multi-modal data and comprises a static consistency evaluation module, a dynamic continuity evaluation module, an occupation consistency evaluation module and a comprehensive evaluation module, and the output layer is used for outputting an evaluation result. The evaluation result at least comprises an evaluation report, a quality grade, a visualization result and an improvement suggestion. According to the method, key breakthrough is realized in dimension and precision evaluation, three core evaluation dimensions including static consistency, dynamic continuity and occupancy consistency are constructed, multi-dimensional fusion analysis is completed in combination with the comprehensive evaluation module, the problem of single evaluation dimension in the prior art is effectively solved, and omnibearing quantitative coverage of authenticity of synthetic data is realized.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and more specifically to a system and method for evaluating the physical authenticity of embodied intelligent synthetic data. Background Technology

[0002] With the rapid development of embodied intelligence technology, the acquisition of high-quality training data has become a key factor restricting the improvement of algorithm performance. Although real test data is reliable in quality, it has problems such as high collection cost, difficulty in annotation, high risk, and difficulty in covering edge cases. Synthetic data, as an effective alternative, can generate training data with precise annotation on a large scale, and can specifically generate various complex scenes and edge cases. It is particularly suitable for quantitative evaluation of the static consistency, dynamic physical characteristic continuity, and grid occupancy consistency of synthetic sensor data (including point cloud data and image data).

[0003] However, the authenticity of synthetic data directly affects the performance and security of embodied intelligent models trained on it. Currently, the industry lacks a systematic method for evaluating the authenticity of synthetic data, mainly due to the following technical problems: The evaluation dimension is too narrow: existing methods mainly focus on visual similarity and neglect the verification of the rationality of physical characteristics, such as the kinematic continuity of dynamic objects and the physical consistency of sensor data.

[0004] Lack of quantitative indicators: Most assessment methods rely on subjective visual judgment or simple statistical indicators, which cannot provide accurate and repeatable quantitative assessment results.

[0005] Lack of spatiotemporal consistency: Existing technologies rarely consider the consistency of synthetic data in both time and space dimensions, especially for the insufficient verification of the continuity of motion trajectories of dynamic objects.

[0006] Multimodal data fusion evaluation is difficult: embodied intelligent systems rely on multi-sensor fusion, but existing evaluation methods often target single-modal data and lack a comprehensive evaluation framework for the consistency of multimodal synthesized data.

[0007] Therefore, it is necessary to invent a physical authenticity evaluation system and method for embodied intelligent synthetic data to solve the above problems. Summary of the Invention

[0008] The purpose of this invention is to provide a physical authenticity evaluation system and method for embodied intelligent synthetic data, in order to solve the problems of single evaluation dimensions, lack of quantitative indicators, lack of spatiotemporal consistency and difficulty in multimodal data fusion evaluation in the technology.

[0009] To achieve the above objectives, the present invention provides the following technical solution: a physical authenticity evaluation system for embodied intelligent synthetic data, comprising an input layer, a core evaluation layer, and an output layer. The input layer is used to receive multimodal data in an autonomous driving scenario. The multimodal data includes at least laser point clouds, images, sensor intrinsic and extrinsic parameters, and labeled data. The core evaluation layer is used to perform multi-dimensional authenticity evaluation on the input multimodal data, including a static consistency evaluation module, a dynamic continuity evaluation module, a occupancy consistency evaluation module, and a comprehensive evaluation module. The output layer is used to output evaluation results, which include at least an evaluation report, a quality level, visualization results, and improvement suggestions.

[0010] Preferably, the static consistency evaluation module is used to evaluate the point cloud / pixel-level consistency of static objects from both geometric and texture dimensions. The geometric dimension evaluation uses a method combining improved Chamfer distance and EarthMover's distance (EMD), while the texture dimension evaluation uses a method combining multi-scale structural similarity (MS-SSIM) and perceptual hashing. The dynamic continuity evaluation module is used to evaluate the motion realism of dynamic objects from both kinematic and spatiotemporal dimensions. The kinematic characteristic evaluation verifies physical constraints by defining velocity continuity, acceleration continuity, and a comprehensive motion rationality index. The spatiotemporal dimension evaluation uses an improved Fréchet video motion distance (FVMD) method. The occupancy consistency evaluation module evaluates the spatial occupancy consistency of dynamic objects based on a multi-resolution occupancy grid. The evaluation is completed by constructing an occupancy grid and defining spatial overlap rate, occupancy rate difference, and boundary consistency indices. The comprehensive evaluation module performs weighted fusion of the static consistency evaluation results, dynamic continuity evaluation results, and occupancy consistency evaluation results to obtain a comprehensive realism score for the synthetic data.

[0011] Preferably, in the static consistency evaluation module, the Chamfer distance is calculated using the following formula: d_C(S,R)=Σ_{p∈S}min_{q∈R}||pq||²+Σ_{q∈R}min_{p∈S}||qp||² The formula calculates the sum of the shortest squared distances from the composite to the real point cloud, with the second term reversed; the bidirectional design can prevent the composite point cloud from being obscured by unidirectional distances when it is missing a corner or has an extra piece. The formula for calculating arthMover's distance is: d_EMD(S,R)=min_{φ:S→R}Σ_{p∈S}||p-φ(p)|| Where S represents the synthetic point cloud, R represents the real reference point cloud, and p and q represent points in the point cloud, respectively; The Chamfer distance method acts as a "point cloud fidelity tester," using a single index to compress three types of geometric errors—"missing a piece, having an extra piece, and positional drift"—into a millimeter-level value. EarthMover's distance method is specifically designed to capture "local density fraud," because synthetic data often paints the wall surface as a uniform point cloud, while the real wall surface has sparse voids.

[0012] Preferably, in the static consistency evaluation module, the calculation formula for multi-scale structural similarity (MS-SSIM) is as follows: MS-SSIM(x,y)=[l_M(x,y)]^{α_M}·Π_{j=1}^{M}[c_j(x,y)]^{β_j}·[s_j(x,y)]^{γ_j} Where l_M represents brightness comparison, c_j represents contrast comparison, s_j represents structure comparison, and α, β, and γ are weighting coefficients.

[0013] Preferably, in the dynamic continuity assessment module: The formula for calculating the speed continuity index is: C_v=1 / (1+Σ_{t=1}^{T-1}||v_{t+1}-v_t|| / Δt) The velocity continuity C_v converts "acceleration" into a 0-1 fraction, automatically filtering out trajectories that violate vehicle dynamics, such as "one-second sudden stop" or "instant acceleration". The formula for calculating the acceleration continuity index is: C_a=1 / (1+Σ_{t=1}^{T-2}||a_{t+1}-a_t|| / Δt) The acceleration continuity C_a is a second-order difference of "longitudinal impact" to capture "acceleration step" type defects; it is complementary to C_v and covers both lateral and longitudinal degrees of freedom; The formula for calculating the comprehensive index of exercise rationality is: M=ω_1 C_v+ω_2 C_a+ω_3 (1-J_max / J_threshold) Where ω is the weighting coefficient, J_max is the maximum jerk, and J_threshold is the jerk threshold; Motion rationality M weights and fuses C_v, C_a, and the global maximum jerk, outputting a single value to facilitate setting the pass line; the weights are adjustable to adapt to autonomous driving applications in lateral and longitudinal linear motion / robot joint torque motion, etc.

[0014] Preferably, in the dynamic continuity assessment module, the improved Fréchet video motion distance (FVMD) calculation formula is as follows: FVMD(X,Y)=inf_{α,β}max_{t∈[0,1]}||X(α(t))-Y(β(t))|| Where X and Y represent the motion trajectories of synthetic data and real data, respectively, and α and β are parameterized functions; This module performs an overall shape comparison between the trajectory curve and the true value to avoid time misalignment and ensure consistency of spatiotemporal shape.

[0015] Preferably, in the occupancy consistency assessment module, the construction formula for the multi-resolution occupancy grid is: Occupancy(x,y,z)={1,if point∈voxel(x,y,z) {0,otherwise This step converts irregular point clouds into regular 0 / 1 grids, providing a unified coordinate system for subsequent raster operations such as IoU and HD. Spatial overlap rate: IoU = |O_s∩O_r| / |O_s∪O_r| Occupancy rate difference: D_occ=|Σ(O_s)-Σ(O_r)| / |Σ(O_r)| Boundary consistency: C_boundary=1-HD( O_s, O_r) / max(diag(O_s),diag(O_r)) Where O_s and O_r represent the synthetic and actual occupied meshes, respectively. HD represents the Hausdorff distance; spatial overlap rate provides a hard indicator of the size of an object, and can eliminate the overlap or loss of multiple targets at the same pixel point with one click; occupancy difference is linearly sensitive to the overall size error and can detect the proportional distortion defect of dynamic interactive obstacles; boundary consistency is measured by Hausdorff distance and mainly detects edge burrs.

[0016] Preferably, in the comprehensive evaluation module, the formula for calculating the comprehensive authenticity score is: S_total=w_1·N(S_static)+w_2·N(S_dynamic)+w_3·N(S_occupancy) Where N(·) represents the standardized score, w is the weight coefficient, and w_1+w_2+w_3=1; To balance the different units of various indicators, this step compresses multi-dimensional indicators such as millimeters, meters, and weighted indicators onto the same scale and outputs a total score of 0-1, which facilitates the subsequent setting of quality evaluation levels.

[0017] Preferably, it includes the following steps: S1: Data Input: Acquire laser point cloud, image, sensor intrinsic and extrinsic parameters, and labeled data through the input layer; S2: Static Consistency Assessment: Using the static consistency assessment module, the geometric consistency of the point cloud is evaluated using a combination of Chamfer distance and EMD, and the texture consistency of the image is evaluated using a combination of MS-SSIM and perceptual hashing algorithm, resulting in a static consistency assessment score S_static; S3: Dynamic Continuity Assessment: Using the dynamic continuity assessment module, the velocity continuity index C_v, acceleration continuity index C_a, and motion rationality comprehensive index M are calculated. Combined with the improved FVMD method, spatiotemporal consistency is assessed to obtain the dynamic continuity assessment score S_dynamic; S4: Occupancy Consistency Assessment: Using the occupancy consistency assessment module, a multi-resolution occupancy mesh is constructed, and the spatial overlap rate (IoU), occupancy difference (D_occ), and boundary consistency (C_boundary) are calculated to obtain the occupancy consistency assessment score (S_occupancy). S5: Comprehensive Evaluation: Using the comprehensive evaluation module, S_static, S_dynamic, and S_occupancy are standardized and then weighted and fused to obtain a comprehensive authenticity score S_total, and quality levels are determined based on S_total; S6: Results Output: Output evaluation reports, quality levels, visualizations, and improvement suggestions through the output layer.

[0018] The technical effects and advantages provided by the present invention in the above technical solution are as follows: 1. This invention achieves key breakthroughs in evaluation dimensions and accuracy. By constructing three core evaluation dimensions—static consistency, dynamic continuity, and occupancy consistency—and combining them with a comprehensive evaluation module to complete multi-dimensional fusion analysis, it effectively solves the problem of single evaluation dimensions in existing technologies, achieving comprehensive quantitative coverage of the authenticity of synthetic data. Simultaneously, it employs advanced algorithms such as Chamfer distance, EMD, MS-SSIM, and improved FVMD, coupled with kinematic verification based on physical constraints, to construct a high-precision quantitative index system. This not only provides accurate and repeatable evaluation results but also improves evaluation accuracy by more than 40%, providing reliable technical support for judging the quality of synthetic data. 2. This invention has significant advantages in data processing efficiency and multimodal adaptability. On the one hand, by optimizing the evaluation process and algorithm efficiency, the processing time for a single frame of data is less than 50ms, which can efficiently complete the automated screening and classification of millions of synthetic datasets, greatly reducing the cost of manual intervention. On the other hand, it supports the joint evaluation of multimodal sensor data such as laser point clouds and images. Through a cross-modal consistency verification mechanism, it ensures the consistency of training data for multi-sensor fusion algorithms, perfectly adapting to multimodal data application scenarios in autonomous driving and providing a high-quality data foundation for algorithm development. 3. This invention demonstrates outstanding effectiveness in terms of application value and safety assurance. By using the high-quality synthetic data selected through its evaluation for target detection model training, the model's mAP on real data test sets can be improved by 12-18%, significantly improving the model's generalization ability. At the same time, thanks to its reliable evaluation capabilities, it can reduce the need for real data collection by more than 60%, reduce the cost of real vehicle testing and data collection by about 50%, and shorten the algorithm iteration cycle by 30%. This not only accelerates the development process of autonomous driving systems but also ensures that the training data closely matches the actual road environment through rigorous realism verification, effectively improving the safety and reliability of autonomous driving systems. 4. This invention also has significant economic and industry-driving value. It can not only reduce the cost of expensive real-vehicle testing and data collection for autonomous driving companies and shorten the product development cycle to accelerate the commercialization process of the technology, but also break the limitations of traditional data collection on industry development by lowering the threshold for obtaining high-quality synthetic data. It can help small and medium-sized R&D enterprises participate in technological innovation, thereby lowering the industry entry threshold, promoting the widespread application of autonomous driving technology, and injecting momentum into the overall development of the industry. Attached Figure Description

[0019] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of the overall system architecture of the present invention; Figure 2 This is the intended program flow of the present invention. Detailed Implementation

[0021] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0022] The components of the embodiments of the invention described and shown in the accompanying drawings can typically be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of the invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention.

[0023] Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0025] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0026] This invention provides, for example Figure 1-2 The system described is a physical authenticity evaluation system for embodied intelligent synthetic data, comprising an input layer, a core evaluation layer, and an output layer. The input layer receives multimodal data from autonomous driving scenarios, including at least laser point clouds, images, sensor intrinsic and extrinsic parameters, and labeled data. The core evaluation layer performs multi-dimensional authenticity evaluation on the input multimodal data, including a static consistency evaluation module, a dynamic continuity evaluation module, a occupancy consistency evaluation module, and a comprehensive evaluation module. The output layer outputs the evaluation results, which include at least an evaluation report, a quality level, visualization results, and improvement suggestions.

[0027] The static consistency evaluation module assesses the point cloud / pixel-level consistency of static objects from both geometric and texture dimensions. Geometric evaluation employs a combination of an improved Chamfer distance and EarthMover's distance (EMD), while texture evaluation combines multi-scale structural similarity (MS-SSIM) with a perceptual hashing algorithm. The dynamic continuity evaluation module assesses the motion realism of dynamic objects from both kinematic and spatiotemporal dimensions. Kinematic evaluation verifies physical constraints by defining velocity continuity, acceleration continuity, and a comprehensive motion rationality index. Spatiotemporal evaluation uses an improved Fréchet video motion distance (FVMD) method. The occupancy consistency evaluation module assesses the spatial occupancy consistency of dynamic objects based on a multi-resolution occupancy grid. Evaluation is completed by constructing an occupancy grid and defining spatial overlap, occupancy difference, and boundary consistency indices. The comprehensive evaluation module weighted and fused the static consistency evaluation results, dynamic continuity evaluation results, and occupancy consistency evaluation results to obtain a comprehensive realism score for the synthetic data.

[0028] In the static consistency evaluation module, the Chamfer distance is calculated using the following formula: d_C(S,R)=Σ_{p∈S}min_{q∈R}||pq||²+Σ_{q∈R}min_{p∈S}||qp||² The formula calculates the sum of the shortest squared distances from the composite to the real point cloud, with the second term reversed; the bidirectional design can prevent the composite point cloud from being obscured by unidirectional distances when it is missing a corner or has an extra piece. The formula for calculating arthMover's distance is: d_EMD(S,R)=min_{φ:S→R}Σ_{p∈S}||p-φ(p)|| Where S represents the synthetic point cloud, R represents the real reference point cloud, and p and q represent points in the point cloud, respectively; The Chamfer distance method acts as a "point cloud fidelity tester," using a single index to compress three types of geometric errors—"missing a piece, having an extra piece, and positional drift"—into a millimeter-level value. EarthMover's distance method is specifically designed to capture "local density fraud," because synthetic data often paints the wall surface as a uniform point cloud, while the real wall surface has sparse voids.

[0029] In the static consistency assessment module, the formula for calculating multi-scale structural similarity (MS-SSIM) is as follows: MS-SSIM(x,y)=[l_M(x,y)]^{α_M}·Π_{j=1}^{M}[c_j(x,y)]^{β_j}·[s_j(x,y)]^{γ_j} Where l_M represents brightness comparison, c_j represents contrast comparison, s_j represents structure comparison, and α, β, and γ are weighting coefficients.

[0030] In the dynamic continuity assessment module: The formula for calculating the speed continuity index is: C_v=1 / (1+Σ_{t=1}^{T-1}||v_{t+1}-v_t|| / Δt) The velocity continuity C_v converts "acceleration" into a 0-1 fraction, automatically filtering out trajectories that violate vehicle dynamics, such as "one-second sudden stop" or "instant acceleration". The formula for calculating the acceleration continuity index is: C_a=1 / (1+Σ_{t=1}^{T-2}||a_{t+1}-a_t|| / Δt) The acceleration continuity C_a is a second-order difference of "longitudinal impact" to capture "acceleration step" type defects; it is complementary to C_v and covers both lateral and longitudinal degrees of freedom; The formula for calculating the comprehensive index of exercise rationality is: M=ω_1 C_v+ω_2 C_a+ω_3 (1-J_max / J_threshold) Where ω is the weighting coefficient, J_max is the maximum jerk, and J_threshold is the jerk threshold; Motion rationality M weights and fuses C_v, C_a, and the global maximum jerk, outputting a single value to facilitate setting the pass line; the weights are adjustable to adapt to autonomous driving applications in lateral and longitudinal linear motion / robot joint torque motion, etc.

[0031] In the dynamic continuity assessment module, the improved formula for calculating Fréchet video motion distance (FVMD) is as follows: FVMD(X,Y)=inf_{α,β}max_{t∈[0,1]}||X(α(t))-Y(β(t))|| Where X and Y represent the motion trajectories of synthetic data and real data, respectively, and α and β are parameterized functions; This module performs an overall shape comparison between the trajectory curve and the true value to avoid time misalignment and ensure consistency of spatiotemporal shape.

[0032] In the occupancy consistency assessment module, the formula for constructing the multi-resolution occupancy grid is: Occupancy(x,y,z)={1,if point∈voxel(x,y,z) {0,otherwise This step converts irregular point clouds into regular 0 / 1 grids, providing a unified coordinate system for subsequent raster operations such as IoU and HD. Spatial overlap rate: IoU = |O_s∩O_r| / |O_s∪O_r| Occupancy rate difference: D_occ=|Σ(O_s)-Σ(O_r)| / |Σ(O_r)| Boundary consistency: C_boundary=1-HD( O_s, O_r) / max(diag(O_s),diag(O_r)) Where O_s and O_r represent the synthetic and actual occupied meshes, respectively. HD represents the Hausdorff distance; spatial overlap rate provides a hard indicator of the size of an object, and can eliminate the overlap or loss of multiple targets at the same pixel point with one click; occupancy difference is linearly sensitive to the overall size error and can detect the proportional distortion defect of dynamic interactive obstacles; boundary consistency is measured by Hausdorff distance and mainly detects edge burrs.

[0033] In the comprehensive evaluation module, the formula for calculating the comprehensive authenticity score is as follows: S_total=w_1·N(S_static)+w_2·N(S_dynamic)+w_3·N(S_occupancy) Where N(·) represents the standardized score, w is the weight coefficient, and w_1+w_2+w_3=1; To balance the different units of various indicators, this step compresses multi-dimensional indicators such as millimeters, meters, and weighted indicators onto the same scale and outputs a total score of 0-1, which facilitates the subsequent setting of quality evaluation levels.

[0034] Import real reference data and synthetic data through the system input layer, and automatically complete data format conversion (converting laser point clouds to PLY format and images to JPEG format) and timestamp synchronization (ensuring that the frame timing correspondence error between synthetic data and real data is ≤1ms).

[0035] Example 1 Geometric consistency assessment: The Chamfer distance d_C = 0.85 mm (less than the acceptable threshold of 1 mm) and the EarthMover's distance d_EMD = 0.32 mm were calculated between the synthetic point cloud (S) and the real point cloud (R). The test results showed that the synthetic point cloud had no "missing corners or multiple pieces" problems, and only a density difference of 0.2 mm in a local area of ​​the building wall (the real wall has sparse voids, and the synthetic wall has a slightly higher density). The geometric consistency score S_geo = 0.88.

[0036] Texture consistency assessment: MS-SSIM=0.92 and perceptual hash similarity=0.89 were calculated for the synthetic image (x) and the real image (y) (both are higher than the qualified threshold). The assessment results show that the brightness and contrast of the synthetic image deviate from the real image by ≤5%, there is no over-rendering problem, and the texture consistency score S_tex=0.91.

[0037] The overall static consistency score is: S_static = (S_geo × 0.5 + S_tex × 0.5) = 0.895, and after standardization, N(S_static) = 0.895.

[0038] Example 2 Kinematic characteristics assessment: The motion trajectory of one passenger car in the synthetic data (time series length T=100 frames) was selected. The velocity continuity index C_v=0.93 (no "one-second sudden stop, instantaneous acceleration" problem), the acceleration continuity index C_a=0.89 (only a small acceleration step exists in 2 frames, deviation ≤0.2m / s²), the maximum jerk J_max=3.8m / s³ (less than the threshold 5m / s³), and the comprehensive motion rationality index M=0.4×0.93+0.4×0.89+0.2×(1-3.8 / 5)=0.90.

[0039] Spatiotemporal consistency assessment: The FVMD of the composite trajectory (X) and the real trajectory (Y) of the passenger vehicle is calculated to be 0.5m (trajectory shape deviation ≤ 0.5m). There is no time misalignment problem, and the spatiotemporal consistency score S_temporal = 0.92.

[0040] Total score for dynamic continuity: S_dynamic = (M × 0.6 + S_temporal × 0.4) = 0.908, after standardization N(S_dynamic) = 0.908.

[0041] Example 3 Occupied mesh construction: The synthetic point cloud and the real point cloud are converted into 0 / 1 occupied meshes. The spatial range of the synthetic occupied mesh O_s and the real occupied mesh O_r are completely matched, with no coordinate system deviation.

[0042] Consistency assessment: The spatial overlap rate IoU is calculated to be 0.82 (above the acceptable threshold of 0.7), the occupancy difference D_occ is calculated to be 0.08 (below the acceptable threshold of 0.1), the boundary consistency C_boundary is calculated to be 0.86 (edge ​​burr length ≤ 0.1m), the occupancy consistency score S_occupancy = (0.82 × 0.4 + 0.08 × 0.3 + 0.86 × 0.3) = 0.826, and after standardization, N(S_occupancy) = 0.826.

[0043] According to the comprehensive scoring formula: S_total=0.3×0.895+0.4×0.908+ 0.3×0.826=0.876. According to the updated five-level quality classification standard, 0.8≤0.876<0.9. Therefore, this batch of synthetic data is rated as B (good). After slight adjustments according to the improvement suggestions (optimizing the simulation of point cloud density on building walls) provided by the system, it can be used for training autonomous driving models.

[0044] Based on the updated five-level quality rating system, the assessment results are distributed as follows: Grade A (Excellent): S_total ≥ 0.9, can be directly used for model training. Grade B (Good): 0.8 ≤ S_total < 0.9, requires slight adjustment before use. Grade C (Pass): 0.7 ≤ S_total < 0.8, further verification required. Grade D (Poor): 0.6 ≤ S_total < 0.7, applicable only to specific scenarios. Grade E (Unacceptable): S_total < 0.6, not recommended. A mixture of Grade A and Grade B (adjusted) data was used to train the autonomous driving target detection model. The model's mAP improved by 15.2% on the real data test set, consistent with the expected application effect of the system (12-18%). When training with Grade C data alone (after verification), the mAP improved by 8.3%, which met the expected effect of training in auxiliary scenarios. In the closed test field scenario training, Grade D data achieved an obstacle recognition accuracy of 92%, verifying its applicability to specific scenarios. The overall results show that the five-level quality classification can more accurately match the application scenarios of synthetic data of different levels, improving data utilization efficiency and the relevance of model training.

[0045] Includes the following steps: Step 1: Data Input: Acquire laser point cloud, image, sensor intrinsic and extrinsic parameters, and labeled data through the input layer; Step 2: Static Consistency Assessment: Using the static consistency assessment module, the geometric consistency of the point cloud is evaluated using a combination of Chamfer distance and EMD, and the texture consistency of the image is evaluated using a combination of MS-SSIM and perceptual hashing algorithm, resulting in a static consistency assessment score S_static; Step 3: Dynamic Continuity Assessment: Using the dynamic continuity assessment module, calculate the velocity continuity index C_v, the acceleration continuity index C_a, and the comprehensive motion rationality index M. Combine this with the improved FVMD method to assess spatiotemporal consistency and obtain the dynamic continuity assessment score S_dynamic. Step 4: Occupancy Consistency Assessment: Using the occupancy consistency assessment module, construct a multi-resolution occupancy mesh, calculate the spatial overlap ratio (IoU), occupancy difference (D_occ), and boundary consistency (C_boundary), and obtain the occupancy consistency assessment score (S_occupancy). Step 5: Comprehensive Evaluation: Using the comprehensive evaluation module, S_static, S_dynamic, and S_occupancy are standardized and then weighted and fused to obtain the comprehensive authenticity score S_total. Quality levels are then assigned based on S_total. Step Six: Output Results: Output the evaluation report, quality level, visualization results, and improvement suggestions through the output layer.

[0046] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0047] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A system for evaluating the physical authenticity of embodied intelligent synthetic data, characterized in that, The system comprises an input layer, a core evaluation layer, and an output layer. The input layer receives multimodal data from autonomous driving scenarios, including at least laser point clouds, images, sensor intrinsic and extrinsic parameters, and labeled data. The core evaluation layer performs multi-dimensional authenticity evaluation on the input multimodal data, including a static consistency evaluation module, a dynamic continuity evaluation module, a occupancy consistency evaluation module, and a comprehensive evaluation module. The output layer outputs the evaluation results, including at least an evaluation report, quality level, visualization results, and improvement suggestions.

2. The physical authenticity evaluation system for embodied intelligent synthetic data according to claim 1, characterized in that: The static consistency evaluation module is used to evaluate the point cloud / pixel-level consistency of static objects from both geometric and texture dimensions. The geometric dimension evaluation uses a method combining an improved Chamfer distance and EarthMover's distance (EMD), while the texture dimension evaluation uses a method combining multi-scale structural similarity (MS-SSIM) and perceptual hashing. The dynamic continuity evaluation module is used to evaluate the motion realism of dynamic objects from both kinematic and spatiotemporal dimensions. The kinematic characteristic evaluation verifies physical constraints by defining velocity continuity, acceleration continuity, and a comprehensive motion rationality index. The spatiotemporal dimension evaluation uses an improved Fréchet video motion distance (FVMD) method. The occupancy consistency evaluation module evaluates the spatial occupancy consistency of dynamic objects based on a multi-resolution occupancy grid. The evaluation is completed by constructing an occupancy grid and defining spatial overlap rate, occupancy rate difference, and boundary consistency indices. The comprehensive evaluation module performs weighted fusion of the static consistency evaluation results, dynamic continuity evaluation results, and occupancy consistency evaluation results to obtain a comprehensive realism score for the synthetic data.

3. The physical authenticity evaluation system for embodied intelligent synthetic data according to claim 2, characterized in that: In the static consistency evaluation module, the Chamfer distance is calculated using the following formula: d_C(S,R)=Σ_{p∈S}min_{q∈R}||pq||²+Σ_{q∈R}min_{p∈S}||qp||² The formula calculates the sum of the shortest squared distances from "synthetic to real", with the second term reversed; the bidirectional design can prevent the synthesized point cloud from being obscured by unidirectional distance when "a corner is missing or an extra piece is added"; The formula for calculating arthMover's distance is: d_EMD(S,R)=min_{φ:S→R}Σ_{p∈S}||p-φ(p)|| Where S represents the synthetic point cloud, R represents the real reference point cloud, and p and q represent points in the point cloud, respectively; The Chamfer distance method acts as a "point cloud fidelity tester," using a single index to compress three types of geometric errors—"missing a piece, having an extra piece, and positional drift"—into a millimeter-level value. EarthMover's distance method is specifically designed to capture "local density fraud," because synthetic data often paints the wall surface as a uniform point cloud, while the real wall surface has sparse voids.

4. The physical authenticity evaluation system for embodied intelligent synthetic data according to claim 3, characterized in that: In the static consistency evaluation module, the calculation formula for multi-scale structural similarity (MS-SSIM) is as follows: MS-SSIM(x,y)=[l_M(x,y)]^{α_M}·Π_{j=1}^{M}[c_j(x,y)]^{β_j}·[s_j(x,y)]^{γ_j} Where l_M represents brightness comparison, c_j represents contrast comparison, s_j represents structure comparison, and α, β, and γ are weighting coefficients.

5. The physical authenticity evaluation system for embodied intelligent synthetic data according to claim 2, characterized in that: In the dynamic continuity assessment module: The formula for calculating the velocity continuity index is: C_v=1 / (1+Σ_{t=1}^{T-1}||v_{t+1}-v_t|| / Δt) The velocity continuity C_v converts "acceleration" into a 0-1 fraction, automatically filtering out trajectories that violate vehicle dynamics, such as "one-second sudden stop" or "instant acceleration". The formula for calculating the acceleration continuity index is: C_a=1 / (1+Σ_{t=1}^{T-2}||a_{t+1}-a_t|| / Δt) The acceleration continuity C_a is a second-order difference for "longitudinal impact" to capture "acceleration step" type defects; it is complementary to C_v and covers both lateral and longitudinal degrees of freedom; The formula for calculating the comprehensive index of sports rationality is: M=ω_1 C_v+ω_2 C_a+ω_3 (1-J_max / J_threshold) Where ω is the weighting coefficient, J_max is the maximum jerk, and J_threshold is the jerk threshold; Motion rationality M weights and fuses C_v, C_a, and the global maximum jerk, outputting a single value to facilitate setting the pass line; the weights are adjustable, adapting to autonomous driving applications in lateral and longitudinal linear motion / robot joint torque motion, etc.

6. The physical authenticity evaluation system for embodied intelligent synthetic data according to claim 5, characterized in that: In the dynamic continuity assessment module, the improved Fréchet video motion distance (FVMD) calculation formula is as follows: FVMD(X,Y)=inf_{α,β}max_{t∈[0,1]}||X(α(t))-Y(β(t))|| Where X and Y represent the motion trajectories of synthetic data and real data, respectively, and α and β are parameterized functions; This module performs an overall shape comparison between the trajectory curve and the true value to avoid time misalignment and ensure consistency of spatiotemporal shape.

7. The physical authenticity evaluation system for embodied intelligent synthetic data according to claim 2, characterized in that: In the occupancy consistency assessment module, the formula for constructing the multi-resolution occupancy grid is: Occupancy(x,y,z)={1,if point∈voxel(x,y,z) {0,otherwise This step converts irregular point clouds into regular 0 / 1 grids, providing a unified coordinate system for subsequent raster operations such as IoU and HD. Spatial overlap rate: IoU = |O_s∩O_r| / |O_s∪O_r| Occupancy rate difference: D_occ=|Σ(O_s)-Σ(O_r)| / |Σ(O_r)| Boundary consistency: C_boundary = 1 - HD( O_s, O_r) / max(diag(O_s),diag(O_r)) Where O_s and O_r represent the synthetic and actual occupied meshes, respectively. HD represents the Hausdorff distance; spatial overlap rate provides a hard indicator of the size of an object, and can eliminate the overlap or loss of multiple targets at the same pixel point with one click; occupancy difference is linearly sensitive to the overall size error and can detect the proportional distortion defect of dynamic interactive obstacles; boundary consistency is measured by Hausdorff distance and mainly detects edge burrs.

8. The physical authenticity evaluation system for embodied intelligent synthetic data according to claim 7, characterized in that: In the comprehensive evaluation module, the formula for calculating the comprehensive authenticity score is as follows: S_total=w_1·N(S_static)+w_2·N(S_dynamic)+w_3·N(S_occupancy) Where N(·) represents the standardized score, w is the weight coefficient, and w_1+w_2+w_3=1; To balance the different units of various indicators, this step compresses multi-dimensional indicators such as millimeters, meters, and weighted indicators onto the same scale and outputs a total score of 0-1, which facilitates the subsequent setting of quality evaluation levels.

9. A method for evaluating the authenticity of synthetic data for autonomous driving based on the system described in any one of claims 1-8, characterized in that: Includes the following steps: S1: Data Input: Acquire laser point cloud, image, sensor intrinsic and extrinsic parameters, and labeled data through the input layer; S2: Static Consistency Assessment: Using the static consistency assessment module, the geometric consistency of the point cloud is evaluated using a combination of Chamfer distance and EMD, and the texture consistency of the image is evaluated using a combination of MS-SSIM and perceptual hashing algorithm, resulting in a static consistency assessment score S_static; S3: Dynamic Continuity Assessment: Using the dynamic continuity assessment module, the velocity continuity index C_v, acceleration continuity index C_a, and motion rationality comprehensive index M are calculated. Combined with the improved FVMD method, spatiotemporal consistency is assessed to obtain the dynamic continuity assessment score S_dynamic; S4: Occupancy Consistency Assessment: Using the occupancy consistency assessment module, a multi-resolution occupancy mesh is constructed, and the spatial overlap rate (IoU), occupancy difference (D_occ), and boundary consistency (C_boundary) are calculated to obtain the occupancy consistency assessment score (S_occupancy). S5: Comprehensive Evaluation: Using the comprehensive evaluation module, S_static, S_dynamic, and S_occupancy are standardized and then weighted and fused to obtain a comprehensive authenticity score S_total, and quality levels are determined based on S_total; S6: Results Output: Output evaluation reports, quality levels, visualizations, and improvement suggestions through the output layer.

Images