A method and system for multimodal data quality assessment and optimization in vehicle-to-everything (V2X) networks

Abstract

This invention discloses a method and system for multimodal data quality assessment and optimization in the field of vehicle-to-everything (V2X) technology. The method includes calculating corresponding quality scores and comprehensive scores for preprocessed multimodal data using five-dimensional factors; identifying the multimodal data to be optimized based on the comprehensive scores; obtaining all quality scores and corresponding weights of the multimodal data to be optimized and inputting them into a factor-aware optimization network to output optimization strategy suggestions; recalculating the quality scores of the optimized and unoptimized multimodal data again using the five-dimensional factors; and determining whether convergence conditions are met based on the change in comprehensive scores and marginal returns. This invention achieves efficient assessment and closed-loop optimization of V2X datasets through quality factor calculation, dynamic weighted scoring, data completion and convergence control strategies based on strategy-aware optimization.

Description

Technical Field

[0001] This invention relates to the field of data quality assessment and optimization technology, and in particular to a method and system for multimodal data quality assessment and optimization in the Internet of Vehicles. Background Technology

[0002] With the rapid development of vehicle-to-everything (V2X) technology, massive amounts of perception data have become a core resource for training perception and decision-making models. The quality of this data directly determines the upper limit of model performance. Especially in complex environments such as low light, occlusion, and extreme weather, low-quality data can severely interfere with model training and deployment. Therefore, evaluating and optimizing V2X data to obtain high-quality data is particularly important.

[0003] Most existing general data quality assessment and optimization methods are geared towards single modalities or static rules. These methods lack unified modeling for multimodal data quality evaluation, making them unsuitable for collaborative quality assessment of multimodal data such as images, point clouds, and labels in connected vehicle scenarios. They fail to comprehensively evaluate the overall quality of multimodal data, leading to inaccurate data evaluation results. In existing scoring methods, most weights are fixed values ​​or subjectively set. However, research has found that different data stages and collection tasks have different focuses on quality factors, and traditional scoring methods cannot dynamically adjust weights or adapt to changes.

[0004] Furthermore, most existing optimization schemes rely on empirical threshold judgments, resulting in fixed optimization strategies that are difficult to combine and cannot be personalized based on the actual defects of the samples. In addition, the existing optimization process lacks a feedback judgment mechanism, which can easily lead to problems such as duplicate optimization, ineffective enhancement, and waste of resources. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a method and system for multimodal data quality assessment and optimization in the Internet of Vehicles (IoV). This invention achieves efficient assessment and closed-loop optimization of IoV datasets through quality factor calculation, dynamic weighted scoring, and data completion and convergence control strategies based on policy awareness optimization.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] In a first aspect, the present invention provides a method for multimodal data quality assessment and optimization in vehicle-to-everything (V2X) networks, comprising:

[0008] Acquire multimodal data and preprocess the multimodal data;

[0009] The preprocessed multimodal data are used to calculate the corresponding quality scores through five dimensions and factors. The quality scores are then weighted and fused with the corresponding weights to obtain a comprehensive score. Based on the comprehensive score, the multimodal data that needs to be optimized is obtained.

[0010] Obtain all quality scores and corresponding weights of the multimodal data to be optimized, concatenate them into a joint input vector, input it into the trained factor-aware optimization network, output optimization strategy suggestions, optimize the multimodal data to be optimized based on the optimization strategy suggestions, and calculate the corresponding quality scores again through the five-dimensional factors based on the optimized multimodal data and the unoptimized multimodal data.

[0011] Obtain the comprehensive score before and after optimization, obtain the change in comprehensive score and marginal benefit, and determine whether the convergence condition is met. If the convergence condition is met, the optimization is completed; otherwise, continue the iterative optimization.

[0012] As a further technical solution, the multimodal data includes image data, lidar point cloud, motion state data, and inertial navigation data; the preprocessing method is as follows: first, the multimodal data is time-synchronized and format-converted to a standard data structure; then, abnormal frames, empty frames, and invalid labeled multimodal data are removed; finally, metadata tags are extracted and attached.

[0013] As a further technical solution, the five dimensional factors include integrity factor, diversity factor, robustness factor, consistency factor, and complexity factor.

[0014] As a further technical solution, the weights are calculated as follows: The weights for each dimension are calculated based on the score fluctuations of historical quality scores, specifically as follows:

[0015] ;in, Indicates the first The weights corresponding to each dimension Indicates the first The standard deviation of the scores in each dimension.

[0016] As a further technical solution, the comprehensive score is calculated as follows:

[0017] ;in, Indicates the first Quality scores in each dimension Indicates the first The weights corresponding to each dimension.

[0018] As a further technical solution, the quality score of the multimodal data to be optimized is calculated using five dimensional factors, and a quality score vector is generated, represented as follows: Obtain the weights corresponding to the quality scores and generate a weight vector, represented as follows: The quality score vector and weight vector are concatenated into a joint input vector, which is then fed into the trained factor-aware optimization network. The network outputs optimization strategy suggestions, which include missing sample supplementation, data augmentation, noise enhancement, spatiotemporal alignment verification, and class-balanced sampling.

[0019] As a further technical solution, the convergence condition is expressed as follows:

[0020] , ;in, This indicates the change in the overall score before and after optimization. Indicates the threshold of change. Indicates marginal revenue. This represents the marginal revenue threshold; where , ; This indicates the current overall score. This indicates the overall score from the previous round.

[0021] Secondly, this invention provides a multimodal data quality assessment and optimization system for vehicle-to-everything (V2X) networks, comprising the following modules:

[0022] The data acquisition module is configured to acquire multimodal data and preprocess the multimodal data.

[0023] The quality factor calculation module is configured to: calculate the corresponding quality scores for the preprocessed multimodal data through five dimensions, and then weight and fuse the quality scores with the corresponding weights to obtain a comprehensive score, and obtain the multimodal data to be optimized based on the comprehensive score;

[0024] The multi-round optimization module is configured to: obtain all quality scores and corresponding weights of the multimodal data to be optimized, concatenate them into a joint input vector, input it into the trained factor-aware optimization network, output optimization strategy suggestions, optimize the multimodal data to be optimized based on the optimization strategy suggestions, and calculate the corresponding quality scores again through the five-dimensional factors based on the optimized multimodal data and the unoptimized multimodal data.

[0025] The convergence control and feedback module is configured to: obtain the comprehensive score before and after optimization, obtain the change in comprehensive score and marginal benefit, and determine whether the convergence condition is met. If the convergence condition is met, the optimization is completed; otherwise, the optimization continues in a loop.

[0026] One or more technical solutions of the present invention have the following beneficial effects:

[0027] (1) To address the lack of unified modeling for multimodal data quality assessment, this invention calculates the corresponding quality scores for multimodal data using five dimensional factors (integrity factor, diversity factor, robustness factor, consistency factor, and complexity factor). By converting multimodal data into unified quantitative indicators, collaborative assessment and unified modeling of multimodal data quality are achieved, thereby improving the accuracy of data assessment results.

[0028] (2) This invention obtains a comprehensive score by weighting and integrating the quality score with the corresponding weight, which makes the response to key quality dimensions more sensitive at different stages, realizes dynamic adjustment of weight, avoids the defects caused by fixed weight or subjective weight setting, and improves the adaptability.

[0029] (3) This invention utilizes a trained factor-aware optimization network to input the quality score and the corresponding weights into the factor-aware optimization network and outputs optimization strategy suggestions. This avoids relying on empirical threshold judgments and fixed optimization strategies. This invention formulates personalized optimization strategies based on the actual defects of the samples.

[0030] (4) This invention calculates the change in comprehensive score and marginal benefit, and formulates a convergence feedback judgment mechanism to avoid problems such as repeated optimization, ineffective enhancement and waste of resources. Attached Figure Description

[0031] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0032] Figure 1 This is a flowchart of the multimodal data quality assessment and optimization method of the present invention;

[0033] Figure 2 This is a framework diagram of the multimodal data quality assessment and optimization system of the present invention. Detailed Implementation

[0034] Example 1

[0035] This embodiment provides a method for multimodal data quality assessment and optimization in vehicle-to-everything (V2X) networks, such as... Figure 1 As shown, the specific steps of the method are as follows:

[0036] S1: Acquire multimodal data and preprocess the multimodal data.

[0037] In step S1, the multimodal data acquired through vehicle-mounted cameras, LiDAR, millimeter-wave radar, GPS, and IMU includes image data, LiDAR point clouds, motion state data, and inertial navigation data. To standardize the raw multimodal data acquired by the connected vehicle and ensure consistent format and structure of the system input data, the preprocessing method in this embodiment is as follows: First, the multimodal data undergoes time synchronization and format conversion to a standardized data structure, aligning data from different sources into a unified structure. Then, the multimodal data is cleaned, including removing abnormal frames, empty frames, and invalidly labeled multimodal data. Finally, metadata tags are extracted and attached, including weather type, time period, and road environment, providing contextual information support for subsequent quality assessment.

[0038] S2: Calculate the corresponding quality scores for the preprocessed multimodal data using five dimensional factors, and then weight and fuse the quality scores with their corresponding weights to obtain a comprehensive score. Based on the comprehensive score, obtain the multimodal data that needs to be optimized.

[0039] In step S2, for each data point, a corresponding quality score is calculated using five dimensional factors, including completeness factor, diversity factor, robustness factor, consistency factor, and complexity factor.

[0040] In this embodiment, the integrity factor is used to assess the missing data and labels for each modality, reflecting the completeness of the data. The calculation method is as follows:

[0041] ;in, This indicates the number of data entries with missing modalities or labels. This represents the total sample size. The closer the completeness factor is to 1, the more complete the data.

[0042] The diversity factor measures the distribution balance of different scenarios (e.g., day / night, urban / rural), modal combinations, or tag types, reflecting the breadth of semantic coverage. The calculation method is as follows:

[0043] in, This indicates semantic categories, such as different scenarios or combinations of tags. This indicates the proportion of each type of sample. A higher diversity factor indicates a richer variety of scene types.

[0044] The robustness factor measures the impact of data perturbations (such as blurring, occlusion, noise, or mode loss) on model performance, reflecting the data's resistance to disturbances. The calculation method is as follows: feature extraction is performed on each sample, and the sample features before perturbation are recorded. The features of the corresponding samples after perturbation are Where j = 1, 2, 3, ... Calculate the distance between the features before and after the perturbation for each sample. : = Set the maximum distance in the feature space. For normalization, the robustness factor is calculated using the following formula:

[0045] The closer the robustness factor score is to 1, the lower the sensitivity of the data to disturbances and the stronger its robustness.

[0046] The consistency factor measures the consistency of expression between multimodal data or between multi-labeled results, reflecting the harmony and reliability of the data. The calculation method is as follows:

[0047] ;in, Indicates sample Features in both modalities, with a total number of samples. A higher consistency factor indicates that the multimodal data provides a consistent description of the same semantic objective.

[0048] The complexity factor measures the structural complexity of the feature space, reflecting variations in sample distribution density, blurred boundaries, and representational difficulty. The calculation method involves extracting features from the sample set and calculating the average distance of each sample to its k nearest neighbors. The calculation formula is as follows:

[0049] ;in, ; Let N be the feature vector of the sample; N is the total number of samples. The higher the complexity factor, the more complex the feature distribution.

[0050] Based on the above method, the corresponding quality scores are calculated according to the five calculation factors: integrity factor, diversity factor, robustness factor, consistency factor, and complexity factor. .

[0051] In step S2, the weights of each dimension are calculated based on the score fluctuations of historical quality scores, specifically as follows:

[0052] ;in, Indicates the first The weights corresponding to each dimension Indicates the first The standard deviation of the scores in each dimension ensures that completeness and consistency are prioritized in the early stages of training, while complexity and robustness are given more attention in the later stages. The comprehensive score is calculated as follows:

[0053] ;in, Indicates the first Quality scores for each dimension.

[0054] Based on the comprehensive score, they can be divided into four categories:

[0055] High quality (overall score ≥ 90): Direct output;

[0056] Acceptable (overall score 75-89): Minor optimization needed;

[0057] Optimization needed (overall score 60-74): Proceed to the enhancement process;

[0058] Unavailable (overall score < 60): Marked as discarded or to be recycled.

[0059] S3: Obtain all quality scores and corresponding weights of the multimodal data to be optimized, concatenate them into a joint input vector, input it into the trained factor-aware optimization network, output optimization strategy suggestions, optimize the multimodal data to be optimized based on the optimization strategy suggestions, and calculate the corresponding quality scores again through the five-dimensional factors based on the optimized multimodal data and the unoptimized multimodal data.

[0060] In step S3, the quality scores of the multimodal data to be optimized are obtained by calculating the scores using five dimensional factors, and a quality score vector is generated, represented as follows: Obtain the weights corresponding to the quality scores and generate a weight vector, represented as follows: The quality score vector and weight vector are concatenated into a joint input vector. The data is input into a trained factor-aware optimization network, a lightweight multilayer perceptual structure that models the interaction patterns between factor scores and generates personalized optimization strategy suggestions based on the quality scores and corresponding weights. The specific structure is as follows: Input layer: Consists of a quality score vector (five dimensions, corresponding to completeness, diversity, robustness, consistency, and complexity) and corresponding weight vectors, concatenated into a 10-dimensional input vector. First layer: Fully connected layer, input dimension 10, output dimension 64, activation function ReLU; Second layer: Fully connected layer, input dimension 64, output dimension 32, activation function ReLU; Third layer: Output layer, dimension 5, corresponding to 5 optimization strategies (sampling supplementation, enhancement, noise addition, alignment, and equalization sampling), activation function Sigmoid, used to represent the execution strength of each strategy (range 0~1). During data processing, in the training phase, the factor-aware optimization network uses the sample's quality score vector and corresponding weight vector as input, and outputs the recommendation strength of each optimization strategy, with a value ranging from 0 to 1. During training, the improvement effects of various optimization strategies on sample quality in historical data are used as supervision signals to construct a 5-dimensional continuous label vector. Mean squared error is used as the loss function to train the network to predict the execution strength of different strategies. In the deployment phase, the quality score and weights of the current sample are input, and the network can quickly output suggested execution strengths for five optimization strategies. The system sorts these output values, prioritizing strategies with higher recommendation strengths for sample optimization, thus achieving a personalized data processing workflow. The output shows the suggested values ​​for the optimization strategies. Output the recommended value for the de1 optimization strategy. Each component in the algorithm corresponds to an optimization action suggestion. Each value is categorized as [0, 1], representing the recommended execution strength of the corresponding strategy. Optimization strategy suggestions include, but are not limited to, missing sample imputation, data augmentation, noise enhancement, spatiotemporal alignment verification, and class-balanced sampling. The output optimization strategy suggestion values ​​are then used to determine the optimal strategy. The strength distribution of each strategy is determined, and the corresponding optimization module is invoked and executed according to the strength allocation. After optimization, the sample data and optimization actions are logged. The optimization operation changes the state of the sample data. These optimized multimodal data, along with other unoptimized multimodal data (high-quality, acceptable, and discarded samples), are again evaluated using five dimensions to calculate their respective quality scores, initiating the next round of evaluation. Subsequent rounds will continue to update the dynamic weights based on the optimized scores and re-enter strategy analysis until the convergence condition is met.

[0061] S4: Obtain the comprehensive score before and after optimization, get the change in comprehensive score and marginal benefit, and determine whether the convergence condition is met. If the convergence condition is met, the optimization is completed; otherwise, continue the iterative optimization.

[0062] In step S4, the convergence condition is satisfied if two consecutive rounds satisfy: , If the multi-round optimization iterations have converged, the entire closed-loop optimization process is terminated, and the optimization is complete. This indicates the change in the overall score before and after optimization. Indicates the threshold of change. Indicates marginal revenue. This represents the marginal revenue threshold; where ; This indicates the current overall score. This represents the overall score from the previous round. In this embodiment, = Number of samples in the dataset × 0.5%, which means that when the change in the average score of the entire dataset is less than 0.5% of the total number of samples, the optimization is considered saturated. = Current score × 1%, representing the marginal benefit threshold, to avoid over-optimization at high score stages.

[0063] If the convergence condition is not met, the next round of evaluation and optimization loop will be started automatically.

[0064] Example 2

[0065] This embodiment provides a multimodal data quality assessment and optimization system for vehicle-to-everything (V2X) networks, including the following modules:

[0066] The data acquisition module is configured to acquire multimodal data and preprocess the multimodal data.

[0067] The quality factor calculation module is configured to: calculate the corresponding quality scores for the preprocessed multimodal data through five dimensions, and then weight and fuse the quality scores with the corresponding weights to obtain a comprehensive score, and obtain the multimodal data to be optimized based on the comprehensive score;

[0068] The multi-round optimization module is configured to: obtain all quality scores and corresponding weights of the multimodal data to be optimized, concatenate them into a joint input vector, input it into the trained factor-aware optimization network, output optimization strategy suggestions, optimize the multimodal data to be optimized based on the optimization strategy suggestions, and calculate the corresponding quality scores again through the five-dimensional factors based on the optimized multimodal data and the unoptimized multimodal data.

[0069] The convergence control and feedback module is configured to: obtain the comprehensive score before and after optimization, obtain the change in comprehensive score and marginal benefit, and determine whether the convergence condition is met. If the convergence condition is met, the optimization is completed; otherwise, the optimization continues in a loop.

[0070] Various modifications and variations of this invention will be apparent to those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for multimodal data quality assessment and optimization in vehicle-to-everything (V2X) networks, characterized in that, include: Acquire multimodal data and preprocess the multimodal data; The preprocessed multimodal data are used to calculate the corresponding quality scores through five dimensions, and the quality scores are weighted and fused with the corresponding weights to obtain a comprehensive score. The multimodal data to be optimized is obtained based on the comprehensive score. The five dimensions include integrity factor, diversity factor, robustness factor, consistency factor, and complexity factor. Obtain all quality scores and corresponding weights of the multimodal data to be optimized, concatenate them into a joint input vector, input it into the trained factor-aware optimization network, output optimization strategy suggestions, optimize the multimodal data to be optimized based on the optimization strategy suggestions, and calculate the corresponding quality scores again through the five-dimensional factors based on the optimized multimodal data and the unoptimized multimodal data. Obtain the comprehensive score before and after optimization, get the change in comprehensive score and marginal benefit, and determine whether the convergence condition is met. If the convergence condition is met, the optimization is completed; otherwise, continue the iterative optimization. The convergence condition is expressed as follows: , ;in, This indicates the change in the overall score before and after optimization. Indicates the threshold of change. Indicates marginal revenue. This represents the marginal revenue threshold; where , ; This indicates the current overall score. This indicates the overall score from the previous round.

2. The method for multimodal data quality assessment and optimization in vehicle networking as described in claim 1, characterized in that, The multimodal data includes image data, lidar point clouds, motion state data, and inertial navigation data. The preprocessing method is as follows: first, the multimodal data is time-synchronized and format-converted to a standard data structure; then, abnormal frames, empty frames, and invalid labeled multimodal data are removed; finally, metadata tags are extracted and attached.

3. The method for multimodal data quality assessment and optimization in vehicle networking as described in claim 1, characterized in that, The weights are calculated as follows: The weights for each dimension are calculated based on the historical quality score fluctuations. Specifically: ;in, Indicates the first The weights corresponding to each dimension Indicates the first The standard deviation of the scores in each dimension.

4. The method for multimodal data quality assessment and optimization in vehicle networking as described in claim 1, characterized in that, The comprehensive score is calculated as follows: ;in, Indicates the first Quality scores in each dimension Indicates the first The weights corresponding to each dimension.

5. The method for multimodal data quality assessment and optimization in vehicle networking as described in claim 1, characterized in that, The quality scores of the multimodal data to be optimized are calculated using five dimensional factors, and a quality score vector is generated, represented as follows: Obtain the weights corresponding to the quality scores and generate a weight vector, represented as follows: The quality score vector and weight vector are concatenated into a joint input vector, which is then fed into the trained factor-aware optimization network. The network outputs optimization strategy suggestions, which include missing sample supplementation, data augmentation, noise enhancement, spatiotemporal alignment verification, and class-balanced sampling.

6. A multimodal data quality assessment and optimization system for vehicle-to-everything (V2X) networks, characterized in that, Includes the following modules: The data acquisition module is configured to acquire multimodal data and preprocess the multimodal data. The quality factor calculation module is configured to: calculate the corresponding quality scores for the preprocessed multimodal data through five dimensions, and then weight and fuse the quality scores with the corresponding weights to obtain a comprehensive score. Based on the comprehensive score, the multimodal data to be optimized is obtained. The five dimensions include integrity factor, diversity factor, robustness factor, consistency factor, and complexity factor. The multi-round optimization module is configured to: obtain all quality scores and corresponding weights of the multimodal data to be optimized, concatenate them into a joint input vector, input it into the trained factor-aware optimization network, output optimization strategy suggestions, optimize the multimodal data to be optimized based on the optimization strategy suggestions, and calculate the corresponding quality scores again through the five-dimensional factors based on the optimized multimodal data and the unoptimized multimodal data. The convergence control and feedback module is configured to: obtain the comprehensive score before and after optimization, obtain the change in comprehensive score and marginal benefit, and determine whether the convergence condition is met. If the convergence condition is met, the optimization is completed; otherwise, the optimization continues in a loop. The convergence condition is expressed as follows: , ;in, This indicates the change in the overall score before and after optimization. Indicates the threshold of change. Indicates marginal revenue. This represents the marginal revenue threshold; where , ; This indicates the current overall score. This indicates the overall score from the previous round.

7. A computer-readable storage medium having a program stored thereon, characterized in that, When executed by the processor, the program implements the steps in the multimodal data quality assessment and optimization method for vehicle networking as described in any one of claims 1-5.

8. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the multimodal data quality assessment and optimization method for vehicle networking as described in any one of claims 1-5.

Images