A vehicle-road-cloud data processing method and device

Abstract

The application provides a vehicle-road cloud data processing method and device, the method comprising: acquiring sensor data collected by a vehicle-mounted sensor, using a respective value evaluation model of each node to perform hierarchical value evaluation on the sensor data to determine whether the sensor data is high-value data; if the sensor data is high-value data, performing feature extraction and attention fusion on the high-value data to determine a scene representation vector; using the scene representation vector to perform scene reconstruction and semantic decoupling to generate scene synthesis data; performing pseudo-label annotation on the high-value data and quality checking on the scene synthesis data; using the high-value data with the pseudo-label and the scene synthesis data that passes the quality checking to optimize the value evaluation model, so as to use the optimized value evaluation model for subsequent evaluation. By using the above vehicle-road cloud data processing method and device, the cloud data processing efficiency is improved, and the data storage cost is reduced.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and more specifically, to a vehicle-road-cloud data processing method and apparatus. Background Technology

[0002] Vehicle-road-cloud data plays a crucial role in the field of autonomous driving, providing comprehensive perception support and overcoming the limitations of single-vehicle perception. In existing technologies, the flow of vehicle-road-cloud data typically follows this pattern: onboard sensors collect raw data, which is then preprocessed using simple rules before being uploaded to the cloud via roadside units. The cloud then analyzes and processes the uploaded multi-source data, ultimately ensuring the safe operation of autonomous driving.

[0003] However, uploading data immediately after preprocessing results in a large amount of redundant and low-value data being uploaded to the cloud, which not only reduces the efficiency of cloud data processing but also increases data storage costs. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a vehicle-road-cloud data processing method and apparatus to overcome at least one of the above-mentioned defects.

[0005] In a first aspect, embodiments of this application provide a vehicle-road-cloud data processing method, applied to a vehicle-road-cloud data processing system. The vehicle-road-cloud data processing system includes multiple nodes, including vehicle-side nodes, roadside nodes, and cloud-based nodes, comprising: Acquire sensor data collected by vehicle-mounted sensors, and use the value assessment model of each node to classify and assess the value of the sensor data to determine whether the sensor data is high-value data. If the sensor data is high-value data, feature extraction and attention fusion are performed on the high-value data to determine the scene representation vector; Scene representation vectors are used for scene reconstruction and semantic decoupling to generate synthetic scene data. High-value data is labeled with pseudo-labels, and the quality of synthesized scene data is verified. By utilizing high-value data labeled with pseudo-labels and scene-synthesized data that has passed quality verification, the value assessment model is optimized for subsequent evaluation.

[0006] Optionally, the value assessment model includes a first value assessment model for vehicle-side node deployment, a second value assessment model for roadside node deployment, and a third value assessment model for cloud node deployment. The model determines whether sensor data is high-value data through the following methods: First, a first value score is determined using the first value assessment model, which characterizes the uncertainty of the data and the degree of difference between the data and a first sample of data; second, a second value score is determined using the second value assessment model, which characterizes the uncertainty of the data, the degree of difference between the data and a second sample of data, and the degree of disagreement between different assessment models regarding the value assessment results; third, a third value score is determined using the third value assessment model, which characterizes the uncertainty of the data and the degree of difference between the data and a third sample of data; based on the third value score, it is determined whether the sensor data is high-value data.

[0007] Optionally, the steps for determining the third value score using the third value assessment model include: analyzing the assessment accuracy of the value assessment model in different scenarios, identifying data characteristics that reflect the model's shortcomings; determining the assessment weights of the value assessment model under the data characteristics, and updating the value assessment model using the assessment weights.

[0008] Optionally, the method also includes: adjusting the data upload conditions for sensor data that conform to the data characteristics.

[0009] Optionally, the third value score can be determined by inputting sensor data, the third difference score, and the identified model weakness data into the third value assessment model to determine the third value score.

[0010] Optionally, the steps of using scene representation vectors to reconstruct the scene and decouple semantics to generate scene synthesis data include: reconstructing the vehicle driving scene using neural radiation fields based on the scene representation vectors to obtain a three-dimensional dynamic scene model; converting the three-dimensional dynamic scene model into an editable scene model through semantic decoupling and parameterized control; and generalizing the editable scene model to generate scene synthesis data.

[0011] Optionally, the scene synthesis data includes truth labels, and the quality of the scene synthesis data is verified in the following ways: the reasonableness of the scene synthesis data is judged by a preset discriminant network to determine whether the scene synthesis data passes the reasonableness verification; it is determined whether there is a logical conflict between the truth labels and the scene synthesis data, and if there is no logical conflict, the scene synthesis data passes the logical consistency verification.

[0012] Optionally, the method also includes: after each value score is determined, determining whether the sensor data meets the data upload conditions; if the data upload conditions are met, uploading the sensor data to the upper-level node.

[0013] Optionally, the data upload conditions include a first data upload condition, which is determined by the following method: comparing the first value score with a first score threshold, and comparing the vehicle-side computing load with a preset load threshold; if the first value score is greater than the first score threshold and the vehicle-side computing load is less than the preset load threshold, then the first data upload condition is determined to be met.

[0014] Secondly, embodiments of this application also provide a vehicle-road-cloud data processing device, applied to a vehicle-road-cloud data processing system, the vehicle-road-cloud data processing system including multiple nodes, including vehicle-side nodes, roadside nodes, and cloud nodes, the device comprising: The value assessment module is used to acquire sensor data collected by vehicle-mounted sensors and to perform graded value assessment of the sensor data using the value assessment model of each node to determine whether the sensor data is high-value data. The first processing module is used to extract features and perform attention fusion on the high-value data if the sensor data is high-value data, and determine the scene representation vector. The second processing module is used to perform scene reconstruction and semantic decoupling using scene representation vectors, and generate scene synthetic data. The labeling module is used to perform pseudo-labeling on high-value data and to perform quality verification on scene-synthesized data. The model optimization module is used to optimize the value assessment model by using high-value data with pseudo-labels and scene-synthesized data that has passed quality verification, so that the optimized value assessment model can be used for subsequent evaluation.

[0015] The embodiments of this application bring the following beneficial effects: This application provides a vehicle-road-cloud data processing method and apparatus that can sequentially perform hierarchical value assessment of sensor data through vehicle-side nodes, roadside nodes, and cloud nodes, and generate scene-synthesized data and pseudo-labels corresponding to high-value data. The scene-synthesized data and pseudo-labels are used to optimize the value assessment model, further improving the assessment accuracy of the value assessment model and reducing redundant data uploaded to the cloud. Compared with existing vehicle-road-cloud data processing methods, this solves the problems of low cloud data processing efficiency and high data storage cost.

[0016] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart of the vehicle-road-cloud data processing method provided in an embodiment of this application is shown; Figure 2 A flowchart illustrating the steps for determining high-value data provided in embodiments of this application is shown; Figure 3 A flowchart illustrating the steps for generating scene synthesis data provided in this application embodiment is shown; Figure 4 This paper shows a schematic diagram of the structure of the vehicle-road-cloud data processing device provided in an embodiment of this application; Figure 5 A schematic diagram of the structure of the electronic device provided in the embodiments of this application is shown. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. Based on the embodiments of this application, every other embodiment obtained by those skilled in the art without inventive effort falls within the scope of protection of this application.

[0020] To facilitate understanding of this embodiment, the following description uses the vehicle-road-cloud data processing method provided in this application embodiment applied to a vehicle-road-cloud data processing system as an example to illustrate the exemplary steps provided in this application embodiment. The vehicle-road-cloud data processing system includes multiple nodes, including vehicle-side nodes, roadside nodes, and cloud nodes.

[0021] Please see Figure 1 , Figure 1 This is a flowchart illustrating a vehicle-road-cloud data processing method provided in an embodiment of this application. Figure 1 As shown in the embodiments of this application, the vehicle-road-cloud data processing method includes: Step S101: Obtain sensor data collected by vehicle-mounted sensors, and use the value assessment model of each node to perform graded value assessment of the sensor data in order to determine whether the sensor data is high-value data.

[0022] Sensor data refers to the data collected by vehicle sensors at the current moment. Taking a single vehicle as an example, vehicle sensors include, but are not limited to: cameras, lidar, and infrared sensors.

[0023] A value assessment model can refer to a model used to assess the value of data. Value assessment models include the first value assessment model for vehicle-side node deployment, the second value assessment model for roadside node deployment, and the third value assessment model for cloud node deployment.

[0024] The following reference Figure 2 This section will introduce the process of determining whether sensor data is high-value data.

[0025] Figure 2 A flowchart illustrating the steps for determining high-value data provided in embodiments of this application is shown, such as... Figure 2 As shown, the steps for identifying high-value data include: Step S1011: Determine the first value score using the first value assessment model.

[0026] The first value score is used to characterize the uncertainty of the data and the degree of difference between the data and the first sample data.

[0027] The first value assessment model, also known as the lightweight value assessment model, is deployed at the vehicle-side node. It is used to assess data uncertainty and the degree of difference between the data and the initial sample data. The uncertainty of the data at the vehicle-side node can be represented by a first uncertainty score. To indicate, Used to quantify the unreliability of evaluation results from data. The magnitude of the value is positively correlated with the uncertainty. The larger the value, the less reliable the assessment of the data value; The smaller the value, the more reliable the assessment of the data's value. The degree of difference between the data and the first sample data can be scored using the first degree of difference rating. To indicate, Used to quantify the difference between the data and the first sample data. The magnitude of the values ​​is positively correlated with the degree of difference. The larger the value, the greater the difference between the data and the first sample data; The smaller the value, the smaller the difference between the data and the first sample data.

[0028] The first sample data may refer to the sample data determined based on the third sample data stored in the cloud node. The third sample data is a feature vector extracted by the cloud from the training data of the value assessment model. The third sample data is used to represent the knowledge that the value assessment model has mastered. For example, the first sample data may be the sample data that the cloud node processes the third sample data according to the characteristics of the vehicle node and then sends it to the vehicle node.

[0029] Specifically, sensor data can be input into the first value assessment model, which will output a first value score. The formula for calculating the first value score can be expressed as: ,in, This indicates the first value rating. This indicates the first uncertainty score. This indicates the first degree of difference score. This indicates the weight of the first uncertainty score. This indicates the weight of the first degree of difference score.

[0030] In calculation In this case, multiple forward propagation inferences can be performed on the sensor data, and random deactivation is applied in each forward propagation inference. The prediction results obtained from each forward propagation inference are different. Then, the average prediction probability of the multiple prediction results is calculated. The formula for calculating the average prediction probability is: ; In the above formula, This represents the average predicted probability; T represents the number of forward propagation inferences. Let x represent the probability that the predicted result is of category c in the t-th inference, and let x represent the sensor data.

[0031] Then, determine the average prediction probability. The prediction entropy of the average prediction probability is used as the first uncertainty score.

[0032] In calculation First, a feature extractor is used to calculate the low-dimensional feature vector of the sensor data and determine multiple cluster centers of the first sample data. Then, the target cluster center that is closest to the low-dimensional feature vector is selected from the multiple cluster centers. The cosine similarity between the low-dimensional feature vector and the target cluster center is calculated. The first difference score is determined based on the cosine similarity. The cosine similarity and the first difference score are negatively correlated.

[0033] The first value score is determined by summing the weights of the first uncertainty score and the first difference score.

[0034] In one embodiment, after each value score is determined, it is determined whether the sensor data meets the data upload conditions. If the data upload conditions are met, the sensor data is uploaded to the upper-layer node.

[0035] Specifically, the data upload conditions include the first data upload condition. After determining the first value score, it is determined whether the sensor data meets the first data upload condition. If the first data upload condition is met, the sensor data is uploaded to the roadside node.

[0036] For example, when determining whether sensor data meets the first data upload condition, a first value score can be compared with a first score threshold. Simultaneously, the vehicle-side computing load can be determined and compared with a preset load threshold. If the first value score is greater than the first score threshold and the vehicle-side computing load is less than the preset load threshold, then the first data upload condition is met, and the sensor data is uploaded to the roadside node. If the first value score is less than or equal to the first score threshold, or the vehicle-side computing load is greater than or equal to the preset load threshold, then the first data upload condition is not met, and the sensor data upload to the roadside node is prohibited.

[0037] Step S1012: Determine the second value score using the second value assessment model.

[0038] The second value score is used to comprehensively characterize the degree of difference between the data and the second sample data, as well as the degree of disagreement among different evaluation models regarding the value assessment results.

[0039] The second valuation model, also known as the standard valuation model, is deployed at roadside nodes. It is used to assess data uncertainty, divergence, and the degree of difference between the data and a second sample. The uncertainty of the data at the roadside nodes can be assessed using a second uncertainty score. To indicate, The unreliability of the evaluation results of quantitative data. The magnitude of the value is positively correlated with the uncertainty. The degree of difference between the data and the second sample data can be scored using the second degree of difference. To indicate, Used to quantify the difference between the first data and the second sample data. The magnitude of the values ​​is positively correlated with the degree of difference. The larger the value of , the greater the difference between the data and the second sample data; The smaller the value, the smaller the difference between the first sample and the second sample. The degree of data dissimilarity can be expressed using a dissimilarity score. To indicate, This is used to quantify the degree of disagreement among different evaluation models regarding value assessment results. By measuring the degree of disagreement, redundant data caused by misjudgments of data value by vehicle-side nodes can be reduced. The magnitude of the value is positively correlated with the degree of divergence.

[0040] The second sample data may refer to the sample data determined based on the third sample data stored in the cloud node. The third sample data is a feature vector extracted by the cloud from the training data of the value assessment model. The third sample data is used to characterize the knowledge already mastered by the value assessment model. For example, the second sample data may be the sample data sent from the cloud node to the vehicle node after the cloud node processes the third sample data according to the characteristics of the roadside node.

[0041] Specifically, the sensor data uploaded by the vehicle-side nodes can be input into the second value assessment model, which will output a second value score. The formula for calculating the second value score can be expressed as follows: ,in, This indicates the second value rating. This indicates the second uncertainty score. This indicates the second degree of difference score. Indicates the degree of divergence score. Indicates consistency score, Indicates the weight of the divergence score. This indicates the weight of the second degree of difference score. Indicates the weight of the divergence score. This represents the weight of the consistency score.

[0042] In calculation At that time, it can be adopted with The calculation method is the same as the one used here, and will not be repeated. In the calculation... At that time, it can be adopted with The calculation method is the same as the one used here, so it will not be repeated here.

[0043] In calculation At that time, multiple evaluation models deployed at roadside nodes can be used to determine the first uncertainty score of sensor data, calculate the variance of the multiple first uncertainty scores output by different evaluation models, and determine the degree of divergence score based on the variance. The variance is positively correlated with the degree of divergence score.

[0044] In calculation In this case, reference data collected by the sensors of the roadside nodes themselves can be used to verify the sensor data, and a consistency score can be determined based on the degree of difference between the reference data and the sensor data.

[0045] The second value score is determined by the sum of the weights of the second uncertainty score, the second difference score, the divergence score, and the consistency score.

[0046] In one embodiment, the data upload conditions include second data upload conditions. After determining the second value score, it is determined whether the sensor data meets the second data upload conditions. If the second data upload conditions are met, the sensor data is uploaded to the cloud node.

[0047] For example, the second value score can be compared with the second score threshold. If the second value score is greater than the second score threshold, the second data upload condition is met, and the sensor data is uploaded to the cloud node. If the second value score is less than or equal to the second score threshold, the second data upload condition is not met, and the sensor data is prohibited from being uploaded to the cloud node.

[0048] Step S1013: Determine the third value score using the third value assessment model.

[0049] The third value score is used to characterize the correlation of the data and the degree of difference between the data and the third sample data.

[0050] The third value assessment model, also known as the advanced value assessment model, is deployed on cloud nodes. It comprises multiple sub-models, including a third value scoring sub-model for assessing data uncertainty, data correlation, and the degree of difference between the data and third-sample data, and a third value assessment sub-model for determining the third value score. The uncertainty of the data on the cloud nodes refers to the uncertainty corresponding to sensor data uploaded from different roadside nodes, and this uncertainty can be represented by a third uncertainty score. To indicate, and The calculation method is the same. The degree of difference between the data and the third sample data can be scored using the third degree of difference. To indicate, Used to quantify the difference between the data and the third sample data. The magnitude of the values ​​is positively correlated with the degree of difference. The larger the value of , the greater the difference between the data and the third sample data; The smaller the value, the smaller the difference between the data and the third sample data.

[0051] The third sample data can refer to the core sample data stored on cloud nodes. The value assessment model is trained using the third sample data. The information breadth of the third sample data is greater than that of the second sample data and the first sample data. Before determining the third value score, it is necessary to identify the model's weakness data. This weakness data is determined by the identification model using reference data to evaluate the accuracy of the value assessment model and determine its recall under different data characteristics, thus pinpointing the model's shortcomings. The reference data can be a test set or real-world road test data. The identification model can identify potential weaknesses for each value assessment model within its respective sample data. For example, if the third sample data contains scarce data on motorcycle driving at night in rainy weather, the value assessment model may struggle to accurately evaluate the value of motorcycle driving data in this scenario. Therefore, the data characteristics of the model's weakness data include nighttime, rainy weather, and motorcycle.

[0052] Specifically, the metadata of the sensor data, the third uncertainty score, the third difference score, and the identified model weakness data can be input into the third value assessment sub-model to determine the third value score. Here, the third value score can be expressed in the following form: ,in, Metadata representing sensor data, This indicates the third uncertainty score. This indicates the third degree of difference score. This represents the model's bottleneck data.

[0053] The third value assessment sub-model can be a large language model, which can output a third value score, assessment weights to be updated, and a score threshold based on the input data. The metadata of the sensor data includes data such as geographic location, time, weather, road type, and traffic flow density; the calculation method for the third degree of difference score is the same as that for the first degree of difference score, and will not be repeated here.

[0054] In one embodiment, to improve the accuracy of the third value score, the third value assessment sub-model can determine the data characteristics of the model's weak points that reflect the model's assessment bias. For example, the data characteristics could be nighttime, rainy weather, and a motorcycle as the target. Then, it determines the assessment weight of the value assessment model under the data characteristics. That is, when there is target data in the sensor data that matches the data characteristics, a new assessment weight is activated. For example, the weight of the first uncertainty score in the first value assessment model is increased. and the weight of the second uncertainty score in the second valuation model The updated evaluation weights are used to update each value assessment model, so that more data that meets the data characteristics can be uploaded to the cloud node, and the shortcomings of the value assessment model can be continuously made up for.

[0055] In one embodiment, in addition to adjusting the evaluation weights, the data upload conditions for sensor data that conform to the data characteristics can also be adjusted.

[0056] For example, the scoring thresholds include at least one of the following: a first scoring threshold, a second scoring threshold, and a third scoring threshold. Based on the scoring thresholds output by the third value assessment sub-model, a first scoring threshold can be set separately for sensor data that meets the data characteristics at the vehicle-side node, a second scoring threshold can be set separately for sensor data that meets the data characteristics at the roadside node, and a third scoring threshold can be set separately for sensor data that meets the data characteristics at the cloud node. The first, second, and third scoring thresholds for sensor data that meets the data characteristics are then lowered, and the lowered first scoring threshold is sent to the vehicle-side node so that when the vehicle-side node detects sensor data that meets the data characteristics, it is easier for the data to be uploaded to the roadside node. Simultaneously, the lowered second scoring threshold is sent to the roadside node so that when the roadside node detects sensor data that meets the data characteristics, it is easier for the data to be uploaded to the cloud node.

[0057] In one embodiment, the third value assessment sub-model can also determine the correlation between sensor data uploaded from different roadside nodes. The correlation between the data can be determined by the third value assessment sub-model using correlation analysis methods, and the correlation between the data can be expressed using a correlation score. To indicate, Used to quantify the correlation between sensor data uploaded from different roadside nodes. The numerical value of the correlation is positively correlated with the correlation score. The correlation score is then input into the formula for calculating the third value score to determine a more accurate third value score.

[0058] Step S1014: Determine whether the sensor data is high-value data based on the third value score.

[0059] The third value score is compared with the third score threshold. If the third value score is greater than the third score threshold, the sensor data is determined to be high-value data; if the third value score is less than or equal to the third score threshold, the sensor data is determined to be low-value data.

[0060] It should be noted that high-value data is obtained based on sensor data, and high-value data is data that has not been manually labeled.

[0061] Step S102: If the sensor data is high-value data, perform feature extraction and attention fusion on the high-value data to determine the scene representation vector.

[0062] High-value data sent from different vehicle nodes and roadside nodes undergoes spatiotemporal alignment processing to unify multiple high-value data points under the same spatiotemporal coordinate system. Then, the spatiotemporally aligned high-value data is input into the encoder to extract multimodal features, including image features and point cloud features.

[0063] Bidirectional cross-modal attention processing is performed on multimodal features to determine the first attention weight from image to point cloud and the second attention weight from point cloud to image. The first attention weight represents the degree of correlation between image features and various parts of point cloud features, while the second attention weight represents the degree of correlation between point cloud features and various parts of image features.

[0064] Image features and point cloud features are weighted and fused according to the first attention weight to obtain image-guided enhanced point cloud features; point cloud features and image features are weighted and fused according to the second attention weight to obtain point cloud-guided enhanced image features.

[0065] The enhanced image features and enhanced point cloud features are input into the decoder, and the decoder outputs a scene representation vector that includes multimodal information.

[0066] Step S103: Use scene representation vectors to reconstruct the scene and decouple semantics to generate scene synthesis data.

[0067] The following reference Figure 3 This section will introduce the process of generating scene synthesis data.

[0068] Figure 3 A flowchart illustrating the steps for generating scene synthesis data provided in this application embodiment is shown, such as... Figure 3 As shown, the steps for generating scene synthesis data include: Step S1031: The vehicle driving scene is reconstructed using the neural radiation field to obtain a three-dimensional dynamic scene model.

[0069] Specifically, a scene representation vector can refer to a vector used to represent scene features. The scene representation vector can be input into a neural radiation field to obtain a high-precision, editable 3D dynamic scene model.

[0070] Step S1032: Through semantic decoupling and parameterized control, the 3D dynamic scene model is converted into an editable scene model.

[0071] The three-dimensional dynamic scene model is semantically decoupled and its dynamic and static elements are separated to obtain structured scene semantic information, dynamic and static element data, and reusable and editable scene components.

[0072] Scene semantic information refers to the category attributes of various elements in the scene, such as roads, buildings, vehicles, and pedestrians. Each element has a clear semantic label, and logical relationships are established between elements, such as vehicles driving on roads and pedestrians on sidewalks, to form an interpretable scene semantic network. Dynamic and static element data includes moving targets such as vehicles, pedestrians, and non-motorized vehicles, as well as dynamic parameters such as the trajectory, speed, and posture of vehicles and pedestrians. Scene components can include vehicles, pedestrians, and weather.

[0073] Then, based on the obtained scene semantic information and dynamic and static element data, the scene components are parametrically controlled to transform the 3D dynamic scene model into a quantifiable, configurable, and repeatable scene model.

[0074] Step S1033: Perform generalization processing on the editable scene model to generate scene synthesis data.

[0075] By adjusting the parameters of the editable scene model, massive and diverse scene synthesis data can be automatically generated. For example, scene synthesis data can be generated in a simulation environment, and the generated scene synthesis data has pixel-level ground truth labels.

[0076] Step S104: Pseudo-label the high-value data and perform quality verification on the scene synthesis data.

[0077] Specifically, a pre-defined guidance model is set up at the cloud nodes. This model is used to assign pseudo-labels to the data and to perform quality checks on the synthesized data. The pre-defined guidance model can be an artificial intelligence model, which acts as a mentor for the entire system, responsible for guiding data labeling and model evolution.

[0078] For example, high-value data includes image data. The pre-defined guidance model will draw boxes around vehicles and pedestrians in the image and give their respective categories and confidence scores. Then, the predicted categories, confidence scores, and bounding boxes are extracted, stored in a standardized labeling format, and pseudo-labels are generated.

[0079] In one embodiment, in order to ensure the quality of pseudo-label annotation, it is necessary to perform consistency verification on the pseudo-labels annotated by the preset guidance model.

[0080] For example, multiple augmented images are obtained by performing different data augmentation processes on the same unlabeled image. Each augmented image is then input into a pre-defined guidance model to make predictions for each image. The consistency of the prediction results across all augmented images is determined. If the prediction results are consistent, the pseudo-label is considered to have high confidence and passes the consistency check; if the prediction results are inconsistent, the pseudo-label is considered to have low confidence and fails the consistency check. Only the high-value data corresponding to pseudo-labels that pass the consistency check are added to the final third-sample data.

[0081] Furthermore, after determining the scene synthesis data, its quality can be verified from multiple dimensions. These multi-dimensional quality verifications include rationality verification and logical consistency verification. During rationality verification, a pre-defined discriminant network is used to judge the rationality of the scene synthesis data, determining whether it conforms to physical laws. If it does, the rationality verification passes; otherwise, it fails. During logical consistency verification, it is determined whether there is a logical conflict between the truth label and the scene synthesis data. If no logical conflict exists, the scene synthesis data passes the logical consistency verification. For example, if the truth label is "truck," but the size of the object identified as a truck in the scene synthesis data does not match the actual size of the truck, then there is a logical conflict between the truth label and the scene synthesis data, and the logical consistency verification fails.

[0082] In one embodiment, the quality verification also includes validity verification. During the validity verification process, the scene synthesis data can be input into the third value assessment model to determine whether the scene synthesis data is high-value data. If it is determined that the scene synthesis data is not high-value data, it means that the scene synthesis data is challenging to the value assessment model and is valid data. Therefore, it is determined that the validity verification is passed.

[0083] Step S105: Optimize the value assessment model using high-value data labeled with pseudo-labels and scene-synthesized data that has passed quality verification, so as to use the optimized value assessment model for subsequent evaluation.

[0084] Specifically, high-value data labeled with pseudo-labels and scenario-synthesized data that has passed quality verification can be used to update the third sample data, which can then be used as new core sample data in the cloud to optimize the value assessment model.

[0085] For example, based on the characteristics of each roadside node, new second sample data for that roadside node is determined using the updated third sample data, and this new second sample data is then distributed to the corresponding roadside node. Simultaneously, based on the characteristics of each vehicle-side node, the corresponding first sample data is determined using the updated third sample data, and this first sample data is distributed to the corresponding vehicle-side node. This timely updates the second sample data of the roadside nodes and the first sample data of the vehicle-side nodes, thereby updating the value assessment models for both the vehicle-side nodes and the roadside nodes. The characteristics of different roadside nodes can refer to the characteristics of different vehicle types. For example, if the vehicle is a passenger car, then the vehicle characteristics for that vehicle type emphasize driving safety and driver assistance functions, requiring updates to data affecting driving safety and improving driver assistance functions; if the vehicle is a public transport vehicle, then the vehicle characteristics emphasize route accuracy and real-time vehicle-to-infrastructure communication, requiring updates to data affecting route accuracy and the real-time performance of vehicle-to-infrastructure communication.

[0086] The vehicle-road-cloud data processing method provided in this application can sequentially perform hierarchical value assessment of sensor data through vehicle-side nodes, roadside nodes, and cloud nodes, and generate scene-synthesized data and pseudo-labels corresponding to high-value data. The scene-synthesized data and pseudo-labels are used to optimize the value assessment model, further improve the assessment accuracy of the value assessment model, reduce redundant data uploaded to the cloud, and solve the problems of low cloud data processing efficiency and high data storage cost.

[0087] Based on the same inventive concept, this application also provides a vehicle-road-cloud data processing device corresponding to the vehicle-road-cloud data processing method. Since the principle of the device in this application is similar to the above-mentioned vehicle-road-cloud data processing method in this application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0088] Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of a vehicle-road-cloud data processing device provided in an embodiment of this application. Figure 4 As shown, the data is applied to a vehicle-road-cloud data processing system, which includes multiple nodes, including vehicle-side nodes, roadside nodes, and cloud nodes. The vehicle-road-cloud data processing device 200 includes: The value assessment module 201 is used to acquire sensor data collected by vehicle-mounted sensors and to perform graded value assessment of the sensor data using the value assessment model of each node to determine whether the sensor data is high-value data. The first processing module 202 is used to perform feature extraction and attention fusion on the high-value data if the sensor data is high-value data, and determine the scene representation vector. The second processing module 203 is used to perform scene reconstruction and semantic decoupling using scene representation vectors to generate scene synthetic data. The labeling module 204 is used to perform pseudo-labeling on high-value data and to perform quality verification on scene-synthesized data. The model optimization module 205 is used to optimize the value assessment model by using high-value data with pseudo-labels and scene synthesis data that has passed quality verification, so as to use the optimized value assessment model for subsequent evaluation.

[0089] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 5 As shown, the electronic device 300 includes a processor 310, a memory 320, and a bus 330.

[0090] The memory 320 stores machine-readable instructions executable by the processor 310. When the electronic device 300 is running, the processor 310 and the memory 320 communicate via the bus 330. When the machine-readable instructions are executed by the processor 310, they can perform the operations described above. Figure 1 The steps of the vehicle-road-cloud data processing method in the method embodiment shown are described in detail in the method embodiment, and will not be repeated here.

[0091] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can perform the above-described actions. Figure 1 The steps of the vehicle-road-cloud data processing method in the method embodiment shown are described in detail in the method embodiment, and will not be repeated here.

[0092] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0093] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0094] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0095] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0096] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0097] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The scope of protection of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for processing vehicle-road cloud data, characterized in that, This is applied to a vehicle-road-cloud data processing system, which includes multiple nodes, including vehicle-side nodes, roadside nodes, and cloud nodes, including: The sensor data collected by the vehicle-mounted sensors is acquired, and the sensor data is graded and valued using the value assessment model of each node to determine whether the sensor data is high-value data. The value assessment model includes a first value assessment model deployed at the vehicle-side nodes, a second value assessment model deployed at the roadside nodes, and a third value assessment model deployed at the cloud nodes. The model determines whether the sensor data is high-value data through the following methods: The first value assessment model is used to determine a first value score, which is used to characterize the uncertainty of the data and the degree of difference between the data and the first sample data. The first sample data is the sample data that the cloud node sends to the vehicle node after processing the third sample data according to the characteristics of the vehicle node. After determining the first value score, it is determined whether the sensor data meets the first data upload conditions. If the first data upload conditions are met, the sensor data is uploaded to the roadside node. The second value score is determined using the second value assessment model. The second value score is used to characterize the uncertainty of the data, the degree of difference between the data and the second sample data, and the degree of disagreement between different assessment models of the roadside node on the value assessment results. The second sample data is the sample data sent to the roadside node after the cloud node processes the third sample data according to the characteristics of the roadside node. After determining the second value score, it is determined whether the sensor data meets the second data upload conditions. If the second data upload conditions are met, the sensor data is uploaded to the cloud node. The third value score is determined using the third value assessment model. The third value score is used to characterize the uncertainty of the data and the degree of difference between the data and the third sample data. The third sample data refers to the core sample data stored on cloud nodes. The information breadth of the third sample data is greater than that of the second sample data and the first sample data. Based on the third value score, determine whether the sensor data is high-value data; If the sensor data is high-value data, feature extraction and attention fusion are performed on the high-value data to determine the scene representation vector; The scene representation vector is used to reconstruct the scene and decouple its semantics, generating scene synthesis data. The high-value data is labeled with pseudo-tags, and the quality of the synthesized scene data is verified. The value assessment model is optimized using high-value data labeled with pseudo-labels and scene-synthesized data that has passed quality verification. This optimized model is then used for subsequent evaluations. Specifically, this includes: updating the third sample data using the high-value data labeled with pseudo-labels and scene-synthesized data that has passed quality verification; determining new second sample data for each roadside node using the updated third sample data based on the characteristics of that roadside node, and distributing the new second sample data to the corresponding roadside node; and determining the first sample data corresponding to each vehicle-end node using the updated third sample data based on the characteristics of that vehicle-end node, and distributing the first sample data to the corresponding vehicle-end node.

2. The method according to claim 1, characterized in that, The method further includes: The accuracy of each of the value assessment models is evaluated, and the data characteristics that reflect the shortcomings of the models are determined. The evaluation weights and scoring thresholds to be updated in the model are determined based on the third value assessment model. When target data that matches the data characteristics exists in the sensor data, the value assessment models are updated using the evaluation weights to be updated.

3. The method according to claim 2, characterized in that, The method further includes: The data upload conditions for sensor data that meet the defined data characteristics are adjusted based on the determined scoring threshold.

4. The method according to claim 2, characterized in that, The third value score is determined using the following methods: The sensor data, the third uncertainty score, the third difference score, and the identified model weakness data features are input into the third value assessment model to determine the third value score.

5. The method according to claim 1, characterized in that, The step of using the scene representation vector to perform scene reconstruction and semantic decoupling to generate scene synthetic data includes: Based on the scene representation vector, the vehicle driving scene is reconstructed using the neural radiation field to obtain a three-dimensional dynamic scene model; Through semantic decoupling and parameterized control, the 3D dynamic scene model is converted into an editable scene model; The editable scene model is generalized to generate scene synthesis data.

6. The method according to claim 1, characterized in that, The scene synthesis data includes truth labels, and the quality of the scene synthesis data is verified in the following ways: The reasonableness of the scene synthesis data is judged by a preset discrimination network to determine whether the scene synthesis data passes the reasonableness check; Determine whether there is a logical conflict between the truth value label and the scene synthesis data. If there is no logical conflict, determine that the scene synthesis data passes the logical consistency check.

7. The method according to claim 1, characterized in that, The sensor data is determined to meet the first data upload condition using the following method: The first value score is compared with the first score threshold, and the vehicle-side computing load is compared with the preset load threshold. If the first value score is greater than the first score threshold and the vehicle-side computing load is less than the preset load threshold, then the first data upload condition is determined to be met.

8. A vehicle-road-cloud data processing device, characterized in that, This is applied to a vehicle-road-cloud data processing system, which includes multiple nodes, including vehicle-side nodes, roadside nodes, and cloud nodes, including: The value assessment module is used to acquire sensor data collected by vehicle-mounted sensors and to perform graded value assessments on the sensor data using a value assessment model specific to each node, in order to determine whether the sensor data is high-value data. The value assessment models include a first value assessment model deployed at the vehicle-side node, a second value assessment model deployed at the roadside node, and a third value assessment model deployed at the cloud node. The determination of whether the sensor data is high-value data is made through the following methods: The first value assessment model is used to determine a first value score, which is used to characterize the uncertainty of the data and the degree of difference between the data and the first sample data. The first sample data is the sample data that the cloud node sends to the vehicle node after processing the third sample data according to the characteristics of the vehicle node. After determining the first value score, it is determined whether the sensor data meets the first data upload condition. If the first data upload condition is met, the sensor data is uploaded to the roadside node. The second value score is determined using the second value assessment model. The second value score is used to characterize the uncertainty of the data, the degree of difference between the data and the second sample data, and the degree of disagreement between different assessment models of the roadside nodes regarding the value assessment results. The second sample data is the sample data that the cloud node sends to the roadside nodes after processing the third sample data according to the characteristics of the roadside nodes. After determining the second value score, it is determined whether the sensor data meets the second data upload conditions. If the second data upload conditions are met, the sensor data is uploaded to the cloud node. The third value assessment model is used to determine the third value score, which is used to characterize the uncertainty of the data and the degree of difference between the data and the third sample data. The third sample data refers to the core sample data stored on cloud nodes. The information breadth of the third sample data is greater than that of the second sample data and the first sample data. Based on the third value score, determine whether the sensor data is high-value data; The first processing module is used to perform feature extraction and attention fusion on the high-value data if the sensor data is high-value data, and determine the scene representation vector. The second processing module is used to perform scene reconstruction and semantic decoupling using the scene representation vector to generate scene synthesis data. The labeling module is used to perform pseudo-labeling on the high-value data and to perform quality verification on the scene-synthesized data. The model optimization module is used to optimize the value assessment model using high-value data labeled with pseudo-labels and scene synthetic data that has passed quality verification, so as to use the optimized value assessment model for subsequent evaluation. Specifically, it includes: updating the third sample data using high-value data labeled with pseudo-labels and scene synthetic data that has passed quality verification; determining new second sample data for each roadside node using the updated third sample data based on the characteristics of each roadside node, and sending the new second sample data to the corresponding roadside node; and determining the first sample data corresponding to each vehicle-end node using the updated third sample data based on the characteristics of each vehicle-end node, and sending the first sample data to the corresponding vehicle-end node.

Images