Automatic driving evaluation method, device, equipment, medium and program product

Abstract

The invention discloses an automatic driving evaluation method, device and equipment, a medium and a program product, and relates to the technical field of automatic driving. The method comprises the following steps: acquiring own vehicle state data, vehicle sensing data and driving decision data of a to-be-detected vehicle in real time through a vehicle end; outputting a first estimated driving intention evaluation, a first estimated driving safety evaluation and a first estimated vehicle operation evaluation of the to-be-detected vehicle according to the own vehicle state data, the vehicle sensing data and the driving decision data by adopting a vehicle end evaluation large model through the vehicle end, and feeding back the first estimated driving intention evaluation, the first estimated driving safety evaluation and the first estimated vehicle operation evaluation to a cloud platform; and integrating the first estimated driving intention evaluation, the first estimated driving safety evaluation and the first estimated vehicle operation evaluation through the cloud platform to obtain a comprehensive evaluation result of the to-be-detected vehicle. According to the technical scheme of the embodiment of the invention, the efficiency and accuracy of automatic driving evaluation are improved.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to an autonomous driving evaluation method, apparatus, device, medium, and program product. Background Technology

[0002] With the rapid development of autonomous driving technology, the evaluation of its safety, reliability, and comfort has become crucial. Traditional evaluation methods mainly rely on simulation testing, closed-track testing, and post-processing of manual road test data. While simulation testing can cover a large number of peripheral cases in a virtual environment, its realism and physical accuracy always fall short of real-world testing, making it difficult to fully simulate complex real-world interactions. Closed-track testing is costly, time-consuming, and limited in its testing scenarios. Post-processing of manual road test data involves professional engineers replaying road test data for subjective evaluation. This method is highly dependent on human labor, suffers from inefficiency and strong subjectivity, and is difficult to scale. Summary of the Invention

[0003] This invention provides an autonomous driving evaluation method, apparatus, device, medium, and program product, which improves the efficiency and accuracy of autonomous driving evaluation.

[0004] According to one aspect of the present invention, an autonomous driving evaluation method is provided, the method comprising:

[0005] Through the vehicle terminal, real-time data on the vehicle status, vehicle perception, and driving decision-making of the vehicle under test can be obtained.

[0006] Using the vehicle-side evaluation model, based on the vehicle status data, vehicle perception data, and driving decision data, the system outputs a first estimated driving intention evaluation, a first estimated driving safety evaluation, and a first estimated vehicle operation evaluation for the vehicle under test, and feeds them back to the cloud platform.

[0007] The cloud platform is used to synthesize the first estimated driving intention evaluation, the first estimated driving safety evaluation, and the first estimated vehicle operation evaluation to obtain a comprehensive evaluation result of the vehicle under test.

[0008] According to another aspect of the present invention, an autonomous driving evaluation apparatus is provided, the apparatus comprising:

[0009] Through the vehicle terminal, real-time data on the vehicle status, vehicle perception, and driving decision-making of the vehicle under test can be obtained.

[0010] Using the vehicle-side evaluation model, based on the vehicle status data, vehicle perception data, and driving decision data, the system outputs a first estimated driving intention evaluation, a first estimated driving safety evaluation, and a first estimated vehicle operation evaluation for the vehicle under test, and feeds them back to the cloud platform.

[0011] The cloud platform is used to synthesize the first estimated driving intention evaluation, the first estimated driving safety evaluation, and the first estimated vehicle operation evaluation to obtain a comprehensive evaluation result of the vehicle under test.

[0012] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0013] At least one processor; and

[0014] A memory communicatively connected to the at least one processor; wherein,

[0015] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the autonomous driving evaluation method according to any embodiment of the present invention.

[0016] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the autonomous driving evaluation method according to any embodiment of the present invention.

[0017] According to another aspect of the present invention, a computer program product is provided, the computer program product comprising a computer program that, when executed by a processor, implements the autonomous driving evaluation method according to any embodiment of the present invention.

[0018] The technical solution of this invention enables multi-dimensional evaluation of autonomous vehicles at the vehicle end, achieving millisecond-level response for autonomous driving evaluation and providing instant feedback to the autonomous driving system, thus improving the real-time performance of autonomous driving evaluation. Furthermore, by employing a unified vehicle-side evaluation model, the subjectivity of manual evaluation can be eliminated, ensuring the objectivity and consistency of autonomous driving evaluation for all vehicles at any location. Simultaneously, by feeding back the multi-dimensional evaluation results of the vehicle under test to the cloud platform, the original sensitive data from the vehicle end is protected from leaving the vehicle; only anonymized multi-dimensional evaluation results are uploaded, greatly protecting user privacy and data security. Through the cloud platform, the first estimated driving intention evaluation, the first estimated driving safety evaluation, and the first estimated vehicle operation evaluation are integrated to obtain the comprehensive evaluation result of the vehicle under test, which can be easily deployed to a fleet of millions, improving the scalability of autonomous driving evaluation.

[0019] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart of an autonomous driving evaluation method provided in Embodiment 1 of the present invention;

[0022] Figure 2 This is a flowchart of an autonomous driving evaluation method provided according to Embodiment 2 of the present invention;

[0023] Figure 3 This is a system architecture diagram of the autonomous driving evaluation method applicable to Embodiment 2 of the present invention;

[0024] Figure 4 This is a flowchart of an autonomous driving evaluation method provided according to Embodiment 2 of the present invention;

[0025] Figure 5 This is a schematic diagram of the structure of an autonomous driving evaluation device according to Embodiment 3 of the present invention;

[0026] Figure 6 This is a schematic diagram of the structure of an electronic device that implements the autonomous driving evaluation method of this invention. Detailed Implementation

[0027] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0029] Example 1

[0030] Figure 1 This is a flowchart illustrating an autonomous driving evaluation method provided in Embodiment 1 of the present invention. This embodiment of the invention is applicable to autonomous driving evaluation based on a vehicle-side and cloud platform. The method is executed by an autonomous driving evaluation device, which is implemented in hardware and / or software and can be configured in an electronic device that carries autonomous driving evaluation functions.

[0031] See Figure 1 The autonomous driving evaluation method shown includes:

[0032] S101. Real-time acquisition of vehicle status data, vehicle perception data, and driving decision data of the vehicle to be tested via the vehicle terminal.

[0033] The vehicle end is the end of the vehicle under test. It is used to acquire real-time data from the vehicle under test and to perform multi-dimensional evaluations of its autonomous driving capabilities. The vehicle under test is the vehicle used for the autonomous driving evaluation.

[0034] Vehicle state data is used to characterize the vehicle state of the vehicle under test. For example, vehicle state data includes vehicle speed, vehicle acceleration, vehicle yaw rate, vehicle steering angle, and / or vehicle heading angle. Vehicle speed is the velocity of the vehicle under test. Vehicle acceleration is the rate of change of the vehicle speed. Vehicle yaw rate is the angular velocity of the vehicle under test rotating about its vertical axis. The vehicle yaw rate describes the rotational dynamics of the vehicle during steering. The vehicle steering angle is the deflection angle of the front wheels of the vehicle under test relative to the vehicle body. The vehicle heading angle characterizes the deviation between the actual direction of travel of the vehicle under test and the reference direction.

[0035] Vehicle perception data is used to characterize the environmental data perceived by the vehicle under test. For example, vehicle perception data includes the location, speed, and type of environmental objects perceived by the vehicle under test. The location of the environmental object characterizes its position. For example, the location of the environmental object is the relative position between the environmental object and the vehicle under test. The speed of the environmental object is its velocity. The type of the environmental object is its category attribute. For example, the type of environmental object includes vehicles, pedestrians, or obstacles. Optionally, the vehicle perception data is collected by the vehicle sensors of the vehicle under test. These vehicle sensors include cameras, LiDAR, or millimeter-wave radar.

[0036] Driving decision data refers to the internal decision-making data of the autonomous driving system of the vehicle under test. For example, driving decision data includes the vehicle's predicted trajectory, trajectory tracking error, and control commands. The planned trajectory is the sequence of trajectories predicted by the vehicle under test. For example, the planned trajectory includes the position, velocity, and acceleration of each trajectory point predicted by the vehicle under test. The trajectory tracking error is the deviation between the predicted trajectory and the actual trajectory of the vehicle under test. For example, the trajectory tracking error includes position error, velocity error, and attitude error. Control commands are used to control the vehicle under test. For example, control commands include motor torque, servo angle, throttle opening, and / or brake opening.

[0037] Specifically, the system acquires real-time vehicle status data and perception data from vehicle sensors. It also acquires real-time driving decision data from the vehicle's infotainment system.

[0038] S102. Through the vehicle end, using the vehicle-side evaluation model, based on the vehicle status data, vehicle perception data, and driving decision data, output the first estimated driving intention evaluation, the first estimated driving safety evaluation, and the first estimated vehicle operation evaluation of the vehicle to be detected, and feed them back to the cloud platform.

[0039] The vehicle-side evaluation model is used to perform multi-dimensional evaluations of the autonomous driving capabilities of the vehicle under test at the vehicle end. Optionally, the vehicle-side evaluation model is pre-deployed at the vehicle end. For example, the vehicle-side evaluation model is pre-deployed in the software stack of the intelligent driving domain of the vehicle under test.

[0040] The first estimated driving intention evaluation characterizes the accuracy of understanding the driving intention of the vehicle under test during autonomous driving. The first estimated driving safety evaluation characterizes the safety of driving the vehicle under test during autonomous driving. The first estimated vehicle operation evaluation characterizes the smoothness of operation of the vehicle under test during autonomous driving. For example, scoring values ​​are used to characterize the first estimated driving intention evaluation, the first estimated driving safety evaluation, and the first estimated vehicle operation evaluation.

[0041] The cloud platform is used to comprehensively evaluate the autonomous driving functions of the vehicles under test. This can be understood as the cloud platform being used to comprehensively evaluate the autonomous driving functions of the vehicles under test from multiple dimensions.

[0042] Specifically, through the vehicle end, the vehicle status data, vehicle perception data, and driving decision data are input into the pre-trained vehicle-side evaluation model, which outputs the first estimated driving intention evaluation, the first estimated driving safety evaluation, and the first estimated vehicle operation evaluation of the vehicle under test, and feeds the first estimated driving intention evaluation, the first estimated driving safety evaluation, and the first estimated vehicle operation evaluation of the vehicle under test back to the cloud platform.

[0043] In an optional embodiment of the present invention, a vehicle-side evaluation model is used to output a first estimated driving intention evaluation, a first estimated driving safety evaluation, and a first estimated vehicle operation evaluation of the vehicle under test based on vehicle status data, vehicle perception data, and driving decision data. This includes: using a first vehicle-side evaluation sub-model to output a first estimated driving intention evaluation of the vehicle under test based on traffic rules, lane information, and the vehicle's planned trajectory; using a second vehicle-side evaluation sub-model to output a first estimated driving safety evaluation of the vehicle under test based on the relative distance, relative speed, estimated collision time, and inter-vehicle time distance between the vehicle under test and environmental objects; and using a third vehicle-side evaluation sub-model to output a first estimated vehicle operation evaluation of the vehicle under test based on the vehicle's longitudinal acceleration and lateral acceleration.

[0044] The vehicle-side evaluation model includes a first vehicle-side evaluation sub-model, a second vehicle-side evaluation sub-model, and a third vehicle-side evaluation sub-model.

[0045] The first vehicle-side evaluation sub-model is used to assess the accuracy of the driving intention of the vehicle under test at the vehicle end. Traffic rules are used to regulate the driving behavior of the vehicle under test. For example, traffic rules include speed limits, lane uses, traffic signal instructions, yielding rules, and prohibited behaviors. Speed ​​limits regulate the speed of a specific lane segment. For example, speed limits include maximum and minimum speeds. Lane uses define the purpose of each lane. For example, lane uses include straight-ahead lanes, left-turn lanes, right-turn lanes, and dedicated lanes for specific vehicles. Traffic signal instructions instruct the vehicle under test to proceed or pass. Yielding rules specify how the vehicle under test should yield in traffic signal-controlled areas. Prohibited behaviors define the behaviors that the vehicle under test is prohibited from performing on the road. For example, prohibited behaviors include driving against traffic, running red lights, or changing lanes arbitrarily. Lane line information is precise information on the geometry, topology, and attribute characteristics of lanes provided by a high-precision map. For example, geometry includes lane line type, lane width, curvature, slope, and elevation information. Topology includes lane connection methods, steering restrictions, and hierarchical relationships. Attribute features include lane function, traffic sign association, regular elm trees, construction areas, weather, and traffic events. The planned trajectory of the vehicle is a sequence of predicted trajectories for the vehicle to be detected. For example, the planned trajectory of the vehicle includes the position, speed, and acceleration of each predicted trajectory point of the vehicle to be detected.

[0046] The second vehicle-side evaluation sub-model is used to assess the driving safety of the vehicle under test at the vehicle end. Relative distance is the distance between the vehicle under test and environmental objects. For example, relative distance includes straight-line distance or longitudinal distance. Relative speed is the speed difference between the vehicle under test and environmental objects. For example, relative speed is the longitudinal speed difference along the road direction. Expected collision time is the remaining time required for the vehicle under test to collide with an environmental object. Inter-vehicle distance is the time interval between the vehicle under test and an environmental object passing the same location. Inter-vehicle distance is used to provide feedback on following safety.

[0047] The third vehicle-side evaluation sub-model is used to evaluate the driving smoothness of the vehicle under test at the vehicle end. The vehicle's longitudinal acceleration is the acceleration of the vehicle under test along its own direction of travel. The longitudinal acceleration of the vehicle under test is used to represent the rate of increase or decrease in the vehicle's speed. The vehicle's lateral acceleration is the acceleration of the vehicle under test perpendicular to its own direction of travel. The lateral acceleration of the vehicle under test is used to represent the centrifugal force effect when the vehicle moves laterally or turns.

[0048] Specifically, on the vehicle side, traffic rules, lane information, and the vehicle's planned trajectory are input into a pre-trained first vehicle-side evaluation sub-model, which outputs a first estimated driving intention evaluation of the vehicle under test. The relative distance, relative speed, estimated collision time, and inter-vehicle distance between the vehicle and environmental objects are input into a pre-trained second vehicle-side evaluation sub-model, which outputs a first estimated driving safety evaluation of the vehicle under test. The longitudinal and lateral accelerations of the vehicle under test are input into a pre-trained third vehicle-side evaluation sub-model, which outputs a first estimated vehicle operation evaluation of the vehicle under test.

[0049] This solution utilizes three vehicle-side evaluation sub-models: a first sub-model to output a preliminary driving intention assessment of the vehicle under test based on traffic rules, lane information, and the vehicle's planned trajectory; a second sub-model to output a preliminary driving safety assessment based on the relative distance, relative speed, estimated collision time, and inter-vehicle distance between the vehicle and environmental objects; and a third sub-model to output a preliminary vehicle performance assessment based on the vehicle's longitudinal and lateral accelerations. By concretizing the overall vehicle-side evaluation model into these sub-models, the solution achieves evaluations of the accuracy of driving intention, driving safety, and driving smoothness of the vehicle under test, thereby improving the accuracy of targeted evaluations of these three aspects.

[0050] S103. Through the cloud platform, the first estimated driving intention evaluation, the first estimated driving safety evaluation, and the first estimated vehicle operation evaluation are integrated to obtain the comprehensive evaluation result of the vehicle to be tested.

[0051] The comprehensive evaluation results are used to comprehensively evaluate the autonomous driving functions of the vehicles under test.

[0052] Specifically, through a cloud platform, the first estimated driving intention evaluation, the first estimated driving safety evaluation, and the first estimated vehicle operation evaluation are weighted and summed to obtain the comprehensive evaluation result of the vehicle to be tested.

[0053] Existing technologies employ simulation testing, closed-track testing, manual road test data post-processing, and cloud-based evaluation. While simulation testing can cover a large number of peripheral cases in a virtual environment, its realism and physical accuracy always fall short of real-vehicle testing, making it difficult to fully simulate complex real-world interactions. Secondly, closed-track testing is costly, time-consuming, and limited in its application scenarios. Furthermore, manual road test data post-processing involves professional engineers replaying road test data for subjective evaluation; this method is highly reliant on human labor, resulting in inefficiency, strong subjectivity, and difficulty in scaling. Finally, cloud-based evaluation involves transmitting complete data collected from the vehicle to a cloud server for analysis; however, this method requires extremely high network bandwidth, suffers from significant transmission latency, cannot provide real-time feedback, and poses data privacy and security risks.

[0054] The technical solution of this invention enables multi-dimensional evaluation of autonomous vehicles through the vehicle end, achieving millisecond-level response for autonomous driving evaluation and providing instant feedback to the autonomous driving system, thus improving the real-time performance of autonomous driving evaluation. Furthermore, by employing a unified vehicle-side evaluation model, the subjectivity of manual evaluation is eliminated, ensuring the objectivity and consistency of autonomous driving evaluation for all vehicles at any location. Simultaneously, by feeding back the multi-dimensional evaluation results of the vehicle under test to the cloud platform, the original sensitive data from the vehicle end is prevented from leaving the vehicle; only anonymized multi-dimensional evaluation results are uploaded, greatly protecting user privacy and data security. Through the cloud platform, the first estimated driving intention evaluation, the first estimated driving safety evaluation, and the first estimated vehicle operation evaluation are integrated to obtain the comprehensive evaluation result of the vehicle under test, which can be easily deployed in fleets of millions, improving the scalability of autonomous driving evaluation.

[0055] In an optional embodiment of the present invention, while using a vehicle-side evaluation model to output a first estimated driving intention evaluation, a first estimated driving safety evaluation, and a first estimated vehicle operation evaluation of the vehicle under test based on vehicle status data, vehicle perception data, and driving decision data, and feeding these evaluations back to the cloud platform, the invention also includes: feeding back the vehicle identifier, timestamp, and vehicle location of the vehicle under test to the cloud platform via the vehicle side; and after integrating the first estimated driving intention evaluation, the first driving safety evaluation, and the first vehicle operation evaluation via the cloud platform to obtain a comprehensive evaluation result of the vehicle under test, the invention further includes: visualizing the vehicle under test based on the vehicle identifier, timestamp, vehicle location, and comprehensive evaluation result via the cloud platform.

[0056] The vehicle identifier is used to uniquely identify the vehicle to be inspected. It distinguishes different vehicles from one another. For example, the vehicle identifier includes a VIN (Vehicle Identification Number) and a temporary identifier. The VIN is a permanent and unique identifier for the vehicle to be inspected. The temporary identifier is used to ensure the privacy of the vehicle during data transmission. The timestamp records the time point at which the real-time data corresponding to the comprehensive evaluation results was collected. The vehicle location identifies the real-time location of the vehicle to be inspected.

[0057] Specifically, on the vehicle side, the vehicle identifier, timestamp, and location of the vehicle to be tested are encapsulated into a data packet and fed back to the cloud platform. The cloud platform then statistically analyzes the vehicle identifier, timestamp, location, and comprehensive evaluation results to generate a visualization of the vehicle under test. This visualization includes a performance evaluation report, performance trend charts, and geographical distribution heatmaps. Optionally, the visualization can be generated periodically or after each autonomous driving trip. For example, a visualization could be generated every 30 seconds.

[0058] Optionally, before statistically analyzing the vehicle identification, timestamp, vehicle location, and comprehensive evaluation results of the vehicle to be inspected through the cloud platform to generate a visual result of the vehicle to be inspected, the cloud platform can be used to clean and aggregate the data of the vehicle identification, timestamp, vehicle location, and comprehensive evaluation results of the vehicle to be inspected.

[0059] This solution uses a cloud platform to visualize the vehicle under test based on its vehicle identification, timestamp, location, and comprehensive evaluation results. It enables the cloud platform to perform aggregated analysis of the vehicle, facilitating subsequent use by R&D and testing personnel.

[0060] Example 2

[0061] Figure 2This is a flowchart of an autonomous driving evaluation method provided in Embodiment 2 of the present invention. Based on the above embodiments, this embodiment further adds the following steps before "using a vehicle-side evaluation model to output a first estimated driving intention evaluation, a first estimated driving safety evaluation, and a first estimated vehicle operation evaluation of the vehicle to be tested based on the vehicle's state data, vehicle perception data, and driving decision data": "Acquiring first vehicle state samples, first vehicle perception samples, and first driving decision samples through a cloud platform; inputting the first vehicle state samples, first vehicle perception samples, and first driving decision samples into a cloud-based evaluation model through the cloud platform, and outputting soft labels for driving intention evaluation, driving safety evaluation, and vehicle operation evaluation; inputting the first vehicle state samples, first vehicle perception samples, and first driving decision samples into an initial vehicle-side evaluation model through the cloud platform." The system outputs a second estimated driving intention evaluation, a second estimated driving safety evaluation, and a second estimated vehicle operation evaluation. The initial vehicle-side large model has fewer model parameters than the cloud-based evaluation large model. Through the cloud platform, based on the differences between the second estimated driving intention evaluation and the driving intention evaluation soft label, the second estimated driving safety evaluation and the driving safety evaluation soft label, and the second estimated vehicle operation evaluation and the vehicle operation evaluation soft label, the initial vehicle-side large model is trained to obtain the vehicle-side evaluation large model. The vehicle-side evaluation large model is then deployed to the vehicle through the cloud platform. By pre-training the cloud-based evaluation large model, a vehicle-side evaluation large model with fewer model parameters is obtained, facilitating its deployment on the vehicle and improving the computational efficiency of multi-dimensional evaluation of autonomous driving. It should be noted that parts not detailed in this embodiment can be found in other embodiments.

[0062] See Figure 2 The autonomous driving evaluation method shown includes:

[0063] S201. Obtain the first autonomous vehicle status sample, the first vehicle perception sample, and the first driving decision sample through the cloud platform.

[0064] The first autonomous vehicle state sample, the first vehicle perception sample, and the first driving decision sample are training samples for the vehicle-side evaluation model. The first autonomous vehicle state sample represents the vehicle state of a historical vehicle. Referring to the example above, the first autonomous vehicle state sample includes the vehicle's speed, acceleration, yaw rate, steering angle, and / or heading angle. The first vehicle perception sample represents the environmental data perceived by the historical vehicle. Referring to the example above, the first vehicle perception sample includes the location, speed, and type of environmental objects perceived by the historical vehicle. The first driving decision sample is the internal decision data of the historical vehicle's autonomous driving system. Referring to the example above, the first driving decision sample includes the vehicle's predicted trajectory, trajectory tracking error, and control commands. The historical vehicle refers to a vehicle that performed autonomous driving during a historical time period.

[0065] Specifically, through the cloud platform, we obtain the first vehicle status sample, the first vehicle perception sample, and the first driving decision sample uploaded by the vehicle terminal corresponding to each historical vehicle.

[0066] S202. Through the cloud platform, the first vehicle state sample, the first vehicle perception sample, and the first driving decision sample are input into the cloud evaluation model, and the driving intention evaluation soft label, driving safety evaluation soft label, and vehicle operation evaluation soft label are output.

[0067] The cloud-based evaluation model is used to train the vehicle-side evaluation model. The cloud-based evaluation model is pre-trained and deployed on a cloud platform. Based on the first vehicle state sample, the first vehicle perception sample, and the first driving decision sample, the cloud-based evaluation model generates soft labels for driving intention evaluation, driving safety evaluation, and vehicle operation evaluation required by the vehicle-side evaluation model. For example, the cloud-based evaluation model may include a BERT (Bidirectional Encoder Representation from Transformers) structure or a convolutional neural network, etc.

[0068] The driving intention evaluation soft label, driving safety evaluation soft label, and vehicle operation evaluation soft label are all multi-dimensional prediction results output by a pre-trained cloud-based evaluation model. Specifically, the driving intention evaluation soft label characterizes the accuracy of understanding the driving intentions of historical vehicles during autonomous driving. The driving safety evaluation soft label characterizes the safety of historical vehicle driving during autonomous driving. The vehicle operation evaluation soft label characterizes the smoothness of historical vehicle operation during autonomous driving.

[0069] In an optional embodiment of the present invention, before inputting the first vehicle state sample, the first vehicle perception sample, and the first driving decision sample into the cloud evaluation model via a cloud platform, and outputting driving intention evaluation soft labels, driving safety evaluation soft labels, and vehicle operation evaluation soft labels, the method further includes: obtaining second vehicle state samples, second vehicle perception samples, second driving decision samples, first driving intention evaluation labels, first driving safety evaluation labels, and first vehicle operation evaluation labels via a cloud platform; inputting the second vehicle state samples, the second vehicle perception samples, and the second driving decision samples into the cloud initial large model via a cloud platform, and outputting reference driving intention evaluation, reference driving safety evaluation, and reference vehicle operation evaluation; and training the cloud reference large model via a cloud platform based on the differences between the reference driving intention evaluation and the first driving intention evaluation labels, the differences between the reference driving safety evaluation and the first driving safety evaluation labels, and the differences between the reference vehicle operation evaluation and the first vehicle operation evaluation labels, to obtain the cloud evaluation large model.

[0070] The second vehicle state sample, the second vehicle perception sample, the second driving decision sample, the first driving intention evaluation label, the first driving safety evaluation label, and the first vehicle operation evaluation label are the training samples for the cloud-based evaluation model.

[0071] The second autonomous vehicle state sample is used to characterize the vehicle state of the historical vehicle. Referring to the example above, the second autonomous vehicle state sample includes the vehicle speed, vehicle acceleration, vehicle yaw rate, vehicle steering angle, and / or vehicle heading angle, etc. The second vehicle perception sample is used to characterize the environmental data perceived by the historical vehicle. Referring to the example above, the second vehicle perception sample includes the location, speed, and type of environmental objects perceived by the historical vehicle, etc. The second driving decision sample is the internal decision data of the historical vehicle's autonomous driving system. Referring to the example above, the second driving decision sample includes the vehicle's predicted trajectory, vehicle trajectory tracking error, and vehicle control commands, etc. The historical vehicle refers to the vehicle that performed autonomous driving during a historical time period. In comparison, the first autonomous vehicle state sample, the first vehicle perception sample, and the first driving decision sample, and the second autonomous vehicle state sample, the second vehicle perception sample, and the second driving decision sample, are historical data corresponding to different historical vehicles, or historical data corresponding to the same historical vehicle in different historical time periods.

[0072] The first driving intention evaluation label, the first driving safety evaluation label, and the first vehicle operation evaluation label are evaluation labels created by technicians based on second autonomous vehicle state samples, second vehicle perception samples, and second driving decision samples. Specifically, the first driving intention evaluation label assesses the accuracy of understanding the driving intentions of historical vehicles during autonomous driving. The first driving safety evaluation label assesses the safety of historical vehicle driving during autonomous driving. The first vehicle operation evaluation label assesses the smoothness of historical vehicle operation during autonomous driving.

[0073] The initial large-scale model in the cloud is an incompletely trained cloud-based evaluation model. The reference driving intention evaluation, reference driving safety evaluation, and reference vehicle operation evaluation are multi-dimensional evaluation results of the cloud-based evaluation model's predictions of the second autonomous vehicle state sample, the second vehicle perception sample, and the second driving decision sample. Specifically, the reference driving intention evaluation is a predictive evaluation result of the accuracy of understanding the driving intentions of historical vehicles during autonomous driving. The reference driving safety evaluation is a predictive evaluation result of the safety of historical vehicle driving during autonomous driving. The reference vehicle operation evaluation is a predictive evaluation result of the smoothness of historical vehicle operation during autonomous driving.

[0074] The difference between the reference driving intention evaluation and the first driving intention evaluation labels is used to characterize the error between the prediction results of the initial large-scale model in the cloud and the labels labeled by the technician, from the dimension of driving intention accuracy. The difference between the reference driving safety evaluation and the first driving safety evaluation labels is used to characterize the error between the prediction results of the initial large-scale model in the cloud and the labels labeled by the technician, from the dimension of driving safety. The difference between the reference vehicle operation evaluation and the first vehicle operation evaluation labels is used to characterize the error between the prediction results of the initial large-scale model in the cloud and the labels labeled by the technician, from the dimension of driving smoothness.

[0075] Specifically, through a cloud platform, second vehicle state samples, second vehicle perception samples, and second driving decision samples uploaded by each historical vehicle are acquired. Pre-annotated labels for first driving intention evaluation, first driving safety evaluation, and first vehicle operation evaluation are also obtained by technicians based on these samples. The second vehicle state samples, second vehicle perception samples, and second driving decision samples are then input into an initial large-scale model in the cloud platform, outputting reference driving intention evaluation, reference driving safety evaluation, and reference vehicle operation evaluation. The cloud-based reference model is then trained using the cloud platform with the goal of minimizing the differences between the reference driving intention evaluation and the first driving intention evaluation labels, the differences between the reference driving safety evaluation and the first driving safety evaluation labels, and the differences between the reference vehicle operation evaluation and the first vehicle operation evaluation labels, resulting in a cloud-based evaluation model.

[0076] This solution generates a large cloud-based evaluation model through pre-training, ensuring the accuracy of the cloud-based evaluation model and thus improving the training efficiency and accuracy of the vehicle-side evaluation model.

[0077] S203. Through the cloud platform, the first vehicle state sample, the first vehicle perception sample, and the first driving decision sample are input into the initial large model on the vehicle side, and the second predicted driving intention evaluation, the second predicted driving safety evaluation, and the second predicted vehicle operation evaluation are output.

[0078] The initial large-scale model on the vehicle side is an untrained large-scale evaluation model on the vehicle side. The initial large-scale model on the vehicle side has fewer model parameters than the large-scale evaluation model on the cloud. This can be understood as follows: to achieve the same functionality, by simplifying the model parameters, the resulting large-scale evaluation model on the vehicle side is easier to deploy on various vehicles, thereby improving the applicability of autonomous driving evaluation methods. For example, the initial large-scale model on the vehicle side may include a Tiny BERT (Tiny Bidirectional Encoder Representation from Transformers) structure or a small convolutional neural network.

[0079] The second predicted driving intention evaluation, the second predicted driving safety evaluation, and the second predicted vehicle operation evaluation are multi-dimensional assessment results of the initial large-scale model on the vehicle for predicting the first vehicle state sample, the first vehicle perception sample, and the first driving decision sample. Specifically, the second predicted driving intention evaluation is a predictive assessment result of the accuracy of understanding the driving intentions of historical vehicles during autonomous driving. The second predicted driving safety evaluation is a predictive assessment result of the safety of historical vehicle driving during autonomous driving. The second predicted vehicle operation evaluation is a predictive assessment result of the smoothness of historical vehicle operation during autonomous driving.

[0080] S204. Through the cloud platform, based on the differences between the second estimated driving intention evaluation and the driving intention evaluation soft label, the differences between the second estimated driving safety evaluation and the driving safety evaluation soft label, and the differences between the second estimated vehicle operation evaluation and the vehicle operation evaluation soft label, the initial large model of the vehicle is trained to obtain the large model of vehicle evaluation.

[0081] The difference between the second estimated driving intention evaluation and the driving intention evaluation soft label is used to characterize the error between the prediction results of the initial large-scale model on the vehicle and the prediction results of the large-scale model in the cloud, from the dimension of driving intention accuracy. The difference between the second estimated driving safety evaluation and the driving safety evaluation soft label is used to characterize the error between the prediction results of the initial large-scale model on the vehicle and the prediction results of the large-scale model in the cloud, from the dimension of driving safety. The difference between the second estimated vehicle operation evaluation and the vehicle operation evaluation soft label is used to characterize the error between the prediction results of the initial large-scale model on the vehicle and the prediction results of the large-scale model in the cloud, from the dimension of driving smoothness.

[0082] Specifically, through a cloud platform, the initial large-scale vehicle-side model is trained with the goal of minimizing the differences between the second estimated driving intention evaluation and the driving intention evaluation soft label, the second estimated driving safety evaluation and the driving safety evaluation soft label, and the second estimated vehicle operation evaluation and the vehicle operation evaluation soft label. This results in a large-scale vehicle-side evaluation model.

[0083] Optionally, before deploying the vehicle-side evaluation model to the vehicle via a cloud platform, the process includes: acquiring third-party vehicle state samples, third-party vehicle perception samples, third-party driving decision samples, third-party driving intention evaluation labels, third-party driving safety evaluation labels, and third-party vehicle operation evaluation labels via a cloud platform; inputting the third-party vehicle state samples, third-party vehicle perception samples, and third-party driving decision samples into the vehicle-side evaluation model via a cloud platform, and outputting third-party predicted driving intention evaluation, third-party predicted driving safety evaluation, and third-party predicted vehicle operation evaluation; and training and updating the vehicle-side evaluation model via a cloud platform based on the differences between the third-party predicted driving intention evaluation and the third-party driving intention evaluation labels, the third-party predicted driving safety evaluation and the third-party driving safety evaluation labels, and the third-party predicted vehicle operation evaluation and the third-party vehicle operation evaluation labels.

[0084] The third autonomous vehicle state sample, third vehicle perception sample, third driving decision sample, third driving intention evaluation label, third driving safety evaluation label, and third vehicle operation evaluation label are training samples for further training of the vehicle-side evaluation model. The third autonomous vehicle state sample represents the vehicle state of a historical vehicle. Referring to the example above, the third autonomous vehicle state sample includes vehicle speed, vehicle acceleration, vehicle yaw rate, vehicle steering angle, and / or vehicle heading angle, etc. The third vehicle perception sample represents the environmental data perceived by the historical vehicle. Referring to the example above, the third vehicle perception sample includes the location, speed, and type of environmental objects perceived by the historical vehicle, etc. The third driving decision sample is the internal decision data of the historical vehicle's autonomous driving system. Referring to the example above, the third driving decision sample includes the vehicle's predicted trajectory, vehicle trajectory tracking error, and vehicle control commands, etc. The historical vehicle refers to a vehicle that performed autonomous driving during a historical time period.

[0085] The third-party driving intention evaluation label, third-party driving safety evaluation label, and third-party vehicle operation evaluation label are assessment labels created by technicians based on third-party vehicle state samples, third-party vehicle perception samples, and third-party driving decision samples. Specifically, the third-party driving intention evaluation label assesses the accuracy of understanding the driving intentions of historical vehicles during autonomous driving. The third-party driving safety evaluation label assesses the safety of historical vehicle driving during autonomous driving. The third-party vehicle operation evaluation label assesses the smoothness of historical vehicle operation during autonomous driving.

[0086] The third predicted driving intention evaluation, the third predicted driving safety evaluation, and the third predicted vehicle operation evaluation are multi-dimensional evaluation results of the vehicle-side evaluation model's predictions of third-party vehicle state samples, third-party vehicle perception samples, and third-party driving decision samples. Specifically, the third predicted driving intention evaluation is a predictive evaluation result of the accuracy of understanding historical vehicle driving intentions during autonomous driving. The third predicted driving safety evaluation is a predictive evaluation result of the safety of historical vehicle driving during autonomous driving. The third predicted vehicle operation evaluation is a predictive evaluation result of the smoothness of historical vehicle operation during autonomous driving.

[0087] The difference between the third predicted driving intention evaluation and the third driving intention evaluation label is used to characterize the error between the prediction results of the vehicle-side assessment model and the updated vehicle-side assessment model, from the dimension of driving intention accuracy. The difference between the third predicted driving safety evaluation and the third driving safety evaluation label is used to characterize the error between the prediction results of the vehicle-side assessment model and the updated vehicle-side assessment model, from the dimension of driving safety. The difference between the third predicted vehicle operation evaluation and the third vehicle operation evaluation label is used to characterize the error between the prediction results of the vehicle-side assessment model and the updated vehicle-side assessment model, from the dimension of driving smoothness.

[0088] Specifically, through a cloud platform, third-party vehicle state samples, third-party vehicle perception samples, and third-party driving decision samples from each historical vehicle are acquired. Furthermore, pre-annotated labels for third-party driving intention evaluation, third-party driving safety evaluation, and third-party vehicle operation evaluation are obtained by technicians based on these samples. These samples are then input into the vehicle-side evaluation model via the cloud platform, outputting third-party predicted driving intention evaluation, third-party predicted driving safety evaluation, and third-party predicted vehicle operation evaluation. The vehicle-side evaluation model is trained and updated via the cloud platform with the goal of minimizing the differences between the third-party predicted driving intention evaluation and its labels, the third-party predicted driving safety evaluation and its labels, and the third-party predicted vehicle operation evaluation and its labels.

[0089] After training the vehicle-side evaluation model based on the cloud-based evaluation model, this solution further adds a process of updating the vehicle-side evaluation model using third-party vehicle state samples, third-party vehicle perception samples, third-party driving decision samples, third-party driving intention evaluation labels, third-party driving safety evaluation labels, and third-party vehicle operation evaluation labels, thereby further improving the accuracy of the vehicle-side evaluation model.

[0090] S205. Deploy the large-scale vehicle-side evaluation model to the vehicle via a cloud platform.

[0091] Specifically, the trained vehicle-side evaluation model is deployed to the software stack of the intelligent driving domain controller on the vehicle via a cloud platform.

[0092] S206. Real-time acquisition of vehicle status data, vehicle perception data, and driving decision data of the vehicle to be tested via the vehicle terminal.

[0093] S207. Through the vehicle end, using the vehicle-side evaluation model, based on the vehicle status data, vehicle perception data, and driving decision data, output the first estimated driving intention evaluation, the first estimated driving safety evaluation, and the first estimated vehicle operation evaluation of the vehicle to be detected, and feed them back to the cloud platform.

[0094] S208. Through the cloud platform, the first estimated driving intention evaluation, the first estimated driving safety evaluation, and the first estimated vehicle operation evaluation are integrated to obtain the comprehensive evaluation result of the vehicle to be tested.

[0095] The technical solution of this invention uses a pre-trained cloud-based evaluation model to pre-train a vehicle-side evaluation model with fewer model parameters, which facilitates the deployment of the vehicle-side evaluation model and improves the computational efficiency of multi-dimensional evaluation of autonomous driving.

[0096] In an optional embodiment of the present invention, after comprehensively evaluating the first estimated driving intention assessment, the first estimated driving safety assessment, and the first estimated vehicle operation assessment through a cloud platform to obtain a comprehensive evaluation result for the vehicle to be tested, the method further includes: periodically acquiring the vehicle status data, vehicle perception data, and driving decision data of each vehicle to be tested through a cloud platform, and acquiring the corresponding second driving intention assessment label, second driving safety assessment label, and second vehicle operation assessment label; updating the cloud-based evaluation model based on the vehicle status data, vehicle perception data, driving decision data, second driving intention assessment label, second driving safety assessment label, and second vehicle operation assessment label through a cloud platform; updating the vehicle-side evaluation model using the updated cloud-based evaluation model through a cloud platform, and deploying the updated vehicle-side evaluation model to the vehicle.

[0097] The second driving intention evaluation label, the second driving safety evaluation label, and the second vehicle operation evaluation label are assessment labels created by technicians based on vehicle status data, vehicle perception data, and driving decision data. Specifically, the second driving intention evaluation label assesses the accuracy of understanding the driving intentions of the vehicle under test during autonomous driving. The second driving safety evaluation label assesses the safety of driving the vehicle under test during autonomous driving. The second vehicle operation evaluation label assesses the smoothness of the vehicle's operation during autonomous driving.

[0098] Specifically, through a cloud platform, the system periodically acquires vehicle status data, vehicle perception data, and driving decision data uploaded by each vehicle under test. It also obtains second driving intention evaluation labels, second driving safety evaluation labels, and second vehicle operation evaluation labels, annotated by technicians based on the vehicle status data, vehicle perception data, and driving decision data. Using the same training method as in the above embodiments, the cloud-based evaluation model is updated based on the vehicle status data, vehicle perception data, driving decision data, second driving intention evaluation labels, second driving safety evaluation labels, and second vehicle operation evaluation labels. Finally, using the updated cloud-based evaluation model, the vehicle-side evaluation model is updated and deployed to the vehicle.

[0099] This solution periodically acquires vehicle-side data, continuously iterates and optimizes the cloud-based evaluation model, and uses the cloud-based evaluation model to continuously update the vehicle-side evaluation model, thereby improving the vehicle-side evaluation model's ability to evaluate autonomous driving and achieving continuous evolution of the vehicle-side evaluation model.

[0100] Based on the above embodiments, Figure 3 and Figure 4 This is a preferred embodiment of the present invention. Figure 3 This is a system architecture diagram for autonomous driving evaluation methods. See also... Figure 3 The system architecture diagram of the autonomous driving evaluation method shown includes: vehicle terminal system (i.e., vehicle terminal) and cloud platform.

[0101] The vehicle terminal system includes a data acquisition module, a lightweight large vehicle model (i.e., a large vehicle evaluation model), an evaluation calculation module, and a vehicle-cloud communication module.

[0102] Specifically, the data acquisition module is used to acquire vehicle sensor data (i.e., vehicle perception data, including data collected by vehicle sensors such as cameras, lidar and millimeter-wave radar), vehicle status data (i.e., vehicle status, such as vehicle speed, acceleration and yaw rate), and internal decision data of the autonomous driving system (i.e., ADS data (Automated Driving System Data), or driving decision data, including planned trajectory, predicted object and control command).

[0103] The lightweight vehicle-side large model learns from a large "teacher model" (i.e., a cloud-based evaluation model with expert-level evaluation capabilities, deployed in the cloud) through knowledge distillation technology. It has the advantages of small size and high computational efficiency, making it suitable for vehicle-side deployment. The vehicle-side evaluation large model has a built-in multi-task learning network (i.e., the first vehicle-side evaluation sub-model, the second vehicle-side evaluation sub-model, and the third vehicle-side evaluation sub-model).

[0104] The evaluation calculation module calculates the input data obtained by the data acquisition module by calling the lightweight large model on the vehicle side, and outputs multi-dimensional lightweight scoring results (i.e., the first estimated driving intention evaluation, the first estimated driving safety evaluation, and the first estimated vehicle operation evaluation of the vehicle under test).

[0105] The vehicle-to-cloud communication module is used to encapsulate lightweight rating results, timestamps, vehicle identifiers, and vehicle locations into lightweight data packets and upload them to the cloud platform via 4G (the 4th Generation Mobile Communication Technology) / 5G (the 5th Generation Mobile Networks).

[0106] The cloud platform comprises an aggregation and analysis module, a visualization and reporting module, and a model optimization and distribution module. Specifically, the aggregation and analysis module receives lightweight evaluation results from vehicle-level subsystems corresponding to a large fleet of vehicles, and performs data cleaning, aggregation, statistics, and analysis. The visualization and reporting module generates fleet-level performance reports, performance trend charts, and / or geographical distribution heatmaps for use by R&D and testing personnel. The model optimization and distribution module continuously optimizes the cloud platform's "teacher model" (i.e., the large cloud-based evaluation model) based on the aggregated massive amounts of real-world data, and uses OTA (Over-The-Air Technology) to periodically push the distilled new version of the "student model" (i.e., the large vehicle-level evaluation model) to the vehicle-level subsystems, enabling iterative evolution of the large vehicle-level evaluation model deployed on the subsystems.

[0107] Figure 4 This is a flowchart of an autonomous driving evaluation method. See also... Figure 4 The autonomous driving evaluation method shown includes:

[0108] S401. Real-time data collection during autonomous driving.

[0109] Specifically, when the vehicle under test is running in autonomous driving mode, relevant data is continuously collected, including vehicle status data, vehicle perception data, and driving decision data.

[0110] S402, Input vehicle-side lightweight large model.

[0111] Specifically, a lightweight, large-scale model on the vehicle side is used for inference on the real-time data stream.

[0112] S403, Parallel computation of multi-dimensional scoring.

[0113] Specifically, the model outputs sub-scores (i.e., first estimated driving intention evaluation, first estimated driving safety evaluation, and first estimated vehicle operation evaluation) from multiple dimensions such as driving intention accuracy, driving safety, and vehicle operation smoothness.

[0114] S404. Generate structured evaluation results.

[0115] Specifically, the lightweight evaluation results will be stored in a structured manner.

[0116] S405. Check if the file has been uploaded to the cloud platform. If yes, compress and upload the file to the cloud platform. If no, cache the file locally and return to execute S401.

[0117] Specifically, it can detect whether network conditions allow uploading to the cloud platform. If network conditions permit, it uploads to the cloud platform; if network conditions do not permit, it caches the data locally.

[0118] S406, Cloud Platform Aggregation and Analysis.

[0119] Specifically, the cloud platform aggregates lightweight evaluation results uploaded by a massive number of vehicles to generate macro-level performance insights.

[0120] For example, taking a smart car equipped with a Level 2+ Autonomous Driving System (L2+) / Level 3 Conditionally Automated Driving System (L3) as an example, the implementation process of the present invention will be described in detail:

[0121] In the model preparation phase, a powerful "teacher model" (i.e., the cloud-based large-scale evaluation model) is trained on a cloud platform using massive amounts of driving data (including driving samples (i.e., second vehicle state samples, second vehicle perception samples, and second driving decision samples) and expert-annotated evaluation labels (i.e., first driving intention evaluation labels, first driving safety evaluation labels, and first vehicle operation evaluation labels)). Then, knowledge distillation technology is used to allow the student model (i.e., the initial large-scale model on the vehicle side, a lightweight network with significantly reduced parameters, such as a Tiny BERT structure or a small convolutional neural network) to learn the outputs of the teacher model (i.e., the cloud-based large-scale evaluation model) (i.e., soft labels for driving intention evaluation, driving safety evaluation, and vehicle operation evaluation) and intermediate layer features. Finally, a lightweight evaluation model (i.e., the vehicle-side large-scale evaluation model) that can be deployed on an in-vehicle chip is obtained.

[0122] During the vehicle deployment phase, the lightweight model obtained from the distillation process is integrated into the software stack of the vehicle's intelligent driving domain controller.

[0123] During the autonomous driving evaluation phase, the input data received by the vehicle-side evaluation model includes vehicle state data, vehicle perception data, and driving decision data. Vehicle perception data includes the position, speed, and type of surrounding vehicles, pedestrians, and obstacles. Vehicle state data includes vehicle speed, acceleration, steering angle, and heading angle. Driving decision data includes preset trajectories, trajectory tracking errors, and control commands (including steering torque and braking pressure). The lightweight evaluation model (i.e., the vehicle-side evaluation model) is a multi-task learning model that simultaneously outputs an autonomous driving score across three dimensions (ranging from 0-100 points or 1-5 stars).

[0124] The input data for the first vehicle-side evaluation sub-model, corresponding to Intent Accuracy, includes the vehicle's planned trajectory, lane line information from a high-precision map, and traffic rules (such as traffic light status and stop lines). This sub-model assesses whether the vehicle's planned trajectory conforms to the current lane lines and the intended navigation route (such as going straight, turning left, or changing lanes). For example, if a straight-going trajectory is generated in a lane where a left turn is required, points will be deducted in this area. It can also assess compliance with traffic signals and signs.

[0125] The input data for the second vehicle-side assessment sub-model for Driving Safety includes the relative distance and speed between the vehicle and all surrounding dynamic objects (i.e., environmental objects), the estimated time to collision (TTC), and the head-to-head distance (THW). The second vehicle-side assessment sub-model is used to calculate a risk metric for the current scenario. For example, this score will significantly decrease if the estimated time to collision is too low (e.g., TTC < 3 seconds), the following distance is too close (e.g., THW < 1 second), or the distance to the adjacent vehicle is too close. The second vehicle-side assessment sub-model can comprehensively assess multiple risk factors, rather than simply using threshold judgments.

[0126] The input data for the third-party vehicle evaluation sub-model for Ride Smoothness includes the vehicle's longitudinal and lateral acceleration. The third-party evaluation sub-model calculates the absolute values ​​of the vehicle's longitudinal and lateral acceleration. Frequent rapid acceleration, hard braking, or excessively fast cornering (high lateral acceleration) will lead to a lower score. Comfortable and smooth driving earns a high score.

[0127] Optionally, traffic efficiency (average vehicle speed / speed limit) and energy economy can also be used as additional output dimensions.

[0128] During the results upload and cloud processing phase, an evaluation report is generated every 30 seconds or after each trip. This report includes time-series dimensional scores and a comprehensive score, along with GPS (Global Positioning System) location information, and is uploaded to the cloud platform via 4G / 5G networks. The cloud platform aggregates evaluation reports from millions of vehicles to create a comprehensive performance heatmap of a vehicle's autonomous driving capabilities, enabling performance bottleneck analysis in different scenarios (such as rainy / snowy weather, congested intersections, etc.).

[0129] This solution deploys a lightweight, large-scale model obtained through knowledge distillation onto the vehicle's computing unit (such as an intelligent driving domain controller). This enables the vehicle to process vehicle sensor data and internal state information of the autonomous driving system in real time, mimicking expert cognitive levels to perform instant reasoning and scoring of the autonomous driving system's performance from multiple key dimensions. Finally, the structured scoring results are uploaded to a cloud platform. This solution addresses the technical problems of existing autonomous driving evaluation methods, such as poor real-time performance, high cost, reliance on human labor, huge bandwidth consumption, and difficulty in conducting continuous and unified evaluations of large fleets. By completing the evaluation on the vehicle side, it achieves millisecond-level response to autonomous driving evaluations, providing instant feedback to the autonomous driving system with strong real-time performance. Furthermore, it only uploads multi-dimensional scoring results with extremely low storage requirements (e.g., a few KB), rather than massive storage requirements. (For example, GB-level) raw data saves over 99% of communication bandwidth, meeting the low bandwidth requirements of cloud platform data transmission; third, it can be easily deployed in fleets of millions, utilizing massive amounts of real road-collected data to build the industry's largest autonomous driving evaluation database, with strong scalability; fourth, it eliminates the subjectivity of manual evaluation, ensuring consistent evaluation standards for all vehicles in any location based on a unified standard vehicle-side evaluation model, guaranteeing the objectivity and consistency of autonomous driving evaluation; fifth, by ensuring that raw sensitive data does not leave the vehicle, only uploading anonymized lightweight scoring results, it greatly protects user privacy and data security; finally, the cloud-based evaluation model can be continuously iterated and optimized through feedback data from the vehicle, and the vehicle-side evaluation model can be updated OTA, realizing the continuous evolution of the vehicle-side evaluation model and continuously improving evaluation capabilities.

[0130] Example 3

[0131] Figure 5 This is a schematic diagram of an autonomous driving evaluation device provided in Embodiment 3 of the present invention. This embodiment of the invention is applicable to situations where autonomous driving evaluation is performed based on a vehicle-side and cloud platform. The device executes an autonomous driving evaluation method and is implemented in hardware and / or software. The device can be configured in an electronic device that carries autonomous driving evaluation functions.

[0132] See Figure 5The autonomous driving evaluation device shown includes a vehicle terminal 501 and a cloud platform 502. The vehicle terminal is used to acquire real-time vehicle status data, vehicle perception data, and driving decision data of the vehicle under test. The vehicle terminal uses a large-scale vehicle evaluation model to output a first estimated driving intention evaluation, a first estimated driving safety evaluation, and a first estimated vehicle operation evaluation of the vehicle under test based on the vehicle status data, the vehicle perception data, and the driving decision data, and feeds these evaluations back to the cloud platform. The cloud platform integrates the first estimated driving intention evaluation, the first estimated driving safety evaluation, and the first estimated vehicle operation evaluation to obtain a comprehensive evaluation result for the vehicle under test.

[0133] The technical solution of this invention enables multi-dimensional evaluation of autonomous vehicles through the vehicle end, achieving millisecond-level response for autonomous driving evaluation and providing instant feedback to the autonomous driving system, thus improving the real-time performance of autonomous driving evaluation. Furthermore, by employing a unified vehicle-side evaluation model, the subjectivity of manual evaluation is eliminated, ensuring the objectivity and consistency of autonomous driving evaluation for all vehicles at any location. Simultaneously, by feeding back the multi-dimensional evaluation results of the vehicle under test to the cloud platform, the original sensitive data from the vehicle end is prevented from leaving the vehicle; only anonymized multi-dimensional evaluation results are uploaded, greatly protecting user privacy and data security. Through the cloud platform, the first estimated driving intention evaluation, the first estimated driving safety evaluation, and the first estimated vehicle operation evaluation are integrated to obtain the comprehensive evaluation result of the vehicle under test, which can be easily deployed in fleets of millions, improving the scalability of autonomous driving evaluation.

[0134] In an optional embodiment of the present invention, the apparatus further includes: a cloud platform 502, configured to acquire a first vehicle state sample, a first vehicle perception sample, and a first driving decision sample before outputting a first estimated driving intention evaluation, a first estimated driving safety evaluation, and a first estimated vehicle operation evaluation of the vehicle to be detected based on the vehicle state data, the vehicle perception data, and the driving decision data using a vehicle-side evaluation model; the cloud platform 502 is configured to input the first vehicle state sample, the first vehicle perception sample, and the first driving decision sample into a cloud-based evaluation model, and output a driving intention evaluation soft label, a driving safety evaluation soft label, and a vehicle operation evaluation soft label .... The vehicle-side initial large model is input into the vehicle-side sample, the first vehicle perception sample, and the first driving decision sample, and outputs a second estimated driving intention evaluation, a second estimated driving safety evaluation, and a second estimated vehicle operation evaluation. The model parameters of the vehicle-side initial large model are fewer than those of the cloud-based evaluation large model. The cloud platform 502 is used to train the vehicle-side initial large model based on the differences between the second estimated driving intention evaluation and the driving intention evaluation soft label, the second estimated driving safety evaluation and the driving safety evaluation soft label, and the second estimated vehicle operation evaluation and the vehicle operation evaluation soft label, to obtain the vehicle-side evaluation large model. The cloud platform 502 is used to deploy the vehicle-side evaluation large model to the vehicle.

[0135] In an optional embodiment of the present invention, the apparatus further includes: a cloud platform 502, configured to acquire a second vehicle state sample, a second vehicle perception sample, a second driving decision sample, a first driving intention evaluation label, a first driving safety evaluation label, and a first vehicle operation evaluation label before inputting the first vehicle state sample, the first vehicle perception sample, and the first driving decision sample into the cloud evaluation large model via the cloud platform and outputting driving intention evaluation soft labels, driving safety evaluation soft labels, and vehicle operation evaluation soft labels; the cloud platform 502 is configured to input the second vehicle state sample, the second vehicle perception sample, and the second driving decision sample into the cloud initial large model and output a reference driving intention evaluation, a reference driving safety evaluation, and a reference vehicle operation evaluation; the cloud platform 502 is configured to train the cloud reference large model based on the differences between the reference driving intention evaluation and the first driving intention evaluation label, the differences between the reference driving safety evaluation and the first driving safety evaluation label, and the differences between the reference vehicle operation evaluation and the first vehicle operation evaluation label, to obtain a cloud evaluation large model.

[0136] In an optional embodiment of the present invention, the device further includes: a cloud platform 502, configured to periodically acquire vehicle status data, vehicle perception data, and driving decision data of each vehicle under test after the cloud platform integrates the first estimated driving intention evaluation, the first estimated driving safety evaluation, and the first estimated vehicle operation evaluation to obtain the comprehensive evaluation result of the vehicle under test; and acquire corresponding second driving intention evaluation labels, second driving safety evaluation labels, and second vehicle operation evaluation labels; the cloud platform 502 is configured to update the cloud-based evaluation model based on the vehicle status data, the vehicle perception data, the driving decision data, the second driving intention evaluation labels, the second driving safety evaluation labels, and the second vehicle operation evaluation labels; the cloud platform 502 is configured to update the vehicle-side evaluation model using the updated cloud-based evaluation model, and deploy the updated vehicle-side evaluation model to the vehicle.

[0137] In an optional embodiment of the present invention, the vehicle terminal 501 includes: the vehicle terminal 501 is configured to use a first vehicle terminal evaluation sub-model to output a first estimated driving intention evaluation of the vehicle to be detected based on traffic rules, lane line information, and the vehicle's planned trajectory; the vehicle terminal 501 is configured to use a second vehicle terminal evaluation sub-model to output a first estimated driving safety evaluation of the vehicle to be detected based on the relative distance, relative speed, estimated collision time, and inter-vehicle time distance between the vehicle to be detected and environmental objects; and the vehicle terminal 501 is configured to use a third vehicle terminal evaluation sub-model to output a first estimated vehicle operation evaluation of the vehicle to be detected based on the vehicle's longitudinal acceleration and lateral acceleration.

[0138] In an optional embodiment of the present invention, the device further includes: a vehicle terminal 501, configured to, while passing through the vehicle terminal, use a vehicle-side evaluation model to output a first estimated driving intention evaluation, a first estimated driving safety evaluation, and a first estimated vehicle operation evaluation of the vehicle to be detected based on the vehicle status data, the vehicle perception data, and the driving decision data, and simultaneously feed these evaluations back to the cloud platform, also feed back the vehicle identifier, timestamp, and vehicle location of the vehicle to be detected to the cloud platform; and a cloud platform 502, configured to, after synthesizing the first estimated driving intention evaluation, the first driving safety evaluation, and the first vehicle operation evaluation through the cloud platform to obtain a comprehensive evaluation result of the vehicle to be detected, visualize the vehicle to be detected based on the vehicle identifier, timestamp, vehicle location, and comprehensive evaluation result of the vehicle to be detected.

[0139] The autonomous driving evaluation device provided in the embodiments of the present invention can execute the autonomous driving evaluation method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0140] In the technical solutions of this invention, the acquisition, storage, and application of data such as vehicle status data, vehicle perception data, driving decision data, first vehicle status sample, first vehicle perception sample, first driving decision sample, second vehicle status sample, second vehicle perception sample, second driving decision sample, first driving intention evaluation label, first driving safety evaluation label, first vehicle operation evaluation label, second driving intention evaluation label, second driving safety evaluation label, and second vehicle operation evaluation label, third vehicle status sample, third vehicle perception sample, third driving decision sample, third driving intention evaluation label, third driving safety evaluation label, third vehicle operation evaluation label, vehicle identifier, timestamp, and vehicle location, etc., all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0141] Example 4

[0142] Figure 6 A schematic diagram of an electronic device 600 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0143] like Figure 6 As shown, the electronic device 600 includes at least one processor 601 and a memory, such as a read-only memory (ROM) 602 or a random access memory (RAM) 603, communicatively connected to the at least one processor 601. The memory stores computer programs executable by the at least one processor. The processor 601 can perform various appropriate actions and processes based on the computer program stored in the ROM 602 or loaded into the RAM 603 from storage unit 608. The RAM 603 may also store various programs and data required for the operation of the electronic device 600. The processor 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0144] Multiple components in electronic device 600 are connected to I / O interface 605, including: input unit 606, such as keyboard, mouse, etc.; output unit 607, such as various types of displays, speakers, etc.; storage unit 608, such as disk, optical disk, etc.; and communication unit 609, such as network card, modem, wireless transceiver, etc. Communication unit 609 allows electronic device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0145] Processor 601 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 601 performs the various methods and processes described above, such as autonomous driving evaluation methods.

[0146] In some embodiments, the autonomous driving evaluation method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by processor 601, one or more steps of the autonomous driving evaluation method described above may be performed. Alternatively, in other embodiments, processor 601 may be configured to perform the autonomous driving evaluation method by any other suitable means (e.g., by means of firmware).

[0147] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0148] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0149] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0150] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0151] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0152] A computing system can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system. It addresses the shortcomings of traditional physical hosts and VPS (Virtual Private Server) services, such as high management difficulty and weak business scalability.

[0153] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0154] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. An automatic driving evaluation method characterized by, The method comprises: obtaining, by a vehicle end, real-time self-vehicle state data, vehicle perception data and driving decision data of a to-be-detected vehicle; using, by the vehicle end, a vehicle-end evaluation large model to output, according to the self-vehicle state data, the vehicle perception data and the driving decision data, a first estimated driving intention evaluation, a first estimated driving safety evaluation and a first estimated vehicle operation evaluation of the to-be-detected vehicle, and feeding back to a cloud end platform; comprehensively evaluating, by the cloud end platform, the first estimated driving intention evaluation, the first estimated driving safety evaluation and the first estimated vehicle operation evaluation to obtain a comprehensive evaluation result of the to-be-detected vehicle.

2. The method of claim 1, wherein, Before the step of using, by the vehicle end, the vehicle-end evaluation large model to output, according to the self-vehicle state data, the vehicle perception data and the driving decision data, the first estimated driving intention evaluation, the first estimated driving safety evaluation and the first estimated vehicle operation evaluation, the method further comprises: obtaining, by the cloud end platform, first self-vehicle state samples, first vehicle perception samples and first driving decision samples; inputting, by the cloud end platform, the first self-vehicle state samples, the first vehicle perception samples and the first driving decision samples into a cloud-end evaluation large model to output driving intention evaluation soft labels, driving safety evaluation soft labels and vehicle operation evaluation soft labels; inputting, by the cloud end platform, the first self-vehicle state samples, the first vehicle perception samples and the first driving decision samples into a vehicle-end initial large model to output second estimated driving intention evaluation, second estimated driving safety evaluation and second estimated vehicle operation evaluation; wherein the vehicle-end initial large model has less model parameters than the cloud-end evaluation large model; training, by the cloud end platform, the vehicle-end initial large model according to differences between the second estimated driving intention evaluation and the driving intention evaluation soft labels, between the second estimated driving safety evaluation and the driving safety evaluation soft labels, and between the second estimated vehicle operation evaluation and the vehicle operation evaluation soft labels to obtain a vehicle-end evaluation large model; deploying, by the cloud end platform, the vehicle-end evaluation large model to the vehicle end.

3. The method of claim 2, wherein, Before the step of inputting, by the cloud end platform, the first self-vehicle state samples, the first vehicle perception samples and the first driving decision samples into the cloud-end evaluation large model to output the driving intention evaluation soft labels, the driving safety evaluation soft labels and the vehicle operation evaluation soft labels, the method further comprises: obtaining, by the cloud end platform, second self-vehicle state samples, second vehicle perception samples, second driving decision samples, first driving intention evaluation annotation labels, first driving safety evaluation annotation labels and first vehicle operation evaluation annotation labels; inputting, by the cloud end platform, the second self-vehicle state samples, the second vehicle perception samples and the second driving decision samples into a cloud-end initial large model to output reference driving intention evaluation, reference driving safety evaluation and reference vehicle operation evaluation; The cloud-end reference large model is trained by the cloud platform according to differences between the reference driving intention evaluation and the first driving intention evaluation label, differences between the reference driving safety evaluation and the first driving safety evaluation label, and differences between the reference vehicle operation evaluation and the first vehicle operation evaluation label, to obtain a cloud-end evaluation large model.

4. The method of claim 3, wherein, After the cloud-end platform synthesizes the first estimated driving intention evaluation, the first estimated driving safety evaluation, and the first estimated vehicle operation evaluation to obtain the comprehensive evaluation result of the to-be-detected vehicle, the method further includes: Periodically, the cloud-end platform acquires self-vehicle state data, vehicle perception data, and driving decision data of each to-be-detected vehicle, and acquires corresponding second driving intention evaluation labels, second driving safety evaluation labels, and second vehicle operation evaluation labels; The cloud-end evaluation large model is updated by the cloud-end platform according to the self-vehicle state data, the vehicle perception data, the driving decision data, the second driving intention evaluation label, the second driving safety evaluation label, and the second vehicle operation evaluation label; The cloud-end platform updates the vehicle-end evaluation large model by using the updated cloud-end evaluation large model, and deploys the updated vehicle-end evaluation large model to the vehicle end.

5. The method of claim 1, wherein, The vehicle-end evaluation large model is used by the vehicle end to output the first estimated driving intention evaluation, the first estimated driving safety evaluation, and the first estimated vehicle operation evaluation of the to-be-detected vehicle according to the self-vehicle state data, the vehicle perception data, and the driving decision data, including: The first vehicle-end evaluation sub-model is used by the vehicle end to output the first estimated driving intention evaluation of the to-be-detected vehicle according to traffic rules, lane line information, and a self-vehicle planned trajectory; The second vehicle-end evaluation sub-model is used by the vehicle end to output the first estimated driving safety evaluation of the to-be-detected vehicle according to a relative distance, a relative speed, an estimated collision time, and an inter-vehicle time interval between the to-be-detected vehicle and an environmental object; The third vehicle-end evaluation sub-model is used by the vehicle end to output the first estimated vehicle operation evaluation of the to-be-detected vehicle according to a self-vehicle longitudinal acceleration and a self-vehicle lateral acceleration.

6. The method of claim 1, wherein, While the vehicle-end evaluation large model is used by the vehicle end to output the first estimated driving intention evaluation, the first estimated driving safety evaluation, and the first estimated vehicle operation evaluation of the to-be-detected vehicle according to the self-vehicle state data, the vehicle perception data, and the driving decision data and is fed back to the cloud-end platform, the method further includes: The vehicle end feeds back a vehicle identifier, a timestamp, and a vehicle position of the to-be-detected vehicle to the cloud-end platform; After the cloud-end platform synthesizes the first estimated driving intention evaluation, the first estimated driving safety evaluation, and the first estimated vehicle operation evaluation to obtain the comprehensive evaluation result of the to-be-detected vehicle, the method further includes: The cloud platform is used for visualizing the vehicle to be detected according to the vehicle identification, the time stamp, the vehicle position and the comprehensive evaluation result of the vehicle to be detected.

7. An automatic driving evaluation device characterized by comprising: The device comprises: The vehicle end is used for acquiring self-vehicle state data, vehicle perception data and driving decision data of a vehicle to be detected in real time; The vehicle end is used for adopting a vehicle-end evaluation large model to output a first estimated driving intention evaluation, a first estimated driving safety evaluation and a first estimated vehicle operation evaluation of the vehicle to be detected according to the self-vehicle state data, the vehicle perception data and the driving decision data, and feeding back to a cloud platform; The cloud platform is used for comprehensively evaluating the first estimated driving intention evaluation, the first estimated driving safety evaluation and the first estimated vehicle operation evaluation to obtain a comprehensive evaluation result of the vehicle to be detected.

8. An electronic device, comprising: The electronic device comprises: at least one processor; and a memory connected with the at least one processor in communication; wherein The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the automatic driving evaluation method in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for enabling the processor to execute the automatic driving evaluation method in any one of claims 1-6 when executed.

10. A computer program product, characterised in that, The computer program product comprises a computer program, and the computer program is executed by the processor to implement the automatic driving evaluation method according to any one of claims 1-6.

Images