Test methods, apparatus, media and procedures for vehicles and autonomous driving systems

Abstract

This invention relates to a testing method, apparatus, medium, and program product for vehicles and autonomous driving systems, and pertains to the field of autonomous driving technology. The method includes: acquiring interaction behavior data between the autonomous driving system and other traffic participants in a test scenario; determining multi-dimensional traffic coordination indicators for the autonomous driving system in the test scenario; weighted fusion of the multi-dimensional traffic coordination indicators to obtain a traffic cooperation index for the autonomous driving system; and generating test results based on the traffic cooperation index. By acquiring relevant interaction data of the autonomous driving system and constructing a unified multi-dimensional evaluation framework based on interaction success rate, interaction timeliness, interaction smoothness, and interaction fairness, the test results can include verification of the autonomous driving system's ability to participate in complex traffic scenarios, forming a more comprehensive quantitative evaluation of the autonomous driving system's performance, and improving the intuitiveness, comparability, and engineering application value of the test results.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and specifically to a testing method, apparatus, medium, and program product for vehicles and autonomous driving systems. Background Technology

[0002] With the development of autonomous driving technology, autonomous driving systems are gradually moving from the laboratory stage to closed-course and real-world road testing. Currently, testing of autonomous driving systems mainly focuses on verifying the performance of sensors such as vision and radar perception capabilities, and functional testing of control strategies or single scenarios, such as emergency braking and lane keeping. However, this type of testing is insufficient to measure the ability of autonomous driving systems to participate in complex traffic scenarios, becoming a bottleneck hindering the comprehensive practical application of autonomous driving technology.

[0003] However, both perception capability verification and functional testing are limited to single functions or partial decision-making links of autonomous driving systems, with the goal of avoiding collisions and maintaining safety. In actual road driving, however, autonomous driving systems often face uncontrollable pedestrians or other vehicles, and safety is only the minimum requirement for testing. Traditional testing methods cannot comprehensively measure the ability of autonomous driving systems to participate in complex traffic scenarios, which is a bottleneck restricting the improvement and commercial application of autonomous driving systems. Summary of the Invention

[0004] One of the objectives of this invention is to provide a testing method, apparatus, medium, and program product for vehicles and autonomous driving systems, in order to solve the problem of one-sided testing results for autonomous driving systems.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A testing method for an autonomous driving system, comprising:

[0007] Acquire data on the interaction behavior of autonomous driving systems with other traffic participants in test scenarios;

[0008] Based on interactive behavior data, multi-dimensional traffic coordination indicators for autonomous driving systems in test scenarios are determined. These multi-dimensional traffic coordination indicators include interaction success rate, interaction timeliness, interaction smoothness, and interaction fairness.

[0009] By weighted and fused multi-dimensional traffic coordination indicators, a traffic coordination index for autonomous driving systems in test scenarios is obtained.

[0010] Based on the traffic cooperation index, test results of the autonomous driving system are generated in the test scenario.

[0011] Furthermore, based on interaction behavior data, multi-dimensional traffic coordination indicators for the autonomous driving system in the test scenario are determined, including:

[0012] The success rate of an interaction is determined by the ratio of the number of successful interactions to the total number of interactions in the interaction behavior data.

[0013] The response time of the autonomous driving system to the interaction signals of other traffic participants in the interaction behavior data is compared with a preset time threshold, and the interaction timeliness is determined based on the comparison results.

[0014] The maximum acceleration of the autonomous driving system during the interaction process is compared with a preset acceleration threshold in the interaction behavior data to determine the smoothness of the interaction.

[0015] The system calculates the efficiency of autonomous driving systems in navigating other traffic participants based on interaction behavior data, and determines the fairness of the interaction based on the efficiency of navigating other traffic participants.

[0016] Furthermore, multi-dimensional traffic coordination indicators are weighted and fused to obtain the traffic coordination index of the autonomous driving system in the test scenario, including:

[0017] Based on the characteristics of the test scenario, the weight coefficients corresponding to each dimension of traffic coordination indicators are adjusted and obtained. The weight coefficients are adapted to the characteristics of the scenario, which include at least one of the following: traffic participant composition, traffic density, road type, and signal control method.

[0018] The traffic coordination index of the autonomous driving system in the test scenario is obtained by weighting and fusing multi-dimensional traffic coordination indicators based on weight coefficients.

[0019] Furthermore, data on the interaction behavior of the autonomous driving system with other traffic participants in the test scenario will be obtained, including:

[0020] Acquire raw behavioral data of the target vehicle and other traffic participants in the test scenario; the target vehicle makes decisions through a pre-deployed autonomous driving system.

[0021] Based on the raw behavioral data, identify the interaction events between the target vehicle and other traffic participants;

[0022] Interactive behavior data is generated based on the timing of the target vehicle's behavioral responses to other traffic participants and the end status of the interaction during the interactive event.

[0023] Furthermore, based on the raw behavioral data, the interaction events between the target vehicle and other traffic participants are identified, including:

[0024] Based on the raw behavioral data, identify interaction request behaviors, which include first request behaviors that characterize the target vehicle's interaction intentions toward traffic participants, and / or, include second request behaviors that characterize the traffic participants' interaction intentions toward the target vehicle.

[0025] Based on the interaction request behavior, the interaction response behavior is extracted. The interaction response behavior includes the first response behavior of the traffic participant after the first request behavior occurs, and / or, the second response behavior of the target vehicle after the second request behavior occurs.

[0026] Interaction events are generated based on the interaction request behavior and the interaction response behavior.

[0027] Furthermore, identify interactive request behaviors, including:

[0028] Detect candidate request behaviors belonging to the preset interaction type in the raw behavior data of the target vehicle;

[0029] If a candidate request behavior is identified as an interactive request behavior, it is determined whether there is an interactive object within the interactive area that can respond to the candidate request behavior.

[0030] Furthermore, based on the traffic cooperation index, test results of the autonomous driving system in the test scenario are generated, including:

[0031] Obtain raw behavioral data of the target vehicle with the deployed autonomous driving system in the test scenario;

[0032] Based on the original behavioral data of the target vehicle, determine the comprehensive indicators of the autonomous driving system. The comprehensive indicators include at least one of the rationality evaluation indicators and the comfort evaluation indicators.

[0033] The comprehensive indicators and traffic coordination indicators are weighted and integrated, and the test results of the autonomous driving system in the test scenario are generated based on the integration results.

[0034] Furthermore, the rationality evaluation indicators include at least one of the following vehicle behavior rationality index, lane change behavior rationality index, lane selection rationality index, and speed control rationality index. The rationality evaluation indicators are determined in the following ways:

[0035] Based on the relationship between the actual following distance and the reference following distance of the target vehicle, a reasonableness index for following behavior is determined.

[0036] The rationality index of lane-changing behavior is determined based on at least one of the following scores: timing rationality score, spatial rationality score, and intention expression rationality score of lane-changing decision-making in autonomous driving systems.

[0037] Based on the relationship between the distance the target vehicle travels in the optimal lane and the total distance traveled, a lane selection rationality index is determined;

[0038] Based on the relationship between the actual speed of the target vehicle and the reference speed, a speed control rationality index is determined. The reference speed is determined based on at least one of the following factors: road speed limit, traffic flow speed, road geometry, and driving scenario.

[0039] Furthermore, the timing rationality score is calculated based on the distance between the target vehicle and the vehicles behind it in the target lane during the target vehicle's lane change process;

[0040] The spatial rationality score is calculated based on the matching degree between the vehicle speed during the target vehicle's lane change process and the available space in the target lane;

[0041] The reasonableness score of the expressed intent is determined based on the timing of the target vehicle turning on and / or off its turn signals.

[0042] Furthermore, the comfort evaluation index includes at least one of the longitudinal comfort index, lateral comfort index, and vertical comfort index, and the comfort evaluation index is determined in the following way:

[0043] The longitudinal comfort index is determined based on the relationship between the longitudinal acceleration change rate and the longitudinal impact threshold of the target vehicle.

[0044] The lateral comfort index is determined based on the relationship between the rate of change of lateral acceleration of the target vehicle and the lateral impact threshold.

[0045] The vertical comfort index is determined based on the relationship between the root mean square value of the vertical weighted acceleration of the target vehicle and the vertical acceleration threshold.

[0046] A testing apparatus for an autonomous driving system, comprising:

[0047] The acquisition module is used to acquire data on the interaction behavior between the autonomous driving system and other traffic participants in the test scenario.

[0048] The indicator determination module is used to determine the multi-dimensional traffic coordination indicators of the autonomous driving system for the test scenario based on the interaction behavior data. The multi-dimensional traffic coordination indicators include interaction success rate, interaction timeliness, interaction smoothness, and interaction fairness.

[0049] The indicator fusion module is used to weight and fuse multi-dimensional traffic coordination indicators to obtain the traffic coordination index of the autonomous driving system in the test scenario.

[0050] The generation module is used to generate test results for autonomous driving systems in test scenarios based on the traffic cooperation index.

[0051] A vehicle includes: a vehicle body and an autonomous driving system deployed in the vehicle body, wherein the autonomous driving system passes a test using any of the above test methods.

[0052] An electronic device includes: a processor, and a memory communicatively connected to the processor;

[0053] The memory stores instructions that the computer executes;

[0054] The processor executes computer execution instructions stored in memory to implement a test method for an autonomous driving system as described above.

[0055] A computer-readable storage medium includes: a test method for storing computer-executable instructions in the computer-readable storage medium, which, when executed by a processor, are used to implement an autonomous driving system as described above.

[0056] A computer program product includes a computer program that, when executed by a processor, implements a test method for an autonomous driving system as described above.

[0057] The beneficial effects of this invention are as follows: By acquiring interaction behavior data between the autonomous driving system and other traffic participants in the test scenario, and determining multi-dimensional traffic coordination indicators including interaction success rate, interaction timeliness, interaction smoothness, and interaction fairness based on the interaction behavior data, the interactive performance of the autonomous driving system in the test scenario can be comprehensively evaluated from multiple dimensions. Then, based on the traffic cooperation index obtained through multi-dimensional traffic coordination indicators, test results are generated, so that the test results include verification of the autonomous driving system's ability to participate in complex traffic scenarios, forming a more comprehensive quantitative evaluation of the autonomous driving system's performance, and improving the intuitiveness and comparability of the test results. Attached Figure Description

[0058] Figure 1 A flowchart illustrating a testing method for an autonomous driving system provided as an exemplary embodiment of the present invention;

[0059] Figure 2 A flowchart illustrating the determination of traffic coordination indicators is provided for an exemplary embodiment of the present invention.

[0060] Figure 3 A flowchart illustrating the determination of rationality evaluation indicators provided for an exemplary embodiment of the present invention;

[0061] Figure 4 A schematic diagram of the architecture of a multi-level testing and evaluation system provided for an exemplary embodiment of the present invention;

[0062] Figure 5A schematic diagram of a multi-level test evaluation system provided for an exemplary embodiment of the present invention;

[0063] Figure 6 A schematic diagram of the structure of a test apparatus for an autonomous driving system provided as an exemplary embodiment of the present invention;

[0064] Figure 7 This is a schematic diagram of the structure of an electronic device provided as an exemplary embodiment of the present invention.

[0065] The accompanying drawings have illustrated specific embodiments of the invention, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the invention in any way, but rather to illustrate the concept of the invention to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0066] The embodiments of the present invention will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention and not for limiting the scope of protection of the present invention.

[0067] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0068] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0069] The terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, product, or apparatus. Without further limitation, the presence of additional identical or equivalent elements in the process, method, product, or apparatus that includes elements is not excluded. For example, the use of terms such as "first," "second," etc., to indicate names does not imply any particular order.

[0070] With the development of intelligent vehicle technology, Level 3 and above high-level autonomous driving systems are gradually moving from laboratories to closed test tracks and real-world road tests. These systems can operate in vehicles to make decisions in place of human drivers, and testing them provides a basis for system iteration, market access certification, and commercial deployment.

[0071] Currently, autonomous driving system testing schemes mostly employ methods such as perception capability verification, control strategy simulation, or specific functional scenario testing. For example, in a closed environment, pre-set lane lines, cones, dummies, or traffic signal devices are used to test whether the vehicle can recognize lanes, detect targets, perform braking, or complete steering. In road testing, human intervention records, event logs, or video playback are often combined to judge the success or failure of the vehicle at a certain functional point, such as whether braking was taken to avoid collisions with obstacles or whether the vehicle maintained lane alignment.

[0072] However, the aforementioned testing methods have certain limitations. They can only reflect the autonomous driving system's ability to perform a single task and identify problems in the local decision-making chain, such as sensor false detections or policy failures (e.g., failure to apply brakes). They fail to provide a more comprehensive assessment of the autonomous driving system's continuous behavioral capabilities. For example, when a pedestrian is crossing the road at a zebra crossing, the autonomous driving system might brake suddenly when it is close to the pedestrian. While this could avoid a collision, it could also cause panic among pedestrians and lead to a safety accident. Another example is in lane-changing scenarios. After the autonomous driving system activates its turn signal and merges into an adjacent lane, a vehicle in the adjacent lane may not slow down, resulting in a very close distance between the two vehicles without a collision. Although no collision occurs, this still poses a certain safety hazard. In traditional testing schemes, such situations would pass the test because they do not result in a collision and meet safety requirements. This type of test result cannot reflect the true capabilities of the autonomous driving system.

[0073] The inventors discovered through research that the aforementioned problems stem from the fact that the testing of autonomous driving systems relies excessively on the single dimension of safety evaluation, neglecting the importance of interaction capabilities in real-world traffic scenarios. Traffic participants with certain interaction capabilities can improve traffic coordination through their interactions with other traffic participants. For example, a following vehicle can proactively slow down based on the interaction signals (such as turn signals) from the vehicle in front to provide sufficient space for lane changing, and the vehicle in front can change lanes only after recognizing the slowing behavior of the following vehicle.

[0074] Based on the aforementioned problems and findings, this paper proposes a technical concept: during the testing of autonomous driving systems, data on the interaction behavior between the autonomous driving system and other traffic participants is acquired. This data is then used to determine multi-dimensional traffic coordination indicators, such as interaction success rate and interaction timeliness. Based on these indicators, a traffic cooperation index characterizing the interactive capabilities of the autonomous driving system is determined, generating test results. By incorporating the interactive performance of the autonomous driving system into the testing framework for comprehensive evaluation, the test results can include an overall judgment of interaction quality, response efficiency, and behavioral consistency, thereby improving the objectivity and intuitiveness of the test results and providing a high-quality data foundation for subsequent system optimization and standardized evaluation.

[0075] The application scenarios described above are only partial examples. Those skilled in the art can expand the applications according to specific needs and scenarios, and the embodiments of the present invention do not impose specific limitations in this regard. The method according to an exemplary embodiment of the present invention will now be described with reference to the accompanying drawings.

[0076] Figure 1 This is a flowchart illustrating a testing method for an autonomous driving system provided as an exemplary embodiment of the present invention. Figure 1 As shown, the method may include:

[0077] Step S101: Obtain data on the interaction behavior between the autonomous driving system and other traffic participants in the test scenario.

[0078] In this embodiment of the invention, the test scenario refers to a traffic environment used to evaluate the performance of an autonomous driving system. It can be a closed test track, an open road test area, a hardware-in-the-loop simulation environment, or a combined virtual and real-world test environment. The test scenario may include road structures, traffic signs and markings, traffic lights, and other traffic participants. The autonomous vehicle can control the target vehicle (i.e., a vehicle equipped with an autonomous driving system, also referred to as an "autonomous driving vehicle") to drive within the test scenario. Other traffic participants may include other vehicles, pedestrians, non-motorized vehicles, etc.

[0079] Interactive behavior data refers to a set of data that reflects the process of mutual influence, yielding, competitive passage, cooperative passage, or conflict avoidance between autonomous driving systems and other traffic participants. This set of data can include not only the autonomous vehicle's own state data, but also trajectory data, relative motion relationship data, intention response data, and event timing data of other traffic participants.

[0080] For example, onboard cameras, LiDAR, high-precision positioning units, and onboard controller log acquisition modules can be deployed on autonomous vehicles, while roadside cameras, roadside radar, edge computing nodes, or traffic light control systems can be configured on the roadside, with clock synchronization based on a unified time standard. During data acquisition, the onboard unit can output real-time data such as the vehicle's position, heading angle, speed, acceleration, steering wheel angle, braking requests, and planned path; the roadside unit can output data such as the position, speed, acceleration, and direction of movement of surrounding traffic participants. For simulation testing environments, the simulation platform can also directly export data such as the true trajectory values, collision determination results, minimum time distance, and minimum collision time between the autonomous vehicle and each virtual traffic participant.

[0081] For example, after all the data has been collected, the data can be synchronized in time and formatted to build a multi-dimensional test truth library.

[0082] In some possible implementations, the formation of interactive behavior data can also incorporate the internal state information of the autonomous driving system. For example, candidate trajectories and target speeds can be read from the planning module, intermediate decision information such as priority passage and yielding decisions can be read from the decision-making module, and execution instructions and feedback can be read from the control module. By correlating external observation data with internal state information, it is possible to identify when the autonomous driving system perceives the interactive object, when it forms an interactive intention, when it issues a control action, and whether the action is actually executed. The interactive behavior data obtained in this way is no longer a simple set of motion trajectories, but a complete interactive time-series data covering the perception, decision-making, planning, and control links.

[0083] Step S102: Based on the interaction behavior data, determine the multi-dimensional traffic coordination indicators of the autonomous driving system for the test scenario.

[0084] Among them, the multi-dimensional traffic coordination indicators include interaction success rate, interaction timeliness, interaction smoothness, and interaction fairness.

[0085] In this embodiment of the invention, the multi-dimensional traffic coordination index is a set of indicators formed by decomposing and quantifying the completion capability, response efficiency, action continuity, and resource allocation rationality exhibited by the autonomous driving system during the interaction process in the test scenario. Interaction success rate can be used to characterize whether the autonomous driving system completes the predetermined interaction task while meeting safety and rule constraints; interaction timeliness can be used to characterize the time efficiency between the autonomous driving system recognizing external interaction needs and implementing an effective response; interaction smoothness can be used to characterize whether the acceleration, jerk, steering changes, and trajectory curvature changes of the autonomous driving system are continuous during the interaction process; and interaction fairness can be used to characterize whether the autonomous driving system exhibits excessive conservatism, excessive aggressiveness, or prolonged occupation of passage opportunities when sharing road resources with other traffic participants.

[0086] For example, interactive event identification can be performed on the acquired interactive behavior data. Based on preset rules, request behaviors made by autonomous vehicles or other traffic participants that may trigger interactive events can be identified. Related behaviors with spatiotemporal correlation can then be identified as response behaviors to the interactive events. Indicators such as interaction success rate, interaction timeliness, interaction smoothness, and interaction fairness can be calculated based on the occurrence time and outcome of these behaviors. For instance, for an interactive event, the start time, critical response time, and end time of the event can be determined first. The start time can be defined as the moment when the autonomous driving system first perceives a traffic participant with whom it has a potential spatiotemporal conflict or cooperative traffic relationship. The critical response time can be defined as the moment when the autonomous driving system first outputs a decision or control action related to the interaction. The end time can be defined as the moment when the conflict is resolved or the interaction is completed.

[0087] Step S103: Weighted fusion of multi-dimensional traffic coordination indicators to obtain the traffic coordination index of the autonomous driving system in the test scenario.

[0088] The traffic cooperation index is a comprehensive evaluation value formed by uniformly quantifying and mapping interaction success rate, interaction timeliness, interaction smoothness and interaction fairness. It is used to characterize the overall level of the autonomous driving system in forming a safe, effective, continuous and reasonable interactive relationship with other traffic participants in the current test scenario.

[0089] For example, interaction success rate, interaction timeliness, interaction smoothness, and interaction fairness can be normalized first, and the normalization results can be numerical indicators. For indicators whose values ​​are between 0 and 1 and which are better the larger the value, such as interaction success rate, the original value can be used directly; for raw quantities like response time, a threshold mapping function can be used to convert them into scores between 0 and 1; for penalty-type quantification results in smoothness and fairness, a piecewise function can be used to complete the normalization. To reduce the impact of outliers, outlier removal and confidence correction can also be performed before normalization.

[0090] For example, the traffic coordination index can be calculated using a linear weighted model, and its calculation formula can be expressed as follows (1):

[0091] (1)

[0092] In the above formula, I coordinate For the traffic coordination index, I success To improve the success rate of the interaction, I timeliness For the timeliness of interaction, I smoothness For smooth interaction, I fairness To ensure fairness in the interaction, ω1, ω2, ω3, and ω4 are the weight coefficients for interaction success rate, interaction timeliness, interaction smoothness, and interaction fairness, respectively, and the sum of these weight coefficients can be 1.

[0093] The weighted fusion mechanism can transform previously scattered multi-dimensional indicators such as interaction success rate and interaction timeliness into a unified traffic cooperation index, so that the interaction capabilities of autonomous driving systems can be presented in a single comparable numerical form, while retaining the adaptability to key capabilities in different scenarios.

[0094] Step S104: Based on the traffic cooperation index, generate test results of the autonomous driving system in the test scenario.

[0095] The test results refer to the quantitative or qualitative evaluation conclusions output based on data such as the traffic coordination index. These conclusions can be expressed as pass / fail, or require retesting, or as ratings, percentage scores, risk labels, problem attribution information, and analytical reports for subsequent optimization. The process of generating test results can not only focus on whether the final index reaches the threshold, but also on the imbalance between indicators across different dimensions, thus ensuring that the output results possess both certification and R&D improvement value.

[0096] For example, result mapping rules corresponding to the test tasks can be pre-established. These mapping rules may include a total score threshold, key safety baselines, scenario level boundaries, and anomaly event penalty mechanisms. For instance, when the traffic coordination index is greater than or equal to the first threshold and there are no rejection events such as collisions, running red lights, or crossing the road, a pass result is output; when the traffic coordination index is lower than the first threshold but higher than the second threshold, a result requiring improvement or suggesting retesting is output; when the traffic coordination index is lower than the second threshold, or although it reaches the first threshold but a key rejection event occurs, a fail result is output.

[0097] In the above embodiments, interactive behavior data between the autonomous driving system and other traffic participants in the test scenario is acquired. Based on the interactive behavior data, multi-dimensional traffic coordination indicators of the autonomous driving system in the test scenario can be determined. By weighted fusion of the multi-dimensional traffic coordination indicators, a traffic cooperation index of the autonomous driving system in the test scenario can be obtained. Based on the traffic cooperation index, the test results of the autonomous driving system in the test scenario can be generated. By structurally collecting the interactive behavior process in the test scenario and constructing a unified multi-dimensional evaluation framework based on interaction success rate, interaction timeliness, interaction smoothness, and interaction fairness, and then obtaining the traffic cooperation index through weighted fusion, the final output test results include verification of the autonomous driving system's ability to participate in complex traffic scenarios, forming a more comprehensive quantitative evaluation of the autonomous driving system's performance, and improving the intuitiveness, comparability, and engineering application value of the test results.

[0098] In one embodiment, acquiring data on the interaction behavior between the autonomous driving system and other traffic participants in a test scenario includes:

[0099] Acquire raw behavioral data of the target vehicle and other traffic participants in the test scenario; determine the interaction events between the target vehicle and other traffic participants based on the raw behavioral data; generate interactive behavioral data based on the behavioral response sequence and interaction end status of the target vehicle and other traffic participants in the interaction events.

[0100] The target vehicle makes decisions through a pre-deployed autonomous driving system.

[0101] In this embodiment of the invention, the raw behavioral data of the target vehicle with the autonomous driving system deployed in the test scenario, as well as the raw behavioral data of other traffic participants, can be obtained first. The raw behavioral data may include the position, speed, orientation, and other data of the target vehicle and other traffic participants from the start to the end of the test scenario.

[0102] For example, for vehicles in the test scenario, data such as the vehicle's position, speed, and heading can be obtained based on onboard cameras, radar, inertial measurement units, and positioning units. Internal communication data of the vehicle's ECU (electronic control unit) and other controllers can also be read to obtain vehicle control data such as turn signals, braking, and accelerator pedal opening. For target vehicles, intermediate decision information (such as avoidance and detour) from the autonomous driving system can also be read.

[0103] For moving objects such as pedestrians and non-motorized vehicles in the test scenario, the location and speed of the moving objects can be obtained through sensing units such as roadside cameras, radar, or vehicle-mounted cameras. The cameras can also capture the body movements (such as gestures) of pedestrians and non-motorized vehicle drivers.

[0104] After obtaining the raw behavioral data of the target vehicle and other traffic participants, clocks can be aligned based on a unified time standard, and interaction events between the target vehicle and other traffic participants can be extracted based on the temporal and spatial correlations between the target vehicle's behavior and the behavior of other traffic participants.

[0105] For example, when a target vehicle approaches an intersection and encounters a pedestrian crossing a crosswalk, an interaction event may occur: First, the target vehicle detects the pedestrian has stepped onto the crosswalk as it approaches, slows down to yield, and then crosses the crosswalk after the pedestrian has passed. Second, the target vehicle detects the pedestrian has stepped onto the crosswalk as it approaches, but does not slow down to yield; the pedestrian stops on the crosswalk to let the target vehicle pass, and the target vehicle crosses the crosswalk before the pedestrian. These two scenarios can be considered two distinct interaction events. The interaction event begins when the pedestrian steps onto the crosswalk and ends when the target vehicle crosses the crosswalk. The first scenario can be considered a successful completion of the interaction, while the second scenario can be considered a completed but not successfully completed interaction.

[0106] Based on the timing of the behavioral responses of the target vehicle and other traffic participants in an interactive event, and the end state of the interaction, relevant interactive behavior data can be extracted. For example, analyzing the behavior of the target vehicle and other traffic participants from a temporal perspective, the pedestrian steps onto the crosswalk first, the target vehicle slows down to yield, then the pedestrian crosses the crosswalk, and finally the target vehicle crosses the crosswalk. The position, speed, and actions of the pedestrian and the target vehicle in these actions together constitute the interactive behavior data of this interactive event.

[0107] In some possible implementations, based on raw behavioral data, interaction events between the target vehicle and other traffic participants are determined, including:

[0108] Based on the raw behavioral data, identify the interaction request behavior; based on the interaction request behavior, extract the interaction response behavior; based on the interaction request behavior and the interaction response behavior, generate interaction events.

[0109] The interaction request behavior may include a first request behavior representing the target vehicle's intention to interact with the traffic participant, or a second request behavior representing the traffic participant's intention to interact with the target vehicle. The interaction response behavior may include a first response behavior of the traffic participant after the first request behavior occurs, or a second response behavior of the target vehicle after the second request behavior occurs.

[0110] In the test scenario, the target vehicle with the autonomous driving system may participate in pedestrian-related interaction events, such as pedestrians crossing the road, and may also participate in interaction events with other vehicles, such as lane changing, yielding, and coordinated passage at intersections.

[0111] In some possible implementations, identifying interactive request behavior may include:

[0112] The system detects candidate request behaviors belonging to a preset interaction type in the raw behavior data of the target vehicle; it identifies whether there is an interactive object that can respond to the candidate request behavior within the interaction area to which the candidate request behavior is directed; if so, the candidate request behavior is identified as an interaction request behavior.

[0113] The interaction types can include slowing down before crossing a zebra crossing, turning on the turn signal, and honking the horn. For each interaction type, corresponding recognition rules can be set for the interaction area and the interaction object. Based on the recognition rules corresponding to multiple interaction types, a rule engine for recognizing interaction request behaviors or a neural network model for recognizing interaction request behaviors can be built. Taking the interaction type of slowing down before crossing a zebra crossing as an example, when the target vehicle makes this candidate request behavior, the interaction area can be the main area of ​​the zebra crossing and the curbs on both sides of the zebra crossing, and the interaction object can be a pedestrian. If there is a pedestrian standing on the curb next to the zebra crossing, it is considered that an interaction object exists within the interaction area; if there is only a motorcycle on the curb next to the zebra crossing and no pedestrian, it is considered that no interaction object exists within the interaction area.

[0114] After identifying the interaction request behavior and the interaction object, it can be determined whether the subsequent behavior of the interaction object constitutes an interaction response behavior. For example, if a target vehicle slows down to a stop before a crosswalk, and a pedestrian stands on the curb next to the crosswalk facing the crosswalk, the target vehicle's slowing down and yielding constitutes an interaction request behavior. However, if the pedestrian does not actually intend to cross the crosswalk and remains stationary outside the crosswalk, then the pedestrian, as the interaction object, is considered not to have responded to the target vehicle's interaction request behavior, and therefore there is no corresponding interaction response behavior. If corresponding interaction request behaviors and interaction response behaviors occur within a certain spatial range and a certain time range, an interaction event can be constituted. If there is only an interaction request behavior and no interaction response behavior, then it is not considered to constitute an interaction event.

[0115] Taking interaction with other vehicles as an example, when there is a vehicle behind in the adjacent lane, the autonomous driving system controls the target vehicle to turn on its turn signal. This can be regarded as the first request behavior of the target vehicle to express its intention to interact with the vehicle behind. After the target vehicle turns on its turn signal, the vehicle behind can choose to accelerate to pass (i.e., not give way) or slow down to give way. Both can be regarded as the first response behavior of the vehicle behind to the target vehicle's intention to interact.

[0116] Taking the interaction with pedestrians as an example, a pedestrian stepping onto a zebra crossing can be regarded as a second request behavior by the pedestrian to express the intention of interaction to the target vehicle. The target vehicle's choice to accelerate through (i.e. not yield) or slow down to yield can be regarded as the target vehicle's second response behavior to the pedestrian's intention of interaction.

[0117] In the above embodiments, by first acquiring the raw behavioral data of the target vehicle and other traffic participants in the test scenario, then identifying interaction events from these data, and finally generating interaction behavior data based on the timing of the behavioral responses of both parties and the end state of the interaction, the final interaction behavior data is not a simple record of the raw data, but rather a processed data that has undergone interaction event identification and structured extraction of the interaction process. This processing method can accurately filter out the interaction segments that truly involve the dynamic game between the autonomous driving system and other traffic participants from massive amounts of raw data, and clearly retain the temporal relationships and end state information during the interaction process. This provides a high-quality, highly relevant data foundation for the accurate calculation of subsequent traffic coordination indicators, avoiding the inclusion of invalid data from non-interaction scenarios in the evaluation system. This not only improves data processing efficiency but also enhances the reliability and relevance of the test results.

[0118] In one embodiment, such as Figure 2 As shown, based on interaction behavior data, multi-dimensional traffic coordination indicators for the autonomous driving system in the test scenario are determined, which may include:

[0119] Step S201: Determine the interaction success rate based on the ratio of the number of successful interactions to the total number of interactions in the interaction behavior data.

[0120] The interaction can be considered successfully completed if the following conditions are met: Condition 1, the autonomous vehicle correctly recognizes the interaction intention of the traffic participant, or the traffic participant correctly recognizes the interaction intention of the autonomous vehicle (such as the autonomous vehicle's turn signal, deceleration, yielding, etc. being responded to by other traffic participants); Condition 2, during the duration of the interaction event, no traffic conflict affecting the normal driving of the autonomous vehicle occurs (such as emergency braking, forced departure from the lane, etc.); Condition 3, the interaction event is completed within a preset threshold of the number of attempts or a preset time window (such as a lane merging operation being successful within 3 attempts).

[0121] Interaction success rate measures whether an autonomous driving system can successfully complete interactions (such as lane changing, yielding, merging, etc.) in scenarios that require interaction with other traffic participants, thus avoiding traffic conflicts or deadlocks caused by interaction failures.

[0122] If we denote the success rate of the interaction as I... success Then it can be calculated using the following formula (2):

[0123] (2)

[0124] In the above formula, N total N represents the total number of interactions with other traffic participants. success This represents the number of times an interaction was successfully completed.

[0125] Step S202: Compare the response time of the autonomous driving system to the interaction signals of other traffic participants in the interaction behavior data with a preset time threshold, and determine the interaction timeliness based on the comparison result.

[0126] Interaction response timeliness can measure the speed at which an autonomous driving system perceives and responds to the intentions of other traffic participants, reflecting the system's interactive agility.

[0127] For example, the interaction response time T can be defined. response The time difference between when other road users send interactive signals (such as turning on turn signals, slowing down to yield, or pedestrians stepping onto crosswalks) and when the system makes a clear response.

[0128] If we denote the timeliness of the interactive response as I... timeliness Then it can be calculated using the following formula:

[0129]

[0130] ,when ; ,when .

[0131] In the above formula, T response,i T represents the response time (in seconds) for the i-th interaction. threshold The response time threshold (can be set according to different interaction scenarios, such as 2.0s for vehicle lane changing scenarios and 1.0s for pedestrian crossing scenarios), and N is the total number of interactions.

[0132] Step S203: Compare the maximum acceleration of the autonomous driving system during the interaction process in the interaction behavior data with the preset acceleration threshold to determine the smoothness of the interaction.

[0133] Interaction smoothness measures the smoothness of motion in an autonomous driving system during interaction, reflecting the comfort and naturalness of the interaction. Sudden acceleration, sudden deceleration, or frequent hesitation all reduce interaction smoothness.

[0134] If we denote the smoothness of interaction as I... smoothness Then it can be calculated using the following formula (3):

[0135] (3)

[0136] In the above formula, a x,i Let a be the maximum longitudinal acceleration (m / s²) during the i-th interaction. y,i Let a be the maximum lateral acceleration (m / s²) during the i-th interaction. max This is the acceleration threshold for interactive scenarios (the default value is 2.5 m / s², and exceeding this value is considered a violent operation).

[0137] Step S204: Calculate the passage efficiency of autonomous vehicles and other traffic participants based on the interaction behavior data, and determine the fairness of the interaction based on the passage efficiency.

[0138] Interaction fairness measures whether an autonomous driving system can reasonably balance its own traffic efficiency with the traffic rights of other traffic participants during the interaction process, avoiding being overly aggressive or overly conservative.

[0139] Let I denote the fairness of interaction. fairness Then it can be calculated using the following formula (4):

[0140] (4)

[0141] In the above formula, t gap,i Let t be the time interval (s) that the system allows for other traffic participants in the i-th interaction. ref The time interval is for reference (set according to different scenarios, such as 4 seconds for yielding at intersections without traffic lights and 3 seconds for merging scenarios).

[0142] In some possible implementations, if the interaction event is a yielding scenario, the yielding rate I can also be set.yield As a simplified indicator equivalent to interactive fairness, the calculation process can be expressed as the following formula (5):

[0143] (5)

[0144] In the above formula, N yield-required N represents the total number of times a right-of-way needs to be yielded. yield This represents the actual number of times the vehicle yielded.

[0145] In the above embodiments, by jointly determining the interaction success rate, interaction timeliness, interaction smoothness, and interaction fairness, the originally scattered interaction performance is transformed into quantifiable and coordinated indicators. This allows the test results to reflect the interaction quality of the autonomous driving system in complex traffic environments from multiple dimensions. Each dimension corresponds to task completion, response speed, action continuity, and traffic allocation relationship, which can simultaneously reflect the availability, stability, and traffic adaptability of the autonomous driving system. Furthermore, it allows for more granular quantitative results of the autonomous driving system's interaction performance in the test scenario, reducing reliance on human experience and improving comparability between different scenarios, vehicle models, and versions of the autonomous driving system.

[0146] It should be noted that both successful interaction and failed interaction are results of interaction events. Interaction completed but not successful and no interaction event generated are two different situations, which will be illustrated with an example below.

[0147] Taking the example of a vehicle yielding to a pedestrian at a crosswalk, if the pedestrian steps onto the crosswalk first and may be affected by the vehicle, the pedestrian's act of stepping onto the crosswalk can be considered an interactive request to the vehicle. If the vehicle chooses to slow down and yield, this act is the second response to the pedestrian's request, and it is also a request to the pedestrian as the interactive object. If the pedestrian crosses the crosswalk before the vehicle after yielding, the pedestrian's act of crossing the crosswalk can be considered a response to the vehicle's request to yield. Finally, the vehicle crosses the crosswalk, and the interaction event is successfully completed. However, if the vehicle accelerates across the crosswalk (i.e., does not yield) when the pedestrian has stepped onto the crosswalk first and may be affected by the vehicle, in this case, the vehicle can no longer yield, the interaction event is completed, but the interaction fails. If a pedestrian is simply standing on the curb next to the zebra crossing, and the target vehicle stops to let the pedestrian pass before crossing the zebra crossing, then the target vehicle's act of letting the pedestrian pass can be regarded as an interactive request. If the pedestrian has no intention of crossing the zebra crossing (keeping standing on the curb without stepping into the zebra crossing), then this situation is considered as no interactive response behavior, not an interactive event, rather than an interactive failure.

[0148] In one embodiment, a traffic coordination index of an autonomous driving system in a test scenario is obtained by weighted fusion of multi-dimensional traffic coordination indicators, including: adjusting the weight coefficients corresponding to each dimension of traffic coordination indicators based on the scenario characteristics of the test scenario; and weighted fusion of multi-dimensional traffic coordination indicators based on the weight coefficients to obtain the traffic coordination index of the autonomous driving system in the test scenario.

[0149] The weighting coefficients are adapted to the scene features. Scene features can be used to characterize information such as the composition of traffic participants, traffic density, road type, and signal control method in the test scene, while the weighting coefficients can be used to characterize the relative importance of interaction success rate, interaction timeliness, interaction smoothness, and interaction fairness in the corresponding scene.

[0150] Scene features can be jointly determined by preset labels for the test scene, environmental perception results, road structure information, or traffic event information, and can be mapped to a scene parameter vector to dynamically adjust the weights of each dimension of indicators. Weight coefficients can be pre-stored in a weight configuration file and retrieved based on the scene parameter vector, or they can be obtained by inputting the scene parameter vector into a weight generation model. For example, scene features can be input into a preset decision tree model or rule engine to obtain the weight coefficients corresponding to each dimension of traffic coordination indicators adapted to the test scene, as output by the decision tree model or rule engine.

[0151] For example, the characteristics of the current interaction environment can be identified first based on the test scenario, then the features can be mapped to a weight combination that fits them. Finally, multi-dimensional traffic coordination indicators can be comprehensively calculated to output a traffic coordination index that reflects the overall traffic coordination level of the autonomous driving system. Since different scenarios have different sensitivities to each dimension of the indicators, this index can avoid a single indicator dominating the evaluation results, thus making the test conclusions more consistent with the interaction requirements in real-world roads.

[0152] In the above embodiments, the traffic cooperation index can adaptively adjust the evaluation focus according to the test scenario, which can not only reflect the interaction capability of the autonomous driving system in a specific environment, but also improve the comparability of results between different test scenarios, different vehicles and different versions of autonomous driving systems.

[0153] In one embodiment, generating test results for an autonomous driving system in a test scenario based on a traffic coordination index includes: acquiring raw behavioral data of a target vehicle with an autonomous driving system deployed in the test scenario; determining a comprehensive index of the autonomous driving system based on the raw behavioral data of the target vehicle; weighted fusion of the comprehensive index and the traffic coordination index; and generating test results for the autonomous driving system in the test scenario based on the fusion result.

[0154] The comprehensive indicators include at least one of the rationality evaluation indicators and the comfort evaluation indicators. The raw behavioral data may include vehicle speed, acceleration, rate of change of acceleration, lateral deviation, steering angle, following distance, lane change timing, turn signal status, trajectory point sequence, and violation records, etc.

[0155] In this invention, based on the aforementioned traffic coordination indicators, more comprehensive test evaluation indicators such as rationality and comfort can be added. These comprehensive indicators can be integrated with the traffic coordination indicators to generate test results, thereby enabling the test results to more comprehensively measure the performance of the autonomous driving system.

[0156] For example, the rationality evaluation index can be set as a score of the consistency of vehicle decisions in following, yielding, lane changing, and passing, while the comfort evaluation index can be set as a smoothness score of longitudinal acceleration, lateral acceleration, and rate of change of acceleration. The traffic coordination index is a traffic coordination index based on interaction success rate, interaction timeliness, interaction smoothness, and interaction fairness. During fusion, weighting coefficients can be set according to the test scenario type, and the comprehensive index and the traffic coordination index can be linearly weighted, or the comprehensive index can be fused internally first and then fused with the traffic coordination index a second time. The final output test result can be a percentage score, a grade conclusion (such as a test pass grade of A), or a binary judgment of pass or fail.

[0157] In the above embodiments, by adding indicators such as rationality and comfort, test results that reflect the driving stability and traffic coordination capabilities of the autonomous driving system can be obtained. This allows the test results to consider both the vehicle's external interaction performance and its own operational performance, reducing the bias that may be caused by relying on a single indicator and improving the objectivity and completeness of the test evaluation.

[0158] Based on the above embodiments, the rationality evaluation index and comfort evaluation index will be further explained below in conjunction with some other embodiments.

[0159] In one embodiment, the rationality evaluation index includes at least one of the following vehicle behavior rationality index, lane change behavior rationality index, lane selection rationality index, and speed control rationality index, such as... Figure 3 As shown, the rationality evaluation indicators can be determined in the following ways:

[0160] Step S301: Determine the following behavior rationality index based on the relationship between the actual following distance and the reference following distance of the target vehicle.

[0161] The actual following distance refers to the real-time distance between the target vehicle and the vehicle in front in the longitudinal direction of travel. The reference following distance can be determined based on vehicle speed, safe following distance, deceleration capability of the vehicle in front, and ride comfort constraints. It is used to characterize the target distance that the vehicle should meet to maintain safe and smooth following.

[0162] Following another vehicle is one of the most fundamental decision-making behaviors of an autonomous driving system, and its rationality is mainly reflected in whether the chosen following distance conforms to traffic flow characteristics and safety expectations. If R... following The rationality index of following another vehicle can be calculated using the following formula (6):

[0163] (6)

[0164] In the above formula, d i d represents the actual following distance (m) at the i-th sampling time. ref (v i (Based on the current vehicle speed v) i Calculated reference following distance (m), d ref (v i The two-second rule (i.e., d) can be adopted. ref =2×v i (or determined by a statistical model based on natural driving data, where N is the total number of sampling points.)

[0165] When the actual following distance is too small (less than the safety threshold) or too large (seriously affecting traffic flow efficiency), the rationality index of following behavior decreases significantly.

[0166] Step S302: Determine the rationality index of lane-changing behavior based on at least one of the timing rationality score, spatial rationality score, and intention expression rationality score of the lane-changing decision of the autonomous driving system.

[0167] Among them, the timing rationality score is used to characterize the rationality of the target vehicle's choice of lane change time under traffic flow gaps, relative positions of vehicles in adjacent lanes, and current road constraints; the spatial rationality score is used to characterize whether there is sufficient available gap, lateral safety margin, and stable driving space after merging in adjacent lanes; and the intention expression rationality score is used to characterize whether the target vehicle clearly expresses its lane change intention through turn signals, trajectory deviation, and changes in lateral acceleration.

[0168] If R is used lanechange The rationality index of lane-changing behavior can be calculated using the following formula (7):

[0169] (7)

[0170] In the above formula, M represents the total number of lane change events; T jScore the reasonableness of the timing of the j-th lane change (0≤T) j ≤1), which can be calculated based on factors such as the time difference between the generation of the lane change intention and its execution, and the distance of vehicles behind the target lane; G j Score the spatial rationality of the j-th lane change (0≤G) j ≤1), can be calculated based on the matching degree between the available clearance of the target lane and the vehicle speed; S j Score the reasonableness of the intention expressed during the j-th lane change (0≤S) j ≤1), which can be based on the turn signal early activation time (should be greater than 3 seconds) and the timing of closing; λ1, λ2 and λ3 are the weight coefficients for the timing rationality score, spatial rationality score and intention expression rationality score, respectively (the default values ​​can be λ1=0.4, λ2=0.4, λ3=0.2, to comprehensively evaluate the rationality of the lane change behavior of the autonomous driving system from three aspects: the rationality of lane change timing, the rationality of lane change spatial, and the rationality of lane change intention expression).

[0171] Step S303: Based on the relationship between the mileage traveled by the target vehicle in the optimal lane and the total mileage, determine the lane selection rationality index.

[0172] The optimal lane can be a target lane suitable for the vehicle's current travel, determined by combining the navigation route, lane efficiency, traffic restrictions, and current traffic conditions. The total mileage is the total distance traveled by the target vehicle in the test scenario, and the ratio of the mileage traveled in the optimal lane to the total mileage reflects the rationality of the vehicle's lane selection.

[0173] In multi-lane scenarios, lane selection rationality is reflected in whether the autonomous driving system chooses the optimal lane based on factors such as the driving route, traffic flow, and obstacles ahead. If using R... laneselect The lane selection rationality index can be calculated using the following formula (8):

[0174] (8)

[0175] In the above formula, S optimal S represents the distance (km) the vehicle travels in the optimal lane. total Total mileage (km).

[0176] For example, the following conditions can be used to determine whether you are in the optimal lane: According to the navigation route, you should enter the corresponding turning lane (such as the left turn lane or the straight lane) in advance; when there is no need to turn, you should avoid occupying the leftmost overtaking lane for a long time; you should avoid driving at high speed in the slow lane for a long time or driving at low speed in the fast lane for a long time; when there are obstacles or slow vehicles ahead, you should change to a lane with higher traffic efficiency in advance.

[0177] Step S304: Determine the speed control rationality index based on the relationship between the actual speed of the target vehicle and the reference speed.

[0178] The reference speed is determined based on at least one factor among road speed limits, traffic flow speed, road geometry, and driving scenario. For example, the reference speed can be given directly by the road speed limit, or it can be adjusted by combining real-time traffic flow speed, curve radius, gradient, or driving scenario such as ramp merging and congestion, so that the evaluation of speed control indicators better reflects the actual road operation requirements.

[0179] Speed ​​control rationality can measure whether the autonomous driving system's speed selection conforms to road speed limits, traffic flow speeds, road geometry (such as road curvature), and driving scenarios (such as deceleration at school zones and pedestrian crossings). If R is used... speed The speed control rationality index can be calculated using the following formula (9):

[0180] (9)

[0181] In the above formula, v(t) is the actual vehicle speed at time t, v ref (t) represents the reference speed at time t, and T represents the total travel time at the actual speed. The reference speed can take into account the following factors: road speed limit, average traffic flow speed (if there is a vehicle in front, use the speed of the vehicle in front as a reference), safe speed corresponding to road curvature, and driving scenario, etc.

[0182] In some possible implementations, the four dimensions of the rationality index—following behavior, lane changing behavior, lane selection, and speed control—can be weighted and fused to obtain a fused human-like index, which serves as the quantitative evaluation result of the rationality assessment indicator. If R... human-like The humanization index can be calculated using the following formula (10):

[0183] (10)

[0184] In the above formula, γ1, γ2, γ3 and γ4 are the fusion weights of the following behavior rationality index, lane change behavior rationality index, lane selection rationality index and speed control rationality index, respectively, which can be adjusted according to the characteristics of the test scenario (each accounts for 25% by default).

[0185] In the above embodiments, by establishing quantitative indicators for following, lane changing, lane selection, and speed control, the rationality evaluation of the autonomous driving system no longer relies on subjective judgment. Instead, it directly reflects whether the vehicle's decisions regarding lane selection, following distance, and lane changing timing in typical driving behaviors conform to human driving habits and logic, and whether it can adapt to complex traffic environments. The determination of reference following distance, reference speed, and optimal lane can be dynamically updated based on the scenario. Therefore, the obtained rationality evaluation results can more accurately adapt to different road types and traffic conditions, and improve the consistency and comparability of test results.

[0186] In one embodiment, the comfort evaluation index includes at least one of the longitudinal comfort index, the lateral comfort index, and the vertical comfort index, and the comfort evaluation index is determined in the following manner:

[0187] The longitudinal comfort index is determined based on the relationship between the longitudinal acceleration rate of change and the longitudinal impact threshold of the target vehicle; the lateral comfort index is determined based on the relationship between the lateral acceleration rate of change and the lateral impact threshold of the target vehicle; and the vertical comfort index is determined based on the relationship between the root mean square value of the vertical weighted acceleration and the vertical acceleration threshold of the target vehicle.

[0188] If C is used long The longitudinal comfort index can be calculated using the following formula (11):

[0189] (11)

[0190] In the above formula, a x J is the longitudinal acceleration (m / s²). x,max The longitudinal impact threshold is set at 2.5 m / s³ (the default value is 2.5 m / s³; exceeding this value is considered significant discomfort).

[0191] If C is used lat The lateral comfort index can be calculated using the following formula (12):

[0192] (12)

[0193] In the above formula, a y J is the longitudinal acceleration (m / s²). y,max This is the longitudinal impact threshold (default value is 2.0 m / s³).

[0194] If C is used vehicle The vertical comfort index can be calculated using the following formula (13):

[0195] (13)

[0196] In the above formula, aw,max The threshold is set (default value is 0.63 m / s², corresponding to the critical value that produces discomfort); a ω a is the root mean square value of the vertical weighted acceleration (m / s²). ω It can be calculated using the following formula (14):

[0197] (14)

[0198] For example, by weighting and summing the above longitudinal comfort index, lateral comfort index, and vertical comfort index, we can obtain a comprehensive comfort evaluation index C that considers the longitudinal, lateral, and vertical aspects. vehicle It can be calculated using the following formula (15).

[0199] (15)

[0200] Wherein, α1, α2 and α3 are the weighting coefficients of the longitudinal comfort index, the lateral comfort index and the vertical comfort index, respectively, which can be adjusted according to the characteristics of the test scenario (the default values ​​are α1=0.4, α2=0.4, α3=0.2).

[0201] In the above embodiments, by establishing a correspondence between the vehicle's motion state and the comfort threshold, the ride smoothness of the vehicle with the autonomous driving system can be directly reflected when accelerating, decelerating, turning, and passing through uneven road surfaces. This transforms the comfort evaluation from an experience-based judgment into a quantifiable and comparable indicator output, which helps the autonomous driving system balance safety, efficiency, and ride experience during testing and optimization.

[0202] The following description, in conjunction with the accompanying drawings, illustrates some possible application examples of the testing method for the autonomous driving system of this invention. Figure 4 This is a schematic diagram of the architecture of a multi-level testing and evaluation system provided for an exemplary embodiment of the present invention. For example... Figure 4 As shown, a multi-level testing and evaluation system for autonomous driving systems can be constructed by comprehensively considering dimensions such as comfort, rationality, and traffic coordination. Each dimension can include several sub-dimensional evaluation indicators. For example, comfort can include sub-dimensions such as longitudinal comfort, lateral comfort, and vertical comfort; rationality can include sub-dimensions such as following rationality, lane changing rationality, lane selection rationality, and speed control rationality; and traffic coordination can include sub-dimensions such as interaction success rate, interaction response timeliness, interaction smoothness, and interaction fairness.

[0203] Figure 5 This is a schematic diagram of a multi-level test evaluation system provided as an exemplary embodiment of the present invention. For example... Figure 5As shown, based on a multi-level testing and evaluation system, hard pass conditions and multi-dimensional evaluation indicators and their weights can be pre-set. Hard pass conditions can be determined according to the ODD (Operational Design Domain) scenario of the autonomous driving system and the function being tested. For example, hard pass conditions may include: no collision, successful arrival at the destination, normal function activation, and no mid-way exit or degradation of the function. Multi-dimensional evaluation indicators may include the longitudinal acceleration change rate, lateral acceleration change rate, speed limit compliance rate, traffic light compliance rate, interaction response time, and interaction success rate of the target vehicle. In some possible implementations, the existence of violations can also be used as a hard pass condition. Violations can refer to driving behaviors that violate safe driving regulations, such as driving in the wrong direction.

[0204] After setting up the conditions and preparing the testing environment, high-precision data acquisition devices can be deployed in closed areas or open roads, among the target vehicle and other traffic participants, to simultaneously record data such as vehicle motion status, environmental perception information, and traffic participant behavior. Next, the collected data can be synchronized in time and standardized in format to construct a multi-dimensional test truth database. Then, it can be determined whether the target vehicle's performance meets the hard pass conditions. If any hard pass condition is not met, the current test of the autonomous driving system can be directly judged as "failure." If all hard pass conditions are met, the next stage of multi-dimensional testing and evaluation can be conducted to determine comfort evaluation indicators, rationality evaluation indicators, and traffic coordination evaluation indicators. These indicators are then weighted and fused to obtain a comprehensive multi-dimensional evaluation of the autonomous driving system.

[0205] For example, if P is used total If the multi-dimensional comprehensive evaluation score is represented, it can be calculated using the following formula (18).

[0206] (18)

[0207] In the above formula, ω C ω R and ω I These are weighting coefficients for each dimension: comfort, rationality, and traffic coordination. The sum of these weighting coefficients can be 1. Ultimately, these dimensional indicators and the multi-dimensional comprehensive evaluation score can provide guidance for the optimization of autonomous driving systems, provide a basis for the standard certification of autonomous driving systems, and promote the commercial application of autonomous driving systems.

[0208] Based on the above, the testing method for the autonomous driving system of the present invention has the following advantages:

[0209] Systematic testing: It realizes a multi-level evaluation system covering dimensions such as comfort, rationality, and traffic coordination, making up for the shortcomings of traditional testing that only focuses on perception capabilities;

[0210] Test quantifiability: Each dimension has clear quantitative evaluation indicators and calculation formulas, which facilitates automated calculation and comparison, and improves the objectivity and repeatability of evaluation results;

[0211] Test scalability: The evaluation system can be flexibly adjusted and expanded according to different levels of autonomous driving and application scenarios, and has good adaptability;

[0212] Engineering applicability of the test: A truth system is built based on actual field test data, and the test results can be directly used for the optimization of autonomous driving systems, standard certification, and product iteration.

[0213] Figure 6 This is a schematic diagram of a testing apparatus for an autonomous driving system provided as an exemplary embodiment of the present invention. Figure 6 As shown, the test apparatus 600 for the autonomous driving system may include:

[0214] The acquisition module 601 is used to acquire data on the interaction behavior between the autonomous driving system and other traffic participants in the test scenario.

[0215] The indicator determination module 602 is used to determine the multi-dimensional traffic coordination indicators of the autonomous driving system for the test scenario based on the interaction behavior data. The multi-dimensional traffic coordination indicators include interaction success rate, interaction timeliness, interaction smoothness and interaction fairness.

[0216] The indicator fusion module 603 is used to perform weighted fusion of multi-dimensional traffic coordination indicators to obtain the traffic coordination index of the autonomous driving system in the test scenario.

[0217] The generation module 604 is used to generate test results of the autonomous driving system in the test scenario based on the traffic cooperation index.

[0218] In some possible implementations, the indicator determination module 602 can also be used to: determine the interaction success rate based on the ratio of the number of successful interactions to the total number of interactions in the interaction behavior data; compare the response time of the autonomous driving system to the interaction signals of other traffic participants in the interaction behavior data with a preset time threshold, and determine the interaction timeliness based on the comparison result; compare the maximum acceleration of the autonomous driving system during the interaction process in the interaction behavior data with a preset acceleration threshold, and determine the interaction smoothness; calculate the traffic efficiency of the autonomous driving system and other traffic participants based on the interaction behavior data, and determine the interaction fairness based on the traffic efficiency.

[0219] In some possible implementations, the index fusion module 603 can also be used to: adjust the weight coefficients corresponding to each dimension of traffic coordination index based on the scene characteristics of the test scenario, and adapt the weight coefficients to the scene characteristics; and perform weighted fusion of multi-dimensional traffic coordination indexes based on the weight coefficients to obtain the traffic coordination index of the autonomous driving system in the test scenario.

[0220] In some possible implementations, the acquisition module 601 can also be used to: acquire raw behavioral data of the target vehicle and other traffic participants in the test scenario, with the target vehicle making decisions through a pre-deployed autonomous driving system; determine the interaction events between the target vehicle and other traffic participants based on the raw behavioral data; and generate interactive behavioral data based on the behavioral response timing and interaction termination status of the target vehicle and other traffic participants in the interactive events.

[0221] In some possible implementations, the acquisition module 601 may also be used to: identify interactive request behaviors based on the original behavioral data, the interactive request behaviors including a first request behavior characterizing the target vehicle's interactive intent towards the traffic participant, and / or including a second request behavior characterizing the traffic participant's interactive intent towards the target vehicle; extract interactive response behaviors based on the interactive request behaviors, the interactive response behaviors including a first response behavior of the traffic participant after the first request behavior occurs, and / or including a second response behavior of the target vehicle after the second request behavior occurs; and generate interactive events based on the interactive request behaviors and interactive response behaviors.

[0222] In some possible implementations, the generation module 604 can also be used to: acquire the original behavior data of the target vehicle deploying the autonomous driving system in the test scenario; determine the comprehensive index of the autonomous driving system based on the original behavior data of the target vehicle, the comprehensive index including at least one of the rationality evaluation index and the comfort evaluation index; weightedly fuse the comprehensive index with the traffic coordination index, and generate the test results of the autonomous driving system in the test scenario based on the fusion result.

[0223] In some possible implementations, the generation module 604 can also be used to: determine a following behavior rationality index based on the relationship between the actual following distance and the reference following distance of the target vehicle; determine a lane change behavior rationality index based on at least one of the timing rationality score, spatial rationality score, and intent expression rationality score of the lane change decision of the autonomous driving system; determine a lane selection rationality index based on the relationship between the mileage traveled by the target vehicle in the optimal lane and the total mileage traveled; and determine a speed control rationality index based on the relationship between the actual speed of the target vehicle and the reference speed, wherein the reference speed is determined based on at least one of the following factors: road speed limit, traffic flow speed, road geometry, and driving scenario.

[0224] In some possible implementations, the generation module 604 can also be used to: determine the longitudinal comfort index based on the relationship between the longitudinal acceleration change rate of the target vehicle and the longitudinal impact threshold; determine the lateral comfort index based on the relationship between the lateral acceleration change rate of the target vehicle and the lateral impact threshold; and determine the vertical comfort index based on the relationship between the root mean square value of the vertical weighted acceleration of the target vehicle and the vertical acceleration threshold.

[0225] The testing device for the autonomous driving system provided in this embodiment is used to execute the technical solutions in any of the aforementioned method embodiments. Its implementation principle and technical effect are similar, and will not be described again here.

[0226] It should be understood that the above-described device embodiments are merely illustrative, and the device of the present invention can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units, modules, or components may be combined, or integrated into another system, or some features may be ignored or not executed.

[0227] Furthermore, unless otherwise specified, the functional units / modules in the various embodiments of the present invention can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or as software program modules.

[0228] Figure 7 This is a schematic diagram of the structure of an electronic device provided as an exemplary embodiment of the present invention. For example... Figure 7 As shown, the electronic device 70 includes:

[0229] Processor 71, memory 72, and communication interface 73;

[0230] The memory 72 is used to store the executable instructions of the processor 71; the executable instructions can be instructions that the computer can execute.

[0231] The processor 71 is configured to execute the technical solutions in any of the foregoing method embodiments by executing executable instructions.

[0232] Optionally, the memory 72 can be either standalone or integrated with the processor 71.

[0233] Optionally, when the memory 72 is a device independent of the processor 71, the electronic device 70 may further include:

[0234] Bus 74, memory 72 and communication interface 73 are connected to processor 71 through bus 74 and complete communication with each other. Communication interface 73 is used to communicate with other devices.

[0235] Optionally, the communication interface 73 can be implemented using a transceiver. The communication interface is used to enable communication between the database access device and other devices (e.g., clients, read-write databases, and read-only databases). The memory may include random access memory (RAM) and may also include non-volatile memory, such as at least one disk drive.

[0236] Bus 74 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, only one line is used in the diagram, but this does not imply that there is only one bus or one type of bus.

[0237] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0238] The electronic device is used to execute the technical solutions in any of the foregoing method embodiments. Its implementation principle and technical effect are similar, and will not be described again here.

[0239] This invention also provides a vehicle, which includes a vehicle body and an autonomous driving system deployed in the vehicle body. The autonomous driving system is tested using the test method of any of the above embodiments and passes the test.

[0240] This invention also provides a readable storage medium, which can be a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements the technical solution provided in any of the foregoing method embodiments.

[0241] This invention also provides a computer program product, including a computer program, which, when executed by a processor, is used to implement the technical solutions provided in any of the foregoing method embodiments.

[0242] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0243] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0244] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the following claims.

[0245] The above embodiments are merely preferred embodiments provided to fully illustrate the present invention, and the scope of protection of the present invention is not limited thereto. Equivalent substitutions or modifications made by those skilled in the art based on the present invention are all within the scope of protection of the present invention.

Claims

1. A testing method for an autonomous driving system, characterized in that, include: Acquire data on the interaction behavior of autonomous driving systems with other traffic participants in test scenarios; Based on the interaction behavior data, a multi-dimensional traffic coordination index for the autonomous driving system in the test scenario is determined. The multi-dimensional traffic coordination index includes interaction success rate, interaction timeliness, interaction smoothness, and interaction fairness. Among them, the interaction success rate represents the ability of the autonomous driving system to successfully complete interaction with other traffic participants; the interaction timeliness represents the perception speed and response speed of the autonomous driving system to the interaction intentions of other traffic participants; the interaction smoothness represents the degree of acceleration change of the vehicle under the control of the autonomous driving system during interaction with other traffic participants; and the interaction fairness represents the ability of the autonomous driving system to weigh its own traffic efficiency and the right-of-way of other traffic participants during the interaction. The multi-dimensional traffic coordination indicators are weighted and fused to obtain the traffic coordination index of the autonomous driving system in the test scenario. Based on the traffic cooperation index, the test results of the autonomous driving system in the test scenario are generated.

2. The method according to claim 1, characterized in that, The step of determining the multi-dimensional traffic coordination indicators of the autonomous driving system for the test scenario based on the interaction behavior data includes: The success rate of the interaction is determined by the ratio of the number of successful interactions to the total number of interactions in the interaction behavior data. The response time of the autonomous driving system to the interaction signals of other traffic participants in the interaction behavior data is compared with a preset time threshold, and the timeliness of the interaction is determined based on the comparison result. The maximum acceleration of the autonomous driving system during the interaction process in the interaction behavior data is compared with a preset acceleration threshold to determine the interaction smoothness. The autonomous driving system calculates the traffic efficiency with other traffic participants based on the interaction behavior data, and determines the fairness of the interaction based on the traffic efficiency.

3. The method according to claim 1 or 2, characterized in that, The weighted fusion of the multi-dimensional traffic coordination indicators to obtain the traffic coordination index of the autonomous driving system in the test scenario includes: Based on the scenario characteristics of the test scenario, the weight coefficients corresponding to the traffic coordination indicators of each dimension are adjusted and obtained. The weight coefficients are adapted to the scenario characteristics, which include at least one of traffic participant composition, traffic density, road type and signal control method. The multi-dimensional traffic coordination indicators are weighted and fused based on the weighting coefficients to obtain the traffic coordination index of the autonomous driving system in the test scenario.

4. The method according to claim 1 or 2, characterized in that, The acquisition of interaction behavior data between the autonomous driving system and other traffic participants in the test scenario includes: Acquire raw behavioral data of the target vehicle and other traffic participants in the test scenario, wherein the target vehicle makes decisions through a pre-deployed autonomous driving system; Based on the original behavioral data, determine the interaction events between the target vehicle and the other traffic participants; Based on the timing of the behavioral responses of the target vehicle and other traffic participants in the interaction event and the end status of the interaction, interaction behavior data is generated.

5. The method according to claim 4, characterized in that, The step of determining the interaction events between the target vehicle and other traffic participants based on the original behavioral data includes: Based on the original behavioral data, interactive request behaviors are identified, including a first request behavior that characterizes the target vehicle's interactive intention towards the other traffic participants, and / or, a second request behavior that characterizes the other traffic participants' interactive intention towards the target vehicle. Based on the interaction request behavior, the interaction response behavior is extracted, which includes the first response behavior of the other traffic participants after the first request behavior occurs, and / or includes the second response behavior of the target vehicle after the second request behavior occurs; An interaction event is generated based on the interaction request behavior and the interaction response behavior.

6. The method according to claim 5, characterized in that, The identification of interactive request behavior includes: Detect candidate request behaviors belonging to a preset interaction type in the original behavior data of the target vehicle; If there is an interactive object within the interactive area targeted by the candidate request behavior that can respond to the candidate request behavior, then the candidate request behavior is determined as an interactive request behavior.

7. The method according to claim 1 or 2, characterized in that, The process of generating test results for the autonomous driving system in the test scenario based on the traffic cooperation index includes: Obtain raw behavioral data of the target vehicle where the autonomous driving system is deployed in the test scenario; Based on the original behavioral data of the target vehicle, a comprehensive index of the autonomous driving system is determined, wherein the comprehensive index includes at least one of a rationality evaluation index and a comfort evaluation index. The comprehensive index and the traffic coordination index are weighted and fused together, and the test results of the autonomous driving system under the test scenario are generated based on the fusion result.

8. The method according to claim 7, characterized in that, The rationality evaluation index includes at least one of the following behavior rationality index, lane change behavior rationality index, lane selection rationality index, and speed control rationality index. The rationality evaluation index is determined in the following way: Based on the relationship between the actual following distance and the reference following distance of the target vehicle, the rationality index of the following behavior is determined; Based on at least one of the timing rationality score, spatial rationality score, and intention expression rationality score of the lane change decision of the autonomous driving system, the rationality index of the lane change behavior is determined; Based on the relationship between the mileage traveled by the target vehicle in the optimal lane and the total mileage traveled, the lane selection rationality index is determined; Based on the relationship between the actual speed of the target vehicle and the reference speed, the speed control rationality index is determined. The reference speed is determined based on at least one factor among road speed limit, traffic flow speed, road geometry, and driving scenario.

9. The method according to claim 7, characterized in that, The comfort evaluation index includes at least one of the longitudinal comfort index, the lateral comfort index, and the vertical comfort index, and the comfort evaluation index is determined in the following manner: The longitudinal comfort index is determined based on the relationship between the longitudinal acceleration change rate and the longitudinal impact threshold of the target vehicle. The lateral comfort index is determined based on the relationship between the lateral acceleration change rate and the lateral impact threshold of the target vehicle. The vertical comfort index is determined based on the relationship between the root mean square value of the vertical weighted acceleration of the target vehicle and the vertical acceleration threshold.

10. A testing apparatus for an autonomous driving system, characterized in that, include: The acquisition module is used to acquire data on the interaction behavior between the autonomous driving system and other traffic participants in the test scenario. The indicator determination module is used to determine multi-dimensional traffic coordination indicators of the autonomous driving system for the test scenario based on the interaction behavior data. The multi-dimensional traffic coordination indicators include interaction success rate, interaction timeliness, interaction smoothness, and interaction fairness. Specifically, the interaction success rate represents the autonomous driving system's ability to successfully interact with other traffic participants; the interaction timeliness represents the autonomous driving system's perception and response speed to the interaction intentions of other traffic participants; the interaction smoothness represents the degree of acceleration change of the vehicle under the control of the autonomous driving system during interaction with other traffic participants; and the interaction fairness represents the autonomous driving system's ability to balance its own traffic efficiency with the right-of-way of other traffic participants during the interaction. The indicator fusion module is used to perform weighted fusion of the multi-dimensional traffic coordination indicators to obtain the traffic coordination index of the autonomous driving system in the test scenario. The generation module is used to generate test results of the autonomous driving system in the test scenario based on the traffic cooperation index.

11. A vehicle, characterized in that, include: The vehicle body and the autonomous driving system deployed in the vehicle body, wherein the autonomous driving system passes the test using the test method as described in any one of claims 1 to 9.

12. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 9.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 9.

14. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1 to 9.

Images