NOA function automatic evaluation method and system and medium

Abstract

The invention discloses an NOA function automatic evaluation method and system and a medium. The method comprises the steps that multiple pieces of signal data in the vehicle NOA function starting process are acquired; preprocessing the plurality of signal data, and identifying a signal jump point of each vehicle state machine based on the preprocessed signal data; based on the identified signal jump point, judging a problem point of the NOA function and carrying out statistical analysis; and generating an evaluation result report of the NOA function. The method is used for solving the problems that manual NOA function evaluation is high in test cost, low in efficiency and not objective enough, and some KPI indexes cannot be counted.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving evaluation technology, and in particular to a NOA function automatic evaluation method, system and medium. Background Art

[0002] The NOA function is a high-level navigation driving assistance function that enables the vehicle to travel horizontally and vertically in areas covered by high-precision maps after the driver sets the navigation destination and selects the navigation route. It can also achieve high-level navigation driving assistance functions from point A to point B in a closed road environment. It has the ability to travel on high-speed elevated main roads, high-speed connecting roads, and on- and off-ramps, and can automatically change lanes and overtake slow vehicles or large vehicles according to actual conditions.

[0003] In order to provide consumers with a better driving experience, it is necessary to conduct functional testing on the actual vehicle, find the problem points, and output the evaluation KPI indicators, and then the NOA function can be optimized. However, many NOA tests are currently manually evaluated, and people are required to follow in the car, and then make notes when encountering unreasonable situations. Because it is a manual record, there are certain subjective ideas. For the same problem, the records at different times or places are likely to be different. In order to measure the function of different software versions, some KPI evaluation indicators need to be counted. The actual vehicle test needs to be counted through video or other methods in the later stage. Some indicators cannot be counted manually. In general, the use of manual evaluation has problems such as high testing cost, low efficiency, lack of objectivity, and some KPI indicators cannot be counted. Summary of the invention

[0004] In view of the above technical problems, the present invention provides a NOA function automatic evaluation method, system and medium, which are used to solve the problems of high testing cost, low efficiency, lack of objectivity, and inability to count some KPI indicators in manual evaluation of NOA functions.

[0005] A first aspect of the present invention provides a method for automatically evaluating NOA function, comprising: Obtain multiple signal data during the vehicle NOA function activation process; Preprocessing the plurality of signal data, and identifying a signal transition point of each vehicle state machine based on the preprocessed signal data; Based on the identified signal transition points, determine the problem points of the NOA function and perform statistical analysis; And, generate an evaluation result report of the NOA function.

[0006] In one possible implementation, the multiple signal data include: perception data, fusion data, map data, positioning data and vehicle status data.

[0007] In a possible implementation manner, the preprocessing the plurality of signal data includes: Time-align each signal data; Combine all signal data into one data table; Fill missing values ​​based on business logic.

[0008] In a possible implementation, the problem points of determining the NOA function at least include: Identify the NOA takeover point and determine the reasons for the takeover; Identify the points where humans and machines co-drive; Identify NOA degradation points and determine the reasons for degradation; Identify autonomous lane change points and determine lane change outcomes; Identify entry and exit ramp points and determine success; and, identifying the acceleration exceeding limit point.

[0009] In a possible implementation, the identifying the NOA takeover point and determining the takeover reason includes: Acquire NOA state machine data, and when the NOA state data changes from an activated state to a passive exit state or a manual takeover state, it is marked as a NOA takeover point; The takeover reason of the NOA takeover point is determined based on a preset takeover reason determination logic.

[0010] In a possible implementation, identifying a human-machine co-driving point includes: The human-machine co-driving point is identified based on the steering wheel torque data and the steering system signal data.

[0011] In a possible implementation, identifying the NOA degradation point and determining the degradation cause includes: Identify the first degradation point from NOA degradation to LCC through the transition of NOA state machine; and determining the downgrade reason of the first downgrade point based on a preset first judgment logic; The second degradation point from NOA to ACC is identified by the transition of NOA state machine; And based on a preset second judgment logic, a demotion reason of the second demotion point is judged.

[0012] In a possible implementation, identifying the autonomous lane change point and determining the lane change result includes: Acquiring vehicle automatic lane change state machine data, and identifying the autonomous lane change point based on the automatic lane change state machine data; The lane change result of the autonomous lane change point is determined based on a preset lane change result determination logic.

[0013] In a possible implementation, the step of identifying the entry and exit ramp points and determining whether the step is successful includes: Identify NOA entry and exit ramp points through the transition of NOA state machine, and filter out misidentified points; If NOA takeover and / or degradation occurs at the on-ramp and off-ramp point, the automatic on-ramp and off-ramp point is deemed to have failed, and the cause of the failure is determined based on a preset evaluation logic.

[0014] In a possible implementation, the method further includes: Statistics are collected on abnormal jump data and line pressure data of each entry and exit ramp point.

[0015] In a possible implementation, identifying the acceleration exceeding limit point includes: Identify the point where the longitudinal acceleration exceeds the limit; Identify the point where the longitudinal deceleration exceeds the limit; Identify the point at which the longitudinal deceleration rate is exceeded; and Identify the point where lateral acceleration is exceeded.

[0016] A second aspect of the present invention provides a NOA function automatic evaluation system, comprising: A data acquisition module, at least used to acquire a plurality of signal data during the activation of the NOA function of the vehicle; a jump identification module, at least used to preprocess the plurality of signal data, and identify the signal jump points of each vehicle state machine based on the preprocessed signal data; A data statistics module, at least used to determine the problem points of the NOA function based on the identified signal transition points and perform statistical analysis; The report generation module is used to generate at least a report on the evaluation results of the NOA function.

[0017] According to a third aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a computer, the method according to the first aspect of the embodiment of the present invention is executed.

[0018] The present invention obtains multi-dimensional signal data during the operation of the vehicle's NOA function, identifies the vehicle's behavior and abnormal state based on the state machine signal jump point after preprocessing, thereby efficiently judging the problem points of the NOA function and performing statistical analysis, and finally generating an evaluation report. Compared with the existing manual evaluation method, this method significantly reduces the test cost, improves the evaluation efficiency and the objectivity of the results, can accurately record the time point of the event, identify problems that cannot be detected manually (such as abnormal degradation, human-machine co-driving points, etc.), and count KPI indicators that are difficult to quantify manually (such as the proportion of driving in the NOA functional area, classification statistics of takeover reasons, etc.), providing more comprehensive and reliable data support for NOA function optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 The present invention is a flowchart of an embodiment of a method for automatically evaluating NOA function.

[0020] Figure 2 A schematic diagram of a scenario of a method for automatically evaluating NOA function according to the present invention.

[0021] Figure 3 It is a schematic diagram of another scenario of a method for automatically evaluating NOA function according to the present invention.

[0022] Figure 4 It is a schematic diagram of another scenario of a method for automatically evaluating NOA function according to the present invention.

[0023] Figure 5 It is a schematic diagram of another scenario of a method for automatically evaluating NOA function according to the present invention.

[0024] Figure 6 It is a schematic diagram of another scenario of a method for automatically evaluating NOA function according to the present invention.

[0025] Figure 7 The figure is a schematic diagram of the structure of an embodiment of a NOA function automatic evaluation system of the present invention. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0027] It should be understood that the terms "first", "second", and "third" etc. in the claims, specifications, and drawings of the present disclosure are used to distinguish different objects rather than to describe a specific order. The terms "include" and "comprise" used in the specification and claims of the present disclosure indicate the presence of the described features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their collections. It should also be understood that the terms used in this disclosure specification are only for the purpose of describing specific embodiments and are not intended to limit the present disclosure.

[0028] Reference Figure 1 , an embodiment of the present invention provides a method for automatic evaluation of NOA function, comprising the following steps.

[0029] S1, obtaining multiple signal data during the process of enabling the NOA function of the vehicle.

[0030] In some embodiments, real-time data of the vehicle during operation may be collected through a data-logger.

[0031] In some embodiments, the internal communication bus (eg, CAN bus, Ethernet) of the vehicle may be used to obtain status signals related to the NOA function.

[0032] In some implementations, ROS2 or other autonomous driving platform interfaces can be used to subscribe to various topic data in vehicle operation, such as perception, fusion, positioning, mapping, vehicle status, etc.

[0033] In some implementations, relevant data on the coverage of a high-precision map may be obtained in real time through a high-precision map interface.

[0034] In some implementations, the message definition file (msg) corresponding to the recorded data may be pulled through the vehicle's software version control platform (such as GitLab) to ensure that the format and structure of the collected data meet subsequent processing requirements.

[0035] In addition, in some embodiments, in order to improve the integrity and reliability of data collection, trigger conditions can be configured before data collection, such as automatically starting packet recording when the NOA function is turned on, and generating and storing relevant metadata information (such as packet recording time, topic list, version information, etc.) in real time.

[0036] S2, preprocessing the plurality of signal data, and identifying the signal transition points of each vehicle state machine based on the preprocessed signal data.

[0037] In some implementations, the recorded data may be decompressed by a data decompression tool to generate a structured data file (such as a db3 file) containing information about each topic and a corresponding metadata information file (such as a yaml file).

[0038] In some embodiments, if metadata is missing, a supplemental file can be generated by an automated tool (e.g., the ROS2 yaml repair command).

[0039] In some implementations, a timestamp alignment algorithm is used on the signal data to align the time axes of each topic, allowing a certain time error range (eg, 0.1 second) to ensure synchronization of multi-dimensional signals.

[0040] In some implementations, the message definition file (msg) of the vehicle software version is combined to extract key signal fields in the target topic.

[0041] In some embodiments, missing values ​​are filled using, for example, interpolation or business logic rules to complete the signal data.

[0042] Finally, based on the transition rules of the state machine signal, the change points of the state machine signal are identified, laying the foundation for subsequent problem point analysis.

[0043] S3, based on the identified signal transition points, determine the problem points of the NOA function and perform statistical analysis.

[0044] In some implementations, the takeover point and reason, such as steering wheel intervention, wiper trigger, geographic fence restriction, etc., can be determined by analyzing the transition of the vehicle state machine signal and combining key signals such as steering wheel torque, brake signal, and throttle position.

[0045] In some implementations, high-precision map data and navigation status may be used to calculate the mileage and time percentage of the NOA function within the drivable area.

[0046] In some embodiments, the success, fallback or failure of the autonomous lane change is identified by analyzing the ALC state machine (automatic lane change state machine) signal.

[0047] In some embodiments, the downgrade cause signal can be used to determine the specific reason why NOA is downgraded to LCC or ACC, and distinguish between normal and abnormal downgrades.

[0048] In some implementations, the vehicle position, ramp signal, and surrounding vehicle speeds may be combined to assess whether ramp entry or exit is successful and whether the failure is caused by congestion.

[0049] In some implementations, it is possible to calculate whether there is an acceleration limit exceeding the limit based on the lateral and longitudinal accelerations, the vehicle speed, and the curvature, and to count the number of frames and times exceeding the limit.

[0050] Finally, all problem points are classified and counted through the algorithm model, and the corresponding key performance indicator (KPI) analysis results are generated for the evaluation.

[0051] S4, generating a NOA function evaluation result report.

[0052] Specifically, the identified NOA problem points (such as takeover events, human-machine co-driving points, degradation conditions, ramp entry and exit performance, etc.) and key performance indicators for statistical analysis (such as the proportion of NOA area driving, takeover number classification, human-machine co-driving times, etc.) are integrated into structured data.

[0053] In some implementations, a script tool may be used to output statistical results as an Excel file, including detailed results of each evaluation package and a long-term summary KPI table.

[0054] In some implementations, the evaluation result files can be stored in NAS or cloud storage platforms through automated procedures to ensure data security and traceability.

[0055] In some embodiments, intuitive charts and graphical results can be generated by visualization tools (such as Matplotlib, Tableau) to facilitate analysis and presentation.

[0056] In some implementations, statistical results and visualization charts can be embedded into a complete evaluation report document according to a standard template, providing key conclusions and optimization suggestions, and providing a detailed reference for NOA function improvement.

[0057] Furthermore, as an embodiment of the present invention, the multiple signal data include: perception data, fusion data, map data, positioning data and vehicle status data.

[0058] Specifically, multiple signal data include perception data, fusion data, map data, positioning data and vehicle status data, which together form the basis of NOA function evaluation. Perception data comes from the vehicle's sensors (such as cameras, radars, and lidars) and is used to identify road environments, obstacles, and lane lines; fusion data is a comprehensive processing of multi-sensor perception results to provide more accurate environmental information; map data includes high-precision map coverage, road geometry information, and geographic fences to determine the status of the vehicle within the NOA drivable area; positioning data combines GPS and inertial navigation information to provide the vehicle's accurate position in the map in real time; vehicle status data includes steering wheel torque, speed, acceleration, brake signals, and other vehicle control and motion status information. After recording, preprocessing, and analysis, these data are used to identify vehicle status changes, takeover points, abnormal events, and NOA key performance indicators, providing data support for function optimization.

[0059] Furthermore, as an implementation mode of the present invention, the preprocessing of the multiple signal data includes: time aligning each signal data; merging all signal data into a data table; and filling in missing values ​​according to business logic.

[0060] In some implementations, a fuzzy time alignment algorithm is used to align the collected signal data based on the timestamp, allowing a certain time difference (such as 0.1 second) to ensure that signals from different sources are synchronized under the same time reference.

[0061] In some implementations, all signal data (such as perception, fusion, positioning, map, vehicle status, etc.) after time alignment are integrated into a unified big data table according to predefined field mapping rules to facilitate subsequent analysis and processing.

[0062] In some embodiments, for missing values ​​in the data, a filling method is selected according to the business logic, such as interpolation to fill in consecutive missing data, or logical deduction and completion of specific signals based on the NOA function context state (such as when the steering wheel torque or status signal is lost, it can be completed by inferring the state at the previous moment).

[0063] In some implementations, for incomplete data segments, small data packets that stop abnormally can be automatically identified and discarded to ensure the integrity and consistency of pre-processed data. These steps lay the foundation for subsequent signal analysis and problem point identification.

[0064] Further, as an embodiment of the present invention, the problem points of determining the NOA function at least include: Identify the NOA takeover point and determine the reasons for the takeover; Identify the points where humans and machines co-drive; Identify NOA degradation points and determine the reasons for degradation; Identify autonomous lane change points and determine lane change outcomes; Identify entry and exit ramp points and determine success; and, identifying the acceleration exceeding limit point.

[0065] Furthermore, as an embodiment of the present invention, the identifying of the NOA takeover point and determining the takeover reason include: obtaining NOA state machine data, and when the NOA state data jumps from an activated state to a passive exit state or a manual takeover state, marking it as a NOA takeover point; and determining the takeover reason of the NOA takeover point based on a preset takeover reason judgment logic.

[0066] Specifically, the NOA state machine signal data is obtained, and when the state signal jumps from the activation state (5) to the passive exit state (2) or the manual takeover state (7), the jump point is marked as the NOA takeover point.

[0067] Subsequently, based on the preset takeover reason judgment logic, the specific reason for the takeover is determined by analyzing key data such as steering wheel torque, brake signal, throttle position signal, wiper status signal and degradation reason signal.

[0068] Exemplarily, the state machine signal s, the wiper level signal, the tts broadcast information, the throttle position signal, the partial reason signal for taking over the downgrade, and the signal for judging whether to brake are obtained.

[0069] Based on the above signals, the logic of each reason is as follows: Takeover caused by the steering wheel: The NOA state machine signal jumps from 5 to 7, filtering out takeovers caused by the throttle and those with unknown reasons for missing data. The rest are defined as takeovers caused by the steering wheel. If two adjacent takeovers caused by the steering wheel do not exceed 3 seconds, they are merged into one. Takeover caused by geo-fencing: The NOA state machine signal jumps from 5 to 2. Whether the vehicle exceeds the ODD area within the next 150 meters, or whether the "intelligent navigation assistance system will exit after 200 meters" is broadcast between the first 300 meters and the first 100 meters; Takeover caused by wiper: The wiper level signal jumps to 4 or 6; During the process of entering the main road from the ramp, all functions are fully withdrawn and taken over: the NOA state machine signal jumps from 5 to 2. Part of the reason for the takeover downgrade is that the signal reports that the latest lane change point was missed when the ramp was entered; Takeover caused by braking: the NOA state machine signal jumps from 5 to 2, and then determines whether the brake signal is 1; Unexplained takeover due to missing data: The NOA state machine signal jumps from 5 to 2, but the intermediate data is lost for more than 1 second. The reason cannot be determined and is classified into this category; Unknown reason: NOA state machine signal jumps from 5 to 2. Those that do not belong to the above items are classified into this category.

[0070] Further, as an embodiment of the present invention, the identifying the human-machine co-driving point includes: identifying the human-machine co-driving point based on steering wheel torque data and steering system signal data.

[0071] Specifically, by collecting steering wheel torque data and steering system signal data (such as EPS signals), the force applied by the driver to the steering wheel and the output of the vehicle steering system are monitored in real time. When the steering wheel torque signal shows that the driver is applying force, and the EPS signal shows that the system is also adjusting the steering wheel, it is determined that the driver and the system are controlling the direction at the same time, thereby identifying the human-machine co-driving point.

[0072] In some implementations, in order to improve the accuracy of recognition, the system can combine the pre-processed co-driving signal topic data to directly read the result of whether co-driving has occurred, thereby reducing the computational complexity and ensuring the consistency of the recognition results. This method can comprehensively monitor the cooperation between the driver and the autonomous driving system and provide support for the evaluation of the NOA function.

[0073] Further, as an implementation mode of the present invention, identifying the NOA degradation point and determining the degradation cause includes: Identify the first degradation point from NOA degradation to LCC through the transition of NOA state machine; and determining the downgrade reason of the first downgrade point based on a preset first judgment logic; The second degradation point from NOA to ACC is identified by the transition of NOA state machine; And based on a preset second judgment logic, a demotion reason of the second demotion point is judged.

[0074] Specifically, the NOA state machine signal is parsed, and when the signal value jumps from the NOA activation state (5) to the LCC state (4), the jump point is marked as the first degradation point from NOA to LCC; then, based on the preset judgment logic, the specific cause of the degradation point is determined by analyzing the degradation cause signal (such as the partial cause signal of the takeover degradation) and other related signals (such as the high-precision map coverage status, vehicle location and surrounding environment data). For example, if the degradation cause signal indicates that the vehicle exceeds the high-precision map or ODD area, it is classified as normal degradation; if the signal shows that the problem is due to an uncompleted lane change or a diagnostic error, it is classified as abnormal degradation. This method ensures the accurate identification of degradation points and degradation causes, and provides data support for the abnormal analysis and optimization of NOA functions.

[0075] Exemplarily, the point at which NOA is downgraded to LCC is identified by a state machine, such as Figure 2 As shown in the figure, the value of the NOA state machine signal drops from 5 (NOA state) to 4 (LCC state), which is a degradation situation. The state machine error signal line shows the cause of degradation. When the signal is degraded, the value jumps from 32 to 160. This value jump shows that the reason is that there is no time to change lanes.

[0076] Among them, the downgrade of NOA to LCC caused by the following reasons is considered normal downgrade: exceeding the high-precision map area, exceeding the ODD area, and being in the emergency lane; the downgrade of NOA to LCC caused by the following reasons is considered abnormal downgrade: missing the latest lane change point, diagnostic error, navigation abnormality, and other reasons.

[0077] Furthermore, the NOA state machine signal is monitored. When the signal jumps from the NOA activation state (5) to the ACC state (3), the jump point is marked as the second degradation point from NOA to ACC. Then, based on the preset second judgment logic, the specific cause of the degradation is determined in combination with the degradation cause signal and the vehicle operation status signal (such as steering wheel torque, lateral acceleration, lane line status, etc.). For example, when the degradation cause signal shows that the lane curvature is too large, the lane line is missing, or the vehicle lane is too narrow, it is classified as normal degradation; if the signal indicates that the degradation is triggered by abnormal behaviors such as excessive steering wheel torque, line pressing in non-lane change scenarios, etc., it is classified as abnormal degradation. Through this process, the ACC degradation point and its cause can be accurately identified, providing support for the reliability analysis and optimization of the NOA function.

[0078] Exemplarily, the point at which NOA is downgraded to ACC is identified by a state machine, such as Figure 3As shown in the figure, the value of the NOA state machine signal drops from 5 (NOA state) to 3 (ACC state), which is a degradation situation. The degradation reason signal line shows the degradation reason. When the signal is degraded, the value jumps from 0 to 32. The value jump shows that the reason is excessive curvature.

[0079] Among them, the downgrade from NOA to ACC caused by the following reasons is counted as normal downgrade: excessive curvature, lane line loss, and the own vehicle lane is too narrow; the downgrade from NOA to ACC caused by the following reasons is counted as abnormal downgrade: excessive steering wheel torque, EPS, pressing the line for 5 seconds in non-lane change scenario, pressing the line for 8 seconds in lane change scenario, excessive steering wheel angular speed, excessive lateral acceleration, manual lane change, lateral override, hands-off detection module failure, and other reasons.

[0080] Furthermore, as an embodiment of the present invention, the identifying of the autonomous lane change point and determining the lane change result includes: acquiring vehicle automatic lane change state machine data, and identifying the autonomous lane change point based on the automatic lane change state machine data; and determining the lane change result of the autonomous lane change point based on a preset lane change result determination logic.

[0081] Specifically, the vehicle's automatic lane change state machine signal is obtained, and the start and end of the autonomous lane change are identified by monitoring the jump of the state machine signal. For example, when the state machine signal jumps from "lane change waiting" (5) to "lane change to the left" (6) or "lane change to the right" (7), it is marked as the start of the autonomous lane change; when the signal further jumps to "lane change completed" (1 or 2) or "lane change return" (8), it is marked as the end of the lane change. Subsequently, based on the preset lane change result judgment logic, the lane change result is classified in combination with other auxiliary signals: if the signal shows that the lane change is completed and the success mark is 1, it is judged as a successful lane change; if the lane change state jumps back to "lane change return", it is judged as a lane change return; if the signal shows that the lane change is not completed or the waiting time is directly timed out, it is judged as a lane change failure. In addition, the rationality of the lane change process can be verified by calculating the lane change duration and the vehicle position change, thereby ensuring the accuracy and comprehensiveness of the judgment result.

[0082] Exemplarily, the ALC state machine identifies the point of autonomous lane change and identifies whether the lane change is to the left or to the right. The following is the logical judgment: Definition of a successful lane change: The automatic lane change state machine signal jumps from 6 / 7 (6 for changing lanes to the left, 7 for changing lanes to the right) to 1 or 2, and the auxiliary signal jumps to 1 at the same time; Definition of lane change rollback: The automatic lane change state machine signal changes from 6 / 7 to 8; Definition of lane change failure: The automatic lane change state machine signal jumps from 6 / 7 to 1 or 2, and the auxiliary signal does not jump to 1 at the same time; Definition of waiting to change lanes but not changing lanes (classified as lane change failure): The automatic lane change state machine signal does not change from 5 to 6 / 7 (i.e. waiting to change lanes but not changing lanes).

[0083] For example, refer to Figure 4 , the automatic lane change state machine signal jumps from 5 (waiting for lane change) to 6 (changing lanes to the left), and then jumps to 1, and at the same time the auxiliary signal jumps to 1, it is judged that the lane change is successful.

[0084] Further, as an embodiment of the present invention, the step of identifying the entry and exit ramp points and determining whether the step is successful includes: Identify NOA entry and exit ramp points through the transition of NOA state machine, and filter out misidentified points; If NOA takeover and / or degradation occurs at the on-ramp and off-ramp point, the automatic on-ramp and off-ramp point is deemed to have failed, and the cause of the failure is determined based on a preset evaluation logic.

[0085] Specifically, by parsing the NOA state machine signal and the ramp sign signal, when the ramp sign signal jumps from 0 (main road) to 1 (ramp), it is marked as the main road entry ramp point; when the signal jumps from 1 to 0, it is marked as the ramp exit main road point.

[0086] In some embodiments, the high-precision map data can be combined to filter out misidentified points. For example, when the ramp sign signal is incorrectly marked due to the lack of high-precision map coverage, these misidentified points are filtered out through the map status. For each entry and exit ramp point, if a NOA takeover event (such as a NOA state machine signal jumps to passive exit or manual takeover) or a degradation event (such as a NOA state machine signal degrades to LCC or ACC) is detected, it is determined that the entry and exit ramp failed.

[0087] The cause of failure is then determined based on the preset evaluation logic. For example, by analyzing the congestion situation in the target lane (calculating vehicle speed and traffic density) or vehicle control signals (such as steering wheel torque and brake signals), it is determined that the failure is caused by congestion, exceeding the high-precision map area, system diagnostic errors, etc.

[0088] In this way, accurate identification of entry and exit ramp points and success rate evaluation are achieved, providing key data support for the optimization of NOA functional performance.

[0089] For example, by checking whether the ramp signal changes, it is possible to identify where the main road enters the ramp and where the ramp enters the main road, such as Figure 5 As shown in the figure, when the ramp sign signal jumps from 0 to 1, it means the main road enters the ramp; when it jumps from 1 to 0, it means the ramp merges into the main road: For example, filtering misidentified parts: some of the recognized main roads entering ramps and ramps entering main roads are actually on the ramps. This is because all parts without high-precision coverage are marked as main roads. Therefore, it is necessary to filter those misidentified parts based on whether they are covered by high-precision. Figure 6 As shown in the figure, it looks like the main road is entering the ramp, but in fact it is always on the ramp. There is no high-precision map before 16:03:25, which makes it impossible to know whether it is on the ramp, so the ramp sign signal gives a default value of 0 (on the main road), but it is actually on the ramp. That is, this section is mistakenly identified as the main road entering the ramp and needs to be filtered out.

[0090] In one implementation, a main road-to-ramp evaluation area (100 meters of main road, all deceleration lanes, and ramp 50) and a ramp-to-main road evaluation area (50 meters of ramp, all acceleration lanes, and main road 100 meters) are defined, and data entering the evaluation area when NOA is turned on is filtered to determine whether the entry and exit of the ramp is successful: if takeover, human-machine co-driving, or abnormal degradation occurs in the evaluation area, it is considered a failure; otherwise, it is considered a success.

[0091] For the failed results, evaluate whether they are caused by congestion: identify the target lane that needs to be changed; if it is necessary to change lanes to the left, calculate the average speed of the vehicles in the area with a lateral distance of 2.5 to 4.5 meters and a longitudinal distance of -30 to 20 meters; if it is necessary to change lanes to the right, calculate the average speed of the vehicles in the area with a lateral distance of -4.5 to -2.5 meters and a longitudinal distance of -30 to 20 meters; when the average speed of the target lane is lower than 10m / s, it is judged to be congested.

[0092] Furthermore, as an implementation mode of the present invention, it also includes: counting abnormal jump data and line pressure data of each entry and exit ramp point.

[0093] Specifically, the entry and exit points are identified through the jumps of NOA state machine signals and ramp sign signals, and the misidentified points are filtered out by combining high-precision map data and the actual vehicle path. Subsequently, for each entry and exit point, the vehicle's abnormal jump signals (such as abnormal changes in steering wheel torque and sudden changes in degradation signals) are extracted and counted, marking jump events caused by instability or environmental changes.

[0094] In some embodiments, lane-crossing behavior can be identified by analyzing the lateral distances between the vehicle's front wheels and rear axles and lane lines: when the lateral distance from the outer side of the front wheel to the inner edge of the lane line is less than zero or the distance from the center of the rear axle to the lane line is less than 0.5 times the rear wheelbase, it is determined to be lane-crossing; if the instantaneous lateral distance change during lane changing exceeds a certain threshold (such as 3 meters), this segment of data is excluded as a normal change caused by lane changing.

[0095] Finally, the statistical abnormal jump and line-crossing data are classified and summarized by ramp section to provide detailed abnormal behavior data support for the ramp scenario performance evaluation of the NOA function.

[0096] Further, as an embodiment of the present invention, the identifying of the acceleration exceeding limit point includes: identifying the longitudinal acceleration exceeding limit point; identifying the longitudinal deceleration exceeding limit point; identifying the longitudinal deceleration change rate exceeding limit point; and identifying the lateral acceleration exceeding limit point.

[0097] In some embodiments, identifying the longitudinal acceleration exceeding limit point includes: extracting the longitudinal acceleration signal of the vehicle (the signal after filtering) and the current vehicle speed data, and calculating frame by frame whether each data point exceeds the safe acceleration range according to a preset exceeding limit judgment logic.

[0098] For example, the specific judgment logic includes: when the vehicle speed is less than 5 m / s, if the longitudinal acceleration exceeds 4 m / s², it is judged as over-limit; when the vehicle speed is between 5-20 m / s, the dynamic threshold is calculated according to the linear formula (for example, longitudinal acceleration>(-2 / 15 × vehicle speed + 14 / 3)) and whether it is over-limit is determined; when the vehicle speed is greater than 20 m / s, the longitudinal acceleration exceeds 2 m / s², it is over-limit. The over-limit points are merged into one over-limit event through continuous over-limit frames to avoid repeated statistics.

[0099] In some embodiments, identifying the longitudinal deceleration exceeding limit point includes: extracting the longitudinal acceleration signal and vehicle speed data of the vehicle, and analyzing frame by frame according to a preset deceleration exceeding limit judgment logic.

[0100] For example, the specific logic is: when the vehicle speed is less than 5 m / s, the longitudinal acceleration is less than -5 m / s² and is judged as exceeding the limit; when the vehicle speed is between 5-20 m / s, the dynamic deceleration threshold is calculated (such as longitudinal acceleration <0.1 × vehicle speed - 5.5), and if it is lower than this value, it is judged as exceeding the limit; when the vehicle speed is greater than 20 m / s, the deceleration is less than -3.5 m / s² and is considered as exceeding the limit. All exceeding points are marked by frame-by-frame analysis, and consecutive exceeding frames are merged into a single deceleration exceeding event to ensure the simplicity and accuracy of the statistical results.

[0101] In some embodiments, identifying the longitudinal deceleration change rate exceeding the limit point includes: extracting the longitudinal acceleration signal of the vehicle, and calculating the acceleration change rate frame by frame (i.e., the difference between the acceleration of the current frame and the previous frame divided by the time difference). Then, based on the change rate of each frame, the average deceleration change rate in the past 1 second is calculated, which is used as a basis for judgment.

[0102] For example, the specific judgment logic is: when the vehicle speed is less than 5 m / s, if the average deceleration change rate is lower than -5 m / s³, it is judged as over-limit; when the vehicle speed is between 5-20 m / s, the dynamic threshold is calculated (such as the deceleration change rate < (1 / 6 × vehicle speed - 35 / 6)), and if it is lower than this threshold, it is over-limit; when the vehicle speed is greater than 20 m / s, it is over-limit if it is lower than -2.5 m / s³. Consecutive over-limit frames are merged into a single event, and these over-limit points are time-sorted and statistically analyzed to provide data support for evaluating the longitudinal dynamic control stability of the NOA function.

[0103] In some embodiments, identifying the lateral acceleration exceeding limit point includes: extracting the lateral acceleration signal of the vehicle and the road curvature data, and calculating the curve radius according to the curvature (curve radius = 1 / curvature); and then analyzing frame by frame whether the lateral acceleration exceeds the threshold according to a preset exceeding limit judgment logic.

[0104] For example, the specific logic is: when the curve radius is between 125m and 250m, the lateral acceleration is over 2.3m / s²; when the curve radius is between 250m and 500m, the dynamic threshold is calculated by the formula (such as lateral acceleration > -0.0012 × curve radius + 2.6), and exceeding this value is over; when the curve radius is greater than 500m, the lateral acceleration exceeds 2 m / s², which is over. Consecutive over-limit frames are merged and recorded as a single over-limit event, and classified and counted by time sequence and event type, providing key data support for the evaluation of the lateral control performance of the NOA function in curve driving.

[0105] Further, see Figure 7 The embodiment of the present invention also discloses a NOA function automatic evaluation system, including a data acquisition module 1, a jump recognition module 2, a data statistics module 3 and a report generation module 4.

[0106] Data acquisition module 1 collects multi-dimensional signal data of the vehicle in the process of NOA function activation in real time through the vehicle data recording tool (data-logger), including perception data, fusion data, map data, positioning data and vehicle status data; uses the vehicle communication bus (such as CAN bus or Ethernet) to obtain internal signals related to vehicle operation; subscribes to various topic data generated during operation through ROS2 or other platform interfaces; combines high-precision map interfaces to obtain map coverage information of the area where the vehicle is located in real time; and automatically pulls the software version message definition file (msg) corresponding to the recording data through the version control platform (such as GitLab) to ensure that the format of the signal data is consistent with subsequent analysis. Data acquisition module 1 can also configure trigger conditions (such as starting the recording when the NOA function is activated) and generate metadata information (such as timestamps, topic lists, etc.) to ensure the integrity and reliability of data acquisition and provide basic data support for subsequent preprocessing and analysis.

[0107] The jump recognition module 2 decompresses the recorded data through the data decompression tool, extracts the structured files (such as db3 files) and metadata files (such as yaml files) containing the information of each topic; if the metadata is missing, it can be completed through the automatic repair tool (such as the yaml repair command of ROS2); then, the jump recognition module 2 aligns the timestamps of the signal data, unifies the time of different topics to the same time axis through the fuzzy matching algorithm; combines the message definition file (msg) of the software version, extracts the key signals in the preset fields; for data with missing values, interpolation or logical rules are used to complete; finally, by parsing the NOA state machine signal and other key signals (such as the NOA state machine signal), the key points of the vehicle state change (such as the takeover point, degradation point, autonomous lane change point, etc.) are identified based on the state jump logic. The jump recognition module 2 outputs the time, type and related signals of the jump point, providing accurate input data for subsequent statistical analysis.

[0108] Based on the jump point data output by the jump recognition module 2, the data statistics module 3 combines the vehicle's multi-dimensional signals (such as steering wheel torque, brake signal, throttle position, wiper status, high-precision map coverage status, etc.) and classifies and counts the problem points of the NOA function according to the preset logical rules. Specifically, it includes: counting the NOA takeover points and their takeover reasons (such as steering wheel intervention, environmental restrictions, etc.); calculating the proportion of NOA mileage and time in the drivable area; analyzing the frequency and reasons of downgrade events (such as downgrade to LCC or ACC) (distinguishing between normal and abnormal downgrades); counting the number of successes, failures and returns of autonomous lane changes; identifying the success rate and reasons for failure of entering and exiting ramps; and counting acceleration over-limit events (such as the number of over-limits and frames of longitudinal acceleration, deceleration and lateral acceleration). The data statistics module 3 classifies and summarizes the statistical results and outputs structured data to provide comprehensive and accurate performance analysis data support for generating NOA function evaluation reports.

[0109] The report generation module 4 integrates the classified statistical results and key performance indicators (such as NOA takeover frequency, distribution of degradation reasons, autonomous lane change success rate, ramp entry and exit success rate and acceleration over-limit events, etc.) output by the data statistics module 3, and automatically generates structured evaluation result files (such as Excel tables) from these data, including daily and long-term evaluation summary tables; generates intuitive charts (such as bar charts, pie charts, line charts) through visualization tools to show the performance of the NOA function in different scenarios; according to the preset template, embeds statistical data and charts into a complete evaluation report document, and attaches conclusive analysis and optimization suggestions; finally, the evaluation report is output in PDF or other formats, and automatically uploaded to NAS or cloud storage platforms to ensure the convenience of data archiving and sharing, and provide professional support for the evaluation and improvement of the NOA function.

[0110] The embodiment of the present invention also discloses a readable storage medium.

[0111] A readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the NOA function automatic evaluation method described in any one of the above embodiments.

[0112] It can be understood that computer-readable storage media may include: any entity or device capable of carrying a computer program, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), and a software distribution medium, etc. A computer program includes a computer program code. The computer program code may be in source code form, object code form, an executable file, or some intermediate form, etc. A computer-readable storage medium may include: any entity or device capable of carrying a computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), and a software distribution medium, etc.

[0113] In certain embodiments of the present invention, the electronic device may include a controller or a processor, and the controller is a single-chip microcomputer chip that integrates a processor, a memory, a communication module, etc. The processor may refer to a processor included in the controller. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc.

[0114] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present invention belong.

[0115] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0116] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for automatically evaluating NOA function, characterized in that: include: Obtain multiple signal data during the vehicle NOA function activation process; Preprocessing the plurality of signal data, and identifying a signal transition point of each vehicle state machine based on the preprocessed signal data; Based on the identified signal transition points, determine the problem points of the NOA function and perform statistical analysis; And, generate an evaluation result report of the NOA function.

2. The NOA function automatic evaluation method according to claim 1, characterized in that: The multiple signal data include: perception data, fusion data, map data, positioning data and vehicle status data.

3. The NOA function automatic evaluation method according to claim 1, characterized in that: The preprocessing of the plurality of signal data comprises: Time-align each signal data; Combine all signal data into one data table; Fill missing values ​​based on business logic.

4. The NOA function automatic evaluation method according to claim 1, characterized in that: The problem points of determining the NOA function at least include: Identify the NOA takeover point and determine the reasons for the takeover; Identify the points where humans and machines co-drive; Identify NOA degradation points and determine the reasons for degradation; Identify autonomous lane change points and determine lane change outcomes; Identify entry and exit ramp points and determine success; and, identifying the acceleration exceeding limit point.

5. The NOA function automatic evaluation method according to claim 4, characterized in that: The identifying of the NOA takeover point and determining the takeover reason include: Acquire NOA state machine data, and when the NOA state data changes from an activated state to a passive exit state or a manual takeover state, it is marked as a NOA takeover point; The takeover reason of the NOA takeover point is determined based on a preset takeover reason determination logic.

6. The NOA function automatic evaluation method according to claim 4, characterized in that: The identifying of the human-machine co-driving point comprises: The human-machine co-driving point is identified based on the steering wheel torque data and the steering system signal data.

7. The NOA function automatic evaluation method according to claim 4, characterized in that: The identification of the NOA downgrade point and determination of the downgrade cause includes: Identify the first degradation point from NOA to LCC through the transition of NOA state machine; and determining the downgrade reason of the first downgrade point based on a preset first determination logic; The second degradation point from NOA to ACC is identified by the transition of NOA state machine; And based on a preset second judgment logic, a demotion reason of the second demotion point is judged.

8. The NOA function automatic evaluation method according to claim 4, characterized in that: The identifying of the autonomous lane change point and determining the lane change result includes: Acquiring vehicle automatic lane change state machine data, and identifying the autonomous lane change point based on the automatic lane change state machine data; The lane change result of the autonomous lane change point is determined based on a preset lane change result determination logic.

9. The NOA function automatic evaluation method according to claim 4, characterized in that: The step of identifying the entry and exit ramp points and determining whether the step is successful includes: Identify NOA entry and exit ramp points through the transition of NOA state machine, and filter out misidentified points; If NOA takeover and / or degradation occurs at the on-ramp and off-ramp point, the automatic on-ramp and off-ramp point is deemed to have failed, and the cause of the failure is determined based on a preset evaluation logic.

10. The NOA function automatic evaluation method according to claim 9, characterized in that: Also includes: Statistics are collected on abnormal jump data and line pressure data of each entry and exit ramp point.

11. The NOA function automatic evaluation method according to claim 4, characterized in that: The identification of the acceleration exceeding limit point comprises: Identify the point where the longitudinal acceleration exceeds the limit; Identify the point where the longitudinal deceleration exceeds the limit; Identify the point at which the longitudinal deceleration rate is exceeded; and Identify the point where lateral acceleration is exceeded.

12. A NOA function automatic evaluation system, characterized in that: include: A data acquisition module, at least used to acquire a plurality of signal data during the activation of the NOA function of the vehicle; a jump identification module, at least used to preprocess the plurality of signal data, and identify the signal jump points of each vehicle state machine based on the preprocessed signal data; A data statistics module, at least used to determine the problem points of the NOA function based on the identified signal transition points and perform statistical analysis; The report generation module is used to generate at least a report on the evaluation results of the NOA function.

13. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a computer, the automatic NOA function evaluation method according to any one of claims 1 to 11 is executed.

Images