Test method and system based on driver's reaction capability of identifying icons of intelligent driving cockpit

Abstract

The invention relates to the technical field of automobile driver data monitoring, and provides a test method and system based on the response capability of a driver to identify an intelligent driving cabin icon, and the method comprises the steps: collecting the feature attributes of the intelligent driving cabin icon of a tested vehicle, and combining a plurality of intelligent driving cabin icons into a target template according to each display position and change time; a road side test is carried out, a target image of the display area of the intelligent driving cabin icon is collected in real time through a first visual sensor, whether the target image is matched with the intelligent driving cabin icon in the target template or not is judged, and if yes, the response capacity of a driver to the intelligent driving cabin icon is collected; otherwise, continuing to collect the target image. According to the invention, the reaction capability of the driver to the intelligent driving cabin icon can be accurately and rapidly obtained under the road test condition.

Description

Technical Field

[0001] This invention relates to the field of automotive driver data monitoring technology, specifically to a testing method and system for assessing a driver's ability to recognize smart cockpit icons. Background Technology

[0002] With the rapid development of the automotive industry, drivers are increasingly reliant on warning icons and prompts displayed on vehicle dashboards and central control screens. Especially with the continuous evolution of intelligent driving technology, the content displayed on dashboards and central control screens has become unprecedentedly rich and diverse, and is increasingly being used in vehicles with HUDs (Heads-Up Displays). These project information such as speed, navigation indicators, and warning signals onto the windshield for easy driver observation. The timely display of this information is crucial for ensuring driving safety. Any delay or omission of navigation indicators and warning signals can lead to missing key intersections, entering restricted areas, or delaying arrival at the destination. This not only affects travel efficiency but may also trigger dangerous driving behaviors such as sudden braking and lane changes, increasing the risk of traffic accidents. The timely and accurate presentation of this information is particularly critical for driving safety in complex road conditions or at night.

[0003] The location of the smart cockpit icons displayed on the instrument panel, center console screen, or HUD generally varies across different vehicles. In existing technology, vehicle design, including the design of the instrument panel, center console screen, or HUD, further involves designing the displayed icons. After production, the vehicle undergoes testing. For testing of vehicles about to leave the factory, most automakers rely on the vehicle's CAN bus signals to verify the accuracy and timeliness of the smart cockpit icon information display, and to test the driver's ability to recognize the smart cockpit icons. Testers typically rely on these signals to determine whether status or warning signals, navigation indication signals, etc., are correctly triggered, ensuring that changes in vehicle status can be accurately and promptly recognized by the driver. However, because different automakers use different signal definitions, this creates inconvenience for testers' data collection. Furthermore, drivers typically react to the smart cockpit icons according to conventional and pre-defined operating manuals, making it impossible to understand the driver's reaction time to the display of one or more smart cockpit icons while driving. This reaction time includes the driver's reaction time and corresponding actions. Summary of the Invention

[0004] The present invention aims to provide a testing method and system for assessing a driver's ability to recognize smart cockpit icons, which can accurately and quickly obtain the driver's ability to react to smart cockpit icons.

[0005] The first approach, a test method based on the driver's reaction ability to recognize intelligent driving cockpit icons, includes: collecting the feature attributes of the intelligent driving cockpit icons of the test vehicle; combining multiple intelligent driving cockpit icons into a target template according to their display positions and change times; conducting roadside testing, acquiring target images of the display area of ​​the intelligent driving cockpit icons in real time through a first visual sensor, and determining whether the target image matches the intelligent driving cockpit icons in the target template. If they match, the driver's reaction ability to the intelligent driving cockpit icons is collected; otherwise, the target image is collected again.

[0006] Beneficial effects: By collecting the feature attributes of icons in the intelligent driving cockpit and combining them with target templates, and using visual sensors to match icons in real time and collect driver reactions, this method effectively solves the problem of inconvenient data collection caused by inconsistent signals among different car manufacturers, without relying on the specific CAN bus signal definitions of different car models. Its core advantage lies in directly locking onto target icons through image recognition, quickly and accurately obtaining the driver's reaction time and processing operations, and adapting to the differences in icon display positions across different car models.

[0007] Preferably, the feature attributes of a single intelligent driving cockpit icon are segmented into multi-dimensional data, and each intelligent driving cockpit icon is combined according to different dimensions to form a target template; the feature attributes include display position, change time, and icon information; wherein, display position is the first dimension data, change time is the second dimension data, and icon information is the third dimension data.

[0008] Beneficial effects: By breaking down the feature attributes of the intelligent driving cockpit icons into three dimensions—display location, change time, and icon information—and combining them with target templates, the template construction becomes more logical and accurate. This multi-dimensional data breakdown and combination not only improves the targeting and matching efficiency of icon recognition but also accurately distinguishes icon changes in different scenarios (such as different icons at the same location or similar icons triggered at different times), avoiding misjudgments caused by matching a single feature. This lays a data foundation for the accurate collection of driver reaction capabilities in subsequent tasks.

[0009] Preferably, the display area of ​​the intelligent driving cockpit icon includes the instrument panel, the central control screen, and the HUD.

[0010] Beneficial effects: The display area of ​​the intelligent driving cockpit icon clearly covers the instrument panel, central control screen, and HUD, comprehensively covering the mainstream display scenarios of current intelligent driving vehicles. This limitation ensures that the test scope is consistent with the driver's actual observation range, avoiding incomplete testing due to the omission of key display areas such as the HUD. At the same time, it is compatible with an increasing number of intelligent vehicle models equipped with HUDs, improving the universality and practical application value of the testing method.

[0011] Preferably, the driving status data of the test vehicle is acquired in real time. The driving status data is used to characterize the vehicle driving status data corresponding to the intelligent driving cockpit icon. The driving status data is parsed to obtain the intelligent driving cockpit icon that should be displayed. Simultaneously, a target image is acquired, and it is determined whether the intelligent driving cockpit icon that should be displayed exists in the target image. If the intelligent driving cockpit icon that should be displayed does not exist in the target image, it is determined that the intelligent driving cockpit icon is not displayed, or the intelligent driving cockpit icons overlap. If the intelligent driving cockpit icon that should be displayed exists in the target image, the driver's reaction ability to the intelligent driving cockpit icon is collected.

[0012] Beneficial effects: By acquiring driving status data in real time and analyzing the icons to be displayed, and simultaneously comparing them with the actual icons in the target image, the system achieves the dual objectives of "driver reaction collection" and "icon display verification." It not only accurately collects the driver's reaction to validly displayed icons but also promptly detects display anomalies such as missing or overlapping icons. This provides automakers with integrated data support for simultaneously optimizing the icon triggering logic and cockpit display effects of intelligent driving systems, enhancing the depth and practicality of the testing.

[0013] Preferably, within a preset first time period, if the intelligent driving cockpit icon that should be displayed is not displayed in the specified position in the target template, and no other intelligent driving cockpit icon is displayed in the specified position, then it is determined that the intelligent driving cockpit icon is not displayed; if there are multiple intelligent driving cockpit icons that should be displayed in any position in the display area of ​​the intelligent driving cockpit icon, and an intelligent driving cockpit icon is displayed in the position, then it is determined that the intelligent driving cockpit icon overlap phenomenon occurs.

[0014] Beneficial effects: By refining the criteria for judging icon display errors and overlaps, and using quantitative conditions such as "first time period" and "whether there are other icons at the specified location," subjective errors in judging icon display anomalies are reduced. This limitation makes the identification of faults such as missing or overlapping icons more objective and operable, helping automakers accurately locate problems in the cockpit icon layout design or display logic (such as icon position conflicts or display delays), and providing a clear direction for improving cockpit human-machine interaction.

[0015] Preferably, if the intelligent driving cockpit icon captured in the target image does not match the intelligent driving cockpit icon that should be displayed after parsing the vehicle driving status data, it is determined that a false intelligent driving cockpit icon has occurred.

[0016] Beneficial effects: The judgment logic for false alarms in the intelligent driving cockpit icons improves the identification dimensions of abnormal icon display. It can accurately capture situations where "the collected icon does not match the icon that should be displayed," helping automakers discover loopholes in the icon triggering logic of the intelligent driving system (such as triggering an alarm when there is no corresponding working condition). This not only ensures the accuracy of driver reaction tests (eliminating the interference of false alarm icons on the driver), but also provides key data for optimizing the fault diagnosis function of the intelligent driving system.

[0017] Preferably, the road test data includes vehicle dynamic data, environmental perception data, intelligent cockpit and human-machine interaction data, autonomous driving system decision and control data, high-precision map and positioning data, and vehicle external environment and traffic information, as well as time synchronization and metadata.

[0018] Beneficial effects: The specific categories of road test data cover multiple dimensions of information, including vehicle dynamics, environmental perception, and intelligent driving decision-making, providing comprehensive and realistic raw data support for testing. This rich road test data not only accurately represents the triggering scenarios of the intelligent driving cockpit icons but also provides ample data sources for subsequent multimodal data construction and scenario parameter modification, ensuring consistency between simulated scenarios and real driving conditions, and enhancing the credibility and reference value of the test results.

[0019] Preferably, by modifying the key parameters of the preprocessed road test data, a road test dataset is formed. Each road test dataset is used to characterize the changes in the intelligent driving cockpit image under at least one real-world scenario. Multiple road test datasets are combined to form multimodal data. The key parameters include weather, lighting conditions, navigation information, and vehicle information.

[0020] Beneficial effects: By modifying key parameters of road test data (weather, lighting, etc.) to construct datasets and combine them into multimodal data, rapid replication and flexible expansion of real-world driving scenarios are achieved. This eliminates the need to rely on complex natural conditions or fault scenarios in actual road tests, significantly reducing testing costs and time. Simultaneously, it can cover scenarios that are difficult to reproduce frequently in real-world road tests, such as extreme weather and complex road conditions, thus improving the richness and coverage of test scenarios.

[0021] Preferably, when the road test involves the vehicle under test undergoing corresponding tests at a vehicle test track, multimodal data is input into the vehicle under test, and the multimodal data is represented as a preset intelligent driving cockpit icon. The preset intelligent driving cockpit icon is an intelligent driving cockpit icon acquired synchronously in the target image. It is then determined whether the intelligent driving cockpit icon matches the intelligent driving cockpit icon in the target template. If they match, the driver's reaction ability to the intelligent driving cockpit icon is collected; otherwise, the target image is collected again and matched with the intelligent driving cockpit icon in the target template.

[0022] Beneficial effects: By limiting the testing scenario to a vehicle testing ground and integrating multimodal data with preset vehicle representation icons, safe and controllable high-frequency, repetitive testing is achieved. This avoids the safety risks and uncontrollable operating conditions of real-world road testing, while addressing the pain points of low efficiency and limited test runs in actual road testing. It ensures driver safety and vehicle integrity, while also improving the stability and reliability of driver reaction data through high-frequency testing, providing more statistically significant data for intelligent driving system optimization.

[0023] The second approach is a test system based on the driver's ability to recognize intelligent cockpit icons, which includes a data analysis module. The data analysis module includes an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it enables the electronic device to implement the test method of the first approach based on the driver's ability to recognize intelligent cockpit icons.

[0024] Beneficial effects: This system provides a hardware implementation system adapted to the aforementioned testing methods. By combining a data analysis module with electronic devices, it transforms abstract testing logic into an automated and implementable execution solution. The system can automate the entire process of data acquisition, parsing, matching, and judgment, reducing errors caused by manual intervention, improving testing efficiency, and supporting standardized testing of batch vehicles. It meets the large-scale production testing needs of automakers, providing hardware support for the practical application of the testing methods. Attached Figure Description

[0025] Figure 1 This is a flowchart illustrating the test method for the driver's ability to recognize intelligent driving cockpit icons in Example 1. Figure 2 A schematic diagram of the structure of the first vision sensor installed inside the vehicle in the test case; Figure 3 A schematic diagram of the target image acquired by the first visual sensor in the experimental example; Figure 4 This is a schematic diagram of the intelligent driving cockpit icons obtained for the test case; Figure 5 This is a schematic diagram showing the information results of the intelligent driving cockpit icon in the test case; Figure 6 A schematic diagram (a) shows the setup for recognizing a single smart cockpit icon in the test case. Figure 7 A schematic diagram (b) showing the setup for recognizing a single smart cockpit icon in the test case. Figure 8 A schematic diagram illustrating the setup for recognizing multiple smart cockpit icons in the test case; Figure 9This is a schematic diagram of the test system for the driver's ability to recognize intelligent driving cockpit icons, as described in Example 3. Figure 10 This is a schematic diagram of the electronic device structure of the test system based on the driver's ability to recognize intelligent driving cockpit icons, as described in Example 3.

[0026] The reference numerals in the accompanying drawings include: Camera fixture 1, camera 2, processor 101, input device 102, output device 103, memory 104, bus 105, computer program 1041. Detailed Implementation

[0027] The embodiments of the technical solution of the present invention will now be described in detail with reference to the accompanying drawings. These embodiments are merely illustrative of the technical solution of the present invention and are therefore intended to limit the scope of protection of the present invention.

[0028] Example 1 like Figure 1 As shown, this embodiment provides a testing method for a driver's ability to recognize intelligent driving cockpit icons, used for testing in real-world roadside scenarios, including: The system collects feature attributes of the intelligent driving cockpit icons on the tested vehicles. These attributes include display position, change time, and icon information. Multiple intelligent driving cockpit icons are then combined into a multi-dimensional target template based on their display position and change time. Within the target template, the change time of the intelligent driving cockpit icon serves as one dimension; the different locations of the intelligent driving cockpit icon in different areas serve as another dimension; and different icon information within the same area serves as yet another dimension. By using multi-dimensional data to construct the feature attributes of the intelligent driving cockpit icons into corresponding target templates, subsequent intelligent driving cockpit icon recognition can achieve accurate and rapid location of the target icon. Simultaneously, the position, color, and size of the intelligent driving cockpit icons in the target templates are labeled to improve the accuracy of subsequent recognition processes.

[0029] During roadside testing, a first vision sensor is used to acquire target images in real time. The target image covers the icon display area. The sensor also determines in real time whether an icon that matches the smart cockpit icon in the target template appears in the target image. If at least one smart cockpit icon appears in the target image, the change time of each smart cockpit icon is collected. The change time includes the time of display and the time of disappearance.

[0030] The driving status data of the test vehicle is acquired. This driving status data is the core data used to characterize the multi-screen linkage and changes in the status of the intelligent driving cockpit icons in the instrument panel, central control screen, and HUD (head-up display) icon display areas. The driving status data consists of signals output from the vehicle system, most of which are threshold signals. When a threshold signal is reached, the display of the intelligent driving cockpit icon signal is activated. This is the underlying data that most accurately reflects the display of the intelligent driving cockpit icon signal, and it represents the driving status data of the test vehicle. In one implementation, driving status data can be collected after road testing and synchronized with video data of the vehicle's movement to verify whether the intelligent driving cockpit icons are mis-displayed, not displayed, or overlapped. In another implementation, during roadside testing, the vehicle's driving status data can be connected wirelessly or via a wired connection to a portable terminal computer. Simultaneously, the portable terminal computer is connected to a first vision sensor, transmitting the vehicle's driving status data and target image data to the computer in real time. The computer calculates in real time whether target image overlap is likely based on the corresponding position of the intelligent driving cockpit icon in the target template. Based on the real-time road conditions, the driver and the tester next to the driver determine whether the intelligent driving cockpit icon is not displayed or is displayed incorrectly.

[0031] Driving status data includes intelligent driving system status data, vehicle dynamic information data, and environmental perception data. Intelligent driving system status data covers autonomous driving levels (e.g., L2, L2+), function activation / deactivation / exit, and system takeover requests, including autonomous driving mode icons, lane keeping status, following distance alerts, lane change assist, and takeover requests. Vehicle dynamic information data is used to trigger icon changes for vehicle assistance functions, including vehicle speed, acceleration, steering angle, and lateral and longitudinal acceleration. Environmental perception data is used to acquire the recognition results of external targets (vehicles, pedestrians, lane lines) from cameras, radar, and lidar, including visual displays of the vehicle's exterior.

[0032] Specifically, the analysis examines whether the driving status data of the tested vehicle causes changes in the intelligent driving cockpit icons. These changes include the display and disappearance of the intelligent driving cockpit icons. Additionally, it examines whether overlapping occurs when intelligent driving cockpit icons appear, or whether the corresponding icons for the driving status data are not displayed.

[0033] Specifically, the system collects icons from the various intelligent driving cockpit icons displayed on the test vehicle's icon display area, including the instrument panel, central control screen, and HUD. These icons include speed, navigation indicator signals, and warning signals. Simultaneously, the system collects the display position and duration of each intelligent driving cockpit icon within its icon display area, and then combines these icons into a target template based on their respective positions within the test vehicle. It is important to note that multiple icons can be displayed simultaneously within the icon display area, but overlapping is not permitted. This overlap can be determined based on the aforementioned vehicle specifications. During the data collection process, a first-vision sensor is installed in the intelligent driving cockpit of the tested vehicle to acquire, label, and record the intelligent driving cockpit icons displayed on the icon display area. The location, features, color, and display time of the intelligent driving cockpit icons are also recorded. The acquired intelligent driving cockpit icons are preprocessed by denoising, enhancing contrast, and generating grayscale images from the images captured by the first-vision sensor to improve the accuracy of subsequent recognition processes.

[0034] Preprocessing of road test data includes secondary verification and data alignment to ensure accuracy and consistency. Road test data includes vehicle dynamics data, environmental perception data, intelligent cockpit and human-machine interaction data, autonomous driving system decision-making and control data, high-precision maps and positioning data, vehicle external environment and traffic information, and time synchronization and metadata. Vehicle dynamics data includes vehicle speed, acceleration, deceleration, steering wheel angle, steering speed, brake pedal status (depression / release, depth), throttle opening, vehicle driving mode, vehicle attitude, and gear information. Environmental perception data includes vehicle camera data, LiDAR data, millimeter-wave radar data, ultrasonic radar data, and infrared / night vision sensor data. Intelligent cockpit and human-machine interaction data includes driver facial expressions, eye movement patterns, heart rate, respiration data, central control screen operation records, HUD (Head-Up Display) information display status and driver gaze area, alarm icon trigger time, and driver response time. High-precision maps and positioning data includes the vehicle's centimeter-level positioning position in the high-precision map, map matching status, positioning reliability, and lane-level positioning information. External environment and traffic information includes real-time weather conditions, road conditions, traffic flow, congestion, traffic light status, and the behavior of surrounding vehicles.

[0035] In this embodiment, the road test data is preprocessed, including denoising, filtering, aligning timestamps, changing image brightness / contrast, simulating weather conditions such as rain and fog, and retaining GPS information. The key parameters in the preprocessed road test data are modified, including weather, lighting conditions, navigation information, and vehicle information.

[0036] Real-time driving status data of the tested vehicle can be acquired. This driving status data characterizes the underlying state data of the intelligent driving cockpit icon display. The driving status data is parsed to obtain the intelligent driving cockpit icon that should be displayed. A target image is acquired simultaneously, and it is determined whether the intelligent driving cockpit icon that should be displayed exists in the target image. If the intelligent driving cockpit icon that should be displayed does not exist in the target image, it is determined that the intelligent driving cockpit icon is not displayed, or the intelligent driving cockpit icons overlap. If the intelligent driving cockpit icon that should be displayed exists in the target image, the driver's reaction ability to the intelligent driving cockpit icon is collected. Within a first time period, if the intelligent driving cockpit icon that should be displayed is not displayed in the specified position in the target template, and no other intelligent driving cockpit icon is displayed in that specified position, it is determined that the intelligent driving cockpit icon is not displayed. The first time period is 1-10 seconds. If multiple intelligent driving cockpit icons that should be displayed exist in any position in the display area of ​​the intelligent driving cockpit icon, and an intelligent driving cockpit icon is displayed in that position, it is determined that intelligent driving cockpit icon overlap occurs.

[0037] Audio-visual sensors were installed in the passenger-side area and steering wheel area of ​​the test vehicle. The audio-visual sensor in the passenger-side area captured images covering the driver's actions, while the audio-visual sensor in the steering wheel area captured the driver's eye movements. These sensors do not interfere with the driver's normal driving. Furthermore, the audio-visual sensors and the primary visual sensor synchronized their capture times. When the intelligent driving cockpit icon appeared, the audio-visual sensors recorded the driver's reaction time, including reaction time to the icon and the processing action. The reaction time was acquired by the audio-visual sensor in the steering wheel area, while the processing action was acquired by the audio-visual sensor in the passenger-side area.

[0038] In current intelligent driving systems (such as NOA, AEB, and lane keeping assist), their normal operation relies on the coordinated work of vehicle sensors (cameras, LiDAR, and millimeter-wave radar), computing platforms, and actuators. The intelligent driving cockpit icon often serves as a direct feedback of "abnormal status" or "function triggering" within the vehicle system. By detecting the intelligent driving cockpit icon, it is possible to verify whether the functionality of the vehicle's intelligent driving cockpit icon conforms to the design logic.

[0039] Specifically, in testing the effectiveness of safety warnings, the verification of the intelligent driving cockpit icons plays a significant role. This reflects the driver's reaction time range during safety warnings, ensuring timely response in real-world road conditions. First, trigger condition verification: comparing the matching degree between actual operating conditions and the triggering of intelligent driving cockpit icons. In one implementation, when simulating "LiDAR obstruction" or "camera dirt," the system checks whether the vehicle's driving status data triggers the corresponding intelligent driving cockpit icon, such as a sensor fault alarm icon, to verify the vehicle's ability to perceive hardware anomalies. Second, when the vehicle approaches an obstacle in autonomous driving mode, the system checks whether a "collision warning alarm" (such as a red triangle warning light) is triggered, verifying the accuracy of the triggering logic of AEB and collision warning functions. Third, when exiting autonomous driving mode due to complex road conditions (such as a pedestrian suddenly appearing at an unprotected intersection), the system checks whether a takeover prompt alarm in the intelligent driving cockpit icons is triggered (such as a flashing steering wheel icon), verifying the compliance of the vehicle's human-machine takeover function logic. Fourth, functional boundary testing: under extreme scenarios (such as heavy rain, strong light, tunnel entrances and exits), continuously monitor whether the smart driving cockpit icon frequently triggers icons such as "system downgrade" and "function unavailable" to determine the applicable boundaries and robustness of intelligent driving.

[0040] Specifically, significant changes in driver reaction time can be observed during long-term or extreme environment testing. Prolonged fatigue driving or sudden environmental changes result in a substantial difference in driver reaction time to the intelligent driving cockpit icons compared to normal conditions. Furthermore, the frequency and type of different alarm icons can reflect the reliability of the vehicle system. First, through long-term reliability monitoring, such as in vehicle durability testing (e.g., 100,000 km road tests), the number of times intelligent driving cockpit icons appear and the corresponding internal vehicle systems (e.g., powertrain, intelligent driving systems) are continuously recorded. This analysis identifies high-frequency, repetitive alarms and tracks driver reaction times to different intelligent driving cockpit icons. Second, in environmental adaptability testing, under extreme environments such as high temperature, high humidity, extreme cold, and dust, the frequent triggering of intelligent driving cockpit icons such as "sensor performance degradation" and "battery system abnormality" can be monitored. This not only assesses the vehicle's system stability under harsh conditions but also determines whether the driver can promptly respond to these icons. Third, verify the reliability of the redundant system: For redundant components of intelligent driving (such as dual lidar and dual power supply), the redundant components are made to fail through fault injection, and it is detected whether "redundant system activation" is triggered. If "redundant system activation" is triggered, the corresponding intelligent driving cockpit icon will be displayed, and the driver's reaction to the intelligent driving cockpit icon will be judged.

[0041] In summary, firstly, traditional testing presents a technical contradiction between "adapting to icon differences across multiple vehicle models" and "ensuring recognition accuracy"—single feature matching (such as recognizing only icon style) either suffers from poor universality due to differences in icon position across vehicle models, or leads to misjudgments due to ignoring temporal changes. This embodiment, however, decomposes the recognition elements of the intelligent driving cockpit icon into three independent modules: "display position (spatial dimension), change time (temporal dimension), and icon information (attribute dimension)," constructing a target template through multi-dimensional combinations. This design allows the system to flexibly adjust the "display position" parameter for different vehicle models, while simultaneously distinguishing different trigger scenarios for similar icons through "change time" (such as the trigger timing of the same collision warning icon under different conditions). This solves the cross-vehicle adaptation problem and avoids the risk of misjudgment from single feature matching, achieving a harmonious balance between "improved universality" and "ensuring accuracy."

[0042] Secondly, traditional testing heavily relies on vehicle manufacturer-specific CAN bus signals. Because different manufacturers define signals differently, cross-vehicle testing requires repeated bus protocol adaptation, resulting in a contradiction of "high adaptation costs and low testing efficiency." This embodiment replaces the reliance on CAN bus signals with image acquisition from a vision sensor: the first vision sensor directly acquires images from display areas such as the instrument panel, central control screen, and HUD, and combines them with pre-built icon templates to complete the matching, eliminating the need to parse vehicle-specific bus signals. This significantly reduces the adaptation costs for cross-vehicle testing.

[0043] Third, traditional testing often employs a "step-by-step testing" model: first verifying the icon display logic, then separately collecting driver reactions, which suffers from "long testing cycles and weak data correlation." This embodiment integrates "icon display verification" and "driver reaction collection" into a single process: by acquiring driving status data in real time and analyzing the "icons that should be displayed," it simultaneously compares the "actually displayed icons" collected by the visual sensor. After confirming that the icon display is normal, it automatically triggers reaction collection (the audio-visual sensor records eye movements and operational actions). This design allows a single testing process to simultaneously output two types of data: "icon display effectiveness" and "driver reaction ability," which shortens the testing cycle, ensures the temporal correlation of data, and resolves the contradiction of "data fragmentation and low efficiency" in traditional step-by-step testing.

[0044] The usage principle of this embodiment The system collects the feature attributes of the intelligent driving cockpit icons in the test vehicle at the time of manufacture, and combines multiple intelligent driving cockpit icons into a target template. Real-time driving status data of the test vehicle is acquired, and the driving status data is compared with the intelligent driving cockpit icons. If the driving status data should display three intelligent driving cockpit icons, but the target image acquired by the first visual sensor only displays fewer than three, a corresponding judgment is made. First, the collected intelligent driving cockpit icons are matched with the intelligent driving cockpit icons that should be displayed in the vehicle's driving status data. If the collected intelligent driving cockpit icons and the expected intelligent driving cockpit icons do not match, a false alarm occurs. If the collected intelligent driving cockpit icons correspond to the expected intelligent driving cockpit icons, but their number is less than the expected number of intelligent driving cockpit icons, it is determined that the intelligent driving cockpit icons are not fully displayed, or that the intelligent driving cockpit icons overlap. Furthermore, if the intelligent driving cockpit icon that should be displayed is not displayed in the specified position in the target template, and no other intelligent driving cockpit icon is displayed in that specified position, then it is determined that the intelligent driving cockpit icon is not displayed. If multiple intelligent driving cockpit icons that should be displayed exist in any position within the display area of ​​the intelligent driving cockpit icons, and an intelligent driving cockpit icon is displayed in that position, then it is determined that intelligent driving cockpit icon overlap occurs. When the intelligent driving cockpit icon that should be displayed, as represented by the driving state data of the tested vehicle, corresponds one-to-one with the intelligent driving cockpit icon acquired in the target image, the driver's reaction ability to the intelligent driving cockpit icon is recorded, and this reaction ability includes reaction time and processing operation.

[0045] Beneficial effects of this embodiment First, it achieves cross-vehicle testing universality, resolving the issue of inconsistent signals among automakers. It breaks away from the traditional testing reliance on CAN bus signals specific to different automakers, avoiding data acquisition obstacles caused by differences in signal definitions. The corresponding technical features are: by collecting the display position, change time, and icon information of the intelligent driving cockpit icons of the vehicle under test, multiple icons are combined into a target template based on display position and change time; during road testing, only the first-line vision sensor is used to collect target images of the instrument panel, central control screen, HUD, and other display areas in real time, and icon matching is performed with the target template. Driver reaction ability data can be collected without adapting to different vehicle bus protocols, adapting to the differences in icon display positions across various vehicle models, and reducing cross-vehicle testing costs.

[0046] Secondly, it improves the accuracy and efficiency of icon recognition and matching, avoiding misjudgments based on single features. Even in complex scenarios where multiple icons are displayed concurrently or similar icons are triggered at different times, it can still accurately locate the target icon. The corresponding technical features are: breaking down the feature attributes of a single intelligent driving cockpit icon into multi-dimensional data of "display location (first dimension), change time (second dimension), and icon information (third dimension)," and combining these dimensions to form a target template; simultaneously, it annotates the position, color, and size of icons in the template, accurately distinguishing scenarios such as "different icons in the same location" and "same type of icons triggered at different times," avoiding misjudgments caused by single-feature matching (such as only looking at icon information and ignoring position), and improving the targeting and efficiency of icon matching.

[0047] Simultaneously, it integrates "icon display verification + driver reaction data collection" to synchronously locate system problems. This breaks through the single objective of traditional testing—"only collecting reactions" or "only verifying displays"—providing dual data support for optimizing the intelligent driving system and cockpit display. The corresponding technical features are: real-time acquisition of driving status data of the tested vehicle (including intelligent driving system status, vehicle dynamic information, and environmental perception data, serving as the underlying data for icon display), parsing to obtain the intelligent driving cockpit icons that should be displayed; synchronously acquiring target images through a first-vision sensor, comparing the "icons that should be displayed" with the "icons that are actually displayed" to determine display anomalies such as non-display, overlap, and false alarms; and collecting driver reaction data only when the icons are displayed normally, ensuring the validity of the reaction data and directly locating problems in the intelligent driving system's icon triggering logic (such as hardware malfunctions failing to trigger alarms) or cockpit display logic (such as icon overlap).

[0048] Furthermore, it achieves three-dimensional acquisition of driver reaction data, capturing real-world driving instincts. This avoids the distortion problems of traditional "simulated reactions according to the operation manual" methods, obtaining reaction data that more closely reflects actual driving. The corresponding technical features are: an audio-visual sensor is installed in the passenger area of ​​the test vehicle (covering driver operation actions), and an audio-visual sensor is installed in the steering wheel area (acquiring driver eye movements), with sensor installation not affecting normal driving; the audio-visual sensor and the first visual sensor acquire data synchronously, and when the target icon is successfully matched, the driver's reaction time (eye movement capture) and processing operation (hand / foot operation capture) can be accurately recorded, truly reflecting the driver's instinctive reaction when facing the icon (such as gaze delay and operational hesitation).

[0049] Finally, the test covers core intelligent driving testing scenarios, supporting the dual optimization of system performance and interaction design. This ensures that the tests fully meet the key requirements of intelligent driving, providing data support for improving vehicle safety and user experience. The corresponding technical features include: test scenarios covering core scenarios such as the effectiveness of safety warnings, long-term reliability monitoring (recording the frequency of icon appearance and corresponding systems during 100,000 km road tests), functional boundary testing (the trigger frequency of the "system degradation" icon under heavy rain / strong light), and redundant system verification (the triggering of the "redundancy activation" icon after dual LiDAR failure); the robustness of the intelligent driving system is inferred from icon changes (such as functional stability under extreme environments), while also statistically analyzing differences in driver reaction under different scenarios (such as prolonged reaction time during fatigued driving). This supports fault diagnosis and performance optimization of the intelligent driving system, and also provides direction for improvement in the interaction design of cockpit icons, such as color and position.

[0050] Example 2 Example 1, which tests the driver's ability to recognize intelligent cockpit icons, is generally suitable for road tests conducted in real-world scenarios. However, real-world road tests use real-world scenario data as the vehicle's driving status data, which leads to low testing efficiency and a limited number of tests. Unlike Example 1, in this example, the test method based on the driver's ability to recognize intelligent cockpit icons is also used in road tests, but only when the tested vehicle is tested at a vehicle testing ground.

[0051] By connecting multimodal data to the vehicle under test, the multimodal data is represented as a preset intelligent driving cockpit icon. The preset intelligent driving cockpit icon is an intelligent driving cockpit icon acquired synchronously in the target image. It is determined whether the intelligent driving cockpit icon matches the intelligent driving cockpit icon in the target template. If they match, the driver's reaction ability to the intelligent driving cockpit icon is collected; otherwise, the target image is collected again and matched with the intelligent driving cockpit icon in the target template.

[0052] Based on the modified key parameters of the preprocessed road test data, road test datasets are formed. Each road test dataset is used to characterize the changes in the intelligent driving cockpit image under at least one real-world scenario. Multiple road test datasets are combined to form multimodal data. Key parameters include weather, lighting conditions, navigation information, and vehicle information. Multimodal data can be used to cover vehicle driving state data to characterize the intelligent driving cockpit icons to be displayed in the test. For example, in Example 1, intelligent driving cockpit icons such as "sensor performance degradation" and "battery system abnormality" appear in the environmental adaptability test. These intelligent driving cockpit icons, which simulate real-world scenarios in the test field under extreme environments, test the driver's reaction ability without causing corresponding damage to the vehicle, and ensure the driver's safety. In other implementations, when verifying the reliability of redundant systems, multimodal data can be used to simulate the intelligent driving cockpit icons that should be displayed under different real-world environments as described in Example 1 (such as the intelligent driving cockpit icons for trigger condition verification scenarios like "LiDAR is blocked" or "camera is dirty", the "collision warning alarm" icon for approaching obstacles in autonomous driving mode, and the intelligent driving cockpit icons for extreme scenarios like "system degradation" or "function unavailable" under functional boundary testing). This can detect the driver's actual reaction ability, enabling multiple repeatable tests, improving test accuracy, ensuring driver safety, and since the multimodal data is real-world data extracted from road test data, it requires minimal modification, reducing test complexity and the difficulty of building test datasets, thus improving test efficiency.

[0053] Specifically, multimodal data is imported into the vehicle under test to cover a portion of the vehicle's driving status data, enabling real-time display of the intelligent driving cockpit icons represented by the multimodal data. The multimodal data packets are connected to the vehicle's control system via the OBD interface. Conversion is performed using the CAN bus protocol to ensure format compatibility between the multimodal data and the vehicle's native driving status data. Ethernet TCP / IP or CANFD high-speed transmission protocols are used to transmit the multimodal data to the vehicle's control system in real time. Timestamp synchronization technology ensures a transmission delay of ≤10ms for the multimodal data, precisely aligning with the image acquisition timing of the first visual sensor to prevent asynchrony between icon display and data triggering. In vehicle test mode, the multimodal data is set to "highest priority." Through the vehicle control system's parameter configuration interface, trigger commands for the intelligent driving cockpit icons from the native driving status data of the CAN interface corresponding to the multimodal data are disabled. This ensures that the intelligent driving system only responds to the multimodal data, displaying the corresponding intelligent driving cockpit icons according to preset scenarios.

[0054] Furthermore, after the multimodal data is accessed, the target image of the intelligent driving cockpit display area is acquired in real time through the first visual sensor and compared with the preset "expected icon" (icon information, position, and change time) of the multimodal data. The comparison method can be consistent with the intelligent driving cockpit icon comparison method that should be displayed for vehicle driving status data as described in Embodiment 1.

[0055] Beneficial effects of this embodiment First, it avoids the safety risks of real-world road testing, ensuring a safe and controllable testing process. It addresses the risks of vehicle damage and traffic accidents that may arise from simulated faults (such as obstructing the LiDAR or disconnecting the controller) or extreme scenarios (heavy rain, strong light) in real-world road testing, while also protecting driver safety. The corresponding technical features are: strictly limiting the testing scenario to a vehicle testing ground; simulating scenarios requiring icon display, such as "sensor performance degradation," "collision warning," and "system degradation," using multimodal data without injecting real faults into the vehicle or subjecting it to extreme natural environments; and connecting the multimodal data to the control system via the vehicle's OBD interface, only overwriting the original driving state data without modifying the vehicle's core control program, thus avoiding damage to the vehicle caused by hardware modifications or fault simulations, and ensuring that the entire test is conducted in a closed and safe testing environment.

[0056] Secondly, it improves testing efficiency and repeatability, overcoming the limitations of reproducing real-world conditions. It addresses the pain points of difficulty in frequently reproducing extreme scenarios (such as heavy rain, strong light at tunnel entrances / exits) and fault scenarios (such as redundant system failures) in real-world road tests, as well as the limited number of tests, significantly shortening the testing cycle. The corresponding technical features are: based on preprocessed real-world road test data, key parameters such as weather, lighting conditions, navigation information, and vehicle information are modified to generate road test datasets representing different real-world scenarios, which are then combined to form multimodal data; multimodal data can be called on demand and repeatedly loaded. For example, scenarios such as "LiDAR being blocked" or "sudden pedestrians at unprotected intersections" can be tested repeatedly without waiting for natural operating conditions or reconstructing the fault environment, and a single test can cover multiple scenarios, significantly improving testing efficiency and data sample size.

[0057] Meanwhile, ensuring consistency between simulated scenarios and real-world conditions guarantees the reliability of test data. This avoids the problem of pure simulation testing being disconnected from real driving logic, making driver reaction data more valuable. The corresponding technical features are: multimodal data originates from preprocessed real-world road test data, preserving the logical connections between core data such as vehicle dynamics, environmental perception, and intelligent driving system decision-making; parameter modifications follow a true mapping relationship of "scenario-parameter-icon" (e.g., the rainstorm parameter corresponds to the "sensor performance degradation" icon trigger, consistent with real-world road test logic); multimodal data is transmitted via Ethernet TCP / IP or CANFD protocols, combined with timestamp synchronization technology to ensure transmission latency ≤10ms, precisely aligned with the image acquisition timing of the first visual sensor, avoiding scene distortion caused by asynchronous icon display and data triggering, and ensuring a high degree of consistency between the logic and timing of the simulated scenario and real-world conditions.

[0058] Furthermore, precise control of icon triggering conditions ensures the effectiveness of driver reaction data collection. This addresses the issue in real-world road tests where native driving state data might interfere with icon triggering (e.g., sudden road conditions causing non-target icons to be displayed), thus affecting the accuracy of driver reaction data. The corresponding technical features are: in vehicle testing mode, multimodal data is set to "highest priority." Through the vehicle control system parameter configuration interface, the native driving state data from the CAN interface corresponding to the multimodal data is shielded, ensuring that the intelligent driving system only responds to multimodal data and displays preset icons. After multimodal data is accessed, a target image is acquired through a first-vision sensor and compared with the preset "expected icons" (type, location, change time) of the multimodal data. Driver reactions are only collected when an icon match is successful, avoiding interference from non-target icons or abnormal displays, ensuring the relevance and effectiveness of the collected data.

[0059] Finally, it reduces testing complexity and cost, adapting to the needs of large-scale testing. It solves the problem of requiring significant manpower and resources for real-world road testing (such as following support vehicles and traveling in extreme environments), and the need for repeated adaptation for different vehicle models. The corresponding technical features are: multimodal data is generated by modifying parameters based on real-world road test data, eliminating the need to build scenario models from scratch and reducing dataset construction complexity; through CAN bus protocol conversion, multimodal data is compatible with the native driving state data formats of different vehicle models, eliminating the need to develop dedicated test interfaces for a single vehicle model; multimodal data adopts a modular design, and adding new test scenarios (such as "dual power redundancy failure") only requires modifying key parameters such as weather and vehicle status, without refactoring data transmission or triggering logic, adapting to the testing needs of different vehicle models and different intelligent driving functions, significantly reducing the time and manpower costs of large-scale testing.

[0060] Example 3 Unlike the previous embodiments, the test method for driver recognition of smart cockpit icons provided in this embodiment is also used to indirectly reflect the driver's recognition of smart cockpit icon responses and judgment of implementation behavior through real data of the tested vehicle.

[0061] Specifically, the test method for driver's reaction capability to recognize smart cockpit icons further includes: acquiring a target image using a first visual sensor and recognizing the smart cockpit icon from the target image; then acquiring the driver's operational behavior within a second time period using an audio-visual sensor. The second time period is defined according to different smart cockpit icons. In this embodiment, the base value of the second time period is set according to the smart cockpit icon type: the base value for emergency icons is within 3 seconds, for navigation icons within 5 seconds, and for comfort icons within 8 seconds. In this embodiment, the audio-visual sensor is a camera device with recording functionality.

[0062] When the first sensor detects the presence of the intelligent driving cockpit icon in the target image, the driver's driving behavior is recorded by the audio-visual sensors during a second time period. This driving behavior includes hand gestures, body posture and gaze, and voice and interaction behaviors. Hand gestures include steering actions, operating the gear shift lever, pressing / rotating the central control screen, buttons (air conditioning, volume, driving mode), and distracting behaviors such as using a mobile phone, eating, and drinking. Body posture and gaze include looking straight ahead and turning the head to check the rearview / side mirrors.

[0063] In the hand gesture behavior determination, when the driver's hand action is directly related to the semantics of the intelligent driving cockpit icon, and the action completion rate is ≥80% (e.g., the "Confirm on Central Control Screen" icon corresponds to "Finger taps the target area on the central control screen and stays for ≥0.5s"), it is determined that the driver has responded to the target intelligent driving cockpit icon. In the eye gaze behavior determination, when the driver's gaze point changes from other positions to the icon display area (instrument panel / central control screen / HUD), and the gaze duration is not less than 1s, it is determined that the driver has responded to the target intelligent driving cockpit icon. In the voice interaction behavior determination, in the second time period, the recognized voice text is compared with the preset semantics of the target intelligent driving cockpit icon (e.g., "Collision Warning Icon" corresponds to "Brake" and "Avoid"), and a similarity of ≥85% is determined to be a match. If the match is successful, it is determined that the driver has responded to the target intelligent driving cockpit icon.

[0064] The system acquires the driver's reaction behavior corresponding to each smart cockpit icon according to the specified rules. It then compares this reaction behavior with the driver's actual reaction behavior for the first time. If the first comparison is successful, the driver's reaction time is recorded; if the comparison fails, a second comparison is performed.

[0065] Furthermore, the second comparison includes acquiring the test vehicle's speed data within the second time period. If the test vehicle's speed data changes during the second time period (meaning acceleration / deceleration is at least 20 km / h), then the driver's actions corresponding to the intelligent driving cockpit icon are analyzed to determine if these actions are related to the vehicle's speed. If not, the second comparison fails, and this comparison is recorded; if they are related, the driver's reaction time is recorded. When the second comparison fails, the relevant records can be saved for manual verification.

[0066] Specifically, because the audio-visual sensor located in the passenger seat and the sensor located at the steering wheel for observing the driver's eye movements are both difficult to detect, the driver's leg movements are often difficult to discern. The driver's leg movements are often related to pressing the accelerator or brake pedals. Pressing the accelerator or brake pedals is closely related to the vehicle's speed. Changes in vehicle speed can reflect the driver's operating behavior. If the test vehicle speed changes, and the acceleration / deceleration changes by at least 20 km / h, it indicates a change in the driver's operating behavior. If this change matches the driver's reaction behavior corresponding to the specified intelligent driving cockpit icon, it indicates that the driver's operation is reasonable. This operation can be considered the driver's reaction ability to the intelligent driving cockpit icon, and the change in vehicle speed is recorded as the driver's reaction time.

[0067] In summary, this method achieves comprehensive coverage of driving reaction behavior, quickly and accurately assesses driver reaction capabilities, and effectively recognizes dynamic smart cockpit icons. By integrating accurate and rapid recognition of smart cockpit icons with accurate and rapid recognition of driver reaction capabilities, this driver-based smart cockpit icon reaction capability testing method can obtain different test results under a single testing approach, significantly improving testing efficiency.

[0068] Beneficial effects of this embodiment First, the system correlates and collects explicit driver behavior with implicit data from the test vehicle to achieve full-dimensional coverage of reaction behavior. At the explicit level, four core behaviors—hand movements, eye gaze, body posture, and voice interaction—are directly collected, and valid reactions are defined using quantitative standards (action completion ≥80%, gaze duration ≥1 second, and voice similarity ≥85%). At the implicit level, vehicle dynamic data (vehicle speed, acceleration changes) are linked to leg movements. The validity of leg movements is indirectly determined through a quantitative threshold of "acceleration / deceleration ≥20 km / h"—this design eliminates the need for additional leg sensors, thus avoiding interference with normal driving and ensuring the safety of road tests. This upgrades the assessment of driver reaction ability from "partial behavior capture" to "full-dimensional behavior reconstruction," achieving comprehensive coverage of reaction behavior.

[0069] Secondly, the two-stage progressive comparison and judgment logic achieves a precise closed loop for the effectiveness of the response, solving the problem of difficulty in judging implicit behaviors and the need for additional equipment. A common problem with existing tests is the subjectivity and susceptibility to distortion in response judgments—judging a response as valid solely based on a single behavior match (such as accidental touch of the central control screen or habitual steering) leads to a disconnect between test results and real driving scenarios, resulting in insufficient data credibility. In this embodiment, the first comparison (forward filtering) uses "icon-behavior semantic mapping" as its core, directly verifying the consistency between the driver's behavior and the expected response of the icon, quickly filtering out clearly valid response data, ensuring the relevance of the judgment, and avoiding "irrelevant behavior interference." The second comparison (reverse completion) addresses scenarios where "direct behavior does not match" (such as the driver not directly viewing the icon but having already braked urgently), using vehicle dynamic data for reverse verification—if the vehicle speed change is strongly correlated with the icon's semantics (such as deceleration corresponding to a collision warning), it is judged as a valid response; if it is unrelated (such as natural deceleration due to road slope), it excludes misjudgments.

[0070] Meanwhile, by introducing a dynamic dual-dimensional parameter system of "icon type - scene complexity," precise alignment with the test scenario is achieved. For icon type: differentiated second time periods are set for icons of different priorities (3s for emergency, 5s for navigation, and 8s for comfort) to match the driver's reaction rhythm to different semantic icons, avoiding judgment bias caused by a "uniform time threshold." For behavior judgment: judgment criteria are adapted according to the semantic differences of the icons (e.g., relaxed posture matching requirements for emergency icons, allowing irregular postures in stress responses; enhanced coordination between line of sight and operation for navigation icons), adapting to the driving needs corresponding to different icons.

[0071] Next, the system can simultaneously test the driver's reaction ability and identify defects in the intelligent driving cockpit icons, enabling rapid iteration of the intelligent driving cockpit. By correlating data on icon features (position / color / semantics), driving behavior (operation / line of sight / voice), and vehicle dynamics (speed / acceleration), it can accurately locate icon design defects (such as a right-leaning icon causing gaze delay) and intelligent driving logic loopholes (such as false alarm icons causing driver fatigue).

[0072] Test case This test case aims to evaluate a driver's ability to react to changes in the vehicle's external environment based on the recognition of icons in the smart cockpit. This method is applicable to autonomous vehicles and advanced driver assistance systems (ADAS) and aims to ensure accurate synchronization of multimodal data, thereby improving testing efficiency and accuracy.

[0073] Specifically, this device is used to detect in-vehicle alarm prompts and employs a high-performance camera with a high-quality image processor. It provides millisecond-level response to various alarm signals, is easy to operate, and offers convenient and efficient system configuration. In this test case, it is used to detect relevant alarm signals from the vehicle's ADAS functions, including LDW, FCW, and BSD. The device used in this test case includes a main unit, a main unit power supply, an image sensor and its fixture, a network cable, an image sensor signal transmission line, and a CAN cable. Features of the device include image recognition triggering, millisecond-level alarm signal response, convenient configuration, easy system operation, the ability to save and reload measurement configuration files at any time, and support for network and CAN communication. The device uses a power-on self-start mode; after correct power connection, the power button indicator light illuminates, and the device automatically starts. It features a USB 3.0 port for connecting the image sensor, an Ethernet port for connecting external devices, and a CAN data transmission port.

[0074] like Figure 2 As shown, camera clamp 1 is magnetically attached to and fixed to the roof or windshield. Camera 2 is mounted on the front end of the clamp. The position and orientation of camera 2 are adjusted using the knob on camera clamp 1. Camera 2 is then fixed in front of the dashboard, HUD, or in-vehicle tablet to detect smart cockpit icons and related information. Furthermore, the mounting position of camera clamp 2 does not obstruct the driver's view or interfere with steering wheel rotation. The camera is connected to the USB 3.0 port on the main unit panel using a camera signal cable.

[0075] like Figure 3 As shown, open the live camera feed. The view can be changed by rotating the camera, with a maximum rotation of 180 degrees. You can select the desired rotation angle, and the live camera feed will rotate accordingly. After rotation, you need to re-record the video for icon selection. You can adjust the camera's exposure intensity and focus by rotating the lens clockwise or counterclockwise, as well as the distance between the camera and the display area (dashboard, central control display, HUD) to ensure the desired icon is clearly and completely displayed in the video.

[0076] like Figure 4 As shown, the intelligent driving cockpit icons in the display area are obtained by adjusting the camera's exposure intensity and focal length. The camera video is acquired and processed using a DESKTOPQLN3HQC computer, as shown... Figure 5 As shown ( Figure 5 for Figure 4 The enlarged diagram on the right shows the processing data, including processing time (ms), average frame rate (fps), total number of matches, number of captured images, still image previews, and captured video. It also includes a global output display, which shows the identified object, matching score, and extended signal matching.

[0077] Record the icons of the intelligent driving cockpit with dynamic changes. After connecting an image sensor, such as a camera and a remote analysis port, capture the video recorded by the image sensor and record the icons of the intelligent driving cockpit to be recognized, such as Figure 6 As shown, taking the right turn signal as an example, start video recording of the right turn signal. Complete the corresponding configuration of the file for recording the video of the right turn signal. According to the recorded video file, box the icons of the intelligent driving cockpit to be recognized and complete the relevant configuration. Specifically, find the icons of the intelligent driving cockpit to be boxed by playing the recorded video or clicking the previous or next frame, select the identification number. In this embodiment, there are 8 identification numbers, and eight icons can be recognized simultaneously, and set the enabled state of icon recognition. Box the icons of the intelligent driving cockpit to be recognized: click the [Box Selection Recognition Area] button. After the mouse cursor becomes a crosshair, click and drag the mouse in the icons of the intelligent driving cockpit to be recognized to draw a rectangular selection box around the icon. The size of the selection box area can be directly adjusted through the four vertices of the selection box, and the boxed icons of the intelligent driving cockpit can be viewed synchronously in the preview area. If you need to re-box the icon, click the [Box Selection Recognition Area] button again and re-box it. The identification number of the right turn signal in this embodiment is "Rec Figure 1 ", the enabled state is on, the matching received value is 70%, color verification is not required, and there is also a preview image of the icons of the intelligent driving cockpit below. This is the target image obtained by the camera 2. Then save the configuration, and then add more icons of the intelligent driving cockpit to form a target template.

[0078] In the real-time preview area of the camera, all configured recognition selection boxes are displayed, and the corresponding identification numbers are displayed on the selection boxes. View the recognition matching results in the global output display area. Global output display area: Display the recognition results of the configured icons of the intelligent driving cockpit and the extended signals corresponding to the icons of the intelligent driving cockpit: green for successful matching, red for failed matching, and display the matching rate. Statistical information result display: Display the recognition statistical information such as the average frame rate of the video, the processing time per frame, the total number of matches, and the number of picture captures. Generally, due to adjustments in the installation position and angle of the camera, it is necessary to re-record the video for icon boxing. After the recognition matching configuration is completed, the configured options can be viewed on the configuration file page. As Figure 7 shown, modify the enabled state, whether to enable color verification, and the matching acceptance value (recognition threshold) of the configured icons of the intelligent driving cockpit, and it will take effect after saving.

[0079] As Figure 8As shown, after collecting multiple smart cockpit icons (including vehicle system status icons and directional indicator icons), eight recognition objects were generated. It can be seen that "Recognition 8" is grayed out, indicating a low recognition matching rate, requiring re-recognition. The global output display area shows the configured smart cockpit icons and the recognition results of the extended signal. Statistical information is displayed, including the average frame rate of the video, processing time per frame, total number of matches, and number of captured images. The extended signal is configured for Ethernet output.

[0080] Example 3 Unlike the previous embodiments, this embodiment provides a testing system 100 based on the driver's ability to recognize intelligent driving cockpit icons, such as... Figure 9 As shown, the test system based on the driver's ability to recognize intelligent cockpit icons includes a data analysis module, which includes an electronic device. The electronic device includes a memory, a processor 101, and a computer program 1041 stored in the memory and executable on the processor. When the processor 101 executes the computer program, it enables the electronic device to implement the test method based on the driver's ability to recognize intelligent cockpit icons as described in any of the above embodiments.

[0081] Specifically, such as Figure 10 As shown, the electronic device may include: one or more processors 101, one or more input devices 102, one or more output devices 103, one or more memories 104, and a computer program stored in the memory 104 and executable on the processor. The processor 101, input devices 102, output devices 103, and memory 104 are interconnected via a bus 105. The memory 104 stores the computer program 1041, which includes program instructions. The processor 101 is configured to invoke the program instructions to execute the method steps described in the above method embodiments.

[0082] It should be understood that, in this embodiment, the processor 101 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0083] Input device 102 may include a keyboard, etc., and output device 103 may include a display (LCD, etc.), a speaker, etc.

[0084] The memory 104 may include read-only memory and random access memory, and provides instructions and data to the processor 101. A portion of the memory 104 may also include non-volatile random access memory. For example, the memory 104 may also store device type information.

[0085] In specific implementations, the processor 101, input device 102, and output device 103 described in the embodiments of the present invention can execute the implementation methods described in the relevant embodiments of the test method and system based on the driver's recognition of intelligent driving cockpit icons provided in the embodiments of the present invention, which will not be repeated here.

[0086] It should be noted that for a more detailed description of the electronic device's workflow and the test method for performing the test based on the driver's ability to recognize the smart cockpit icon, please refer to the aforementioned method implementation section, which will not be repeated here.

[0087] Example 4 Unlike the previous embodiments, the memory described in this embodiment should be interpreted broadly. It can be not only a hardware component in a computer system used for temporary data storage, but also a physical medium capable of storing digital information and being read by a computer. These media can be permanent or temporary, including but not limited to hard disks and solid-state drives.

[0088] Specifically, the memory can be an internal storage unit of the electronic device described in any of the embodiments, such as a system hard drive or memory. The memory can also be an external storage device of the system, such as a plug-in hard drive, SmartMediaCard (SMC), Secure Digital (SD) card, FlashCard, etc., equipped on the system. Furthermore, the memory can include both internal storage units and external storage devices. The memory is used to store the computer program and other programs and data required by the system. The memory can also be used to temporarily store data that has been output or will be output.

[0089] Storage devices include: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media that can store program code.

[0090] Numerous specific details are set forth in this specification. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, systems, and techniques have not been shown in detail so as not to obscure the understanding of this specification. In the description of this specification, references to the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., mean that a specific feature, method, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this specification.

[0091] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be covered within the scope of the claims and specification of the present invention.

Claims

1. A test method for assessing a driver's responsiveness to icons in a smart driving cockpit, characterized in that, include: Collect the feature attributes of the smart cockpit icons of the tested vehicle, and combine multiple smart cockpit icons into a target template according to their display positions and change times; Roadside testing is conducted by using a first visual sensor to capture target images of the display area of ​​the intelligent driving cockpit icon in real time, and to determine whether the target image matches the intelligent driving cockpit icon in the target template. If they match, the driver's reaction ability to the intelligent driving cockpit icon is collected; otherwise, the target image is collected again.

2. The test method for the driver's reaction ability to recognize intelligent driving cockpit icons according to claim 1, characterized in that, The feature attributes of a single intelligent driving cockpit icon are segmented into multi-dimensional data, and each intelligent driving cockpit icon is combined according to different dimensions to form a target template; the feature attributes include display position, change time, and icon information; wherein, display position is the first dimension data, change time is the second dimension data, and icon information is the third dimension data.

3. The test method for the driver's reaction ability to recognize intelligent driving cockpit icons according to claim 1, characterized in that, The display areas for the intelligent driving cockpit icons include the instrument panel, central control screen, and HUD.

4. The test method for the driver's reaction ability to recognize intelligent driving cockpit icons according to claim 1, characterized in that, Real-time acquisition of driving status data of the test vehicle. The driving status data is used to characterize the vehicle driving status data corresponding to the intelligent driving cockpit icon. The driving status data is parsed to obtain the intelligent driving cockpit icon that should be displayed. The target image is acquired synchronously, and it is determined whether the intelligent driving cockpit icon to be displayed exists in the target image. If the target image does not contain the intelligent driving cockpit icon that should be displayed, it is determined that the intelligent driving cockpit icon is not displayed or the intelligent driving cockpit icon is overlapping. If the target image contains a smart cockpit icon that should be displayed, then the driver's reaction ability to the smart cockpit icon is collected.

5. The test method for the driver's ability to recognize intelligent driving cockpit icons according to claim 4, characterized in that, If, within a preset first time period, the intelligent driving cockpit icon that should be displayed is not displayed in the specified position in the target template, and no other intelligent driving cockpit icon is displayed in the specified position, then it is determined that the intelligent driving cockpit icon is not displayed. If multiple smart driving cockpit icons are displayed at any position in the display area of ​​the smart driving cockpit icons, and a smart driving cockpit icon is displayed at that position, then it is determined that a smart driving cockpit icon overlap phenomenon has occurred.

6. The test method for the driver's reaction capability based on recognizing intelligent driving cockpit icons according to claim 4, characterized in that, If the intelligent driving cockpit icon captured in the target image does not match the intelligent driving cockpit icon that should be displayed after parsing the vehicle driving status data, it is determined that a false intelligent driving cockpit icon has occurred.

7. The test method for the driver's reaction ability to recognize intelligent driving cockpit icons according to claim 1, characterized in that, Road test data includes vehicle dynamic data, environmental perception data, intelligent cockpit and human-machine interaction data, autonomous driving system decision and control data, high-precision map and positioning data, and vehicle external environment and traffic information, as well as time synchronization and metadata.

8. The test method for the driver's ability to recognize intelligent driving cockpit icons according to claim 1, characterized in that, By modifying the key parameters of the preprocessed road test data, a road test dataset is formed. Each road test dataset is used to characterize the changes in the intelligent driving cockpit image in at least one real-world scenario. Multiple road test datasets are combined to form multimodal data. Key parameters include weather, lighting conditions, navigation information, and vehicle information.

9. The test method for the driver's ability to recognize intelligent driving cockpit icons according to claim 8, characterized in that, When the road test involves the vehicle under test undergoing corresponding tests at a vehicle test track, multimodal data is input into the vehicle under test, and the multimodal data is represented as a preset intelligent driving cockpit icon. The preset intelligent driving cockpit icon is an intelligent driving cockpit icon acquired synchronously in the target image. It is then determined whether the intelligent driving cockpit icon matches the intelligent driving cockpit icon in the target template. If they match, the driver's reaction ability to the intelligent driving cockpit icon is collected; otherwise, the target image is collected again and matched with the intelligent driving cockpit icon in the target template.

10. A testing system for a driver's ability to recognize icons in a smart driving cockpit, characterized in that, The device includes a data analysis module, which includes an electronic device. The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it enables the electronic device to implement any of the test methods for the driver's ability to recognize intelligent driving cockpit icons as described in claims 1-8.

Images