Vehicle-mounted automobile instrument control display system

Abstract

The invention discloses a vehicle-mounted automobile instrument control display system, which relates to the field of instrument control display and is used for acquiring face, eye movement, grip strength and physiological data, identifying a driving state, acquiring road, weather and traffic target information, analyzing an operation intention, predicting path conflicts and dynamically adjusting an instrument interface according to risks. According to the system, the driver state and environment information are collected, behavior prediction is combined, operation intention recognition and risk early warning are achieved, the content of an instrument interface can be dynamically adjusted before danger occurs, multi-channel prompt intervention is conducted through voice and light, the driving safety is improved, and compared with the prior art, the system has the advantages of being high in practicability and easy to popularize. The vehicle-mounted instrument control system has the advantages of prospective early warning, interface self-adaption and personalized prompt, and the intelligent and situation-aware vehicle-mounted instrument control function is achieved.

Description

Technical Field

[0001] This invention relates to the field of instrument control and display technology, and in particular to an in-vehicle instrument control and display system. Background Technology

[0002] As the level of automotive intelligence continues to improve, in-vehicle information systems have gradually shifted from traditional fixed instrument panels to multifunctional digital display platforms that integrate navigation, warnings, and driver assistance. However, existing in-vehicle instrument systems generally suffer from static information display, delayed risk perception, and limited prompting responses, making it difficult to personalize adjustments based on the driver's current state and changes in the driving environment. On the one hand, drivers may face safety risks such as distraction and fatigue in complex traffic scenarios, and existing systems struggle to monitor their physiological and operational states in real time, resulting in delayed prompts and limited methods, thus affecting intervention effectiveness. On the other hand, traditional display interfaces are fixed, and the hierarchy of prompt information is unclear, which can easily lead to excessive cognitive load during multitasking driving, hindering the rapid acquisition of key information.

[0003] Currently, Chinese invention patent application number CN202211342998.4 discloses a startup method, device, and storage medium for an automotive instrument display system. This method involves obtaining pre-loading instructions from the automotive instrument display system, compiling these instructions to obtain Linux kernel attribute information, virtualizing the host machine based on the Linux kernel attribute information to obtain an Lxc container, and creating a partition within the guest machine corresponding to the Lxc container, storing the root directory denoted as rootfs. The Linux kernel mounts the Lxc container to the partition where rootfs is located and starts the operating system corresponding to the Lxc container and the partition where rootfs is located to complete the normal boot of the automotive instrument display system. This ensures that software deployed in any operating system environment can always run normally, improving the startup speed, security, and flexibility of the automotive instrument display system.

[0004] The aforementioned technologies struggle to achieve comprehensive perception and coordinated response to the driver's state, operational intentions, and driving environment, resulting in a lack of dynamic adaptability and forward-looking safety warning capabilities in the display of in-vehicle instrument information. Summary of the Invention

[0005] The technical problem solved by this invention is that the existing technology is difficult to achieve comprehensive perception and linkage response of driver status, operation intention and driving environment, resulting in the lack of dynamic adaptability and forward-looking safety prompt capability in the display of in-vehicle instrument information.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: An in-vehicle instrument control and display system includes a driver state perception module, an environmental perception and analysis module, a behavior prediction module, an adaptive display control module, and a multimodal interactive response module; The driver state perception module is used to collect facial expression data, eye movement trajectory data, hand grip strength data and physiological data, and output driving state feature data through a fusion classification algorithm; The environmental perception and analysis module is used to acquire road condition information, weather type, visibility data, vehicle motion status data, and surrounding traffic flow density information, analyze the road condition information, weather type, visibility data, vehicle motion status data, and surrounding traffic flow density information, and output an environmental risk distribution map. The behavior prediction module is used to perform semantic analysis based on vehicle motion state data and driving state feature data, and output operation intention labels and path conflict prediction results. The operation intention labels include lane change operation intention, emergency braking operation intention and lane change operation intention. The adaptive display control module is used to input the operation intention label and path conflict result to the adjustment instrument interface, and generate a risk level signal by combining driving state characteristic data, and output the interface dynamic adjustment command. The multimodal interactive response module is used to dynamically adjust instructions and risk level signals according to the interface, output warning prompts through voice and light signals, and transmit the driver's voice or gaze feedback signals back to the driver's state perception module, forming a closed-loop interactive link.

[0007] Preferably, the driver state perception module includes a facial recognition unit, an eye-tracking unit, a grip strength recognition unit, and a physiological indicator analysis unit. The facial recognition unit collects real-time facial image data of the driver and outputs facial expression data through an expression recognition algorithm. The facial expression data includes the mouth opening range, the degree of eyebrow raising, and the eyelid closing time. The eye-tracking unit acquires the driver's eye movement trajectory data, which includes time series data of gaze time distribution, gaze angle range, and frequent deviation direction. The grip strength recognition unit collects grip strength data from both hands, including grip pressure value, rate of change, maximum grip strength value, grip strength ratio between the left and right hands, and grip pressure change amplitude. The physiological index analysis unit collects physiological data, including heart rate data, heart rate variability, and skin conductance value sequences. Based on the driver's real-time facial image data, eye movement trajectory data, and hand grip strength data, it outputs fatigue index and fluctuation frequency, and generates driving state characteristic data, including fatigue index, attention distraction, visual stability, grip strength change amplitude, physiological fluctuation frequency, and emotional tension parameters. Among them, the fatigue index is calculated based on eyelid closure time and heart rate variability; the attentional distraction is calculated from the fixation time distribution variance and fixation deviation frequency obtained from eye movement trajectory data; the visual stability is determined based on the rate of change of angular velocity of eye movement trajectory obtained from eye movement trajectory data; the grip strength variation amplitude is calculated from the grip pressure curve fluctuation obtained from the grip strength data of both hands; the physiological fluctuation frequency is obtained from the joint analysis of skin conductance value and heart rate; and the emotional tension parameter is deduced by coupling facial expression data and skin conductance transient response.

[0008] Preferably, the environmental perception and analysis module includes a road condition acquisition unit, a weather condition perception unit, and a traffic flow association unit: The road status acquisition unit is used to acquire road status information, which includes lane information, traffic sign information, and obstacle information ahead. The weather condition sensing unit is used to acquire weather type and visibility data; The traffic flow association unit is used to collect vehicle motion state data and surrounding traffic flow density information. The vehicle motion state data includes vehicle speed, acceleration, direction angle and brake pedal travel. The surrounding traffic flow density information includes the speed, direction and density of surrounding vehicles. The road condition information, weather type, visibility data, vehicle motion state data and surrounding traffic flow density information are aligned with driving state feature data under a unified time reference. The geometric correspondence between road scene and traffic object is realized through spatial coordinate mapping. Using attention distraction and fatigue index from driving state characteristic data as weighting factors, the risk contribution of preset road condition information, weather type, visibility data, vehicle motion state data and surrounding traffic flow density information is weighted and integrated to establish a dynamic correlation between driving state characteristic data, road condition information, weather type, visibility data, vehicle motion state data, surrounding traffic flow density information and local risk labels, and an environmental risk distribution map is generated.

[0009] Preferably, the behavior prediction module includes a behavior recording learning unit, an operation intention recognition unit, a path conflict assessment unit, and a pre-prompt generation unit; The behavior recording learning unit is used to collect the driver's past operating patterns and corresponding environmental features to establish a behavior feature library, which includes scene labels, vehicle dynamics parameters, gaze features and risk response results. The behavior recording learning unit uses the correspondence between the environmental risk distribution map and historical behavior data to match similar scenarios and form a behavior pattern mapping matrix, providing a reference feature set for the operation intention recognition unit.

[0010] Preferably, the operation intention recognition unit is used to receive speed change curves, acceleration curves, direction angle change sequences, brake pedal travel change trajectories and gaze trajectories, perform time synchronization and linear normalization on each input data, use the sliding window method to detect local extreme points and derivative change trends, and identify behavioral turning points; Identify points of sudden velocity drop in velocity change curves, detect points of sudden change in direction angle in direction angle change sequences, extract acceleration and deceleration transition segments in acceleration curves, and detect points of sudden line of sight deviation in gaze trajectories. The behavioral turning points are used to form a behavioral key point index. The maximum value, minimum value, change range, duration and average change rate within the window are extracted to form a feature vector. The system uses a classification model to determine the operational intent label corresponding to the feature vector and outputs a confidence score.

[0011] Preferably, the path conflict assessment unit is used to generate a sequence of trajectory points within a preset time period based on the vehicle's current speed, direction angle, and operating intention, and to calculate a future path prediction sequence by combining the surrounding vehicle motion state data output by the traffic flow association unit. The vehicle path prediction sequence is overlapped with the path of external traffic objects to detect overlap and output a conflict score matrix. The conflict score matrix includes the intersection location, time overlap segment, speed difference, angle data and corresponding conflict level. The pre-warning generation unit determines the warning method and display window based on the conflict level, operation intention label, driver's current fatigue index and preset warning logic, and outputs dynamic warning layer signals, which include flashing borders, risk area highlighting and guiding arrows.

[0012] Preferably, the adaptive display control module includes an information priority evaluation unit, an interface structure adjustment unit, and a simplified display switching unit; The information priority evaluation unit outputs a priority vector based on conflict level, operation intention label, and fatigue index. The interface structure adjustment unit adjusts the display position, font size, brightness value, border thickness and color value of speed information, forward distance data and navigation trajectory according to the interface layout priority vector, and determines whether to output light effect border signal, element magnification signal or trajectory arrow signal based on the difference of the driver's current gaze area. After detecting that the fatigue index exceeds the threshold for a preset number of consecutive times, the simplified display switching unit outputs simplified display signals for retaining speed, gear, and following distance while hiding the other items.

[0013] Preferably, the multimodal interactive response module includes a voice broadcast execution unit, an in-cabin light signal linkage unit, and a fatigue wake-up intervention unit; The voice broadcast execution unit converts the warning prompt into a voice signal output. The cabin light signal linkage unit outputs ambient light color value, brightness value and flashing frequency control signals according to the conflict level; The fatigue wake-up intervention unit outputs voice prompts, seat vibration, and instrument interface warning switching signals when the fatigue index rises continuously.

[0014] Preferably, the interface structure adjustment unit divides the instrument interface into a main display area, a secondary area, and a hidden area; The main display area is used to display speed information and navigation path; When it is detected that the driver is looking at a secondary or hidden area for more than a preset duration, a high-frequency border flashing signal, a content scaling animation signal, and a directional arrow prompt are triggered to guide the driver's vision back to the main display area.

[0015] Preferably, the inputs to the fatigue wake-up intervention unit include heart rate variability, blink rate, fixation time, and skin conductance change trends. The fatigue wake-up intervention unit performs fatigue trend analysis based on the above inputs and calculates the fatigue trend slope value. Based on the fatigue trend slope value, the fatigue state is divided into Level 1 fatigue, Level 2 fatigue, and Level 3 fatigue, which trigger voice broadcast signal, seat vibration signal, and instrument interface warning switching signal respectively.

[0016] The beneficial effects of this invention are as follows: By collecting driver status and environmental information and combining it with behavior prediction, this invention can realize operation intention recognition and risk warning. It can dynamically adjust the content of the instrument interface before danger occurs and provide multi-channel prompts and interventions through voice and lights to improve driving safety. Compared with the prior art, it has the advantages of forward warning, interface adaptation and personalized prompts, and realizes intelligent and context-aware vehicle instrument control functions. Attached Figure Description

[0017] Figure 1 This is a basic flowchart of an in-vehicle instrument control and display system provided in one embodiment of the present invention. Detailed Implementation

[0018] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0019] Example, refer to Figure 1 This invention provides an in-vehicle instrument control and display system, including a driver state perception module, an environmental perception and analysis module, a behavior prediction module, an adaptive display control module, and a multimodal interactive response module.

[0020] The driver state perception module is used to collect facial expression data, eye movement trajectory data, hand grip strength data, and physiological data, and outputs driving state feature data through a fusion classification algorithm.

[0021] The environmental perception and analysis module is used to acquire road condition information, weather type, visibility data, vehicle motion status data, and surrounding traffic flow density information. It analyzes the road condition information, weather type, visibility data, vehicle motion status data, and surrounding traffic flow density information, and outputs an environmental risk distribution map.

[0022] The behavior prediction module is used to perform semantic analysis based on vehicle motion state data and driving state feature data, and output operation intention labels and path conflict prediction results. The operation intention labels include lane change operation intention, emergency braking operation intention and lane change operation intention.

[0023] The adaptive display control module is used to input the operation intention label and path conflict result into the adjustment instrument interface, and generate a risk level signal by combining driving status characteristic data, and output dynamic adjustment instructions to the interface.

[0024] The multimodal interaction response module is used to dynamically adjust instructions and risk level signals according to the interface, output warning prompts through voice and light signals, and transmit the driver's voice or gaze feedback signals back to the driver's state perception module, forming a closed-loop interaction link.

[0025] This invention collects driver status and environmental information, combines it with behavior prediction, and realizes operation intention recognition and risk warning. It can dynamically adjust the instrument interface content before danger occurs and provide multi-channel prompts and interventions through voice and lights to improve driving safety. Compared with existing technologies, it has the advantages of forward-looking warning, interface self-adaptation and personalized prompts, and realizes intelligent and context-aware vehicle instrument control functions.

[0026] The driver state perception module includes a facial recognition unit, an eye-tracking unit, a grip strength recognition unit, and a physiological indicator analysis unit. The facial recognition unit collects real-time facial image data of the driver and outputs facial expression data through an expression recognition algorithm. The facial expression data includes the degree of mouth opening, the degree of eyebrow raising, and the duration of eyelid closure.

[0027] The facial recognition unit collects real-time facial image data of the driver and executes expression recognition algorithms to identify facial expression parameters such as the degree of mouth opening, the degree of eyebrow raising, and the duration of eyelid closure.

[0028] This unit enables the visual detection of facial muscle activity and fatigue state, which can be used to determine the driver's emotional tension and drowsiness level, providing key inputs for the calculation of fatigue index and emotional tension parameters.

[0029] The eye-tracking unit acquires the driver's eye movement trajectory data, which includes time series data on gaze duration distribution, gaze angle range, and frequent deviation directions.

[0030] The eye-tracking unit continuously monitors the driver's eye movement trajectory to obtain time-series data on gaze duration distribution, gaze angle range, and frequent deviation directions, enabling real-time assessment of attention distribution and visual concentration.

[0031] This unit can detect gaze deviation frequency and eye movement angular velocity changes to calculate attentional distraction and visual stability, thereby accurately reflecting the driver's visual focus level and potential attention decline trend.

[0032] The grip strength recognition unit collects grip strength data from both hands, including grip pressure value, rate of change, maximum grip strength value, left and right hand grip strength ratio, and grip pressure change amplitude.

[0033] The grip strength recognition unit collects grip strength data from both hands, including grip pressure values, rate of change, maximum grip strength, left and right hand grip strength ratio, and grip pressure change amplitude, which can reflect the driver's tension and control stability during operation.

[0034] This unit realizes the dynamic feature extraction of grip behavior, and calculates the range of grip force change by using grip force curve fluctuations, providing a quantitative basis for identifying sudden tension, relaxation or unstable posture.

[0035] The physiological index analysis unit collects heart rate data, heart rate variability, and skin conductance value sequences. Based on the driver's real-time facial image data, eye movement trajectory data, and hand grip strength data, it outputs fatigue index and fluctuation frequency, and generates driving state characteristic data, including fatigue index, attention distraction, visual stability, grip strength change amplitude, physiological fluctuation frequency, and emotional tension parameters.

[0036] Among them, the fatigue index is calculated based on eyelid closure time and heart rate variability; the attentional distraction is calculated from the fixation time distribution variance and fixation deviation frequency obtained from eye movement trajectory data; the visual stability is determined based on the rate of change of angular velocity of eye movement trajectory obtained from eye movement trajectory data; the grip strength variation amplitude is calculated from the grip pressure curve fluctuation obtained from the grip strength data of both hands; the physiological fluctuation frequency is obtained from the joint analysis of skin conductance value and heart rate; and the emotional tension parameter is deduced by coupling facial expression data and skin conductance transient response.

[0037] The physiological index analysis unit can generate driving state characteristic data by synchronously collecting heart rate data, heart rate variability and skin conductance value sequences, and combining them with facial images, eye movement trajectory and grip strength data.

[0038] Based on the dynamic changes of physiological signals, this unit uses the joint analysis of skin conductance and heart rate to assess the frequency of physiological fluctuations. By coupling facial expressions and transient responses of skin conductance, it calculates emotional tension parameters, thus achieving a refined analysis of the driver's physiological fatigue and emotional stress state.

[0039] The driver state perception module can identify the driver's fatigue level, distraction state, emotional fluctuations and physiological stability in real time by simultaneously collecting and fusing multimodal data such as the driver's facial images, eye movement trajectory, grip force signals and physiological indicators.

[0040] This module achieves a deep correlation between driving behavior characteristics and physiological signals, generating driving state characteristic data including fatigue index, attention distraction, line of sight stability, grip strength variation, physiological fluctuation frequency, and emotional tension parameters, providing a quantitative basis for subsequent behavior prediction, risk assessment, and interface adaptation.

[0041] The environmental perception and analysis module includes a road condition acquisition unit, a weather condition perception unit, and a traffic flow correlation unit: The road condition acquisition unit is used to acquire road condition information, which includes lane information, traffic sign information, and obstacle information. Lane information includes lane geometry, lane width, lane boundaries, and lane markings. Traffic sign information includes traffic sign type and traffic sign location. Obstacle information includes obstacle type, obstacle distance, obstacle shape, obstacle relative speed, and obstacle relative position.

[0042] The road condition acquisition unit achieves accurate perception of road structure and traffic elements by detecting and identifying lane lines, traffic signs and obstacles ahead in real time.

[0043] This unit can dynamically capture the geometric and semantic information of road boundaries, speed limit signs, and potential obstacles, ensuring that the system can accurately establish driving space boundaries in different road scenarios.

[0044] The weather condition sensing unit is used to acquire weather type and visibility data.

[0045] The weather condition sensing unit establishes a real-time sensing mechanism for the external climate environment by collecting meteorological type and visibility data.

[0046] This unit can identify weather conditions such as rain, snow, fog, and clear skies, as well as visibility trends, providing external environmental correction factors for risk assessment.

[0047] The traffic flow association unit is used to collect vehicle motion state data and surrounding traffic flow density information. Vehicle motion state data includes vehicle speed, acceleration, steering angle, and brake pedal travel. Surrounding traffic flow density information includes the speed, direction, and density of surrounding vehicles. The unit aligns road condition information, weather type, visibility data, vehicle motion state data, and surrounding traffic flow density information with driving state feature data under a unified time reference. The geometric correspondence between road scenes and traffic objects is realized through spatial coordinate mapping.

[0048] Using attention distraction and fatigue index from driving state characteristic data as weighting factors, the risk contribution of preset road condition information, weather type, visibility data, vehicle motion state data and surrounding traffic flow density information is weighted and integrated to establish a dynamic correlation between driving state characteristic data, road condition information, weather type, visibility data, vehicle motion state data, surrounding traffic flow density information and local risk labels, and an environmental risk distribution map is generated.

[0049] The analysis process is as follows: The data on driving state characteristics, road conditions, weather types, visibility, vehicle motion, and surrounding traffic flow density are aligned to a unified time base. Time synchronization is performed to ensure all data points are aligned at the same timestamp. Spatial coordinate mapping technology ensures a one-to-one correspondence between vehicle position and the geometric position of the external environment. The driver's current state is assessed by analyzing driver distraction and fatigue index. Key information describing environmental conditions is extracted based on road conditions, weather types, visibility, vehicle motion, and surrounding traffic flow density. After these preprocessing steps, the data is fused in a weighted manner, with the weights determined by distraction and fatigue index. Fatigue index and distraction, as weighting factors, reflect the driver's mental state and thus influence the risk contribution of environmental factors to driving behavior. Based on distraction and fatigue index in the driving state characteristic data as weighting factors, the risk contribution of each feature is calculated as follows: Road condition information: The risk contribution of the external road environment increases when the driver is fatigued or inattentive, and decreases when the driver is not fatigued or inattentive.

[0050] Weather type and visibility: Reduced visibility in rainy, snowy, and foggy weather conditions significantly increases the risk of traffic accidents, especially when drivers are not paying attention.

[0051] Vehicle motion data: When driving at high speed, accelerating rapidly, or braking suddenly, the driver's behavior conflicts with the environment, increasing the likelihood of an accident.

[0052] Surrounding traffic density information: The density of traffic flow directly affects the driver's decision space, especially when attention is distracted or fatigue is high, the impact of surrounding traffic density is more obvious.

[0053] After weighted fusion of road condition information, weather type, visibility data, vehicle motion status data, and surrounding traffic flow density information, an environmental risk distribution map is output, reflecting the overall risk level of the environment in which the driver is currently located.

[0054] The functions of a risk distribution map include: Regional risk prediction: Through spatiotemporal data modeling, weighted and fused data is mapped onto road scenarios, generating spatiotemporal risk distribution maps based on different timestamps and geographical locations. This part involves geographic coordinate mapping of spatiotemporal data to ensure accuracy in both geographic space and time dimensions. The risk contribution of each region is quantified, converted into a risk intensity value, and these values ​​are mapped to the corresponding geographic areas. The risk intensity value for each region represents the probability of a potential risk occurring in that region.

[0055] Multi-dimensional risk analysis: Combining the driver's physiological and psychological states, the system assesses the driver's performance in different environments. When evaluating environmental risk, the driver's physiological and psychological data serve as weighting factors to influence the risk contribution of other environmental features. When the fatigue index exceeds a preset threshold, the weight of environmental factors increases. When the driver's attention is diverted, the system may increase the weight of surrounding traffic flow density and road sign information to reflect the decreased ability of the driver to perceive changes in the external environment when attention is diverted. When the emotional tension parameter exceeds a preset threshold, the predicted risk value of actions such as sudden braking and sudden acceleration increases. Through weighted fusion, the risk contribution of each input data point is adjusted according to the driver's state. Low visibility, rainy or snowy weather, and traffic congestion may have a small impact on the driver under normal circumstances, but they significantly increase the risk when the driver is fatigued or diverted. Sudden acceleration and sudden braking are affected by the fatigue index and the degree of attention diversion. Fatigued or diverted drivers are more likely to engage in these unsafe behaviors. An environmental risk score is generated using this weighted fused data.

[0056] The traffic flow association unit achieves dynamic modeling of the local traffic flow field by collecting vehicle motion state data and surrounding traffic flow density information.

[0057] This unit aligns various environmental data with driving state feature data under a unified benchmark through time synchronization and spatial coordinate mapping, realizing the geometric correspondence between road scenes and traffic objects.

[0058] Using attention distraction and fatigue index from driving state characteristic data as weighting factors, the risk contribution of weather, visibility, traffic density and vehicle dynamics characteristics is weighted and integrated to establish a dynamic correlation between environmental factors and driving state, thereby generating an environmental risk distribution map.

[0059] The environmental perception and analysis module achieves dynamic perception of the driving environment and spatial expression of risks by collecting and semantically modeling multi-source data on road information, meteorological conditions, and traffic flow characteristics.

[0060] This module can weightedly fuse driver state characteristics with external environmental elements and output an environmental risk distribution map, thereby realizing a two-way correlation between driving behavior and the external environment, providing a scene perception basis for subsequent behavior prediction and adaptive display control.

[0061] The behavior prediction module includes a behavior recording and learning unit, an operation intention recognition unit, a path conflict assessment unit, and a pre-prompt generation unit.

[0062] The behavior recording learning unit is used to collect drivers' past operating patterns and corresponding environmental characteristics to build a behavior feature library, which includes scene labels, vehicle dynamics parameters, gaze characteristics, and risk response results.

[0063] The behavior recording learning unit uses the correspondence between the environmental risk distribution map and historical behavior data to match similar scenarios and form a behavior pattern mapping matrix, providing a set of reference features for the operation intention recognition unit.

[0064] The behavior recording learning unit collects drivers' past operating patterns and corresponding environmental features to establish a behavior feature library that includes scene labels, vehicle dynamics parameters, gaze features, and risk response results, thus enabling the long-term accumulation of driving behavior experience.

[0065] This unit utilizes the correspondence between environmental risk distribution maps and historical behavioral data to match similar scenarios and form a behavioral pattern mapping matrix.

[0066] Through this mapping relationship, the system can automatically retrieve historical behavior patterns that are most similar to the current environment during real-time driving, providing a priori reference feature set for operation intention recognition, thereby enhancing the contextual accuracy of intention recognition.

[0067] The operation intent recognition unit is used to receive speed change curves, acceleration curves, direction angle change sequences, brake pedal travel change trajectories and gaze trajectories, perform time synchronization and linear normalization on each input data, and use the sliding window method to detect local extreme points and derivative change trends to identify behavioral turning points.

[0068] Identify points of sudden velocity drops in velocity change curves, detect points of abrupt changes in direction angle in direction angle change sequences, extract acceleration and deceleration transition segments in acceleration curves, and detect abrupt changes in gaze deviation in gaze trajectories.

[0069] The behavioral turning points are used to form a behavioral key point index. The maximum value, minimum value, change range, duration and average change rate within the window are extracted to form a feature vector.

[0070] The classification model determines the operation intention label corresponding to the feature vector. The operation intention label includes lane change operation intention, emergency braking operation intention and lane change operation intention, and outputs a confidence score.

[0071] The detailed process of generating operation intent labels is as follows: The operation intent recognition unit analyzes the speed change curve, acceleration curve, direction angle change sequence, brake pedal travel change trajectory and gaze trajectory, and uses the sliding window method to detect local extreme points and derivative change trends in the speed change curve, acceleration curve, direction angle change sequence, brake pedal travel change trajectory and gaze trajectory.

[0072] Local extreme points and derivative trends reflect changes in driver behavior, helping to identify behavioral turning points.

[0073] In the speed change curve, a sudden drop in speed indicates that the driver may be performing an emergency braking operation. In the steering angle change sequence, a sudden change in steering angle indicates that the driver is performing a sharp turn or lane change operation. In the acceleration curve, the acceleration and deceleration transition segments are the behavioral characteristics of acceleration or deceleration. The abrupt shift in the line of sight trajectory can determine whether the driver intends to change the driving direction or focus.

[0074] The identified behavioral turning points are compiled into a behavioral key point index, and the maximum, minimum, magnitude of change, duration, and average rate of change are extracted from the time windows in which these key points are located. Maximum and minimum values ​​represent the extreme points of data change, helping to determine the magnitude and intensity of driver behavior.

[0075] The magnitude, duration, and average rate of change quantify the degree and duration of changes in driving behavior, helping to assess the urgency and risk of the behavior.

[0076] Feature vectors are generated based on behavioral keypoint indexes. These feature vectors contain velocity, acceleration, extreme points of direction change, amplitude of behavior, rate of change, and duration. Through a trained support vector machine, these feature vectors are used to predict the operation intention label.

[0077] The classification model will determine the driver's current operating intention based on the content of the feature vector. The operating intention label includes: Lane change intention: When the steering angle changes abruptly or the line of sight changes abruptly, it indicates that the driver intends to change lanes.

[0078] Intent for emergency braking: A sudden drop in speed and abrupt change in the rate of acceleration indicate that the driver intends to brake suddenly.

[0079] Lane change intention: When acceleration and a change of direction occur together, it indicates that the driver intends to change lanes or merge into another lane.

[0080] Finally, the support vector opportunity outputs a confidence score for each operational intent label, which represents the degree of confidence in the prediction result.

[0081] A high confidence score indicates a high degree of certainty that the intention to perform the action has occurred, while a low confidence score may indicate uncertainty, suggesting that more information is needed to confirm the intention.

[0082] The driver's intention recognition unit achieves real-time recognition of the driver's intention by performing time-series analysis on speed change curves, acceleration curves, direction angle change sequences, brake pedal travel trajectory, and gaze trajectory.

[0083] This unit uses a sliding window algorithm to detect local extreme points and derivative change trends, identify points of sudden speed drop, points of sudden change in direction angle, acceleration and deceleration transition segments, and points of sudden change in line of sight deviation, and extract key behavioral inflection points.

[0084] By constructing a behavioral key point index and generating feature vectors, a classification model is used to calculate and output the labels of operational intentions such as lane changing, sudden braking, and lane merging, as well as their confidence scores.

[0085] This process achieves semantic mapping from low-level motion signals to high-level driving intentions, possessing real-time performance and high interpretability, and providing basic input for subsequent path conflict prediction.

[0086] The path conflict assessment unit is used to generate a sequence of trajectory points within a preset time period based on the vehicle's current speed, direction angle, and operational intent, and to calculate the future path prediction sequence by combining the surrounding vehicle motion state data output by the traffic flow association unit.

[0087] The vehicle path prediction sequence is overlapped with the path of external traffic objects, and a conflict score matrix is ​​output. The conflict score matrix includes the intersection location, time overlap segment, speed difference, angle data and corresponding conflict level.

[0088] The path conflict assessment unit generates a sequence of trajectory points within a preset time period based on the vehicle's current speed, direction angle, and operational intent, and constructs a future path prediction sequence by combining the motion status data of surrounding vehicles, thus realizing the detection of potential trajectory conflicts between the vehicle and external traffic objects.

[0089] This unit uses an overlap detection algorithm to output a conflict score matrix, which includes the intersection location, time overlap segment, velocity difference, included angle data, and conflict level information.

[0090] Through this analysis process, the system can identify potential collision paths before the vehicle enters the risk area, enabling quantitative assessment and graded early warning of the spatiotemporal distribution of risks, and providing predictive prevention and control basis for driver assistance systems.

[0091] The pre-warning generation unit determines the warning method and display window based on the conflict level, operation intention label, driver's current fatigue index and preset warning logic, and outputs dynamic warning layer signals, including flashing borders, risk area highlighting and guide arrows.

[0092] The advance warning generation unit generates dynamic warning layer signals adapted to different risk levels by comprehensively considering the conflict level, operation intention label, driver fatigue index, and preset warning logic.

[0093] This unit can automatically select semantic prompts and visual display windows based on the risk level, guiding the driver's attention back to the key areas through layer signals such as flashing borders, highlighting risk areas, and guiding arrows.

[0094] This process enables proactive risk information perception and visual visualization prompts, enhancing the system's interactivity, immediacy, and security response capabilities.

[0095] The behavior prediction module integrates the driver's historical operating behavior with the real-time environmental status to identify driving intentions, dynamically predict future paths, and provide advance warnings of potential risks and conflicts.

[0096] This module can proactively analyze the interaction between the current driving status and environmental risks, and output behavioral trend prediction results and early warning instructions before the risks manifest, thereby achieving predictability of driving behavior and forward-looking risk control.

[0097] The adaptive display control module includes an information priority evaluation unit, an interface structure adjustment unit, and a simplified display switching unit.

[0098] The information priority assessment unit outputs a priority vector based on conflict level, operational intent label, and fatigue index.

[0099] The information priority assessment unit generates an interface layout priority vector by comprehensively analyzing conflict level, operation intention tags, and fatigue index, thereby realizing dynamic hierarchical classification of interface display content.

[0100] This unit can determine the display priority of different information items based on the current driving risk level and operational intention, so that the interface content is closely related to the driving task.

[0101] By outputting priority vectors in real time, a quantitative basis is provided for subsequent adjustments to the interface layout and visual effects, ensuring that drivers receive the most important information first at critical moments.

[0102] The interface structure adjustment unit adjusts the display position, font size, brightness value, border thickness, and color value of speed information, forward distance data, and navigation trajectory based on the interface layout priority vector, and determines whether to output light effect border signal, element magnification signal, or trajectory arrow signal based on the difference in the driver's current gaze area.

[0103] The interface structure adjustment unit divides the instrument interface into a main display area, a secondary area, and a hidden area.

[0104] The main display area is used to show speed information and navigation path.

[0105] When it is detected that the driver is looking at a secondary or hidden area for more than a preset duration, a high-frequency border flashing signal, a content scaling animation signal, and a directional arrow prompt are triggered to guide the driver's vision back to the main display area.

[0106] The interface structure adjustment unit dynamically adjusts the display position, font size, brightness, border thickness, and color value of speed information, forward distance data, and navigation trajectory in the interface based on the interface layout priority vector, thereby achieving adaptive optimization of the information visibility hierarchy.

[0107] This unit detects the difference between the driver's current gaze area and the high-priority area, and intelligently determines whether to trigger light effect border signals, element magnification signals, or trajectory arrow prompts, actively guiding visual attention back to the key area.

[0108] Meanwhile, the unit divides the instrument panel into a main display area, a secondary area, and a hidden area. When the driver looks at the secondary or hidden area for more than a preset time, it triggers high-frequency flashing, content scaling, and directional prompt animations to form a visual feedback mechanism.

[0109] This mechanism effectively avoids perceptual omissions caused by attention shifts, and enhances the coupling and interactive responsiveness between the interface and driving behavior.

[0110] After detecting that the fatigue index exceeds the threshold for a preset number of consecutive times, the simplified display switching unit outputs simplified display signals for speed, gear, and following distance while hiding the other items.

[0111] The simplified display switching unit monitors the changing trend of the fatigue index in real time. When the fatigue index exceeds the preset threshold multiple times in a row, it automatically switches to the simplified display mode, retaining only core information such as speed, gear and following distance, while hiding the rest of the display items.

[0112] This unit implements a visual load adjustment mechanism based on the driver's state, enabling the interface to present the simplest information structure when the driver is fatigued, thereby reducing cognitive burden, delaying fatigue accumulation, and improving operational safety in high-risk situations.

[0113] The adaptive display control module achieves dynamic adaptive adjustment of the instrument interface through comprehensive analysis of driving state characteristics, risk level, and attention distribution.

[0114] This module can automatically assess information priority based on driving scenarios, adjust interface layout, visual focus, and interaction feedback methods, and automatically switch to a simplified display mode when driver fatigue is detected, thereby effectively reducing information interference, improving attention concentration, and enhancing driving safety.

[0115] The multimodal interactive response module includes a voice broadcast execution unit, an in-cabin light signal linkage unit, and a fatigue wake-up intervention unit.

[0116] The voice broadcast execution unit converts the warning prompts into voice signals for output.

[0117] The voice broadcast execution unit converts the warning prompts generated by the system into voice signals for output, thereby enabling real-time risk transmission through the auditory channel.

[0118] This unit can generate corresponding voice broadcast commands based on risk level, operational intent, and environmental changes. It adopts a phrase-based and hierarchical voice output strategy to provide auxiliary prompts when the driver's visual load is high.

[0119] Its effect is to achieve a non-visual risk perception approach, enabling drivers to be aware of potential risk events in a timely manner in complex driving scenarios.

[0120] The cabin light signal linkage unit outputs ambient light color value, brightness value and flashing frequency control signals according to the conflict level.

[0121] The cabin light signal linkage unit controls the color value, brightness value and flashing frequency of the cabin ambient light according to the conflict level, so as to realize the corresponding expression of light signals and risk levels.

[0122] In low-to-medium risk scenarios, the unit provides early warnings with gradual or flashing light effects, while in high-risk situations, it triggers high-brightness or high-frequency flashing warnings, enabling drivers to perceive the intensity of risk through changes in the visual environment.

[0123] This mechanism enhances the immersive feedback experience in the cockpit, creating an intuitive and non-intrusive risk warning channel that helps improve driver concentration without interfering with operation.

[0124] When the fatigue index rises continuously, the fatigue wake-up intervention unit outputs voice prompts, seat vibration, and instrument panel warning switching signals.

[0125] The inputs to the fatigue wake-up intervention unit include heart rate variability, blink rate, fixation time, and skin conductance trends. The fatigue wake-up intervention unit performs fatigue trend analysis based on the above inputs and calculates the fatigue trend slope value.

[0126] Based on the fatigue trend slope value, the fatigue state is divided into Level 1 fatigue, Level 2 fatigue, and Level 3 fatigue, which trigger voice broadcast signal, seat vibration signal, and instrument interface warning switching signal respectively.

[0127] The fatigue wake-up intervention unit quantitatively analyzes the driver's fatigue trend by receiving physiological signals such as heart rate variability, blink rate, fixation time, and skin conductance changes in real time.

[0128] This unit uses the fatigue trend slope to classify fatigue state, dividing the driver's state into Level 1 fatigue, Level 2 fatigue and Level 3 fatigue, respectively triggering voice broadcast signals, seat vibration signals and instrument interface warning switching signals, thus realizing adaptive intervention based on physiological state.

[0129] This graded response mechanism enables the system to automatically match the intervention intensity according to the severity of fatigue, from mild reminders to active wake-up, forming a progressive safety protection system that effectively prevents delayed reactions or driving errors caused by fatigue.

[0130] The multimodal interactive response module achieves multi-channel prompts and active wake-up under driving risk conditions by using voice, light signals and physical intervention in a coordinated manner.

[0131] This module can dynamically allocate prompts across different modalities based on the level of conflict and the driver's fatigue trend, achieving a multi-level warning response from visual to auditory to tactile.

[0132] The system forms a closed-loop intervention mechanism through voice broadcasting, cabin light signal linkage, and seat vibration feedback, which not only improves the driver's timeliness in perceiving risk events, but also significantly enhances driving safety and the initiative of interactive experience.

[0133] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0134] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. An in-vehicle automobile instrument control display system characterized by comprising: The driver state perception module, the environment perception analysis module, the behavior prediction module, the adaptive display control module and the multi-modal interaction response module are comprised. The driver state perception module is used for collecting facial expression data, eye movement trajectory data, two-hand grip data and physiological data, and outputting driving state feature data through fusion classification algorithm. The environment perception analysis module is used for obtaining road state information, weather type, visibility data, vehicle motion state data and surrounding traffic flow density information, analyzing the road state information, weather type, visibility data, vehicle motion state data and surrounding traffic flow density information, and outputting an environment risk distribution map. The behavior prediction module is used for performing semantic analysis according to the vehicle motion state data and the driving state feature data, outputting an operation intention label and a path conflict prediction result, and the operation intention label comprises a lane changing operation intention, a sudden braking operation intention and a merging operation intention. The adaptive display control module is used for inputting the operation intention label and the path conflict result into an adjustment instrument interface, generating a risk level signal in combination with the driving state feature data, and outputting an interface dynamic adjustment instruction. The multi-modal interaction response module is used for outputting a pre-warning prompt through voice and light signals according to the interface dynamic adjustment instruction and the risk level signal, and feeding back the driver's voice or gaze feedback signal to the driver state perception module to form a closed-loop interaction link.

2. The vehicle-mounted automobile gauge control display system according to claim 1, wherein The driver state perception module comprises a facial recognition unit, a gaze tracking unit, a grip recognition unit and a physiological index analysis unit. The facial recognition unit collects real-time facial image data of the driver, and outputs facial expression data through an expression recognition algorithm, wherein the facial expression data comprises mouth opening amplitude, eyebrow lifting degree and eyelid closing time. The gaze tracking unit obtains eye movement trajectory data of the driver, and the eye movement trajectory data comprises gaze time distribution, gaze angle range and time sequence data of frequent deviation direction. The grip recognition unit collects two-hand grip data, and the two-hand grip data comprises holding pressure value, change rate, maximum grip value, left-right hand grip ratio and holding pressure change amplitude. The physiological index analysis unit collects physiological data, and the physiological data comprises heart rate data, heart rate variability and skin conductance value sequence, outputs fatigue index and fluctuation frequency according to real-time facial image data, eye movement trajectory data and two-hand grip data, and generates driving state feature data, wherein the driving state feature data comprises fatigue index, attention dispersion, gaze stability, grip change amplitude, physiological fluctuation frequency and emotional tension parameter. The fatigue index is calculated based on the eyelid closing time and the heart rate variability, the attention dispersion is calculated based on the gaze time distribution variance and the gaze deviation frequency obtained from the eye movement trajectory data, the gaze stability is determined based on the angular velocity change rate of the eye movement trajectory calculated based on the eye movement trajectory data, the grip change amplitude is calculated based on the holding pressure curve fluctuation obtained from the two-hand grip data, the physiological fluctuation frequency is obtained based on the joint analysis of the skin conductance value and the heart rate, and the emotional tension parameter is calculated based on the coupling of the facial expression data and the skin conductance transient response.

3. An in-vehicle car meter control display system according to claim 2, wherein The environment perception analysis module comprises a road state acquisition unit, a weather condition perception unit and a traffic flow correlation unit: The road state acquisition unit is configured to acquire road state information, which comprises lane information, traffic sign information and front obstacle information; The weather condition perception unit is configured to acquire meteorological type and visibility data; The traffic flow correlation unit is configured to acquire vehicle motion state data and surrounding traffic flow density information, wherein the vehicle motion state data comprises vehicle speed, acceleration, direction angle and brake pedal stroke, and the surrounding traffic flow density information comprises speed, direction and density of surrounding vehicles; the road state information, meteorological type, visibility data, vehicle motion state data and surrounding traffic flow density information are aligned with driving state feature data under a unified time reference, and the geometric correspondence between the road scene and the traffic object is realized through spatial coordinate mapping; The risk contribution degrees of the preset road state information, meteorological type, visibility data, vehicle motion state data and surrounding traffic flow density information are weighted and fused by taking the distraction degree and fatigue index in the driving state feature data as weight factors, a dynamic correlation relationship between the driving state feature data and the road state information, meteorological type, visibility data, vehicle motion state data, surrounding traffic flow density information and local risk label is established, and an environment risk distribution map is generated.

4. A vehicle mounted control and display system for a vehicle instrument according to claim 3, wherein, The behavior prediction module comprises a behavior record learning unit, an operation intention recognition unit, a path conflict evaluation unit and a pre-warning generation unit; The behavior record learning unit is configured to collect past operation modes of drivers and corresponding environment features, and establish a behavior feature library, wherein the behavior feature library comprises scene labels, vehicle dynamics parameters, gaze features and risk response results; The behavior record learning unit matches similar scenes by using the corresponding relationship between the environment risk distribution map and historical behavior data, forms a behavior mode mapping matrix, and provides a reference feature set for the operation intention recognition unit.

5. An in-vehicle car meter control display system according to claim 4, wherein The operation intention recognition unit is configured to receive a speed change curve, an acceleration curve, a direction angle change sequence, a brake pedal stroke change trajectory and a gaze trajectory, perform time synchronization and linear normalization on each input data, detect local extreme points and derivative change trends by using a sliding window method, and recognize behavior turning points; The speed change curve is used to identify a speed drop point, the direction angle change sequence is used to detect a direction angle change point, the acceleration curve is used to extract acceleration and deceleration transition sections, and the gaze trajectory is used to detect a line of sight deviation mutation point; The behavior turning points are used to form a behavior key point index, and the maximum value, minimum value, change amplitude, duration and average change rate in a window are extracted to form a feature vector; The classification model is used to determine an operation intention label corresponding to the feature vector and output a confidence score.

6. A vehicle mounted control and display system for a vehicle instrument as claimed in claim 5, wherein, The path conflict evaluation unit is configured to generate a trajectory point sequence in a future preset time period according to the current speed, direction angle and operation intention of the vehicle, and calculate a future path prediction sequence in combination with the vehicle motion state data of surrounding vehicles output by the traffic flow correlation unit; The vehicle path prediction sequence is overlapped with the external traffic object path to perform overlap detection, and a conflict score matrix is output, the conflict score matrix including intersection position, time overlap section, speed difference, angle data, and corresponding conflict level; The front prompt generation unit determines the prompt mode and display window according to the conflict level, operation intention label, current fatigue index of the driver, and preset warning logic, and outputs a dynamic prompt layer signal, the dynamic prompt layer signal including a flashing border, a risk area highlight, and a guide arrow.

7. An in-vehicle car meter control display system according to claim 6, wherein The adaptive display control module includes an information priority evaluation unit, an interface structure adjustment unit, and a simplified display switching unit. The information priority evaluation unit outputs an interface layout priority vector based on the conflict level, the operation intention label, and the fatigue index. The interface structure adjustment unit adjusts the display position, font size, brightness value, border thickness, and color value of the speed information, front distance data, and navigation trajectory according to the interface layout priority vector, and determines whether to output a light effect border signal, an element enlargement signal, or a trajectory arrow signal based on the current gaze area difference of the driver. The simplified display switching unit outputs a simplified display signal that retains the speed, gear, and following distance and hides the remaining items after detecting that the fatigue index exceeds the threshold value for a continuous preset number of times.

8. A vehicle mounted control and display system for a vehicle instrument according to claim 7, wherein, The multi-modal interaction response module includes a voice broadcast execution unit, a cabin light signal linkage unit, and a fatigue awakening intervention unit. The voice broadcast execution unit converts the warning prompt mode into a voice signal output. The cabin light signal linkage unit outputs an ambient light color value, brightness value, and flashing frequency control signal according to the conflict level. The fatigue awakening intervention unit outputs a voice prompt, seat vibration, and instrument interface warning switching signal when the fatigue index continuously increases.

9. A vehicle mounted control and display system for a vehicle instrument according to claim 8, wherein, The interface structure adjustment unit divides the instrument interface into a main display area, a secondary area, and a hidden area. The main display area is used to display speed information and navigation path. When it is detected that the driver's gaze on the secondary area or hidden area exceeds a preset gaze duration, a high-frequency border flashing signal, a content scaling animation signal, and a direction arrow prompt are triggered to guide the driver's visual return to the main display area.

10. A vehicle mounted control and display system for a vehicle instrument according to claim 9, wherein, The input of the fatigue awakening intervention unit includes heart rate variability, blink rate, gaze stabilization time, and skin conductance change trend. The fatigue awakening intervention unit performs fatigue trend analysis according to the above inputs and calculates a fatigue trend slope value. According to the fatigue trend slope value, the fatigue state is divided into first-level fatigue, second-level fatigue, and third-level fatigue, and voice broadcast signals, seat vibration signals, and instrument interface warning switching signals are triggered respectively.

Images