An apparatus and system for dynamic human factors driving analysis

Abstract

This application provides a device and system for dynamic human-factor driving analysis. The device includes a data acquisition unit, a data processing unit, an artificial intelligence analysis unit, and a human-machine interface. The data acquisition unit collects data through sensors deployed on various driving interaction interfaces. The data processing unit receives the raw data and performs data processing and feature extraction. The artificial intelligence analysis unit analyzes the associated features using a multimodal fusion neural network model, obtaining analysis results and related prompts and / or human-factor driving analysis reports, which are then transmitted to the human-machine interface. These results are displayed and / or prompted on the human-machine interface. The associated features include the driver's psychological characteristics, physiological characteristics, and driving behavior characteristics. The technical solution of this application enables comprehensive perception and intelligent analysis of the driver's psychological, physiological, and driving behavior states, thereby improving driving safety.

Description

Technical Field

[0001] This invention relates to the technical fields of human factors engineering and traffic psychology, specifically to a device and system for dynamic human factors driving analysis. Background Technology

[0002] Dynamic human factors analysis in driving simulators plays a crucial role in several ways. It allows for a better understanding of driver reactions and behavioral patterns in different situations, such as decision-making processes in emergency situations. Understanding driver habits and preferences helps improve the human-machine interface design of vehicles, making control systems more intuitive and user-friendly, reducing driver distraction, and thus improving driving comfort. Automakers can also use these analyses to optimize new vehicle designs, ensuring new features are both practical and easy to use. Furthermore, these analyses can be used in the testing and validation phases of new systems, contributing to the design of more human-centered safety systems, such as automatic emergency braking and lane-keeping assist, and ultimately enhancing road safety.

[0003] By analyzing driver behavior data before and after an accident, the causes of accidents can be determined more accurately, providing a scientific basis for future accident prevention. For novice drivers or drivers in specific environments (such as inclement weather or nighttime driving), dynamic human factors analysis can be used to develop targeted training courses, helping them adapt to various driving conditions more quickly and enhancing their coping abilities. Furthermore, government agencies can collect and analyze large amounts of dynamic human factors data to formulate more scientific and reasonable traffic rules and standards, thereby promoting the continuous improvement of public transportation safety.

[0004] In conclusion, dynamic human factors analysis performed in driving simulators is not only crucial for improving individual driving skills and experience, but also has a profound impact on traffic safety for the entire society.

[0005] Traditional technologies rely heavily on pre-set rules and operational standards for feedback and judgment, making it difficult to conduct human factor analysis, personalized training, and subsequent behavioral correction interventions based on individual differences and actual scenario conditions. Furthermore, traditional technologies have limited data sources, making it difficult to deeply synchronize and integrate with other physiological data (such as heart rate and skin conductance) or environmental interaction signals. This limits their analytical capabilities and robustness in complex driving, working conditions, and emergency scenarios.

[0006] Therefore, a technological solution is needed that can achieve comprehensive perception and intelligent analysis of the driver's psychological, physiological, and driving behavior states to improve driving safety. Summary of the Invention

[0007] This application aims to provide a device and system for dynamic human factors driving analysis, which can achieve comprehensive perception and intelligent analysis of the driver's psychological, physiological and driving behavior states, thereby improving driving safety.

[0008] According to one aspect of this application, an apparatus for dynamic human-factor driving analysis is provided, the apparatus comprising: a data acquisition unit, a data processing unit, an artificial intelligence analysis unit, and a human-computer interaction interface, wherein...

[0009] The data acquisition unit collects data through sensors deployed on each driving interaction interface, obtains raw data information from different driving interaction interfaces, and sends it to the data processing unit.

[0010] The data processing unit receives the raw data information from the data acquisition unit and processes the data to obtain processed raw data. It then extracts features from the raw data and transmits the extracted related features to the artificial intelligence analysis unit.

[0011] The artificial intelligence analysis unit receives the associated features from the data processing unit, analyzes them through a multimodal fusion neural network model, obtains analysis results and related prompts and / or human factors driving analysis reports, and transmits them to the human-computer interaction interface.

[0012] The human-computer interaction interface receives the analysis results, related prompts, and / or human-cause driving analysis reports from the artificial intelligence analysis unit, and displays and / or prompts them on the human-computer interaction interface.

[0013] The sensor includes a flexible thin-film pressure sensor array, allowing the device to collect data without affecting the driver's perception. The associated features include the driver's psychological characteristics, physiological characteristics, and driving behavior characteristics.

[0014] According to some embodiments, the device further includes: a warning unit,

[0015] The early warning unit receives the associated features from the data processing unit, generates danger alarms and early warning information through a time-based prediction model, and sends them to the human-machine interface to send alarms and warnings to the driver.

[0016] According to some embodiments, the sensor includes an eye tracker for acquiring raw data information.

[0017] According to some embodiments, the data acquisition unit further includes a digital-to-analog conversion subunit, which samples, quantizes, and digitizes the analog signals from the flexible thin-film pressure sensor and the eye tracker to obtain the original data information on different driving interaction interfaces.

[0018] According to some embodiments, the data processing unit is configured as follows:

[0019] The system receives the raw data information from the data acquisition unit and processes the data to obtain processed raw data. The data processing includes filtering, interpolation, and normalization.

[0020] Based on the human factors analysis requirements of different application scenarios, the original data is subjected to feature extraction related to the application scenario to obtain corresponding associated features.

[0021] According to some embodiments, the artificial intelligence analysis unit is configured as follows:

[0022] A multimodal integrated analysis neural network is constructed, which integrates multimodal fusion neural networks, association rule mining, sequence pattern mining, cluster analysis, structural equation modeling, time series analysis modeling, and causal graph modeling. The multimodal fusion neural network is a convolutional neural network, a long short-term memory network, and a sequence model network.

[0023] According to some embodiments, the artificial intelligence analysis unit is configured as follows:

[0024] The multimodal integrated analysis neural network is pre-trained using pre-collected standard safe driving reaction data to establish a safe driving reaction benchmark.

[0025] The driving features in the associated features are identified and analyzed by the multimodal integrated analysis neural network to obtain the driver's current operation data;

[0026] The operation data is compared with the safe driving reaction benchmark in real time to obtain real-time safe driving analysis results and related prompts, which are then transmitted to the human-machine interface.

[0027] According to some embodiments, the artificial intelligence analysis unit is further configured as follows:

[0028] Based on the safe driving analysis results and related prompts, the multimodal integrated analysis neural network performs correlation and causal analysis on the psychological and physiological features among the associated features to obtain the human-cause driving analysis report and transmits it to the human-computer interaction interface.

[0029] According to some embodiments, the artificial intelligence analysis unit is further configured as follows:

[0030] Based on the associated features, fatigue driving analysis is performed, hazard alarm information is generated and sent to the human-machine interface to send an alert to the driver.

[0031] According to another aspect of this application, a system for dynamic human factors driving analysis is provided, the system comprising the apparatus as described in any of the preceding claims.

[0032] According to embodiments of this application, this solution employs multimodal sensors such as flexible thin-film pressure sensor arrays and eye trackers for non-invasive data acquisition. It combines deep learning models (such as CNN and LSTM), time-series analysis, and multimodal fusion technology to intelligently identify and comprehensively analyze the driver's psychological, physiological, and behavioral states. This achieves comprehensive perception and real-time warning of driving status, effectively improving driving safety. This solution supports personalized feedback and driving behavior optimization, is applicable to various driving scenarios, and possesses excellent user experience and broad application expansion potential, providing solid technical support for intelligent driving assistance systems.

[0033] It should be understood that the above general description and the following detailed description are merely exemplary and do not limit this application. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below.

[0035] Figure 1 The diagram illustrates a device for dynamic human factors driving analysis according to an example embodiment.

[0036] Figure 2 A schematic flowchart of a method for dynamic human factors driving analysis according to an example embodiment is shown.

[0037] Figure 3 A block diagram of a computing device according to an exemplary embodiment is shown. Detailed Implementation

[0038] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, they are provided so that this application will be thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted.

[0039] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0040] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0041] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0042] It should be understood that although the terms first, second, third, etc., may be used herein to describe various components, these components should not be limited by these terms. These terms are used to distinguish one component from another. Therefore, the first component discussed below may be referred to as the second component without departing from the teachings of this application. As used herein, the term "and / or" includes all combinations of any one and more of the associated listed items.

[0043] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0044] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of exemplary embodiments, and the modules or processes in the drawings are not necessarily necessary for implementing this application, and therefore cannot be used to limit the scope of protection of this application.

[0045] Dynamic human factors analysis in driving simulators plays a crucial role in several ways. It allows for a better understanding of driver reactions and behavioral patterns in different situations, such as decision-making processes in emergency situations. Understanding driver habits and preferences helps improve the human-machine interface design of vehicles, making control systems more intuitive and user-friendly, reducing driver distraction, and thus improving driving comfort. Automakers can also use these analyses to optimize new vehicle designs, ensuring new features are both practical and easy to use. Furthermore, these analyses can be used in the testing and validation phases of new systems, contributing to the design of more human-centered safety systems, such as automatic emergency braking and lane-keeping assist, and ultimately enhancing road safety.

[0046] By analyzing driver behavior data before and after an accident, the causes of accidents can be determined more accurately, providing a scientific basis for future accident prevention. For novice drivers or drivers in specific environments (such as inclement weather or night driving), dynamic human factors analysis can be used to develop targeted training courses to help them adapt to various driving conditions more quickly and enhance their coping abilities.

[0047] In addition, government agencies can collect and analyze large amounts of dynamic human factors data to formulate more scientific and reasonable traffic rules and standards, thereby promoting the continuous improvement of public transportation safety.

[0048] In conclusion, dynamic human factors analysis performed in driving simulators is not only crucial for improving individual driving skills and experience, but also has a profound impact on traffic safety for the entire society.

[0049] Traditional driver simulation analysis primarily relies on collecting data on operational behaviors (such as levers, steering wheels, buttons, pedals, etc.), environmental conditions, video images, or post-event questionnaires and subjective evaluations. It offers very limited objective quantification of the driver's (or operator's) physiological and psychological state, failing to directly reflect internal states such as tension, fatigue, and stress. Combined with static analysis or simple attribution based on rules and statistical characteristics, the analysis is coarse-grained, unable to capture the deep-seated driving forces behind behavior, and struggles to distinguish whether "operational errors" stem from insufficient skill, excessive stress, fatigue, or excessive cognitive load.

[0050] Moreover, feedback and judgment in traditional technologies rely more on preset rules and operational standards, making it difficult to achieve human factor analysis, personalized training, and subsequent behavioral correction intervention based on different individual differences and actual scenario conditions.

[0051] Moreover, traditional technologies rely on a single source of data, making it difficult to achieve deep synchronization and fusion with other physiological data (such as heart rate and skin conductance) or environmental interaction signals. Consequently, their analytical capabilities and robustness are limited when faced with complex driving, working conditions, and emergency scenarios.

[0052] To address this, a device and system for dynamic human-factor driving analysis are proposed, enabling comprehensive perception and intelligent analysis of the driver's psychological, physiological, and driving behavior states, thereby improving driving safety. According to some embodiments, a flexible thin-film pressure sensor array is deployed on the driving operation interface to collect operational data from the driver or operator. Multimodal fusion analysis is then performed using an artificial intelligence model, taking into account the driving environment and driver psychological factors, to comprehensively analyze the operational data from multiple perspectives.

[0053] According to some embodiments, by adding pre / alarm programs and human-machine interaction display interfaces, dangerous or illegal behaviors during driving or simulated driving can be prompted or alarmed, thereby further improving driving safety.

[0054] Before describing the embodiments of this application, some terms or concepts involved in the embodiments of this application will be explained.

[0055] Multimodal Fusion Neural Network (MFNN): A deep learning model that can simultaneously process multiple types of input data (such as images, text, speech, sensor signals, etc.) and perform feature fusion to improve the understanding of complex tasks.

[0056] Association Rule Mining (ARM): A data mining technique used to discover frequent co-occurrence relationships between variables from large amounts of data. It is commonly used in scenarios such as market basket analysis and behavioral pattern recognition.

[0057] Sequential Pattern Mining (SPM) is a method for mining frequently occurring patterns in time series data to identify the sequential patterns of events, such as the analysis of driving operation sequences.

[0058] Clustering Analysis (CA): An unsupervised learning method that groups similar data objects into a class to discover natural grouping structures in data, such as identifying groups of people with different driving styles.

[0059] Structural Equation Modeling (SEM) is a statistical modeling method used to verify complex causal relationships and underlying structures among variables. It is widely used in psychology, social science research, and human factors engineering.

[0060] Time Series Analysis Model (TSAM): Used to model and predict data sequences arranged in chronological order, capturing trends, periodicity, and random fluctuations. It is suitable for fatigue monitoring, reaction delay prediction, etc.

[0061] Causal Graphical Model (CGM): A graphical modeling tool used to represent causal relationships between variables. Common examples include Bayesian Networks and Causal Diagrams, which can help infer the effectiveness of interventions.

[0062] Convolutional Neural Network (CNN): A deep learning model designed specifically for processing data with spatial structure (such as images). It excels at extracting local features and is often used for classification tasks such as eye-tracking images and pressure distribution maps.

[0063] Long Short-Term Memory Network (LSTM): An improved recurrent neural network (RNN) that can effectively capture long-term dependent information and is suitable for processing time-series data, such as changes in driver physiological indicators and analysis of operational behavior sequences.

[0064] Sequence-to-Sequence Model (Seq2Seq): A deep learning architecture that maps input sequences to output sequences. It typically consists of an encoder and a decoder and is suitable for tasks such as action sequence generation and driving behavior prediction.

[0065] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of this application.

[0066] Figure 1 The diagram illustrates a device for dynamic human factors driving analysis according to an example embodiment.

[0067] See Figure 1 The figure shows a device for dynamic human factors driving analysis, which includes: a data acquisition unit 01, a data processing unit 05, an artificial intelligence analysis unit 07, and a human-computer interaction interface 03.

[0068] According to some embodiments, the data acquisition unit 01 collects data through sensors 0101 deployed on various driving interaction interfaces, obtaining raw data information from different driving interaction interfaces and sending it to the data processing unit 05. The sensors 0101 include a flexible thin-film pressure sensor array, allowing the device to collect data without affecting the driver's sensory experience. The associated features include the driver's psychological characteristics, physiological characteristics, and driving behavior characteristics. The data acquisition unit 01 is the basic module of the entire dynamic human-factor driving analysis device, responsible for acquiring multimodal raw data from the driver-vehicle interaction process. By deploying sensors 0101 on multiple driving interaction interfaces, comprehensive perception of the driver's operational behavior and physiological state is achieved. The driving interaction interfaces include, but are not limited to, key locations such as the steering wheel, pedals, seat, and central control area, ensuring coverage of the driver's main input points during driving.

[0069] According to some embodiments, the sensor 0101 further includes an eye tracker for acquiring raw data. The sensor 0101 used in the data acquisition unit 01 includes a flexible thin-film pressure sensor array, an eye tracker, and other optional physiological monitoring devices. The flexible thin-film pressure sensor array has good flexibility and conformability, and can be embedded in key locations of the driving interface such as the steering wheel, seat cushion, and pedals to collect real-time pressure distribution information generated by the driver's hand grip, foot pedals, and body posture during operation. The eye tracker is used to track the driver's gaze trajectory, blink frequency, and pupil changes, thereby reflecting their attention state and cognitive load level.

[0070] According to some embodiments, the data acquisition unit 01 can also integrate a heart rate sensor, a skin conductance sensor, etc., to obtain richer physiological characteristic data. The data acquisition unit 01 synchronously collects various behavioral signals during the driving process through the aforementioned sensors, forming multi-source raw data information containing timestamps. This data covers multiple dimensions such as the driver's operating force, operating rhythm, gaze focus, and physiological indicators, providing a rich and accurate information foundation for subsequent data processing and intelligent analysis. After unified time synchronization and preliminary format conversion, the collected raw data is transmitted to the data processing unit 05 for the next step of data cleaning, feature extraction, and modeling analysis.

[0071] According to some embodiments, the data acquisition unit 01 is further configured with a digital-to-analog conversion subunit 0102. The digital-to-analog conversion subunit 0102 samples, quantizes, and digitizes the analog signals from the flexible thin-film pressure sensor and the eye tracker to obtain the raw data information on different driving interaction interfaces. In actual driving interaction, the raw signals output by the flexible thin-film pressure sensor and the eye tracker are mostly continuously changing analog quantities, such as voltage and current physical signals. These signals cannot be directly recognized and used by subsequent data processing and analysis modules. Therefore, the function of the digital-to-analog conversion subunit 0102 is to sample, quantize, and digitize these analog signals to form standard format digital signals for transmission and storage.

[0072] According to some embodiments, in the entire system, the data acquisition unit 01 acts as the "sensing front end," and its deployment method and the selection of sensor 0101 directly affect the accuracy and completeness of subsequent analysis results. Therefore, in practical applications, the rationality of the sensor 0101 arrangement, the stability of data acquisition, and the degree of impact on the driver's normal operation should be fully considered to ensure high-precision perception of driving behavior and psychophysiological state without interfering with the driving experience.

[0073] According to some embodiments, the data processing unit 05 receives the raw data information from the data acquisition unit 01 and processes the data to obtain processed raw data. It then extracts features from the raw data and transmits the extracted related features to the artificial intelligence analysis unit 07. As a key intermediate module in the dynamic human factors driving analysis device, the data processing unit 05 undertakes the important tasks of cleaning, organizing, and extracting features from the raw collected data.

[0074] According to some embodiments, the data processing unit 05 is configured to: receive the raw data information from the data acquisition unit 01 and perform data processing to obtain processed raw data. The data processing includes: filtering, interpolation, and normalization. Based on the human factors analysis requirements of different application scenarios, the raw data is used to extract features related to the application scenario to obtain corresponding associated features. Since raw data often suffers from noise interference, inconsistent sampling frequencies, and time asynchrony, the data processing unit 05 first performs necessary preprocessing operations on this raw data information, including filtering to eliminate abnormal fluctuations introduced during the acquisition process by the sensor 0101; interpolation to fill in potentially missing data points; and normalization to unify data from different sources to the same numerical range or dimension, thereby improving the consistency and accuracy of subsequent analysis.

[0075] According to some embodiments, after completing basic data cleaning, the data processing unit 05 further performs a feature extraction step to identify and extract key features reflecting the driver's state from the raw data. These features include psychological aspects such as attention level and emotional changes, physiological aspects such as heart rate fluctuations and fatigue levels, and behavioral aspects such as operational standardization and reaction speed. By setting specific feature extraction algorithms, such as time-domain statistics, frequency-domain analysis, and image feature extraction, the system can extract representative related features from a large amount of raw data. Finally, the data processing unit 05 integrates and formats these extracted related features and transmits them to the artificial intelligence analysis unit 07, providing high-quality input data for subsequent intelligent modeling, state recognition, and risk assessment.

[0076] According to some embodiments, the artificial intelligence analysis unit 07 receives the associated features from the data processing unit 05, analyzes them through a multimodal fusion neural network model, obtains analysis results and related prompts and / or human factors driving analysis reports, and transmits them to the human-machine interface 03.

[0077] The artificial intelligence analysis unit 07 is configured to construct a multimodal integrated analysis neural network. This network integrates a multimodal fusion neural network, association rule mining, sequence pattern mining, cluster analysis, structural equation modeling, time series analysis, and causal graph modeling. Specifically, the multimodal fusion neural network comprises a convolutional neural network, a long short-term memory network, and a sequence model network. In its implementation, the artificial intelligence analysis unit 07 employs a multimodal fusion neural network (MFNN) model to jointly model data features from different sensor channels 0101. This model effectively integrates visual information (such as eye movement trajectory), spatial pressure distribution (such as hand grip strength and posture changes), and time series signals (such as heart rate trends and operating rhythm), thereby providing a more comprehensive understanding and identification of the driver's true state. Generally, the multimodal fusion neural network combines various deep learning structures for in-depth analysis of the associated features. For example, it combines convolutional neural networks (CNNs) to extract local features in the spatial dimension (such as operational patterns in stress distribution images); it combines temporal models such as long short-term memory networks (LSTM) or Transformers to capture dynamic changes in the temporal dimension (such as trends in eye movement frequency or fluctuations in heart rate); and it incorporates attention mechanisms to help the model focus on the most discriminative features from multimodal inputs, improving recognition accuracy. Furthermore, it can integrate traditional data analysis methods, such as association rule mining, cluster analysis, and causal graph models, to enhance the model's interpretability and reasoning capabilities.

[0078] Through the aforementioned technical means, the AI ​​analysis unit 07 can generate analysis results including driving safety assessments, risk level judgments, and operational compliance scores, and further form relevant guidance prompts or complete human factor driving analysis reports. For example, when it detects signs of decreased driver attention or fatigue, the system can automatically generate a real-time prompt such as "Current attention level is low, please take a rest." After completing a full driving process, it can also output a detailed analysis report including dimensions such as psychological state, physiological reactions, and operational behavior, providing drivers with intuitive, timely, and personalized feedback to help them understand their own state and make corresponding adjustments, thereby improving driving safety and operational efficiency. Simultaneously, it also provides solid data support and a technical foundation for subsequent driving behavior research, training evaluation, and optimization of vehicle intelligent assistance systems.

[0079] According to some embodiments, the artificial intelligence analysis unit 07 can be further configured to: pre-train the multimodal comprehensive analysis neural network using pre-collected standard safe driving reaction data to establish a safe driving reaction benchmark; identify and analyze driving features in the associated features using the multimodal comprehensive analysis neural network to obtain the driver's current operation data; compare the operation data with the safe driving reaction benchmark in real time to obtain real-time safe driving analysis results and related prompts, and transmit them to the human-machine interface 03. The safe driving reaction benchmark is formed by deep learning modeling of a large number of data samples of standardized and safe driving behaviors, covering typical patterns of drivers in multiple dimensions such as operating force, reaction time, eye movement trajectory, and pressure distribution during normal driving. On this basis, the artificial intelligence analysis unit 07 uses the trained multimodal comprehensive analysis neural network to identify and analyze driving behavior-related features in the associated features from the data processing unit 05, including but not limited to steering wheel control stability, brake response speed, gaze focus change trend, and pressure distribution patterns of hands and feet, which can comprehensively reflect the driver's actual operating state during the current driving process.

[0080] Subsequently, the system compares the extracted driver's current operation data with preset safe driving reaction benchmarks in real time. This comparative analysis identifies whether the driver deviates from the standard driving mode, such as slow operation, inattention, or uncoordinated movements. Based on the degree of deviation, the system determines the risk level of the current driving behavior and generates corresponding safe driving analysis results and prompts, such as "Current steering operation is unstable; please pay attention to steering control" or "Inattention detected; it is recommended to take a break when appropriate." Finally, the analysis results and prompts are transmitted to the human-machine interface 03, presented to the driver in a visual or voice prompt manner, helping them to promptly perceive their own state and take appropriate adjustment measures. This not only enhances the system's intelligent early warning capabilities but also provides technical support for personalized driving assistance, behavior correction training, and risk prevention.

[0081] According to some embodiments, the artificial intelligence analysis unit 07 can be further configured to: based on the safe driving analysis results and related prompts, perform correlation and causal analysis on the psychological and physiological features in the associated features using the multimodal comprehensive analysis neural network, obtain the human-cause driving analysis report, and transmit it to the human-computer interaction interface 03. Specifically, based on the safe driving analysis results and related prompts, the system uses a multimodal comprehensive analysis neural network to perform deeper modeling and analysis on the associated features from the data processing unit 05. The focus is on key indicators such as the driver's psychological state (e.g., level of attention, emotional fluctuations, cognitive load) and physiological state (e.g., fatigue level, heart rate variability, pupillary changes). By introducing correlation analysis and causal reasoning mechanisms, the artificial intelligence analysis unit 07 can identify the synergistic change patterns between different psychological and physiological features and further determine how these changes affect actual driving operations. For example, if it detects that the driver's attention decreases while accompanied by a slowed heart rate and increased blinking frequency over a certain period, it infers that the driver is in a state of mild fatigue, which may lead to delayed operational reactions or unstable directional control. Based on the above analysis results, the system automatically generates a structured human factors driving analysis report, which includes an assessment of the driver's behavior during the driving process, as well as a detailed record of the changing trends of their psychological and physiological states and their potential influencing factors.

[0082] According to some embodiments, the human factors driving analysis report will be transmitted to the human-computer interaction interface 03 and presented to the driver or relevant management personnel in the form of a combination of graphics and text or voice broadcast, so as to help them fully understand their own status, optimize driving behavior, and provide scientific basis and technical support for subsequent behavioral intervention, driving training or health management.

[0083] According to some embodiments, the artificial intelligence analysis unit 07 can be further configured to: perform fatigue driving analysis based on the associated features, generate hazard alarm information, and send it to the human-machine interface 03 to send an alarm to the driver. Based on the associated features obtained from the data processing unit 05, identification and analysis of fatigue driving state are carried out. These associated features include, but are not limited to, the driver's eye movement behavior (such as blinking frequency, eye closure duration, and gaze drift), physiological indicators (such as heart rate variability and skin conductance), and operational behavior (such as steering wheel control stability and pedal response delay), which can comprehensively reflect the driver's alertness and cognitive state during driving. Through a multimodal comprehensive analysis neural network model, the associated features are fused and modeled, and combined with preset fatigue recognition rules and dynamic threshold judgment mechanisms, the system assesses in real time whether the driver has signs of fatigue. For example, when multiple consecutive long periods of eye closure, abnormal head posture, or significantly irregular steering wheel operation are detected, the system will determine it as a potential fatigue state and classify the risk level according to the severity. Once a medium- to high-risk fatigue driving behavior is identified, the artificial intelligence analysis unit 07 will immediately generate corresponding danger alarm information and send it to the human-machine interface 03. The alarm will be issued to the driver in a timely manner through visual prompts (such as warning icons and color changes), sound alarms (such as voice reminders and beeps), or tactile feedback (such as seat vibration), prompting the driver to increase vigilance or take rest measures.

[0084] According to some embodiments, the device further includes a warning unit, which receives the associated features from the data processing unit 05, generates hazard alarms and warning information through a time-based prediction model, and sends it to the human-machine interface 03 to send alarms and warnings to the driver. In specific implementations, the warning unit employs a time-based prediction model, such as a Long Short-Term Memory (LSTM) network, a Transformer, or other deep learning structures with temporal modeling capabilities. These models excel at capturing the patterns of data evolution over time and can predict potential anomalies that may occur in the near future based on the current state, such as impending decline in attention, increased probability of operational errors, or sudden fatigue events.

[0085] According to some embodiments, based on the prediction results, the early warning unit generates corresponding danger alarm information and graded warning prompts, such as "Attention is expected to decrease significantly within 30 seconds, please remain vigilant" or "There is a potential risk of fatigue, it is recommended to rest soon." This information is divided into different priorities according to the risk level and sent to the human-machine interface 03 through visual interfaces, voice broadcasts, or haptic feedback, thereby achieving timely reminders and guidance for the driver. The early warning unit is an important component of the entire system for achieving active safety control, further enhancing the intelligent response capability of the dynamic human-factor driving analysis device in complex driving environments.

[0086] According to some embodiments, the warning unit receives associated features extracted from the data processing unit 05. These features not only cover the driver's operational behaviors (such as steering wheel operation and pedal usage rhythm), but also their psychological state (such as attention level and cognitive load) and physiological reactions (such as heart rate changes and fatigue level). Through these multi-dimensional data inputs, the warning unit can comprehensively perceive the driver's current behavioral patterns and potential risk trends. The warning unit adopts a forward-looking risk identification mechanism, enabling the device to move beyond passively responding to abnormalities that have already occurred, and instead intervene before risks manifest, effectively improving driving safety and reducing the likelihood of accidents. Simultaneously, it provides key technical support for building a more intelligent and user-friendly driver assistance system.

[0087] Figure 2 The illustration shows a schematic diagram of a dynamic human factors driving analysis process according to an example embodiment.

[0088] See Figure 2 In S101, data is acquired through sensors, using various sensors (such as flexible thin-film pressure sensor arrays and eye trackers) to collect multimodal data in real time, including the driver's operational behavior, visual attention, and physiological state. This data forms the basis for subsequent analysis.

[0089] In S103, data transformation and preprocessing are performed. The collected raw data typically undergoes preprocessing steps such as format conversion, noise removal, and standardization to ensure data quality and consistency, providing reliable data support for subsequent feature extraction and model analysis.

[0090] In step S105, feature extraction is performed to obtain the associated features. Key features, such as operational force, reaction time, eye movement trajectory, and heart rate changes, are extracted from the preprocessed data. These features reflect the driver's current state and behavioral patterns, serving as an important basis for intelligent analysis.

[0091] In S107, multi-model analysis is combined with different research directions. Deep learning models (such as CNN, LSTM, etc.) are used to comprehensively analyze the extracted features, including safety assessment of driving behavior, as well as correlation and causal analysis of psychological and physiological characteristics, thereby generating a comprehensive human factors driving analysis report.

[0092] In S109, further dangerous dummy operation alarms and warnings are issued. Based on the analysis results, the system predicts potential risks using a time-based predictive model (such as LSTM) and generates corresponding hazard alarm information and warning prompts. This information is classified according to risk level and sent to the driver in a timely manner to help them take measures in advance to avoid accidents.

[0093] In S111, the human-machine interface provides feedback and prompts. Ultimately, the analysis results, alarm information, and warning prompts are presented to the driver through the human-machine interface. The interface can use various methods such as visual charts, voice broadcasts, or haptic feedback to ensure that the driver can clearly and intuitively receive the system's feedback, thereby adjusting their driving behavior and improving driving safety.

[0094] The above processes are interconnected to form a closed-loop system. From data collection to final feedback, each link is closely connected, together constituting an intelligent and personalized driving assistance and safety management system.

[0095] According to some embodiments, the technical solution of the present invention can also be applied to the design of a dynamic human factor driving analysis system or an intelligent driving system. The system includes the device described above, which can realize comprehensive perception and intelligent analysis of the driver's psychological, physiological and driving behavior states, thereby improving driving safety.

[0096] According to some embodiments, the technical solution of the present invention achieves comprehensive perception and intelligent judgment of the driver's state by integrating multi-sensor acquisition, data processing, artificial intelligence analysis, and early warning mechanisms. By collecting driver's operational behavior, visual attention, and physiological state through multi-modal sensors such as flexible thin-film pressure sensor arrays and eye trackers, high-risk behaviors such as fatigued driving, inattentiveness, and improper operation can be accurately identified. By adding the aforementioned early warning unit combined with a time-based prediction model (such as LSTM), potential dangers can be predicted in advance and warnings can be issued, realizing a shift from "post-event response" to "pre-event early warning," effectively reducing the traffic accident rate.

[0097] According to some embodiments, by applying the multimodal fusion neural network and combining it with deep learning structures such as CNN and LSTM to comprehensively analyze psychological, physiological, and behavioral characteristics, the accuracy and robustness of state recognition are improved. By modeling the relationship between psychological and physiological characteristics, the intrinsic factors affecting driving safety are deeply explored, providing a basis for personalized driving assistance. Through the synergistic effect of multimodal data acquisition, intelligent feature extraction, deep learning modeling, and early warning mechanisms, comprehensive perception and accurate recognition of the driver's psychological, physiological, and behavioral states are achieved, significantly improving driving safety and intelligence levels, and possessing excellent user experience and broad application prospects.

[0098] Figure 3 A block diagram of a computing device according to an example embodiment of this application is shown.

[0099] like Figure 3As shown, the computing device 30 includes a processor 12 and a memory 14. The computing device 30 may also include a bus 22, a network interface 16, and an I / O interface 18. The processor 12, memory 14, network interface 16, and I / O interface 18 can communicate with each other via the bus 22.

[0100] Processor 12 may include one or more general-purpose CPUs (Central Processing Units), microprocessors, or application-specific integrated circuits, for executing relevant program instructions. According to some embodiments, computing device 30 may also include a high-performance display adapter (GPU) 20 for accelerating processor 12.

[0101] Memory 14 may include a machine system readable medium in the form of volatile memory, such as random access memory (RAM), read-only memory (ROM), and / or cache memory. Memory 14 is used to store one or more programs containing instructions, as well as data. Processor 12 may read the instructions stored in memory 14 to perform the methods described above according to embodiments of this application.

[0102] The computing device 30 can also communicate with one or more networks via the network interface 16. The network interface 16 can be a wireless network interface.

[0103] Bus 22 can include address bus, data bus, control bus, etc. Bus 22 provides a path for exchanging information between components.

[0104] It should be noted that, in specific implementations, the computing device 30 may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the device described above may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0105] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method. The computer-readable storage medium may include, but is not limited to, any type of disk, including floppy disks, optical disks, DVDs, CD-ROMs, microdrives, as well as magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic cards or optical cards, nanosystems (including molecular memory ICs), network storage devices, cloud storage devices, or any type of medium or device suitable for storing instructions and / or data.

[0106] This application also provides a computer program product including a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps of any of the methods described in the above method embodiments.

[0107] Those skilled in the art will clearly understand that the technical solutions of this application can be implemented using software and / or hardware. In this specification, "unit" and "module" refer to software and / or hardware capable of independently performing or cooperating with other components to perform a specific function, where the hardware may be, for example, a field-programmable gate array (FPGA), integrated circuit, etc.

[0108] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0109] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0110] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some service interface; the indirect coupling or communication connection between apparatuses or units may be electrical or other forms.

[0111] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0112] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0113] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application.

[0114] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0115] The exemplary embodiments of this application have been specifically shown and described above. It should be understood that this application is not limited to the detailed structures, arrangements, or implementation methods described herein; rather, this application is intended to cover various modifications and equivalent arrangements contained within the spirit and scope of the appended provisions.

Claims

1. An apparatus for dynamic human factors driving analysis, characterized by, The device is used for a driving simulator and includes: a data acquisition unit, a data processing unit, an artificial intelligence analysis unit, and a human-machine interface. The data acquisition unit collects data through sensors deployed on various driving interaction interfaces, obtains raw data information from different driving interaction interfaces, and sends it to the data processing unit. The driving interaction interface includes a steering wheel, pedals, seat, and central control area, thereby covering the driver's action input points during the driving process. The data processing unit receives the raw data information from the data acquisition unit and processes the data to obtain processed raw data. It then extracts features from the raw data and transmits the extracted related features to the artificial intelligence analysis unit. The artificial intelligence analysis unit receives the associated features from the data processing unit, analyzes them through a multimodal integrated analysis neural network, obtains the analysis results, compares the analysis results with the preset safe driving reaction benchmark in real time, judges the risk level of the current driving behavior based on the degree of deviation from the standard driving mode, and generates relevant prompt information and / or human factor driving analysis report and transmits it to the human-machine interface. The analysis results reflect the driver's actual operating status in the current driving process and include steering wheel control stability, brake response speed, gaze focus change trend, and pressure distribution pattern of hands and feet. The human-computer interaction interface receives the analysis results, related prompts, and / or human-cause driving analysis reports from the artificial intelligence analysis unit, and displays and / or prompts them on the human-computer interaction interface. The sensor includes a flexible thin-film pressure sensor array, allowing the device to collect data without affecting the driver's perception. The associated features include the driver's psychological characteristics, physiological characteristics, and driving behavior characteristics. The safe driving reaction benchmark is established by pre-training the multimodal integrated analysis neural network using pre-collected standard safe driving reaction data.

2. The apparatus according to claim 1, characterized in that, The device also includes: an early warning unit, The early warning unit receives the associated features from the data processing unit, generates danger alarms and early warning information through a time-based prediction model, and sends them to the human-machine interface to send alarms and warnings to the driver.

3. The apparatus according to claim 1, characterized in that, The sensor also includes an eye tracker, through which raw data information is collected.

4. The apparatus according to claim 2, characterized in that, The data acquisition unit further includes an analog-to-digital conversion subunit, which samples, quantizes, and digitizes the analog signals from the flexible thin-film pressure sensor and the eye tracker to obtain the original data information on different driving interaction interfaces.

5. The apparatus according to claim 1, characterized in that, The data processing unit is configured as follows: The system receives the raw data information from the data acquisition unit and processes the data to obtain processed raw data. The data processing includes filtering, interpolation, and normalization. Based on the human factors analysis requirements of different application scenarios, the original data is subjected to feature extraction related to the application scenario to obtain corresponding associated features.

6. The apparatus according to claim 1, characterized in that, The artificial intelligence analysis unit is configured as follows: The multimodal comprehensive analysis neural network is constructed, which integrates association rule mining, sequence pattern mining, cluster analysis, structural equation modeling, time series analysis modeling, and causal graph modeling. The multimodal comprehensive analysis neural network is a convolutional neural network, a long short-term memory network, and a sequence model network.

7. The apparatus according to claim 1, characterized in that, The artificial intelligence analysis unit is also configured to: Based on the analysis results and related prompts, the multimodal integrated analysis neural network performs correlation and causal analysis on the psychological and physiological features in the associated features to obtain the human-cause driving analysis report and transmits it to the human-computer interaction interface.

8. The apparatus according to claim 1, characterized in that, The artificial intelligence analysis unit is also configured to: Based on the associated features, fatigue driving analysis is performed, hazard alarm information is generated and sent to the human-machine interface to send an alert to the driver.

9. A system for dynamic human factors driving analysis, characterized in that, The system includes the apparatus as described in any one of claims 1-8.

Images