Driving state identification intervention method and system

Abstract

The invention provides a driving state recognition intervention method and system. The method comprises the steps that at least two sets of electromyographic signals of different parts of a driver are collected; after the electromyographic signals are preprocessed, time domain features and frequency domain features of the electromyographic signals are extracted, and real-time signal features are obtained; the real-time signal features are compared with the personal feature baseline value, the comparison result and the real-time signal features are input into the trained classification model, and the current state of the driver is judged; the personal feature baseline value is a personal electromyographic feature baseline value calculated based on the acquired electromyographic signals when the driver is in an initial normal driving state; and executing a corresponding intervention strategy according to the current state of the driver. According to the scheme, the specific distraction state of the driver can be accurately distinguished, and the real-time performance of driver state recognition is guaranteed.

Description

Technical Field

[0001] This invention belongs to the field of intelligent driving technology, and in particular relates to a driving state recognition and intervention method and system. Background Technology

[0002] With the rapid development of intelligent driving technology, driver status monitoring has become a core element in improving road safety. Traditional methods often rely on monitoring the driver's facial features (such as eyelids and head posture), steering wheel operation behavior, or basic physiological signals such as heart rate to determine fatigue or distraction. However, these methods have significant limitations. For example, facial monitoring is easily affected by lighting and occlusion, behavioral analysis is lagging, and heart rate-based monitoring is difficult to accurately distinguish between different types of psychological states (such as tension and fatigue).

[0003] Currently, electromyography (EMG) signals are being used for driver monitoring because they can directly reflect the activation state and intention of neuromuscular muscles. For example, some publicly available technical solutions (publication number CN 114081513A) propose acquiring EMG data through EMG sensors and then using neural network models to determine whether the driver is exhibiting abnormal behavior. While such solutions can detect and identify abnormal driving states, they cannot accurately distinguish the driver's specific distraction state from signals that are similar in amplitude, such as "weakness due to fatigue" and "stiffness due to tension," making it difficult to implement personalized intervention. Furthermore, these solutions still suffer from real-time bias in driver state recognition. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a driving state recognition intervention method and system to solve the problem that current driving monitoring cannot accurately distinguish the specific distraction state of the driver and has real-time deviation.

[0005] In a first aspect of the present invention, a driving state recognition intervention method is provided, comprising: At least two sets of electromyographic signals from different parts of the driver's body should be collected; After preprocessing the electromyographic signal, the time-domain and frequency-domain features of the electromyographic signal are extracted to obtain the real-time signal features; The real-time signal features are compared with the baseline values ​​of individual features, and the comparison results and real-time signal features are input into the trained classification model to determine the driver's current state. The personal characteristic baseline value is the personal electromyographic characteristic baseline value calculated by the driver based on the collected electromyographic signals under the initial normal driving state. Execute the appropriate intervention strategy based on the driver's current condition.

[0006] In a second aspect of the present invention, a driving state recognition and intervention system is provided, comprising: The signal acquisition module is used to acquire at least two sets of electromyographic signals from different parts of the driver's body. The feature extraction module is used to extract the time-domain and frequency-domain features of the electromyographic signal after preprocessing, so as to obtain the real-time signal features. The state recognition module is used to compare real-time signal features with personal feature baseline values, and input the comparison results and real-time signal features into the trained classification model to determine the driver's current state. The personal characteristic baseline value is the personal electromyographic characteristic baseline value calculated by the driver based on the collected electromyographic signals under the initial normal driving state. The intervention execution module is used to execute corresponding intervention strategies based on the driver's current state.

[0007] In a third aspect of the present invention, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the processor executes the computer program to implement the steps of the method as described in the first aspect of the present invention.

[0008] In a fourth aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the method provided in the first aspect of the present invention.

[0009] In this embodiment of the invention, by directly monitoring muscle signals most relevant to driving control, features of electromyography (EMG) signals are extracted in two dimensions, and a personal feature baseline is introduced for comparison. Through a trained classification model, the driver's current state is determined, and corresponding intervention strategies are executed. This not only accurately distinguishes the driver's distracted state based on differences in time-domain frequency EMG signals, thus enabling personalized intervention, but also effectively improves the real-time performance of driving state recognition based on the comparison of personal feature baselines. Furthermore, by establishing a personal EMG baseline, the system can adapt to the muscle activity characteristics of different drivers, effectively improving the accuracy and reliability of the judgment. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a flowchart illustrating a driving state recognition and intervention method according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a driving state recognition and intervention system according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of an electronic device provided in one embodiment of the present invention. Detailed Implementation

[0012] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0013] It should be understood that the terms "comprising" and other similar expressions in the specification, claims, and accompanying drawings of this invention are intended to cover a non-exclusive inclusion, such as a process, method, system, or apparatus that includes a series of steps or units and is not limited to the listed steps or units. Furthermore, "first" and "second" are used to distinguish different objects and are not intended to describe a specific order.

[0014] Please see Figure 1 A flowchart illustrating a driving state recognition and intervention method provided in this embodiment of the invention includes: S101. Collect at least two sets of electromyographic signals from different parts of the driver's body; Electromyography (EMG) sensors are used to collect EMG signals from different parts of the driver's body, and the driver's driving status can be determined based on these EMG signals.

[0015] At least the driver's hand electromyography (EMG) signal and neck EMG signal were collected.

[0016] A surface electromyography (EMG) sensor is positioned in the driver's steering wheel grip area. The EMG sensor uses a flexible electrode patch embedded in the steering wheel grip to collect hand EMG signals. A neck EMG sensor is integrated into the front of the seat headrest, contacting the back of the driver's neck, to collect neck EMG signals. Two sets of EMG signals can be obtained through non-invasive acquisition of EMG signals using surface electrodes.

[0017] S102. After preprocessing the electromyographic signal, extract the time-domain and frequency-domain features of the electromyographic signal to obtain the real-time signal features; Preprocessing of electromyographic signals can include amplification, filtering, and interference removal. Signal preprocessing can improve signal quality and eliminate noise interference.

[0018] Temporal features refer to the characteristics of a signal in the time dimension, while frequency domain features refer to the characteristics of a signal in the frequency dimension. Extracting the temporal and frequency domain features of electromyographic signals allows us to characterize the driver's muscle activation level and fatigue state, respectively.

[0019] S103. Compare the real-time signal features with the baseline values ​​of personal features, and input the comparison results and real-time signal features into the trained classification model to determine the driver's current state; The personal characteristic baseline value is the personal electromyographic characteristic baseline value calculated based on the collected electromyographic signals when the driver is in an initial normal driving state.

[0020] The real-time feature vector is compared with an individual feature baseline. The comparison result and signal features are then input into a trained classification model to determine the driver's current state in real time. The classification model can be a random forest, neural network, etc. The input real-time signal features can include corresponding data source labels (such as neck, hands) and preliminary judgment labels corresponding to the comparison results (e.g., suspected fatigue relaxation, suspected tension stiffness).

[0021] By comparing the deviation of signal features from their individual dynamic baseline in real time, a rapid initial screening can be achieved. Real-time signal features can reflect the driver's muscle strength and fatigue level. After comparing the signal features with the individual feature baseline, the data is input into a classification model, which outputs a probability distribution of the driver's state.

[0022] The driver's current state or driving state can include normal, fatigued and relaxed state, and tense and stiff state.

[0023] The personal characteristic baseline value is calculated by collecting electromyographic signals from the driver's hands and neck during the driver's normal focused driving phase. After the driver gets into the vehicle, the system can default to learning mode during the first 10 minutes of normal driving, collecting and calculating the driver's electromyographic characteristic baseline.

[0024] In this embodiment, an individual electromyography baseline is established, enabling the system to adapt to the muscle activity characteristics of different drivers, thereby improving the accuracy and reliability of the judgment.

[0025] S104. Execute the corresponding intervention strategy based on the driver's current state.

[0026] It stores a mapping table of different driver states and intervention strategies. When the driver's current state is determined, it generates corresponding control commands and executes specific intervention operations.

[0027] Optionally, if the driver is currently in a state of fatigue and relaxation, the air conditioning can be controlled to blow cold air in a specific direction and / or play music of a predetermined type. If the driver is currently in a tense or stiff state, adjust the air conditioning temperature, seat temperature, or play the corresponding voice guidance prompts.

[0028] If the driver is identified as being in a state of fatigue-induced relaxation, the air conditioning system will direct cold air in a specific direction and may automatically play specific types of music (such as upbeat music) to enhance the driver's alertness. If the driver is identified as being in a state of tension-induced stiffness, the air conditioning or seat will direct hot air and may play synthesized or pre-recorded guiding voice messages (such as "We suggest you take two deep breaths and relax your shoulders") to relieve muscle and psychological tension.

[0029] In this embodiment, the driver's electromyographic signals are monitored by non-invasive surface electrodes. By comprehensively utilizing the time-domain features that represent intensity and the frequency-domain features that represent quality (fatigue), the algorithm can clearly separate the two states, "weakness due to fatigue" and "stiffness due to tension," which have similar amplitudes but completely opposite physiological causes. This ensures the accuracy of driver state recognition and enables the execution of personalized intervention strategies to improve the intervention effect.

[0030] It directly monitors the electrophysiological activity of the hands and neck muscles, which are most relevant to driving control and reflect overall tension. The signals are direct and objective, ensuring accurate judgment and strong anti-interference capabilities. At the electromyographic signal level, it clearly defines and distinguishes two distinct sub-states of distraction: "fatigue-induced relaxation" and "tension-induced stiffness," achieving in-depth differentiation of distraction types. It establishes an individual electromyographic baseline, enabling the system to adapt to the muscle activity characteristics of different drivers, achieving both accurate judgment and improved classification model efficiency. Based on physiological principles (the effects of hot and cold stimuli on the nervous system) and psychological principles, the intervention method is more humane and targeted, with better results than a single alarm or music switch.

[0031] In one embodiment, step S102, extracting the time-domain and frequency-domain features of the electromyographic signal, includes: Extract the integral electromyography (EMG) value and root mean square (RMS) value of the time-domain EMG signal, extract the median frequency of the frequency-domain EMG signal, and calculate the complexity of the frequency-domain EMG signal.

[0032] We extract the integral electromyography (iEMG) and root mean square amplitude (RMS) in the time domain to represent intensity, and the intermediate frequency (MF) and complexity in the frequency domain to represent quality. Integral EMG reflects the total electrical discharge of a muscle over a certain period of time, the RMS value reflects the degree of muscle activation, and the median frequency and complexity of the signal can reflect the state of muscle fatigue.

[0033] Optionally, if the integrated electromyography (EMG) value and root mean square value of the EMG signal are consistently lower than the individual characteristic baseline value, and the median frequency of the EMG signal shows a monotonically decreasing trend, then the driver's current state is initially marked as suspected fatigue relaxation. If the integrated electromyography (EMG) value and root mean square (RMS) value of the EMG signal are consistently higher than the individual's baseline values, and the complexity of the EMG signal is consistently lower than the individual's baseline values, then the driver's current state is initially labeled as suspected tension-related rigidity. Based on the preliminary category labels, the classification model can quickly determine the corresponding EMG feature signals.

[0034] It is understandable that introducing a comparison between personal characteristic baseline values ​​and real-time signal characteristics can enable a preliminary judgment of signal characteristics. Inputting the comparison results into the classification model can further improve the efficiency of classification judgment, thereby ensuring the real-time performance of driver status recognition.

[0035] Optionally, real-time signal features with suspected labels are obtained, and the real-time signal features with suspected labels are input into the trained classification model to output the discrimination result with probability distribution. If a driving state with a probability distribution that is consistently greater than a predetermined value exists in the judgment results, then the corresponding driving state will be taken as the driver's current state.

[0036] The trained classification model uses the suspected labels from the initial screening as input features, combined with the real-time input feature vectors, to output triples containing the probability distribution of the categories. For example, a triple of [Normal: 0.10, Fatigue-Relaxed: 0.85, Tension-Stiff: 0.05] indicates an 85% confidence level that the current state is one of fatigue-relaxed.

[0037] A judgment is considered valid only when the probability value of the highest-probability category exceeds a preset threshold (e.g., >0.80). Low-confidence results are considered "uncertain," and the system maintains the previous state. To prevent misjudgments caused by instantaneous actions, a high-confidence state must be maintained continuously for at least one time window (e.g., 3 seconds) before it is finally confirmed as the current driving state.

[0038] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0039] Figure 2 This is a schematic diagram of a driving state recognition and intervention system provided in an embodiment of the present invention. The system includes: Signal acquisition module 210 is used to acquire at least two sets of electromyographic signals from different parts of the driver's body; At least the driver's hand electromyography (EMG) signal and neck EMG signal were collected.

[0040] The feature extraction module 220 is used to extract the time-domain and frequency-domain features of the electromyographic signal after preprocessing the electromyographic signal to obtain real-time signal features; Preferably, the integral electromyography (EMG) value and root mean square (RMS) value of the time-domain EMG signal are extracted, the median frequency of the frequency-domain EMG signal is extracted, and the complexity of the frequency-domain EMG signal is calculated.

[0041] The state recognition module 230 is used to compare real-time signal features with personal feature baseline values, and input the comparison results and real-time signal features into the trained classification model to determine the driver's current state. The personal characteristic baseline value is the personal electromyographic characteristic baseline value calculated by the driver based on the collected electromyographic signals under the initial normal driving state. The intervention execution module 240 is used to execute the corresponding intervention strategy according to the driver's current state.

[0042] In one embodiment, if the integrated electromyography (EMG) value and root mean square (RMS) value of the EMG signal are consistently lower than the baseline value of the individual characteristics, and the median frequency of the EMG signal shows a monotonically decreasing trend, then the driver's current state is initially marked as suspected fatigue relaxation. If the integrated electromyography (EMG) value and root mean square value of the EMG signal are consistently higher than the individual characteristic baseline value, and the complexity of the EMG signal is consistently lower than the individual characteristic baseline value, then the driver's current state is initially marked as suspected tension-related stiffness.

[0043] Optionally, the state recognition module 230 includes: The feature input unit is used to acquire real-time signal features with suspected labels, input the real-time signal features with suspected labels into the trained classification model, and output the discrimination result with probability distribution. The state discrimination unit is used to determine the current state of the driver when there is a driving state in the discrimination result whose probability distribution is continuously greater than a predetermined value.

[0044] Optionally, the intervention execution module 240 includes The fatigue intervention unit is used to control the air conditioning to blow cold air in a directional manner and / or play a predetermined type of music if the driver's current state is one of fatigue relaxation. The tension intervention unit is used to adjust the air conditioning temperature, seat temperature, or play corresponding voice guidance prompts if the driver is currently in a tense or stiff state.

[0045] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the system and modules described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0046] Figure 3 This is a schematic diagram of an electronic device according to an embodiment of the present invention. The electronic device is used for driving status recognition. Figure 3As shown, the electronic device 30 of this embodiment includes: a memory 310, a processor 320, and a system bus 330. The memory 310 includes an executable program 3101 stored thereon. As those skilled in the art will understand, Figure 3 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0047] The following is combined Figure 3 A detailed description of each component of the electronic device 30 is provided below: The memory 310 can be used to store software programs and modules. The processor 320 executes various functional applications and data processing of the electronic device by running the software programs and modules stored in the memory 310. The memory 310 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device (such as cached data), etc. In addition, the memory 310 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0048] The memory 310 contains an executable program 3101 for a driving state recognition method. This executable program 3101 can be divided into one or more modules / units, which are stored in the memory 310 and executed by the processor 320 to perform driving state determination, intervention, etc. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, describing the execution process of the executable program 3101 in the electronic device 30. For example, the executable program 3101 can be divided into functional modules such as a signal acquisition module, a feature extraction module, a state recognition module, and an intervention execution module.

[0049] The processor 320 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 310, and by calling data stored in the memory 310, it performs various functions and processes data, thereby monitoring the overall status of the electronic device. Optionally, the processor 320 may include one or more processing units; preferably, the processor 320 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, application programs, etc., and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into the processor 320.

[0050] The system bus 330 is used to connect various functional components within the computer, transmitting data, address, and control information. Its type can be, for example, a PCI bus, an ISA bus, or a CAN bus. Instructions from the processor 320 are transmitted to the memory 310 via the bus, and the memory 310 sends data back to the processor 320. The system bus 330 is responsible for data and instruction exchange between the processor 320 and the memory 310. Of course, the system bus 330 can also connect to other devices, such as network interfaces and display devices.

[0051] In this embodiment of the invention, the executable program executed by the processor 320 included in the electronic device includes: At least two sets of electromyographic signals from different parts of the driver's body should be collected; After preprocessing the electromyographic signal, the time-domain and frequency-domain features of the electromyographic signal are extracted to obtain the real-time signal features; The real-time signal features are compared with the baseline values ​​of individual features, and the comparison results and real-time signal features are input into the trained classification model to determine the driver's current state. The personal characteristic baseline value is the personal electromyographic characteristic baseline value calculated by the driver based on the collected electromyographic signals under the initial normal driving state. Execute the appropriate intervention strategy based on the driver's current condition.

[0052] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0053] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0054] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A driving state recognition and intervention method, characterized in that, include: At least two sets of electromyographic signals from different parts of the driver's body should be collected; After preprocessing the electromyographic signal, the time-domain and frequency-domain features of the electromyographic signal are extracted to obtain the real-time signal features; The real-time signal features are compared with the baseline values ​​of individual features, and the comparison results and real-time signal features are input into the trained classification model to determine the driver's current state. The personal characteristic baseline value is the personal electromyographic characteristic baseline value calculated by the driver based on the collected electromyographic signals under the initial normal driving state. Execute the appropriate intervention strategy based on the driver's current condition.

2. The method according to claim 1, characterized in that, The acquisition of at least two sets of electromyographic signals from different parts of the driver's body includes: At least the driver's hand electromyography (EMG) signal and neck EMG signal should be collected.

3. The method according to claim 1, characterized in that, The extracted time-domain and frequency-domain features of the electromyographic signals include: Extract the integral electromyography (EMG) value and root mean square (RMS) value of the time-domain EMG signal, extract the median frequency of the frequency-domain EMG signal, and calculate the complexity of the frequency-domain EMG signal.

4. The method according to claim 3, characterized in that, The step of comparing real-time signal features with baseline values ​​of individual features, and inputting the comparison results and real-time signal features into the trained classification model to determine the driver's current state includes: If the integrated electromyography (EMG) value and root mean square value of the EMG signal are consistently lower than the individual characteristic baseline value, and the median frequency of the EMG signal shows a monotonically decreasing trend, then the driver's current state is preliminarily marked as suspected fatigue relaxation. If the integrated electromyography (EMG) value and root mean square value of the EMG signal are consistently higher than the individual characteristic baseline value, and the complexity of the EMG signal is consistently lower than the individual characteristic baseline value, then the driver's current state is initially marked as suspected tension-related stiffness.

5. The method according to claim 4, characterized in that, The step of comparing real-time signal features with baseline values ​​of individual features, and inputting the comparison results and real-time signal features into the trained classification model to determine the driver's current state includes: The real-time signal features with suspected labels are obtained, and the real-time signal features with suspected labels are input into the trained classification model to output the discrimination result with probability distribution. If a driving state with a probability distribution that is consistently greater than a predetermined value exists in the judgment results, then the corresponding driving state will be taken as the driver's current state.

6. The method according to claim 1, characterized in that, The step of implementing the corresponding intervention strategy based on the driver's current state includes: If the driver is currently in a state of fatigue and relaxation, control the air conditioning to blow cold air in a specific direction and / or play music of a predetermined type; If the driver is currently in a tense or stiff state, adjust the air conditioning temperature, seat temperature, or play the corresponding voice guidance prompts.

7. A driving state recognition and intervention system, characterized in that, include: The signal acquisition module is used to acquire at least two sets of electromyographic signals from different parts of the driver's body. The feature extraction module is used to extract the time-domain and frequency-domain features of the electromyographic signal after preprocessing, so as to obtain the real-time signal features. The state recognition module is used to compare real-time signal features with personal feature baseline values, and input the comparison results and real-time signal features into the trained classification model to determine the driver's current state. The personal characteristic baseline value is the personal electromyographic characteristic baseline value calculated by the driver based on the collected electromyographic signals under the initial normal driving state. The intervention execution module is used to execute corresponding intervention strategies based on the driver's current state.

8. The system according to claim 7, characterized in that, The acquisition of at least two sets of electromyographic signals from different parts of the driver's body includes: At least the driver's hand electromyography (EMG) signal and neck EMG signal should be collected.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of a driving state recognition intervention method as described in any one of claims 1 to 6.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed, it implements the steps of a driving state recognition intervention method as described in any one of claims 1 to 6.

Images