Method and device for starting auxiliary driving based on driver action

Abstract

The invention relates to a method and a device for starting auxiliary driving based on actions of a driver. The method comprises the following steps: acquiring action data of a body part of the driver when a vehicle runs through a sensor; analyzing the action data to determine a target emotional state of the driver; if it is determined that the driver is in an abnormal state according to the target emotional state, auxiliary driving operation is started. According to the invention, the safety of vehicle driving is improved.

Description

Technical Field

[0001] The present application relates to the field of vehicle safety technology, and more particularly to a method and device for activating assisted driving based on driver actions. Background Art

[0002] With the rapid growth of car ownership, road traffic safety issues are becoming increasingly prominent. Driver distraction, fatigue driving, misoperation and other behaviors during driving are one of the important causes of traffic accidents. Although traditional driving assistance systems can improve driving safety to a certain extent, they mostly focus on monitoring and controlling the vehicle's own status, and do not fundamentally solve the problem from the driver's perspective, resulting in low driving safety. Summary of the invention

[0003] The present application provides a method and device for activating assisted driving based on driver actions to solve the problem of low driving safety.

[0004] In a first aspect, the present application provides a method for enabling assisted driving based on a driver's action, the method comprising:

[0005] The motion data of the driver's body parts while the vehicle is driving is obtained through sensors;

[0006] Determining a target emotional state of the driver by analyzing the motion data;

[0007] If it is determined that the driver is in an abnormal state according to the target emotional state, the assisted driving operation is started.

[0008] Optionally, obtaining the motion data of the driver's body parts when the vehicle is driving through the sensor includes:

[0009] When the vehicle is driving, the driver's hand motion data is obtained through a first sensor located on the steering wheel of the cockpit;

[0010] Acquiring hip movement data of the driver through a second sensor located on the cockpit seat;

[0011] The driver's leg movement data is obtained through a third sensor located on the cockpit floor.

[0012] Optionally, determining the target emotional state of the driver by analyzing the action data includes:

[0013] Generate an actual hand waveform graph according to the hand motion data, generate an actual hip waveform graph according to the hip motion data, generate an actual leg waveform graph according to the leg motion data, and generate an actual fusion waveform graph by combining the hand motion data, the hip motion data and the leg motion data;

[0014] Comparing the actual hand waveform with the standard hand waveform, comparing the actual hip waveform with the standard hip waveform, comparing the actual leg waveform with the standard leg waveform, and comparing the actual fusion waveform with the standard fusion waveform, to obtain four comparison results;

[0015] Determine the error rate for each comparison result;

[0016] The target emotional state of the driver is determined according to the relationship between the error rate and a preset error rate range.

[0017] Optionally, after the driver completes a set number of drivings, the method further includes:

[0018] Using the waveform graph generated by the driver each time driving as a historical waveform graph, and determining a comparison result between the historical waveform graph and a standard waveform graph, wherein the historical waveform graph is one of the actual hand waveform graph, the actual hip waveform graph, and the actual leg waveform graph;

[0019] If it is determined that the comparison result of the historical waveform graph is always a comparison failure, the historical waveform graph is used as a reference waveform graph, wherein the reference waveform graph is used to indicate that the specific action data of the driver is abnormal but does not affect the driver's emotions;

[0020] The target emotional state of the driver is determined by comparing the target waveform graph generated by the driver's second driving with the historical waveform graph, wherein the body part corresponding to the target waveform graph and the historical waveform graph is the same.

[0021] Optionally, comparing the target waveform generated by the driver's second driving with the historical waveform to determine the driver's target emotional state includes:

[0022] Comparing the target waveform generated by the driver's driving again with the historical waveform, and determining an error rate of comparison failure;

[0023] The target emotional state of the driver is determined according to the relationship between the error rate and a preset error rate range.

[0024] Optionally, determining the target emotional state of the driver according to the relationship between the error rate and a preset error rate range includes:

[0025] If the error rate of each comparison result is within the first error rate range, it is determined that the driver's emotion is in a normal state;

[0026] If there are at least two comparison results whose error rates are within the second error rate range, and the error rates of the remaining comparison results are within the first error rate range, it is determined that the driver's emotion is in a slightly abnormal state, wherein the range threshold of the second error rate range is greater than the range threshold of the second error rate range;

[0027] If there are at least two comparison results whose error rates are within a third error rate range, it is determined that the driver's emotion is in an abnormal state, wherein the range threshold of the third error rate range is greater than the range threshold of the second error rate range.

[0028] Optionally, generating an actual hand waveform diagram according to the hand motion data includes:

[0029] Determine the setting weight of each hand movement data;

[0030] Determining overall hand motion data at the target moment according to the multiple hand motion data acquired at the target moment and the set weight of each hand motion data;

[0031] The actual hand waveform diagram is generated according to each moment and the overall hand motion data at each moment.

[0032] Optionally, combining the hand motion data, the hip motion data, and the leg motion data to generate an actual fusion waveform graph includes:

[0033] Get the preset weights for each body part;

[0034] Generate fusion data at the target moment according to the hand motion data, hip motion data and leg motion data acquired at the target moment and the preset weight of each body part;

[0035] The actual fusion waveform diagram is generated according to each moment and the fusion data at each moment.

[0036] Optionally, acquiring the driver's hand motion data through a first sensor located on a steering wheel in a cockpit includes:

[0037] The vibration sensor on the steering wheel in the cockpit is used to obtain data of hand hitting the steering wheel and data of hand rubbing the steering wheel;

[0038] Acquiring palm humidity data through a humidity sensor located on the steering wheel of the cockpit;

[0039] The data of whether the hands are off the steering wheel is obtained through an infrared sensor located on the steering wheel of the cockpit.

[0040] Optionally, obtaining the driver's hip movement data through a second sensor located on the cockpit seat includes:

[0041] Acquiring hip movement data via a position sensor located on the cockpit seat;

[0042] The hip contraction data is acquired by an infrared sensor located on the cockpit seat.

[0043] Optionally, acquiring the driver's leg movement data through a third sensor located on the cockpit floor includes:

[0044] acquiring leg movement data via a position sensor located on the cockpit floor;

[0045] Acquiring stomping data via a pressure sensor located on the cockpit floor;

[0046] Leg vibration data is acquired via a vibration sensor located on the cockpit floor.

[0047] Optionally, if it is determined according to the target emotional state that the driver is in an abnormal state, starting the assisted driving operation includes:

[0048] If it is determined that the driver is in a first abnormal state according to the target emotional state, adjusting the driver's emotion through prompt information;

[0049] If it is determined that the driver is in a second abnormal state according to the target emotional state, the auxiliary driving system is turned on to perform auxiliary driving.

[0050] In a second aspect, the present application provides a device for starting assisted driving based on a driver's action, the device comprising:

[0051] An acquisition module, used for acquiring motion data of the driver's body parts when the vehicle is driving through sensors;

[0052] An analysis module, configured to determine a target emotional state of the driver by analyzing the motion data;

[0053] The activation module is used to activate the assisted driving operation if it is determined that the driver is in an abnormal state according to the target emotional state.

[0054] In a third aspect, the present application provides an electronic device comprising: at least one communication interface; at least one bus connected to the at least one communication interface; at least one processor connected to the at least one bus; and at least one memory connected to the at least one bus.

[0055] In a fourth aspect, the present application also provides a vehicle, which is used to execute any of the above-mentioned methods for starting assisted driving based on driver actions.

[0056] The above technical solution provided by the embodiment of the present application has the following advantages compared with the prior art: the motion data of the driver's key body parts are accurately captured by sensors, and then these motion data are analyzed and evaluated, so as to timely discover and identify the driver's abnormal emotional state. Finally, according to the analysis results, corresponding assisted driving operations are performed and feedback and warning information are provided to guide the driver to adopt correct driving behavior and restore normal emotional state. The present application can reduce the risk of traffic accidents caused by emotional fluctuations of the driver and improve traffic driving safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0058] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0059] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0060] Figure 1 A flow chart of a method for enabling assisted driving based on driver actions provided in an embodiment of the present application;

[0061] Figure 2 A schematic diagram of an overall process of activating assisted driving based on driver actions provided in an embodiment of the present application;

[0062] Figure 3 A schematic diagram of the structure of a device for activating assisted driving based on driver actions provided in an embodiment of the present application;

[0063] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0064] The following will describe the implementation methods of the present application with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be understood that the preferred embodiments are only for illustrating the present application, not for limiting the scope of protection of the present application.

[0065] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present application, and thus the drawings only show components related to the present application rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed at will, and the component layout may also be more complicated.

[0066] For ease of description, spatial relative terms may be used in the text to describe the relative position or movement of an element or feature as shown in the figure relative to another element or feature, such as "inside", "outside", "inner side", "outer side", "below", "below", "above", "above", "front", "back", etc. Such spatial relative terms are intended to include different orientations of the device in use or operation in addition to the orientation depicted in the figure. For example, if the device in the figure undergoes a position flip or a change in posture or a change in motion state, then these directional indications will also change accordingly, for example: an element described as "below other elements or features" or "below other elements or features" will subsequently be oriented as "above other elements or features" or "above other elements or features". Therefore, the example term "below..." can include both upper and lower orientations. The device can be oriented otherwise (rotated 90 degrees or in other directions) and the spatial relative descriptors used in the text are interpreted accordingly.

[0067] In order to solve the technical problem of low vehicle driving safety in the prior art, the embodiment of the present application provides a method for starting assisted driving based on driver actions, such as Figure 1 As shown, the auxiliary driving system applied to a vehicle comprises the following steps:

[0068] Step 101: Acquire motion data of the driver's body parts when the vehicle is driving through sensors;

[0069] Step 102: Determine the target emotional state of the driver by analyzing the motion data;

[0070] Step 103: If it is determined that the driver is in an abnormal state according to the target emotional state, the assisted driving operation is started.

[0071] The assisted driving system consists of three modules: motion acquisition module, behavior analysis module and assisted driving module.

[0072] Motion acquisition module: At least one sensor is installed on the steering wheel, seat and floor of the vehicle cockpit. These sensors can capture the motion data of the driver's hands, hips, feet and other key body parts in real time during the vehicle's driving. For example, the sensor on the steering wheel can detect whether the driver's hands leave the steering wheel; the sensor on the seat can monitor whether the driver's hips have irregular contractions due to tension, and the sensor on the floor can monitor whether the driver's legs are shaking.

[0073] Behavior Analysis Module: Based on the output of the motion acquisition module, the advanced artificial intelligence algorithm is used to conduct in-depth analysis of the driver's motion data. By comparing the difference between the motion pattern under normal driving conditions and the current motion data, the system can evaluate the driver's target emotional state. These emotional states may include abnormal emotions such as anxiety, fatigue, anger, or normal emotional states. When drivers have abnormal emotions while driving, it often means that they may have potential safety risks. Therefore, an important function of the behavior analysis module is to detect and identify these abnormal emotional states in a timely manner.

[0074] Assisted driving module: Based on the output of the behavior analysis module, the assisted driving module will perform corresponding assisted driving operations through the vehicle control system. For example, if the driver is detected to be anxious or sad, the system can automatically adjust the speed or play soothing music to help the driver relax; if the driver is detected to be angry or excited, the system can temporarily take over the driving rights to ensure driving safety. At the same time, the system will also provide timely feedback and warnings to the driver through voice prompts, display information, etc., to guide them to take correct driving behavior and return to normal emotional state as soon as possible.

[0075] This application uses sensors to accurately capture the motion data of key body parts of the driver, and then analyzes and evaluates these motion data, which can timely detect and identify the driver's abnormal emotional state. Finally, according to the analysis results, the corresponding assisted driving operations are performed and feedback and warning information are provided to guide the driver to adopt correct driving behaviors and restore normal emotional states. This application can reduce the risk of traffic accidents caused by driver emotional fluctuations and improve traffic driving safety.

[0076] Among them, in step 101, the motion data of three body parts of the driver when the vehicle is driving are obtained through sensors, as follows.

[0077] 1. The first body part: hands.

[0078] The system obtains the driver's hand movement data through the first sensor located on the steering wheel in the cockpit. Specifically, vibration sensors, humidity sensors and infrared sensors are installed around the steering wheel in the cockpit. The vibration sensor monitors the driver's hand slapping and rubbing data on the steering wheel in real time; the humidity sensor monitors the driver's palm humidity data in real time; and the infrared sensor monitors whether the driver's hands are off the steering wheel in real time.

[0079] Generally speaking, hand movement data reflects a person's emotional state. For example, when a person is nervous, his hands will constantly rub the steering wheel, and the moisture in his palms will increase. When a person is sad, he may lose focus, causing his hands to grip the steering wheel less forcefully or even let his hands go off the steering wheel. When a person is angry or anxious, he may slap the steering wheel, and the moisture in his palms will also increase.

[0080] 2. The second body part: the buttocks.

[0081] The system obtains the driver's hip movement data through a second sensor located on the cockpit seat. Specifically, a position sensor and an infrared sensor are installed on the cockpit seat respectively. The position sensor is used to monitor in real time whether the driver's hips move unconsciously, and the infrared sensor is used to monitor in real time whether the driver's hips contract irregularly.

[0082] Generally speaking, hip movement data reflects a person's emotional state. For example, when a person is nervous, the hips will unconsciously move left and right, and the hips will contract irregularly.

[0083] 3. The third body part: the legs.

[0084] The system obtains the driver's leg movement data through a third sensor located on the cockpit floor. Specifically, a position sensor, a pressure sensor and a vibration sensor are installed on the cockpit floor respectively. The position sensor is used to monitor in real time whether the driver's legs move back and forth, the pressure sensor is used to monitor in real time whether the driver frequently stamps his feet, and the vibration sensor is used to monitor in real time whether the driver's legs shake.

[0085] Generally speaking, leg movement data reflects a person's emotional state. For example, when a person is nervous, his legs will move back and forth unconsciously; when a person is angry, he will stomp his feet; when a person is afraid, his legs will shake.

[0086] In the present application, the driver's hand motion data, hip motion data and leg motion data can all reflect the driver's emotional state. Therefore, the driver's emotions can be analyzed based on the motion data to determine whether the driver has abnormal emotions, thereby providing active assisted driving when the driver's emotions are abnormal to ensure safe driving of the vehicle.

[0087] Among them, in step 102, the process of determining the target emotional state of the driver by analyzing the motion data is as follows: first, a standard waveform of the motion data is obtained, and then an actual waveform is generated according to the actual collected motion data of the driver, and by comparing the actual waveform with the standard waveform, it is determined whether the driver's motion data is abnormal. Figure 1 It is abnormal, indicating that the abnormality should be attributed to the driver's personal behavioral habits. For example, the driver has the habit of shaking his legs while driving, or the driver usually sweats on his palms, or the driver habitually rubs the steering wheel while driving. These behavioral habits do not affect the driver's emotions. Therefore, in order to improve the accuracy of subsequent comparison waveforms, the actual waveform that seems abnormal can be used as a reference waveform. The actual waveform collected subsequently is compared with the reference waveform, thereby taking the driver's personal behavioral habits into account and improving the accuracy of the comparison results.

[0088] Therefore, the analysis of motion data includes two major steps. The first major step is the comparison between the actual waveform and the standard waveform, and the second major step is the comparison between the actual waveform and the reference waveform. The two steps are described below.

[0089] 1. The process of comparing the actual waveform and the standard waveform is as follows:

[0090] Step S11: generating an actual hand waveform graph according to the hand motion data, generating an actual hip waveform graph according to the hip motion data, generating an actual leg waveform graph according to the leg motion data, and generating an actual fusion waveform graph by combining the hand motion data, the hip motion data and the leg motion data;

[0091] Step S12: comparing the actual hand waveform with the standard hand waveform, comparing the actual hip waveform with the standard hip waveform, comparing the actual leg waveform with the standard leg waveform, and comparing the actual fusion waveform with the standard fusion waveform to obtain four comparison results;

[0092] Step S13: determining the error rate of each comparison result;

[0093] Step S14: determining the target emotional state of the driver according to the relationship between the error rate and a preset error rate range.

[0094] Specifically, after obtaining the motion data, the system generates four waveform graphs, including a waveform graph generated by the motion data of each body part and a waveform graph generated after the fusion of multiple motion data, that is, an actual hand waveform graph is generated according to the hand motion data, an actual hip waveform graph is generated according to the hip motion data, and an actual leg waveform graph is generated according to the leg motion data, and an actual fused waveform graph is generated by combining the hand motion data, hip motion data and leg motion data.

[0095] Among the four actual waveforms generated, each actual waveform has its own corresponding standard waveform. The standard waveform is a reference waveform generated based on a large amount of normal driving behavior data, representing a standard driving action pattern. By comparing the actual waveform with the corresponding standard waveform, it can be determined whether the actual waveform is normal. For example, the actual hand waveform is compared with the standard hand waveform. If the comparison result fails, it indicates that the actual hand waveform is abnormal and the hand has abnormal movements, such as frequent hand friction of the steering wheel and high humidity in the palm. Similarly, similar comparative analysis can be performed for waveforms of other parts of the body.

[0096] Since there are four actual waveforms, each actual waveform and the corresponding standard waveform will generate an error rate when compared, so four error rates can be generated. The emotional state of the driver can be determined according to the error rate range where the error rate is located. Specifically, if the error rate of each comparison result is within the first error rate range (for example, 0% to 5%), the driver's emotion is determined to be in a normal state; if there are at least two comparison results whose error rates are within the second error rate range (for example, 5% to 15%), and the error rates of the remaining comparison results are within the first error rate range, the driver's emotion is determined to be in a slightly abnormal state, the range threshold of the second error rate range is greater than the range threshold of the second error rate range, and the slightly abnormal refers to emotions such as tension or sadness; if there are at least two comparison results whose error rates are within the third error rate range (for example, 15% to 50%), the driver's emotion is determined to be in an abnormal state, the range threshold of the third error rate range is greater than the range threshold of the second error rate range, and the abnormal refers to emotions such as anger or anxiety.

[0097] In the embodiments of the present application, the specific emotions corresponding to the micro-abnormal state and the abnormal state can be determined according to the actual situation. For example, anxiety can be regarded as a micro-abnormal emotion, or sadness can be regarded as an abnormal emotion. The present application does not make any specific limitations on this.

[0098] In addition, if the actual waveform corresponding to one body part is abnormal, the actual fused waveform will also be abnormal; if the actual waveforms corresponding to two body parts are abnormal, the actual fused waveform may be abnormal, or the actual fused waveform may not be abnormal due to data cancellation. Therefore, if the comparison fails, at least two of the four comparison results will show waveform comparison failure. In other words, as long as at least two error rates are within the second error rate range or the third error rate range, it indicates that the driver's emotions are abnormal.

[0099] In the present application, when the driver initially uses the system, a preset standard waveform is used as a comparison with the actual waveform to detect whether the driver has abnormal emotions.

[0100] Second, the process of comparing the actual waveform and the reference waveform is as follows:

[0101] Step S21: after the driver completes a set number of drivings, the waveform graph generated by the driver each time driving is used as a historical waveform graph, and a comparison result between the historical waveform graph and the standard waveform graph is determined, wherein the historical waveform graph is one of an actual hand waveform graph, an actual hip waveform graph, and an actual leg waveform graph;

[0102] Step S22: if it is determined that the comparison result of the historical waveform graph is always a comparison failure, the historical waveform graph is used as a reference waveform graph, wherein the reference waveform graph is used to indicate that the specific action data of the driver is abnormal but does not affect the driver's emotions;

[0103] Step S23: Determine the target emotional state of the driver by comparing the target waveform generated by the driver's second driving with the historical waveform, wherein the target waveform and the historical waveform correspond to the same body part.

[0104] Specifically, after the driver completes a set number of driving times, the system stores the waveform graph generated by the driver each time driving as a historical waveform graph, and stores the comparison results of the historical waveform graph, wherein the number of driving times can be pre-set, such as 10 times, 20 times, etc.

[0105] The system obtains the comparison result of the historical waveform graph with the standard waveform graph before, and the comparison result can be "comparison success" or "comparison failure". If the system determines that the comparison result of the historical waveform graph is always comparison failure, that is, the waveform graph of each driving is significantly different from the standard waveform graph, indicating that the driver's specific action data is abnormal, but these abnormalities do not affect the driver's emotions. In other words, although the driver's action pattern is different from the standard, these differences do not mean that the driver is in a bad emotional state. The system uses these historical waveform graphs as control waveform graphs for subsequent comparative analysis.

[0106] After generating the control waveform, the system will continue to monitor the driver's driving behavior. When the driver drives again, the system will generate a new target waveform. The target waveform corresponds to the same body part as the historical waveform, that is, if the historical waveform is a hand waveform, the target waveform should also be a hand waveform. The system compares the target waveform with the historical waveform to determine the driver's target emotional state. By comparing, the system can determine whether the driver's emotions have changed. If there is a significant difference between the target waveform and the historical waveform, it may indicate that the driver's emotions have changed, and the system can promptly remind the driver to rest or provide assisted driving.

[0107] When the target waveform is compared with the historical waveform, the target emotional state of the driver is determined according to the preset error rate range of the error rate of the comparison result. For example, if the error rate is between 5% and 15%, the driver's emotion is determined to be slightly abnormal.

[0108] In this application, the generation of the control waveform provides a personalized benchmark for each driver. Even if the driver's action pattern is different from the standard, as long as these differences do not affect the emotion, the system can treat it as normal behavior to avoid false alarms. By generating a control waveform, the system adapts to individual differences and improves the accuracy and reliability of emotion analysis. In addition, by generating a control waveform, the system has the ability to self-learn. Over time, the system can continuously optimize the control waveform to better adapt to the driver's behavioral changes and improve the accuracy and effectiveness of emotion monitoring.

[0109] As an optional implementation, generating an actual hand waveform graph based on hand motion data includes: determining a set weight for each hand motion data; determining the overall hand motion data at the target moment based on multiple hand motion data acquired at the target moment and the set weight for each hand motion data; generating an actual hand waveform graph based on each moment and the overall hand motion data at each moment.

[0110] In order to more comprehensively evaluate the driver's hand movements and their potential emotional state, the system uses multiple hand movement data and sets weights for each movement data. In this way, the system can comprehensively consider different types of movement data to more accurately reflect the driver's actual hand behavior.

[0111] Specifically, the action of slapping the steering wheel with the hands may reflect the driver's anger, and the frequency and strength of this action can be used as important indicators. Therefore, the system can set a higher weight for it, such as 0.3. The action of rubbing the steering wheel with the hands may reflect the driver's anxiety or restlessness. The duration and frequency of this action are also important indicators, and the system can set a medium weight for it, such as 0.25. Palm humidity is an important indicator to measure the driver's physiological state. High humidity may indicate that the driver is in a state of tension or anxiety. The system can set a higher weight for it, such as 0.3. The time and frequency of taking both hands off the steering wheel may reflect the driver's sadness. This data is crucial for evaluating driving safety, and the system can set a medium weight for it, such as 0.15.

[0112] The system collects the above four types of hand motion data at each target moment (for example, every second), and these data can be obtained in real time through sensors installed on the steering wheel.

[0113] The system calculates the overall hand motion data at the target moment based on the set weight of each hand motion data. The specific formula is as follows: Overall hand motion data = (hand slapping steering wheel data * 0.3) + (hand rubbing steering wheel data * 0.25) + (palm humidity data * 0.3) + (whether both hands are off the steering wheel data * 0.15).

[0114] The system records the overall hand motion data at each target moment and stores it in a time series data structure. The system generates an actual hand waveform based on the time series data. The horizontal axis of the waveform represents time, and the vertical axis represents the value of the overall hand motion data. In this way, the changes in the driver's hand motions in different time periods can be intuitively displayed.

[0115] In this application, by comprehensively considering multiple hand motion data, the system can more comprehensively evaluate the driver's hand behavior; by setting reasonable weights for each type of data, the accuracy and reliability of the evaluation are ensured.

[0116] The generation of the actual hip waveform graph based on the hip movement data and the generation of the actual leg waveform graph based on the leg movement data are also generated in the same manner as the above-mentioned actual hand waveform graph, and this application will not elaborate on them.

[0117] As an optional implementation, generating an actual fusion waveform graph by combining hand motion data, hip motion data and leg motion data includes: obtaining a preset weight for each body part; generating fusion data at the target moment based on the hand motion data, hip motion data and leg motion data obtained at the target moment and the preset weight for each body part; generating an actual fusion waveform graph based on each moment and the fusion data at each moment.

[0118] In order to more comprehensively evaluate the driver's overall behavior and its potential emotional state, the system not only considers the hand motion data, but also combines the hip motion data and leg motion data to generate an actual fusion waveform. In this way, the system can more accurately reflect the driver's comprehensive behavioral characteristics.

[0119] The hand motion data includes data of hand slapping the steering wheel, data of hand rubbing the steering wheel, data of palm humidity, and data of whether both hands are off the steering wheel. The system can set a higher weight for the hand motion data, such as 0.4. The hip motion data includes hip contraction data and hip movement data. The system can set a medium weight for the hip motion data, such as 0.3. The leg motion data includes leg shaking data, leg movement data, and stomping data. The system can set a lower weight for the leg motion data, such as 0.3.

[0120] Hand motion data: Assume that the hand motion data at a certain target moment is H.

[0121] Hip movement data: Assume that the hip movement data at a certain target moment is B.

[0122] Leg movement data: Assume that the leg movement data at a certain target moment is L.

[0123] The system calculates the fusion data at the target moment based on the preset weights of each body part. The specific formula is as follows: fusion data = (H*0.4)+(B*0.3)+(L*0.3).

[0124] The system records the fusion data at each target moment and stores it in a time series data structure. The system generates the actual fusion waveform based on the time series data. The horizontal axis of the waveform represents time, and the vertical axis represents the value of the fusion data. In this way, the comprehensive behavior changes of the driver in different time periods can be intuitively displayed.

[0125] By comprehensively considering the motion data of the hands, hips and legs, the system can more comprehensively evaluate the driver's overall behavior; by setting reasonable weights for each body part, the accuracy and reliability of the evaluation are ensured; by generating a fused waveform graph, the system can display the driver's comprehensive behavior in real time, more accurately detect the driver's abnormal behavior, assist the driver in managing his emotions, and reduce driving risks caused by emotional fluctuations.

[0126] As an optional implementation, if the driver is determined to be in an abnormal state according to the target emotional state, starting the assisted driving operation includes: if the driver is determined to be in a first abnormal state according to the target emotional state, adjusting the driver's emotions through prompt information; if the driver is determined to be in a second abnormal state according to the target emotional state, starting the assisted driving system for assisted driving.

[0127] The system will have different assisted driving strategies based on different analysis results to help the driver drive the vehicle safely.

[0128] While driving, the driver's feet did not shake abnormally, his legs did not lift abnormally, his hips did not move frequently, his hands were placed normally on the steering wheel, he did not sweat obviously or have abnormal behaviors such as honking the horn multiple times or accelerating suddenly, so it was judged that the driver was in a normal state.

[0129] When people are nervous, they will shake their feet involuntarily, their hips will move left and right unconsciously, their hands will keep turning the steering wheel, and the moisture in their palms will increase. This can be used to judge that the driver is in a slightly abnormal state. A slightly abnormal state, such as nervousness or sadness, etc.

[0130] When a person is angry, he or she may stomp his or her feet, slap the steering wheel, and other abnormal behaviors. These abnormal behaviors can be used to determine that the driver is in an abnormal state, such as anger or anxiety.

[0131] To ensure safety during driving, when the system determines that the driver is in an abnormal state based on the target emotional state, the corresponding assisted driving operation will be initiated. These operations are designed to help the driver restore his normal emotional state in different ways and avoid driving accidents caused by emotional fluctuations. Depending on the degree of abnormality, assisted driving operations are divided into two main stages: intervention measures in micro-abnormal states and emergency responses in abnormal states.

[0132] When the system detects that the driver is in a slightly abnormal state, that is, the driver's emotions fluctuate slightly but have not yet reached a level that seriously affects driving safety, the system will take the following measures to help the driver adjust his emotions:

[0133] 1. The system automatically switches the voice settings of the navigation system to a gentler, more soothing voice. The gentle navigation voice helps to relieve the driver's nervousness and make him feel more relaxed and calm;

[0134] 2. The system automatically plays some soothing music, such as light music, natural sounds, etc., based on the pre-configured music library. Music can effectively regulate people's emotions and help drivers recover from tension or anxiety;

[0135] 3. The on-board display screen will display some driving suggestions, such as "Please drive steadily", "The road ahead is complicated, please pay attention", etc. These prompts can help drivers concentrate and reduce operational errors caused by emotional fluctuations.

[0136] When the system detects that the driver is in an abnormal state, that is, the driver's emotions fluctuate greatly, which may seriously affect driving safety, the system will take the following emergency measures to ensure driving safety:

[0137] 1. The on-board display and speakers will issue a warning message to clearly inform the driver that the current state is not suitable for continuing to drive. This prompt can immediately attract the driver's attention, prompting them to be aware of their emotional state and take necessary actions;

[0138] 2. The assisted driving system will automatically take over the control of the vehicle, plan a safe route, and drive to the nearest service area or parking spot. The system taking over driving can effectively avoid traffic accidents caused by the driver's emotional loss of control and ensure the safety of the people in the car;

[0139] 3. After stopping at a safe location, the system will continue to play soothing music and provide some relaxation techniques, such as deep breathing exercises, through the in-car display. These measures help the driver to quickly regain calm and regain control;

[0140] 4. The system will continuously monitor the driver's emotional state. When it detects that the driver's emotions have returned to normal, it will gradually exit the assisted driving mode and return to the normal monitoring state. This ensures that the driver can regain control of the vehicle after his emotions have fully recovered, further improving driving safety;

[0141] 5. Even after the assisted driving ends, the system will continue to monitor the driver's emotional changes to ensure that they always remain within a safe range. Through continuous monitoring, any new problems that may arise can be discovered and dealt with in a timely manner to ensure the safety of the entire driving process.

[0142] After each driving session, the system will provide the driver with a detailed report, including the trend of mood changes, the effectiveness of intervention measures, etc. These suggestions will help the driver understand his or her own mood management and improve his or her self-regulation ability, so as to better control his or her emotions during future driving.

[0143] Through the above-mentioned assisted driving operations, the system can take corresponding intervention measures under different emotional states to ensure that the driver is always in the best mental state, thereby improving driving safety and comfort. This intelligent assisted driving system can not only effectively prevent traffic accidents caused by emotional fluctuations, but also help drivers better manage their emotions and improve the overall driving experience.

[0144] Figure 2 This is the overall flow chart of starting assisted driving based on driver actions in this application, including the following steps:

[0145] Step S31: Leg movement data collection: Sensors on the cockpit floor collect leg movement data, foot stamping data, and leg shaking data.

[0146] Step S32: hip movement data collection: The sensors on the cockpit seats collect hip movement data and hip contraction data.

[0147] Step S33: Hand motion data collection: The sensor on the cockpit steering wheel collects data on the hand hitting the steering wheel, the hand rubbing the steering wheel, the palm humidity data, and whether the hands are off the steering wheel.

[0148] Step S34: Merging the motion data: Merge the motion data of the hands, hips and legs at the same time.

[0149] Step S35: motion data analysis. The four actual waveforms are compared with the corresponding standard waveforms to determine whether the motion data is abnormal. If the collected data are abnormal, the collected actual data are used as the reference waveforms to judge whether the motion data is abnormal, and thus judge whether the driver's emotions are abnormal.

[0150] Step S36: If the driver's mood is slightly abnormal, music, lights, sounds, etc. are used to appease the driver.

[0151] Step S37: If the driver's emotions are abnormal, the system takes over the driving and calms the driver's emotions. After the driver's emotions return to calm, the system exits the assisted driving and continues to control the driver's emotional changes.

[0152] Based on the same technical concept, the present application provides a device for starting assisted driving based on the driver's action, such as Figure 3 As shown, the device comprises:

[0153] An acquisition module 301 is used to acquire the motion data of the driver's body parts when the vehicle is driving through sensors;

[0154] An analysis module 302 is used to determine the target emotional state of the driver by analyzing the motion data;

[0155] The activation module 303 is used to activate the assisted driving operation if it is determined that the driver is in an abnormal state according to the target emotional state.

[0156] Optionally, the acquisition module 301 is used to:

[0157] When the vehicle is driving, the driver's hand movement data is obtained through a first sensor located on the steering wheel in the cockpit;

[0158] The driver's hip movement data is obtained through a second sensor located on the cockpit seat;

[0159] The driver's leg movement data is obtained through a third sensor located on the cockpit floor.

[0160] Optionally, the analysis module 302 is used to:

[0161] Generate an actual hand waveform according to the hand motion data, generate an actual hip waveform according to the hip motion data, generate an actual leg waveform according to the leg motion data, and generate an actual fusion waveform by combining the hand motion data, the hip motion data and the leg motion data;

[0162] The actual hand waveform is compared with the standard hand waveform, the actual hip waveform is compared with the standard hip waveform, the actual leg waveform is compared with the standard leg waveform, and the actual fusion waveform is compared with the standard fusion waveform, to obtain four comparison results;

[0163] Determine the error rate for each comparison result;

[0164] The target emotional state of the driver is determined based on the relationship between the error rate and a preset error rate range.

[0165] Optionally, the device is also used for:

[0166] The waveform graph generated by the driver each time driving is used as a historical waveform graph, and a comparison result between the historical waveform graph and the standard waveform graph is determined, wherein the historical waveform graph is one of an actual hand waveform graph, an actual hip waveform graph, and an actual leg waveform graph;

[0167] If it is determined that the comparison result of the historical waveform graph is always a comparison failure, the historical waveform graph is used as a control waveform graph, wherein the control waveform graph is used to indicate that the specific action data of the driver is abnormal but does not affect the driver's emotions;

[0168] The target emotional state of the driver is determined by comparing the target waveform generated by the driver's second driving with the historical waveform, wherein the body parts corresponding to the target waveform and the historical waveform are the same.

[0169] Optionally, the device is also used for:

[0170] Compare the target waveform generated by the driver's re-driving with the historical waveform, and determine the error rate of comparison failure;

[0171] The target emotional state of the driver is determined based on the relationship between the error rate and a preset error rate range.

[0172] Optionally, the device is also used for:

[0173] If the error rate of each comparison result is within the first error rate range, it is determined that the driver's emotion is in a normal state;

[0174] If there are at least two comparison results whose error rates are within the second error rate range, and the error rates of the remaining comparison results are within the first error rate range, it is determined that the driver's emotion is in a slightly abnormal state, wherein the range threshold of the second error rate range is greater than the range threshold of the second error rate range;

[0175] If there are at least two comparison results whose error rates are within a third error rate range, it is determined that the driver's emotion is in an abnormal state, wherein the range threshold of the third error rate range is greater than the range threshold of the second error rate range.

[0176] Optionally, the device is also used for:

[0177] Determine the setting weight of each hand movement data;

[0178] Determine the overall hand motion data at the target moment according to the multiple hand motion data acquired at the target moment and the set weight of each hand motion data;

[0179] Based on the overall hand motion data at each moment and at each moment, an actual hand waveform graph is generated.

[0180] Optionally, the device is also used for:

[0181] Get the preset weights for each body part;

[0182] Generate fusion data at the target moment according to the hand motion data, hip motion data and leg motion data acquired at the target moment and the preset weight of each body part;

[0183] According to each moment and the fused data at each moment, an actual fused waveform graph is generated.

[0184] Optionally, the device is also used for:

[0185] The vibration sensor located on the steering wheel in the cockpit is used to obtain data of the hand hitting the steering wheel and the hand rubbing the steering wheel;

[0186] The palm humidity data is obtained through the humidity sensor located on the steering wheel in the cockpit;

[0187] The infrared sensor located on the steering wheel in the cockpit is used to obtain data on whether the hands are off the steering wheel.

[0188] Optionally, the device is also used for:

[0189] Hip movement data is obtained through position sensors located on the cockpit seats;

[0190] Hip contraction data is obtained via infrared sensors located on the cockpit seats.

[0191] Optionally, the device is also used for:

[0192] Leg movement data is acquired through position sensors located on the cockpit floor;

[0193] The stomping data is obtained through a pressure sensor located on the cockpit floor;

[0194] Leg vibration data were obtained via a vibration sensor located on the cockpit floor.

[0195] Optionally, the opening module 303 is used to:

[0196] If it is determined according to the target emotional state that the driver is in the first abnormal state, adjusting the driver's emotion through prompt information;

[0197] If it is determined that the driver is in the second abnormal state according to the target emotional state, the assisted driving system is turned on for assisted driving.

[0198] An embodiment of the present application provides a vehicle, which is used to execute the method of starting assisted driving based on driver actions provided by any of the aforementioned method embodiments.

[0199] like Figure 4 As shown, an embodiment of the present application provides an electronic device, including a processor 401, a communication interface 402, a memory 403 and a communication bus 404, wherein the processor 401, the communication interface 402, and the memory 403 communicate with each other through the communication bus 404.

[0200] The memory 403 is used to store computer programs.

[0201] In one embodiment of the present application, the processor 401 is used to implement the method of starting assisted driving based on driver actions provided by any of the aforementioned method embodiments when executing the program stored in the memory 403.

[0202] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the method for activating assisted driving based on driver actions as provided in any of the aforementioned method embodiments are implemented.

[0203] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place, or may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0204] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiment.

[0205] It should be understood that the terms used herein are only for the purpose of describing specific example embodiments and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "" as used herein may also be meant to include plural forms. The terms "include", "comprise", "contain", and "have" are inclusive and therefore specify the presence of stated features, steps, operations, elements and / or parts, but do not exclude the presence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the execution order is clearly indicated. It should also be understood that additional or alternative steps may be used.

[0206] The above is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features applied herein.

Claims

1. A method for enabling assisted driving based on driver actions, characterized in that: The method comprises: The motion data of the driver's body parts while the vehicle is driving is obtained through sensors; Determining a target emotional state of the driver by analyzing the motion data; If it is determined that the driver is in an abnormal state according to the target emotional state, the assisted driving operation is started.

2. The method according to claim 1, characterized in that The motion data of the driver's body parts obtained by sensors while the vehicle is driving include: When the vehicle is driving, the driver's hand motion data is obtained through a first sensor located on the steering wheel of the cockpit; Acquiring hip movement data of the driver through a second sensor located on the cockpit seat; The driver's leg movement data is obtained through a third sensor located on the cockpit floor.

3. The method according to claim 2, characterized in that Determining the target emotional state of the driver by analyzing the action data includes: Generate an actual hand waveform graph according to the hand motion data, generate an actual hip waveform graph according to the hip motion data, generate an actual leg waveform graph according to the leg motion data, and generate an actual fusion waveform graph by combining the hand motion data, the hip motion data and the leg motion data; Comparing the actual hand waveform with the standard hand waveform, comparing the actual hip waveform with the standard hip waveform, comparing the actual leg waveform with the standard leg waveform, and comparing the actual fusion waveform with the standard fusion waveform, to obtain four comparison results; Determine the error rate for each comparison result; The target emotional state of the driver is determined according to the relationship between the error rate and a preset error rate range.

4. The method according to claim 3, characterized in that After the driver completes a set number of drivings, the method further includes: Using the waveform graph generated by the driver each time driving as a historical waveform graph, and determining a comparison result between the historical waveform graph and a standard waveform graph, wherein the historical waveform graph is one of the actual hand waveform graph, the actual hip waveform graph, and the actual leg waveform graph; If it is determined that the comparison result of the historical waveform graph is always a comparison failure, the historical waveform graph is used as a reference waveform graph, wherein the reference waveform graph is used to indicate that the specific action data of the driver is abnormal but does not affect the driver's emotions; The target emotional state of the driver is determined by comparing the target waveform graph generated by the driver's second driving with the historical waveform graph, wherein the body part corresponding to the target waveform graph and the historical waveform graph is the same.

5. The method according to claim 4, characterized in that Determining the target emotional state of the driver by comparing the target waveform generated by the driver's driving again with the historical waveform includes: Comparing the target waveform generated by the driver's driving again with the historical waveform, and determining an error rate of comparison failure; The target emotional state of the driver is determined according to the relationship between the error rate and a preset error rate range.

6. The method according to claim 3 or 5, characterized in that: Determining the target emotional state of the driver according to the relationship between the error rate and a preset error rate range includes: If the error rate of each comparison result is within the first error rate range, it is determined that the driver's emotion is in a normal state; If there are at least two comparison results whose error rates are within the second error rate range, and the error rates of the remaining comparison results are within the first error rate range, it is determined that the driver's emotion is in a slightly abnormal state, wherein the range threshold of the second error rate range is greater than the range threshold of the second error rate range; If there are at least two comparison results whose error rates are within a third error rate range, it is determined that the driver's emotion is in an abnormal state, wherein the range threshold of the third error rate range is greater than the range threshold of the second error rate range.

7. The method according to claim 3, characterized in that Generating an actual hand waveform diagram according to the hand motion data includes: Determine the setting weight of each hand movement data; Determining overall hand motion data at the target moment according to the multiple hand motion data acquired at the target moment and the set weight of each hand motion data; The actual hand waveform diagram is generated according to each moment and the overall hand motion data at each moment.

8. The method according to claim 3, characterized in that Combining the hand motion data, the hip motion data, and the leg motion data to generate an actual fusion waveform graph includes: Get the preset weights for each body part; Generate fusion data at the target moment according to the hand motion data, hip motion data and leg motion data acquired at the target moment and the preset weight of each body part; The actual fusion waveform diagram is generated according to each moment and the fusion data at each moment.

9. The method according to claim 2, characterized in that: Acquiring the driver's hand motion data through a first sensor located on a steering wheel in a cockpit includes: The vibration sensor on the steering wheel in the cockpit is used to obtain data of hand hitting the steering wheel and data of hand rubbing the steering wheel; Acquiring palm humidity data through a humidity sensor located on the steering wheel of the cockpit; The data of whether the hands are off the steering wheel is obtained through an infrared sensor located on the steering wheel of the cockpit.

10. The method according to claim 2, characterized in that Acquiring the driver's hip movement data through a second sensor located on the cockpit seat includes: Acquiring hip movement data via a position sensor located on the cockpit seat; The hip contraction data is acquired by an infrared sensor located on the cockpit seat.

11. The method according to claim 2, characterized in that Acquiring the driver's leg movement data through a third sensor located on the cockpit floor includes: acquiring leg movement data via a position sensor located on the cockpit floor; Acquiring stomping data via a pressure sensor located on the cockpit floor; Leg vibration data is acquired via a vibration sensor located on the cockpit floor.

12. The method according to claim 1, characterized in that If it is determined according to the target emotional state that the driver is in an abnormal state, starting the assisted driving operation includes: If it is determined that the driver is in a first abnormal state according to the target emotional state, adjusting the driver's emotion through prompt information; If it is determined that the driver is in a second abnormal state according to the target emotional state, the auxiliary driving system is turned on to perform auxiliary driving.

13. A device for activating assisted driving based on driver actions, characterized in that: The device comprises: An acquisition module, used for acquiring motion data of the driver's body parts when the vehicle is driving through sensors; An analysis module, configured to determine a target emotional state of the driver by analyzing the motion data; The activation module is used to activate the assisted driving operation if it is determined that the driver is in an abnormal state according to the target emotional state.

14. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-12 when executing a program stored in a memory.

15. A vehicle, characterized in that: The vehicle is used to execute the method according to any one of claims 1-12.

Images