Driving behavior recognition method, vehicle and program product

Abstract

Provided in the embodiments of the present disclosure are a driving behavior recognition method, a vehicle and a program product. The driving behavior recognition method is applied to a vehicle, and comprises: acquiring channel state information of a preset space corresponding to a target part of a driver in a cabin of a vehicle; on the basis of the channel state information, acquiring motion information of the target part; and on the basis of the motion information of the target part, recognizing a driving behavior of the driver.

Description

Driving behavior recognition methods, vehicles, and software products

[0001] This application claims priority to Chinese patent application No. 202410681991.8, filed on May 28, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This disclosure relates to the field of communication technology, and in particular to a driving behavior recognition method, vehicle, and software product. Background Technology

[0003] With the significant increase in vehicle sales, the number of traffic accidents is also rising. Traffic accidents are sometimes caused by dangerous driving behaviors; therefore, it is necessary to detect dangerous driving behaviors to reduce the occurrence of traffic accidents.

[0004] Currently, in the field of driving behavior detection, image detection and image-based algorithms are mainly used to detect whether a driver is engaging in dangerous driving behavior. Summary of the Invention

[0005] On the one hand, a driving behavior recognition method is provided for use in a vehicle. The driving behavior recognition method includes: acquiring channel state information (CSI) of a preset space corresponding to a target part of the driver in the driver's cockpit of the vehicle; acquiring motion information of the target part based on the channel state information; and recognizing the driver's driving behavior based on the motion information of the target part.

[0006] On the other hand, a driving behavior recognition device is provided, comprising an acquisition module and a recognition module. The acquisition module is used to acquire channel state information of a preset space corresponding to a target part of the driver in the cockpit. The acquisition module is also used to acquire motion information of the target part based on the channel state information. The recognition module is used to recognize the driver's driving behavior based on the motion information of the target part.

[0007] On another front, an electronic device is provided, comprising: a memory and a processor. The memory is coupled to the processor. The memory stores a computer program; when the processor executes the computer program, it implements the aforementioned driving behavior recognition method.

[0008] On another front, a computer-readable storage medium is provided, on which computer program instructions are stored, which, when executed by a processor, implement the aforementioned driving behavior recognition method.

[0009] On another front, a computer program product is provided, which includes computer program instructions that, when executed by a processor, implement the aforementioned driving behavior recognition method. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in this disclosure, the accompanying drawings used in some embodiments of this disclosure will be briefly described below. Obviously, the drawings described below are merely drawings of some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings.

[0011] Figure 1 is a schematic diagram of a system architecture according to some embodiments of the present disclosure.

[0012] Figure 2 is a flowchart illustrating a driving behavior recognition method according to some embodiments of the present disclosure.

[0013] Figure 3 is a flowchart illustrating another driving behavior recognition method according to some embodiments of the present disclosure.

[0014] Figure 4 is a schematic diagram of the amplitude variation of channel state information according to some embodiments of the present disclosure.

[0015] Figure 5 is a flowchart illustrating another driving behavior recognition method according to some embodiments of the present disclosure.

[0016] Figure 6 is a schematic diagram of another channel state information amplitude variation according to some embodiments of the present disclosure.

[0017] Figure 7 is a schematic diagram of an interactive interface according to some embodiments of the present disclosure.

[0018] Figure 8 is a flowchart illustrating another driving behavior recognition method according to some embodiments of the present disclosure.

[0019] Figure 9 is a flowchart illustrating another driving behavior recognition method according to some embodiments of the present disclosure.

[0020] Figure 10 is a schematic diagram of the structure of a driving behavior recognition device according to some embodiments of the present disclosure.

[0021] Figure 11 is a schematic diagram of the structure of another driving behavior recognition device according to some embodiments of the present disclosure. Detailed Implementation

[0022] The technical solutions of this disclosure will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0023] It should be noted that in this disclosure, the terms "exemplary" or "for example" are used to describe examples, illustrations, or descriptions. Any embodiment or design described in this disclosure using the terms "exemplary" or "for example" should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0024] In the following text, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined by terms such as "first," "second," etc., may explicitly or implicitly include one or more of that feature.

[0025] In the description of this disclosure, unless otherwise stated, " / " means "or," for example, A / B can mean A or B. "And / or" in this document is used only to describe the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: only A, only B, and A and B. Furthermore, "at least one" means one or more, and "more than one" means two or more.

[0026] With the significant increase in vehicle sales, the number of traffic accidents is also rising, and most of these accidents are caused by dangerous driving behavior. To prevent traffic accidents resulting from dangerous driving, it is necessary to monitor drivers' driving behavior.

[0027] In the field of driving behavior detection, current methods mainly rely on image detection and image algorithms to detect whether a driver is engaging in dangerous driving behavior. If such behavior is detected, the system alerts the driver to achieve intelligent assistance in driving and reduce the occurrence of traffic accidents.

[0028] Currently, methods for recognizing driving behavior using image detection have a high risk of privacy exposure. Furthermore, they also suffer from high costs, light interference, and incomplete coverage (blind spots or obstructions).

[0029] In one related technology, driver behavior can also be detected through wearable devices. These devices incorporate sensors such as accelerometers and gyroscopes. However, wearable devices suffer from inconvenience and can disrupt driver habits.

[0030] To at least address the aforementioned technical problems, this disclosure provides a driving behavior recognition method, which can be applied to vehicles or any other suitable means of transportation or cabins. The driving behavior recognition method includes: acquiring channel state information of a preset space corresponding to a target part of the driver in the vehicle's cockpit; acquiring action information of the target part based on the channel state information; and recognizing the driver's driving behavior based on the action information of the target part. Since the wireless signal corresponding to the channel state information can comprehensively and without blind spots cover the driver's target part, the driver's driving behavior can be more accurately identified through the channel state information, thereby at least solving the problem of low accuracy in driving behavior recognition in related technologies.

[0031] In some embodiments, the vehicle's cockpit can be a smart cockpit.

[0032] The driving behavior recognition method provided in this embodiment can be applied to the driving behavior recognition system shown in FIG1. ​​As shown in FIG1, the driving behavior recognition system may include: electronic device 101, vehicle main control device 102, and vehicle response device 103.

[0033] The electronic device 101 includes a wireless signal transmitter 1011, a wireless signal receiver 1012, and a processing module 1013 (or identification module).

[0034] The wireless signal transmitter 1011 is used to transmit wireless signals to a preset space in the cockpit where the driver's part to be identified is located. In Figure 1, taking the driver's head as the part to be identified as an example, the wireless signal transmitter 1011 and the wireless signal receiver 1012 are located behind and above the driver's head, respectively.

[0035] For example, the wireless signal can be a Wi-Fi (wireless fidelity) signal.

[0036] The wireless signal receiving device 1012 is used to receive the signal scattered by the driver's part to be identified, thereby obtaining the channel state information of the preset space of the driver's part to be identified.

[0037] The processing module 1013 is used to process channel state information, for example, to obtain the action information of the part to be identified based on the channel state information of the preset space of the part to be identified; or to identify whether the driver has committed dangerous driving behavior based on the action information of the part to be identified.

[0038] In some embodiments, the wireless signal transmitting device 1011 is referred to as the WI-FI signal transmitter, and the wireless signal receiving device 1012 is referred to as the WI-FI signal receiver.

[0039] In some embodiments, the wireless signal transmitting device 1011 is a WI-FI module in the wireless access point (AP) working mode, and the wireless signal receiving device 1012 is a WI-FI module in the station (SRA) working mode.

[0040] In some embodiments, the wireless signal receiving device 1012 sends real-time channel state information to the processing module 1013. The processing module 1013 sends the processing result of the real-time channel state information to the vehicle main control device 102.

[0041] Electronic device 101 is used to send the identification result of electronic device 101 to vehicle main control device 102.

[0042] The vehicle's main control device 102 responds to the recognition result sent by the electronic device 101 and sends an operation command corresponding to the recognition result to the vehicle's response device 103 to realize intelligent auxiliary control.

[0043] In some embodiments, the vehicle infotainment main control device 102 is also referred to as the vehicle infotainment main control system.

[0044] The vehicle-mounted system response device 103 is used to receive the operation command corresponding to the recognition result sent by the vehicle-mounted system main control device 102, and execute the operation corresponding to the operation command.

[0045] For example, if the identification result indicates that the driver is engaging in dangerous driving behavior, the vehicle-mounted response device 103 can execute a voice reminder command to alert the driver to the dangerous driving behavior (at this time, the vehicle-mounted response device 103 is a voice device), or execute an assisted driving command to brake the vehicle (at this time, the vehicle-mounted response device 103 is a vehicle braking device), or execute other corresponding operations that can prompt the user, such as turning on the hazard lights.

[0046] In some embodiments, the vehicle-mounted response device 103 is also referred to as a controllable element.

[0047] For example, the vehicle infotainment response device 103 may also include one or more of the following: a camera module, a microphone module, a window control module, a sunroof control module, a voice reminder device, a vehicle infotainment screen interface control module (which may also include various applications, such as music applications, video applications and other multimedia applications), vehicle peripherals (external devices) and other controllable devices.

[0048] In some embodiments, the Wi-Fi signal transmitter and the Wi-Fi signal receiver can be collectively referred to as a wireless transceiver.

[0049] In some embodiments, the driving behavior recognition system may include multiple sets of wireless transceivers, each set of wireless transceivers including at least one wireless signal transmitter and at least one wireless signal receiver, and each set of wireless transceivers is used to acquire channel state information of a preset space of different parts of the driver (e.g., the driver's head, hands, or feet).

[0050] For example, when the part to be identified is the driver's head, the wireless signal transmitter 1011 and the wireless signal receiver 1012 are located around a predetermined space around the driver's head. For example, the wireless signal transmitter 1011 and the wireless signal receiver 1012 are located around the front of the driver's headrest (e.g., on the left and right sides in front of the headrest).

[0051] When the part to be identified is the driver's hand, the wireless signal transmitter 1011 and the wireless signal receiver 1012 are located around a predetermined space around the driver's hand. For example, the wireless signal transmitter 1011 and the wireless signal receiver 1012 are located around the area above the steering wheel.

[0052] When the part to be identified is the driver's legs, the wireless signal transmitter 1011 and the wireless signal receiver 1012 are located around a preset space for the driver's legs. For example, the wireless signal transmitter 1011 and the wireless signal receiver 1012 are located around the lower half of the driver's seat (the preset space for the driver's legs).

[0053] In some embodiments, the electronic device 101 provided in this disclosure can also acquire the driver's hand gesture based on the channel state information of the preset space of the driver's hand, and send the gesture to the vehicle main control device 102. The vehicle main control device 102, in response to the type of gesture, controls the vehicle response device 103 to respond, thereby completing the control and management of the upper-layer applications of the vehicle system.

[0054] For example, in response to a gesture action that corresponds to a music playback response, the vehicle infotainment system main control device 102 controls the music application included in the vehicle infotainment system screen interface to play music; or in response to a gesture action that corresponds to a window control response, the vehicle infotainment system main control device 102 controls the window control device to perform window control operations (e.g., closing the window, opening the window, or controlling the window opening ratio).

[0055] In some embodiments, the electronic device 101 can acquire the hand gestures of all occupants, including the driver. In this case, the electronic device 101 includes at least one set of wireless transceivers, each used to acquire channel status information of a preset space for the hands of occupants inside the vehicle cabin. For example, the electronic device may include three sets of wireless transceivers. Of the three sets of wireless transceivers, the first set is used to acquire channel status information of the space above the steering wheel; the second set is used to acquire channel status information of the space above the central control device in the vehicle cabin (or the space between the driver's seat and the passenger seat); and the third set is used to acquire channel status information of the rear passenger space in the vehicle cabin.

[0056] It should be noted that Figure 1 is only an exemplary framework diagram, and the number of devices included in Figure 1 and the names of each device are not limited.

[0057] The application scenarios of the embodiments disclosed herein are not limited. The system architecture and business scenarios described in the embodiments of this disclosure are for the purpose of more clearly illustrating the technical solutions of the embodiments of this disclosure, and do not constitute a limitation on the technical solutions provided by the embodiments of this disclosure. As those skilled in the art will know, with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided by the embodiments of this disclosure are also applicable to similar technical problems.

[0058] The driving behavior recognition method provided in the embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0059] The driving behavior recognition method provided in this disclosure can be applied to the driving behavior recognition system shown in FIG1. ​​FIG2 shows a schematic flowchart of a driving behavior recognition method. As shown in FIG2, the driving behavior recognition method includes the following S201 to S203.

[0060] In S201, the channel state information of the preset space corresponding to the driver's target part in the vehicle's cockpit is obtained.

[0061] In some embodiments, the target body part includes at least one of the following: head, hand, and leg.

[0062] It should be understood that the preset space corresponding to the target part is the space in which the driver's target part may move during the driving process.

[0063] It should be understood that the channel state information of the preset space corresponding to the driver's target location is the channel state information of the wireless signal within the preset space.

[0064] It is understood that the wireless signals in this preset space include wireless signals scattered by the target part of the driver. The channel state information of the wireless signals scattered by the target part includes the characteristic information of the target part.

[0065] In S202, action information of the target part is obtained based on channel state information.

[0066] It should be understood that when the wireless signal in the preset space includes the wireless signal scattered by the target part of the driver, since the channel state information of the wireless signal scattered by the target part includes the characteristic information of the target part, the electronic device can obtain the action information of the target part based on the channel state information.

[0067] In S203, the driver's driving behavior is identified based on the motion information of the target part.

[0068] It should be noted that, since the wireless signal can fully and without blind spots cover the driver's target area, the channel state information of this preset space contains more comprehensive and accurate information about the target area, thus enabling more accurate identification of the driver's driving behavior.

[0069] In some embodiments, dangerous driving behaviors include one or more of the following: the driver turning their head for a long time, looking down for a long time, operating the steering wheel with one hand for a long time, taking both hands off the steering wheel, and suddenly stepping on the brake with their leg.

[0070] In some embodiments, operations corresponding to the driver's driving behavior can also be performed based on the identification results. For example, if the identification result indicates that the driver's driving behavior is dangerous, the operation instructions corresponding to the identification result include at least one of the following: sending a warning message to the driver, controlling the vehicle to brake, activating autonomous driving takeover of the vehicle, activating driver assistance functions, or sending a message to the vehicle network via the vehicle communication device that the vehicle is exhibiting dangerous driving behavior.

[0071] For example, in cases where the dangerous driving behavior involves the driver looking down for an extended period, a voice message is played to remind the driver that the driver has been looking down for too long. In one implementation, the target body part is the head. Referring to the embodiment shown in Figure 2, as shown in Figure 3, the above-mentioned S202, based on channel state information, obtains the action information of the target body part, including: S301 and S302.

[0072] In S301, the turning motion of the target part is identified based on the channel state information.

[0073] In S302, in response to the detected head-turning action of the target part, the position of the target part after the head-turning action is determined based on the channel state information.

[0074] It should be understood that responding to the detection of a head-turning action at the target location indicates that the driver's head may have changed from a normal position to an abnormal position, or vice versa. In this case, the electronic equipment can determine the position of the target location after the head-turning action based on channel state information, thereby enabling it to judge whether the driver's head position is abnormal after the head-turning action.

[0075] In some embodiments, in S302 above, in response to the detection of a head turning action, the following method is used: based on the amplitude change rate of the channel state information being greater than or equal to a first change rate threshold, the head turning action of the target part is detected.

[0076] It should be understood that, due to the change in head position, the amplitude of the channel state information of the wireless signal scattered by the head will change accordingly.

[0077] Understandably, the amplitude change rate is used to indicate the magnitude of change in the amplitude of channel state information within a preset time period. The preset time period is the typical duration of a head turning motion. A larger amplitude change rate indicates a greater change in head position within the preset time period (or time threshold). Therefore, when the amplitude change rate is greater than or equal to a first change rate threshold, the electronic device recognizes the head and performs a head turning motion. When the amplitude change rate is less than the first change rate threshold, the head movement is considered minor and does not affect the vehicle's driving status.

[0078] For example, the formula for calculating the rate of change of amplitude satisfies the following formula (1):

[0079] Where t1 is the moment when the amplitude changes, t2 is the moment when the amplitude change ends, t1-t2 is the preset duration (constant), A t1 -A t2 The amplitude change corresponds to a preset duration, where δ is the rate of change. Furthermore, a first rate of change threshold can be set as δ. max .

[0080] In some embodiments, the first rate of change threshold is an empirical parameter obtained from experimental data. For example, the range of amplitude change rates of the channel state information corresponding to the driver's head detected during normal vehicle behavior for most of the time (or within a certain time threshold) is determined as the normal range, and the first rate of change threshold is determined based on the normal range.

[0081] For example, as shown in FIG4, it is a schematic diagram of the amplitude change of channel state information according to an embodiment of the present disclosure. During the turn operation, the amplitude of the channel state information shows a significant increase and decrease; when there is no turn operation, the amplitude of the channel state information does not change significantly. Both an increase and a decrease in amplitude indicate that a turn operation has occurred.

[0082] The above-mentioned S203, based on the motion information of the target part, identifies the driver's driving behavior, including: S303.

[0083] In S303, in response to determining that the position of the target part after the head-turning action is an abnormal position, the driver's driving behavior is identified as dangerous driving behavior.

[0084] In some embodiments, the electronic device determines the head position as an abnormal position based on the fact that the head position location information is the same as the location information of an abnormal position; and determines the head position as a normal position based on the fact that the head position location information is the same as the location information of a normal position.

[0085] In some embodiments, the position information of the driver's head during most of the time when the vehicle is traveling on a normal road or performing normal actions is determined as the position information of the normal position.

[0086] In some embodiments, the position information of the driver's head for most of the time when the vehicle is driving on a special road section or performing a special behavior is determined as abnormal position information.

[0087] A general road segment or general behavior refers to a road segment or behavior where the driver's driving operations are relatively fixed, while a special road segment or special behavior refers to a road segment or behavior where the driver's driving operations are relatively variable. For example, a general road segment is a straight-ahead road segment, in which the driver does not need to turn their head or turn the steering wheel; a special road segment is a turning road segment or a road segment where reversing is required, in which the driver's head or hands are not fixed.

[0088] In some embodiments, when the vehicle is performing normal behavior and the amplitude change rate of the channel state information corresponding to the driver's head is lower than a first change rate threshold, the position information of the driver's head can be determined as the position information of the normal position.

[0089] In some embodiments, location information of abnormal locations and location information of normal locations are stored inside the memory of the electronic device.

[0090] In some embodiments, the location information of abnormal locations and the location information of normal locations stored in the memory can be updated periodically.

[0091] In some embodiments, the electronic device uses machine learning to identify abnormal head positions. Data on the heads of various populations at different positions is acquired as the initial state of the dataset, and continuously updated data on normal or abnormal positions are acquired as the iterative dataset to obtain a head position dataset. The head position dataset is then binary-classified to obtain normal position labels and abnormal position labels. Based on the head position dataset, normal position labels, and abnormal position labels, a neural network is trained to obtain a neural network model for identifying whether a head position is abnormal.

[0092] In some embodiments, the electronic device can perform online inference. By inputting real-time acquired channel state information into a neural network model used to identify whether the head position is abnormal, a recognition result is obtained.

[0093] In some embodiments, the above-described S203, which identifies the driver's driving behavior based on the action information of the target part, can also be achieved by: determining that the position of the target part after turning the head is an abnormal position and the duration of the target part being in the abnormal position is greater than a first duration threshold, and identifying the driver's driving behavior as dangerous driving behavior.

[0094] It should be understood that when the driver's target area (i.e., head) is in an abnormal position for a duration exceeding a first duration threshold, it indicates that the driver has not been paying attention to driving information such as vehicle status and road conditions for an extended period. In this case, the electronic equipment identifies the driver as engaging in dangerous driving behavior. Thus, it is possible to determine abnormal driving behavior when the driver has not been properly provided with the driving information necessary for safe driving for an extended period.

[0095] In some embodiments, the electronic device uses a timer to obtain the duration of the head being in an abnormal position. When the driver's head is in an abnormal position, the electronic device controls the timer to start counting. If the duration is less than or equal to a first duration threshold, the electronic device again detects a head turning motion. At this point, the electronic device again determines whether the driver's head position is abnormal, and if the driver's head position is not abnormal, the electronic device controls the timer to reset to zero.

[0096] In some embodiments, after identifying the driver's driving behavior as dangerous based on the abnormal head position of the driver, a voice prompt is given to indicate that the driver's head position is abnormal.

[0097] In one implementation, the target part is the hand. Referring to the embodiment shown in Figure 2, as shown in Figure 5, the above-mentioned S202, based on channel state information, obtains the action information of the target part, including: S501.

[0098] In S501, the location information of the target part of the driver is identified based on the channel state information.

[0099] The above-mentioned S203 identifies the driver's driving behavior based on the motion information of the target part, including: S502 and S503.

[0100] In S502, based on the position information of the target part and the preset position information of the steering wheel, it is determined that the driver's grip on the steering wheel is abnormal.

[0101] It should be understood that the driver's hand position information indicates the position of the driver's hands in space, while the preset steering wheel position information indicates the position of the steering wheel in space. By combining the hand position information and the preset steering wheel position information, the relative positional relationship between the hands and the steering wheel can be determined, i.e., the driver's grip on the steering wheel, and it can be determined whether the driver's grip on the steering wheel is abnormal.

[0102] In some embodiments, if the driver's grip on the steering wheel is abnormal, it is determined that the driver's grip on the steering wheel is abnormal. An abnormal grip on the steering wheel includes at least one of the following: the driver takes both hands off the steering wheel, or the driver takes one hand off the steering wheel.

[0103] In S503, in response to determining that the driver's grip on the steering wheel is abnormal, the driver's driving behavior is identified as dangerous driving behavior.

[0104] In some embodiments, the driver's grip on the steering wheel is normal (the driver's grip is not abnormal), including both hands on the steering wheel.

[0105] In one implementation, the electronic device detects hand position anomalies using a hand position anomaly recognition model. This model can be a neural network model. The electronic device can collect a large amount of hand-related channel state information data and filter the data to obtain a dataset. The electronic device then trains the neural network model based on a machine learning algorithm to obtain the hand position anomaly recognition model. The process of recognizing hand position anomalies includes: the electronic device filtering the real-time collected hand channel state information and inputting the processing result into the hand position anomaly recognition model to determine whether there is an anomaly in the current hand recognition result, i.e., whether there is an anomaly in the driver's grip on the steering wheel.

[0106] In some embodiments, the above-mentioned S203, which identifies the driver's driving behavior based on the action information of the target part, includes S503, which can be implemented in the following way: in response to determining that the driver's grip on the steering wheel is abnormal and the duration of the abnormal grip on the steering wheel is greater than a second duration threshold, the driver's driving behavior is identified as dangerous driving behavior.

[0107] In one implementation, the electronic device includes a timer. When an abnormality in the driver's grip on the steering wheel is detected, the electronic device starts the timer. If the timer's duration is less than a second duration threshold, and the electronic device detects that the driver's grip on the steering wheel is normal, the electronic device resets the timer to zero.

[0108] In one implementation, in response to determining that the driver's grip on the steering wheel is abnormal when the driver has one hand on the steering wheel and the duration of the abnormal grip on the steering wheel exceeds a second duration threshold, the abnormal grip on the steering wheel is determined to be when the driver removes one hand from the steering wheel.

[0109] In one implementation, if the driver's hands are both on the steering wheel, no action is taken; if the driver's hands are off the steering wheel, the electronic device does not need to time the event and directly identifies the driver's driving behavior as dangerous driving behavior.

[0110] In some embodiments, when the vehicle's cockpit is a smart cockpit, the vehicle has autonomous driving capabilities. Therefore, when the vehicle is in autonomous driving mode, if the driver's grip on the steering wheel is detected as the driver taking both hands off the steering wheel, the electronic equipment does not recognize the driver's driving behavior as dangerous driving behavior.

[0111] In some embodiments, after identifying the driver's driving behavior as dangerous based on an abnormal state of the driver's grip on the steering wheel, a voice prompt is given to the driver to indicate that the abnormal state of the driver's grip on the steering wheel is detected. For example, if the driver takes both hands off the steering wheel, the voice prompt will remind the driver to take both hands off the steering wheel; if the driver takes one hand off the steering wheel, the voice prompt will remind the driver to take one hand off the steering wheel; if the driver takes one hand off the steering wheel for a duration greater than or equal to a second duration threshold, the voice prompt will remind the driver to take one hand off the steering wheel for a longer duration.

[0112] In some embodiments, if the driver's grip on the steering wheel is abnormal, the vehicle may activate autonomous driving; or the vehicle may activate driver assistance functions to control the stability of the steering wheel.

[0113] In some embodiments, in response to an abnormal head position of the driver and the driver taking one hand off the steering wheel, the electronic device identifies the driver's driving behavior as dangerous driving behavior and reports the abnormal head and hand position of the driver to the vehicle's main control device.

[0114] In one implementation, the target part is the hand, and the motion information of the target part acquired by the electronic device can also be the hand gesture.

[0115] In some embodiments, electronic devices can recognize user control commands based on the type of gesture. This enables users to complete human-vehicle interaction based on gestures, improving the user experience during this process.

[0116] For example, an electronic device can send a gesture to the vehicle's main control device. Based on the gesture type, the main control device identifies the corresponding control command and executes it. For instance, if the control command corresponding to the gesture type is to play music, the main control device can control a music application to play music.

[0117] For example, the gesture type of the gesture action includes one or more of the following: swiping, tapping, clenching a fist, etc. Swiping includes at least one of the following: swiping up, swiping down, swiping left, swiping right, swiping forward, swiping backward. Tapping includes at least one of the following: single tapping, double tapping.

[0118] It should be noted that the above-mentioned gesture types include the following categories: Hand swipe upward: Hand naturally open, palm facing outward, fingers pointing upward, swipe upward; Hand swipe downward: Hand naturally open, palm facing outward, fingers pointing upward, swipe downward; Hand swipe left: Hand naturally open, palm facing outward, fingers pointing upward, swipe to the left side of the body; Hand swipe right: Hand naturally open, palm facing outward, fingers pointing upward, swipe to the right side of the body; Hand swipe forward: Hand naturally open, palm facing outward, fingers pointing upward. 1. Hand moves forward towards the body; 2. Hand moves backward: Hand naturally open, palm facing outward, fingers pointing upward, hand moves backward towards the body; 3. Single finger tap: Hand naturally open, palm facing outward, index finger extended upward, other fingers curled, index finger taps once; 4. Double finger tap: Hand naturally open, palm facing outward, index finger extended upward, other fingers curled, index finger taps twice; 5. Hand makes a fist: Hand naturally open, palm facing outward, fingers gradually curl.

[0119] In some embodiments, in order to accurately obtain the information corresponding to the gesture from the channel state information, it is set that there should be a preset duration (e.g., more than 1 second) after the gesture occurs. In this way, the electronic device (or the detection module or processing module of the electronic device) can quickly determine the channel state information corresponding to the gesture and discard or delete other irrelevant data, reduce the amount of data to be processed, and increase processing efficiency.

[0120] In some embodiments, the electronic device determines the channel state information corresponding to the gesture by using the standard deviation of the channel state information collected by two data windows with a time interval. For example, the size of both data windows can be 0.2 seconds, and the time interval between the two data windows can be 0.6 seconds.

[0121] It should be understood that the standard deviation of a data window is the standard deviation of the amplitudes of the multiple channel state information values ​​included within that data window.

[0122] It should be understood that the electronic device can determine the channel state information corresponding to the second data window as the channel state information for which a pause action occurred, based on the fact that the ratio between the first standard deviation of the previous data window and the second standard deviation of the subsequent data window meets a preset ratio range. At this point, the electronic device determines the channel state information corresponding to the gesture action as the channel state information for a fixed duration preceding the second data window. The fixed duration is the duration required for the human body to perform the gesture action.

[0123] It should be understood that the standard deviation of a data window is used to represent the dispersion of the amplitude of the channel state information within that data window. A larger standard deviation indicates a greater dispersion of the channel state information amplitude. A greater dispersion of the channel state information amplitude indicates a larger range of hand movements by the driver corresponding to that channel state information; conversely, a smaller dispersion of the channel state information amplitude indicates a smaller range of hand movements by the driver.

[0124] Understandably, assuming the electronic device determines the ratio of the first standard deviation to the second standard deviation as the target ratio, the preset ratio range includes the first ratio (the lower limit of the preset ratio range, the first ratio is greater than 1) and the second ratio (the upper limit of the preset ratio range). In this case, if the target ratio is greater than the first ratio and less than the second ratio, the electronic device can determine that the data window corresponding to the second standard deviation is the window corresponding to the paused action after the gesture. If the target ratio is greater than the first ratio, meaning the first standard deviation is a multiple of the first ratio of the second standard deviation, it indicates that the data window corresponding to the first standard deviation is the data window where the gesture occurred, and the data window corresponding to the second standard deviation is the data window where the paused action occurred. If the target ratio is less than the second ratio, meaning the first standard deviation is less than the second ratio of the second standard deviation, it indicates that the data window corresponding to the first standard deviation is not the data window corresponding to a non-gesture action (such as a rapidly moving other object). This avoids the rapid movement of other objects affecting the judgment of the channel state information corresponding to the gesture.

[0125] For example, the first standard deviation and the second standard deviation satisfy the following formula (2):

[0126] Where s1 is the first standard deviation, s2 is the second standard deviation, and r min As the first ratio, r max The second ratio is s3, and the standard deviation threshold is s3. The first standard deviation is greater than the standard deviation threshold, which indicates that the channel state information corresponding to the first standard deviation is channel state information with gesture action.

[0127] Figure 6 illustrates an amplitude diagram of another channel state information according to this disclosure, showing a channel state information portion including gesture information, a channel state information portion including hand position information, and two sets of data windows. Each set of data windows includes two data windows. The latter set of data windows is the next set of data windows to be calculated after the former set of data windows. In some embodiments, the two data windows move according to their own duration, thereby enabling the traversal of the channel state information.

[0128] In some embodiments, the electronic device recognizes the driver's hand gestures using a neural network model. By collecting hand gesture data from different groups of people, filtering the data, and inputting the filtered result into the neural network model for training, a hand gesture recognition model is obtained. The output of the hand gesture recognition model is a label for the hand gesture. The electronic device inputs the filtered channel state information acquired in real time into the hand gesture recognition model to obtain the hand gesture recognition result (i.e., the hand gesture label).

[0129] In some embodiments, the in-vehicle interface includes an interactive interface that can obtain feedback from the driver (or user) on the gesture recognition results. In the event of an error in the detection result, the electronic device stores the user-submitted feedback as sample data of the recognition error and uses it to periodically update and train the gesture recognition model. This improves the accuracy of gesture recognition.

[0130] For example, as shown in FIG7, which is a schematic diagram of an interactive interface according to an embodiment of the present disclosure, the following are shown: the area where the gesture action occurs, including the central control position and the rear seat position; the gesture recognition result; and the actual gesture action, including: swiping up, swiping down, swiping left, swiping right, swiping forward, swiping backward, clicking, double-clicking, and clenching a fist.

[0131] In some embodiments, the electronic device can recognize the hand gestures of all occupants in the vehicle.

[0132] For example, the electronic device can recognize hand gestures in the upper center console space and the rear passenger space. In this case, the wireless transceivers in the upper center console space are located on the roof and center console, respectively; the wireless transceivers in the rear passenger space are located on the inside of the left and right rear doors, respectively. The wireless transceivers in the upper center console space can collect channel status information corresponding to the driver's right-hand hand gestures and the front passenger's left-hand hand gestures, while the wireless transceivers in the rear passenger space can collect channel status information corresponding to the hand gestures of the rear passengers.

[0133] In one implementation, the target body part is the leg. Referring to the embodiment shown in Figure 2, as shown in Figure 8, the above-mentioned S202, based on channel state information, obtains the action information of the target body part, including: S801.

[0134] In S801, based on the amplitude change rate of the channel state information being greater than or equal to the second change rate threshold, the action information of the target part is determined, instructing the target part to perform an emergency braking action.

[0135] It should be understood that the greater the rate of change of the amplitude of the channel state information of the driver's legs, the greater the speed of the driver's leg movement. If the rate of change of the amplitude of the channel state information of the driver's legs is greater than or equal to a second rate of change threshold, it indicates that the driver's leg movement is an emergency braking action. The second rate of change threshold can be the minimum rate of change of amplitude corresponding to an emergency braking action. In this way, electronic devices can accurately identify whether the driver's legs are engaged in emergency braking.

[0136] In some embodiments, the second rate of change threshold is obtained from experimental data.

[0137] The above-mentioned S203 identifies the driver's driving behavior based on the motion information of the target part, including: S802.

[0138] In S802, in response to the motion information of the target part, the target part is instructed to perform an emergency braking action, and the driver's driving behavior is identified as dangerous driving behavior.

[0139] It should be understood that sudden braking at a target location could cause injury to occupants of the vehicle. In such cases, the driver's behavior is identified as dangerous driving.

[0140] In some embodiments, after identifying the driver's driving behavior as dangerous based on the presence of a sudden braking action in the legs, a voice prompt is sent to the driver or passengers in the vehicle to indicate the presence of a sudden braking action, or to reassure the passengers via voice.

[0141] In some embodiments, upon detecting a driver's sudden braking action, the vehicle activates an autonomous driving function; or the vehicle activates a driver assistance function to assist the driver in safely stopping the vehicle or in smoothly braking under safe conditions.

[0142] It should be noted that, in addition to the above embodiment that uses a single body part of the driver as the target part and identifies whether the driver's driving behavior is dangerous, the target part may include at least two of the above-mentioned hands, head and legs, and the electronic device identifies the driver's driving behavior based on these at least two parts.

[0143] In some embodiments, when there are multiple target locations and at least two locations are abnormal at the same time, the electronic device does not need to perform timing and can directly identify the driver's driving behavior as dangerous driving behavior.

[0144] Referring to the embodiment shown in Figure 2, as shown in Figure 9, S203, based on the motion information of the target part, the driver's driving behavior is identified, including S901 to S903.

[0145] In S901, the current driving status of the vehicle is obtained.

[0146] In some embodiments, the current driving state of the vehicle includes at least one of the following: driving straight, turning, reversing, parking in a parking space, waiting for a traffic light, changing lanes, driving at high speed, and autonomous driving.

[0147] In S902, based on the vehicle's current driving state, an anomaly judgment condition is determined to indicate that the motion information of the target part is abnormal.

[0148] It should be understood that the driver's corresponding driving operations are different under different driving conditions. Therefore, the criteria for judging the driver's driving behavior as dangerous driving behavior are also different under different driving conditions.

[0149] For example, if the vehicle is currently traveling straight, the judgment condition corresponding to the driver's dangerous driving behavior while the vehicle is traveling straight is determined as an abnormal judgment condition indicating that the action information of the target part is abnormal. For instance, if the driver's dangerous driving behavior includes turning his head to look out the window while the vehicle is traveling straight, the head position corresponding to turning the head to look out the window is taken as an abnormal head position, and this abnormal head position is determined as one of the abnormal judgment conditions indicating that the head's action information is abnormal.

[0150] In S903, the driver's driving behavior in the current driving state of the vehicle is identified based on the anomaly judgment conditions and the action information of the target part.

[0151] It should be understood that, since the anomaly judgment conditions corresponding to the vehicle's current driving state include the possibility of abnormal motion information of the target part, if the motion information of the target part meets the anomaly judgment conditions, the electronic equipment identifies the driver's driving behavior as dangerous driving behavior; if the motion information of the target part does not meet the anomaly judgment conditions, the electronic equipment identifies the driver's driving behavior as normal driving behavior. In this way, by combining the vehicle's current driving state, the driver's driving behavior can be identified more accurately and reliably.

[0152] In some embodiments, the vehicle's current driving state (or driving process) includes a first driving state and a second driving state. The first driving state indicates a special behavior of the vehicle (or a special road segment in which the vehicle travels), and the second driving state indicates a general behavior of the vehicle (or a general road segment in which the vehicle travels). Special behaviors include one or more of the following: turning, stopping at a red light, changing lanes, reversing, etc. General behaviors include all vehicle behaviors other than special behaviors. When a special behavior occurs, the vehicle's current driving state is determined to be the first driving state; when no special behavior occurs, the vehicle's current driving state is determined to be the second driving state.

[0153] In some embodiments, the specific behavior of a vehicle can be determined based on at least one of the following characteristics: turn signal illuminated, windshield wipers activated, reverse gear engaged, vehicle stationary, engine off, door open, etc.

[0154] In some embodiments, the driver's driving actions are not fixed when the vehicle is performing special behaviors. In such cases, the driver may take various actions to ensure driving safety, making it impossible to use a uniform standard to identify (or detect) dangerous driving actions. When the vehicle is performing normal behaviors, the driver's driving actions are relatively fixed. Therefore, in some embodiments, only when the vehicle is performing normal behaviors, the electronic equipment identifies whether the driver has engaged in dangerous driving behavior based on motion information of the target body part.

[0155] In some embodiments, when the vehicle's current driving state is a first driving state (the vehicle exhibits special behavior), the electronic equipment does not identify dangerous driving actions to avoid interfering with the driver's normal driving behavior. When the vehicle's current driving state is a second driving state (the vehicle does not exhibit special behavior, i.e., the vehicle exhibits normal behavior), the electronic equipment identifies dangerous driving actions. This ensures that the driver's driving operations are not interfered with when the vehicle exhibits special behavior, while also ensuring the identification of potential dangerous driving actions by the driver when the vehicle exhibits normal behavior. Furthermore, since the driver's driving behavior is relatively fixed when the vehicle is exhibiting normal behavior, the electronic equipment can accurately and reliably identify potential dangerous driving behaviors by the driver.

[0156] This disclosure embodiment can divide the driving behavior recognition device into functional modules according to the above method embodiment. For example, each function can be divided into a separate functional module, or two or more functions can be integrated into one functional module. The integrated module can be implemented in hardware or software. It should be noted that the module division in this disclosure embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods. The following description uses the example of dividing each function into a separate functional module.

[0157] Figure 10 is a schematic diagram of a driving behavior recognition device according to an embodiment of the present disclosure. This driving behavior recognition device can execute the driving behavior recognition method provided in the above-described method embodiments. As shown in Figure 10, the driving behavior recognition device 100 includes: an acquisition module 1001 and a recognition module 1002.

[0158] The acquisition module 1001 is used to acquire channel status information of a preset space corresponding to the target part of the driver in the cockpit.

[0159] The acquisition module 1001 is also used to acquire the action information of the target part based on the channel state information.

[0160] The recognition module 1002 is used to recognize the driver's driving behavior based on the action information of the target part.

[0161] In implementing the functions of the integrated modules described above in hardware, this disclosure provides another structure for the driving behavior recognition device involved in the above embodiments. As shown in FIG11, the driving behavior recognition device 110 includes: a processor 1102 and a bus 1104. In some embodiments, the driving behavior recognition device may further include a memory 1101. In some embodiments, the driving behavior recognition device may further include a communication interface 1103.

[0162] Processor 1102 may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with embodiments of this disclosure. Processor 1102 may be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with embodiments of this disclosure. Processor 1102 may also be a combination that implements computing functions, for example, including one or more microprocessor combinations, a combination of a DSP (digital signal processor) and a microprocessor, etc.

[0163] Communication interface 1103 is used to connect with other devices via a communication network. This communication network can be Ethernet, wireless access network, wireless local area network (WLAN), etc.

[0164] The memory 1101 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), disk storage medium or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto.

[0165] In one implementation, the memory 1101 can exist independently of the processor 1102. The memory 1101 can be connected to the processor 1102 via a bus 1104 and is used to store instructions or program code. When the processor 1102 calls and executes the instructions or program code stored in the memory 1101, it can implement the driving behavior recognition method provided in this embodiment of the present disclosure.

[0166] In another implementation, the memory 1101 can also be integrated with the processor 1102.

[0167] Bus 1104 can be an extended industry standard architecture (EISA) bus, etc. Bus 1104 can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in Figure 11, but this does not mean that there is only one bus or one type of bus.

[0168] Some embodiments of this disclosure provide a computer-readable storage medium (e.g., a non-transitory computer-readable storage medium) storing computer program instructions. When executed on a computer, the computer program instructions cause the computer to perform the driving behavior recognition method as described in any of the above embodiments.

[0169] Exemplary examples show that the aforementioned computer-readable storage media may include, but are not limited to: magnetic storage devices (e.g., hard disks, floppy disks, or magnetic tapes), optical disks (e.g., compact disks (CDs), digital versatile disks (DVDs), etc.), smart cards, and flash memory devices (e.g., erasable programmable read-only memory (EPROMs), cards, sticks, or key drives, etc.). The various computer-readable storage media described in this disclosure may represent one or more devices for storing information and / or other machine-readable storage media. The term "machine-readable storage media" may include, but is not limited to, wireless channels and various other media capable of storing, containing, and / or carrying instructions and / or data.

[0170] This disclosure provides a computer program product containing instructions that, when run on a computer, cause the computer to execute the driving behavior recognition method described in any of the above embodiments.

[0171] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any changes or substitutions within the technical scope disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A driving behavior recognition method, applied to a vehicle, comprising: Obtain channel state information of a preset space corresponding to the target part of the driver in the cockpit of the vehicle; Based on the channel state information, obtain the action information of the target part; Based on the motion information of the target area, the driver's driving behavior is identified.

2. The method according to claim 1, wherein, The target body part includes at least one of the following: head, hand, or leg.

3. The method according to claim 2, wherein, When the target part is the head, obtaining the action information of the target part based on the channel state information includes: identifying a head-turning action of the target part based on the channel state information; and, in response to identifying the head-turning action of the target part, determining the position of the target part after the head-turning action based on the channel state information. The step of identifying the driver's driving behavior based on the motion information of the target body part includes: In response to determining that the target part is in an abnormal position after the head-turning action, the driver's driving behavior is identified as dangerous driving behavior; or, In response to determining that the target part is in an abnormal position after turning the head and that the duration of the target part being in an abnormal position is greater than a first duration threshold, the driver's driving behavior is identified as dangerous driving behavior.

4. The method according to claim 3, wherein, The head-turning action that identifies the target area includes: Based on the fact that the amplitude change rate of the channel state information is greater than or equal to the first change rate threshold, the head-turning action of the target part is identified.

5. The method according to claim 2, wherein, When the target body part is the hand, obtaining the motion information of the target body part based on the channel state information includes: identifying the position information of the driver's target body part based on the channel state information. The step of identifying the driver's driving behavior based on the motion information of the target body part includes: Based on the location information of the driver's target part and the preset position information of the steering wheel, it is determined that the driver's grip on the steering wheel is abnormal. In response to determining that the driver's grip on the steering wheel is abnormal, the driver's driving behavior is identified as dangerous driving behavior; or, in response to determining that the driver's grip on the steering wheel is abnormal and the duration of the abnormal grip on the steering wheel is greater than a second duration threshold, the driver's driving behavior is identified as dangerous driving behavior.

6. The method according to claim 5, wherein, The driver's grip on the steering wheel is abnormal, including at least one of the following: the driver takes both hands off the steering wheel, or the driver takes one hand off the steering wheel.

7. The method according to claim 2, wherein, When the target body part is the leg, obtaining the motion information of the target body part based on the channel state information includes: determining that the motion information of the target body part instructs the target body part to perform an emergency braking action based on the fact that the amplitude change rate of the channel state information is greater than or equal to a second change rate threshold. The step of identifying the driver's driving behavior based on the motion information of the target body part includes: In response to the motion information of the target part, the target part is instructed to perform an emergency braking action, and the driver's driving behavior is identified as dangerous driving behavior.

8. The method according to claim 1, wherein, The step of identifying the driver's driving behavior based on the motion information of the target body part includes: Obtain the current driving status of the vehicle; Based on the current driving state of the vehicle, an anomaly judgment condition is determined to be that the motion information of the target part is abnormal; Based on the anomaly detection conditions and the action information of the target part, the driving behavior of the driver in the current driving state of the vehicle is identified.

9. A vehicle comprising: The system includes a memory, a processor, and at least one set of wireless transceivers; wherein the memory is coupled to the processor; the memory is used to store instructions executable by the processor; the processor executes the instructions to perform the method according to any one of claims 1-8; and the at least one set of wireless transceivers is used to acquire channel state information of a preset space corresponding to a target part of the driver.

10. A computer program product, wherein, The computer program product includes computer program instructions that, when executed by a processor, implement the method according to any one of claims 1-8.

Images