Driving monitoring method, device, computer-readable storage medium, and vehicle

Abstract

The present invention relates to the field of vehicle control technology, and specifically provides a driving monitoring method, device, medium and vehicle, aiming to solve the problem of how to accurately analyze whether a driving error has occurred and how to safely control the vehicle's driving. To this end, the method of the present invention includes predicting the driver's current driving intention based on vehicle environmental information and driver status information, determining a safe driving behavior to achieve the current driving intention, predicting the vehicle's safe movement state and safe movement trajectory based on the safe driving behavior, and judging whether a driving error has occurred based on the actual movement state and the safe movement state; if so, issuing an alarm and / or controlling the vehicle to run according to the safe movement trajectory. By comparing the actual movement state with the safe movement state, it is possible to accurately judge whether a driving error has occurred, and controlling the vehicle to run according to the safe movement trajectory can avoid the driver's inability to correct the error in time, resulting in the vehicle losing control and causing a traffic accident.

Description

Technical Field

[0001] The present invention relates to the field of vehicle control technology, and specifically provides a vehicle driving monitoring method, device, computer-readable storage medium, and vehicle. Background Art

[0002] During driving, factors such as environmental interference and fatigue can cause the driver to make an error, resulting in loss of control and an accident. To reduce the risk of accidents, conventional vehicle control methods currently rely on alerting the driver or forcibly stopping the vehicle when the vehicle is detected to be close to an obstacle. However, since this method only considers information about obstacles around the vehicle, it cannot fully assess whether the driver has made an error. Forcibly stopping the vehicle even when the driver has not made an error can seriously affect the driver's ability to operate normally. In some cases, forced stopping can even cause the driver or passengers to be impacted by objects inside the vehicle, resulting in injury. Summary of the Invention

[0003] In order to overcome the above-mentioned defects, the present invention is proposed to provide a driving monitoring method, device, computer-readable storage medium and vehicle that solve or at least partially solve the technical problems of how to accurately analyze whether the driver has made a driving error during vehicle driving and how to safely control the vehicle driving in the event of a driving error to avoid vehicle loss of control and traffic accidents.

[0004] In a first aspect, the present invention provides a vehicle driving monitoring method, the method comprising:

[0005] During vehicle travel, predicting the driver's current driving intention based on vehicle environment information and driver state information, the driver state information including at least driving behavior information determined based on the driver's historical driving operation data under the vehicle environment information;

[0006] Determine safe driving behaviors to achieve current driving intentions, and predict the safe motion state and safe motion trajectory of the vehicle over a period of time in the future based on the safe driving behaviors;

[0007] Determining whether there is a driving error based on the actual motion state of the vehicle and the safe motion state;

[0008] If so, an alarm is issued and / or the vehicle is controlled to run according to the safe motion trajectory.

[0009] In one technical solution of the above-mentioned driving monitoring method, the step of "predicting the driver's current driving intention based on vehicle environment information and driver status information" specifically includes:

[0010] Acquire training samples, wherein the training samples include driver state information of a driver when the vehicle is running under different vehicle environment information, and the sample labels of the training samples include driving intention;

[0011] Performing driving intention classification training on a preset classification model based on the training samples and their sample labels to obtain a driving intention classification model;

[0012] The driving intention classification model is used to predict the driver's current driving intention based on the vehicle environment information and the driver status information.

[0013] In a technical solution of the above-mentioned driving monitoring method, the driver status information also includes the driver's biological information, and the biological information at least includes emotional information determined based on the driver's facial image and the driver's bioelectric information.

[0014] In one technical solution of the above-mentioned vehicle driving monitoring method, the step of “determining whether there is a driving error according to the actual motion state of the vehicle and the safe motion state” specifically includes:

[0015] Determine whether a state deviation value between the actual motion state and the safe motion state is greater than or equal to a preset deviation threshold; if so, determine that a driving error occurs; if not, determine that no driving error occurs;

[0016] and / or,

[0017] After determining that a driving error has occurred, the steps of "issuing an alarm and / or controlling the vehicle to move according to the safe motion trajectory" specifically include:

[0018] Determine whether the vehicle is in a preset critical driving state based on the actual motion state and the vehicle environment information; if so, issue an alarm and control the vehicle to run according to the safe motion trajectory; if not, issue an alarm;

[0019] The preset critical driving state indicates that the actual motion state of the vehicle needs to be changed immediately to ensure the safety of the vehicle.

[0020] In one technical solution of the above-mentioned vehicle driving monitoring method, the step of "controlling the vehicle to run according to the safe motion trajectory" specifically includes:

[0021] Taking the safe motion state as a reference quantity, the actual motion state as a controlled quantity, and the state deviation value between the safe motion state and the actual motion state as a feedback quantity, closed-loop feedback control is performed on the actual motion state of the vehicle until the state deviation value falls within a preset deviation range.

[0022] In a second aspect, a vehicle driving monitoring device is provided, the device comprising:

[0023] a driving intention prediction module configured to predict the driver's current driving intention during vehicle driving based on vehicle environment information and driver state information, the driver state information including at least driving behavior information determined based on historical driving operation data of the driver under the vehicle environment information;

[0024] a safe motion state and trajectory prediction module, configured to determine safe driving behavior to achieve the current driving intention and predict the safe motion state and safe motion trajectory of the vehicle in the future based on the safe driving behavior;

[0025] The driving error operation judgment and control module is configured to judge whether there is a driving error operation based on the actual movement state of the vehicle and the safe movement state; if so, to issue an alarm and / or control the vehicle to run according to the safe movement trajectory.

[0026] In one technical solution of the above-mentioned vehicle driving monitoring device, the driving intention prediction module is further configured to perform the following operations:

[0027] Acquire training samples, wherein the training samples include driver state information of a driver when the vehicle is running under different vehicle environment information, and the sample labels of the training samples include driving intention;

[0028] Performing driving intention classification training on a preset classification model based on the training samples and their sample labels to obtain a driving intention classification model;

[0029] The driving intention classification model is used to predict the driver's current driving intention based on the vehicle environment information and the driver status information.

[0030] In a technical solution of the above-mentioned driving monitoring device, the driver status information also includes the driver's biological information, and the biological information at least includes emotional information determined based on the driver's facial image and the driver's bioelectric information.

[0031] In one technical solution of the above-mentioned vehicle driving monitoring device, the driving error operation judgment and control module includes a driving error operation judgment submodule and / or a driving error operation control submodule;

[0032] The driving error operation judgment submodule is configured to judge whether a state deviation value between the actual motion state and the safe motion state is greater than or equal to a preset deviation threshold; if so, it is judged that a driving error operation exists; if not, it is judged that no driving error operation exists;

[0033] The driving misoperation control submodule is configured to, after determining that a driving misoperation exists, determine whether the vehicle is in a preset critical driving state based on the actual motion state and the vehicle environment information; if so, issue an alarm and control the vehicle to operate according to the safe motion trajectory; if not, issue an alarm;

[0034] The preset critical driving state indicates that the actual motion state of the vehicle needs to be changed immediately to ensure the safety of the vehicle.

[0035] In one technical solution of the above-mentioned vehicle driving monitoring device, the driving error operation judgment and control module is configured to control the vehicle to run according to the safe motion trajectory by performing the following operations:

[0036] Taking the safe motion state as a reference quantity, the actual motion state as a controlled quantity, and the state deviation value between the safe motion state and the actual motion state as a feedback quantity, closed-loop feedback control is performed on the actual motion state of the vehicle until the state deviation value falls within a preset deviation range.

[0037] In a third aspect, a vehicle driving monitoring device is provided, which includes a processor and a storage device, wherein the storage device is suitable for storing multiple program codes, and the program codes are suitable for being loaded and run by the processor to execute the vehicle driving monitoring method described in any one of the technical solutions of the above-mentioned vehicle driving monitoring method.

[0038] In a fourth aspect, a computer-readable storage medium is provided, which stores a plurality of program codes, wherein the program codes are suitable for being loaded and run by a processor to execute the vehicle monitoring method described in any one of the technical solutions of the above-mentioned vehicle monitoring method.

[0039] In a fifth aspect, a vehicle is provided, comprising a vehicle driving monitoring device as described in any one of the technical solutions of the above-mentioned vehicle driving monitoring device.

[0040] Solution 1. A vehicle driving monitoring method, characterized in that the method comprises:

[0041] During vehicle travel, predicting the driver's current driving intention based on vehicle environment information and driver state information, the driver state information including at least driving behavior information determined based on the driver's historical driving operation data under the vehicle environment information;

[0042] Determine safe driving behaviors to achieve current driving intentions, and predict the safe motion state and safe motion trajectory of the vehicle over a period of time in the future based on the safe driving behaviors;

[0043] Determining whether there is a driving error based on the actual motion state of the vehicle and the safe motion state;

[0044] If so, an alarm is issued and / or the vehicle is controlled to run according to the safe motion trajectory.

[0045] Solution. The driving monitoring method according to Solution 1 is characterized in that the step of "predicting the driver's current driving intention based on vehicle environment information and driver status information" specifically includes:

[0046] Acquire training samples, wherein the training samples include driver state information of a driver when the vehicle is running under different vehicle environment information, and the sample labels of the training samples include driving intention;

[0047] Performing driving intention classification training on a preset classification model based on the training samples and their sample labels to obtain a driving intention classification model;

[0048] The driving intention classification model is used to predict the driver's current driving intention based on the vehicle environment information and the driver status information.

[0049] Solution 3. The driving monitoring method according to Solution 2 is characterized in that the driver status information also includes the driver's biological information, and the biological information at least includes emotional information determined based on the driver's facial image and the driver's bioelectric information.

[0050] Solution 4. The vehicle driving monitoring method according to Solution 1 is characterized in that the step of "determining whether there is a driving error based on the actual motion state of the vehicle and the safe motion state" specifically includes:

[0051] Determine whether a state deviation value between the actual motion state and the safe motion state is greater than or equal to a preset deviation threshold; if so, determine that a driving error occurs; if not, determine that no driving error occurs;

[0052] and / or,

[0053] After determining that a driving error has occurred, the steps of "issuing an alarm and / or controlling the vehicle to move according to the safe motion trajectory" specifically include:

[0054] Determine whether the vehicle is in a preset critical driving state based on the actual motion state and the vehicle environment information; if so, issue an alarm and control the vehicle to run according to the safe motion trajectory; if not, issue an alarm;

[0055] The preset critical driving state indicates that the actual motion state of the vehicle needs to be changed immediately to ensure the safety of the vehicle.

[0056] Solution 5. The vehicle driving monitoring method according to any one of Solutions 1 to 4, wherein the step of "controlling the vehicle to run according to the safe motion trajectory" specifically includes:

[0057] Taking the safe motion state as a reference quantity, the actual motion state as a controlled quantity, and the state deviation value between the safe motion state and the actual motion state as a feedback quantity, closed-loop feedback control is performed on the actual motion state of the vehicle until the state deviation value falls within a preset deviation range.

[0058] Solution 6. A vehicle monitoring device, characterized in that the device comprises:

[0059] a driving intention prediction module configured to predict the driver's current driving intention during vehicle driving based on vehicle environment information and driver state information, the driver state information including at least driving behavior information determined based on historical driving operation data of the driver under the vehicle environment information;

[0060] a safe motion state and trajectory prediction module, configured to determine safe driving behavior to achieve the current driving intention and predict the safe motion state and safe motion trajectory of the vehicle in the future based on the safe driving behavior;

[0061] The driving error operation judgment and control module is configured to judge whether there is a driving error operation based on the actual movement state of the vehicle and the safe movement state; if so, to issue an alarm and / or control the vehicle to run according to the safe movement trajectory.

[0062] Solution 7. The vehicle driving monitoring device according to Solution 6, wherein the driving intention prediction module is further configured to perform the following operations:

[0063] Acquire training samples, wherein the training samples include driver state information of a driver when the vehicle is running under different vehicle environment information, and the sample labels of the training samples include driving intention;

[0064] Performing driving intention classification training on a preset classification model based on the training samples and their sample labels to obtain a driving intention classification model;

[0065] The driving intention classification model is used to predict the driver's current driving intention based on the vehicle environment information and the driver status information.

[0066] Solution 8. The driving monitoring device according to Solution 7 is characterized in that the driver status information also includes the driver's biological information, and the biological information at least includes emotional information determined based on the driver's facial image and the driver's bioelectric information.

[0067] Solution 9. The vehicle monitoring device according to Solution 6, wherein the driving error operation judgment and control module includes a driving error operation judgment submodule and / or a driving error operation control submodule;

[0068] The driving error operation judgment submodule is configured to judge whether a state deviation value between the actual motion state and the safe motion state is greater than or equal to a preset deviation threshold; if so, it is judged that a driving error operation exists; if not, it is judged that no driving error operation exists;

[0069] The driving misoperation control submodule is configured to, after determining that a driving misoperation exists, determine whether the vehicle is in a preset critical driving state based on the actual motion state and the vehicle environment information; if so, issue an alarm and control the vehicle to operate according to the safe motion trajectory; if not, issue an alarm;

[0070] The preset critical driving state indicates that the actual motion state of the vehicle needs to be changed immediately to ensure the safety of the vehicle.

[0071] Solution 10. The vehicle driving monitoring device according to any one of Solutions 6 to 9, wherein the driving error operation judgment and control module is configured to control the vehicle to operate according to the safe motion trajectory by performing the following operations:

[0072] Taking the safe motion state as a reference quantity, the actual motion state as a controlled quantity, and the state deviation value between the safe motion state and the actual motion state as a feedback quantity, closed-loop feedback control is performed on the actual motion state of the vehicle until the state deviation value falls within a preset deviation range.

[0073] Solution 11. A vehicle driving monitoring device, comprising a processor and a storage device, wherein the storage device is suitable for storing multiple program codes, and is characterized in that the program codes are suitable for being loaded and run by the processor to execute the vehicle driving monitoring method described in any one of Solutions 1 to 5.

[0074] Solution 12. A computer-readable storage medium storing a plurality of program codes, wherein the program codes are suitable for being loaded and run by a processor to execute the vehicle driving monitoring method according to any one of Solutions 1 to 5.

[0075] Solution 13. A vehicle, characterized in that the vehicle includes the driving monitoring device according to any one of Solutions 6 to 11.

[0076] The above one or more technical solutions of the present invention have at least one or more of the following beneficial effects:

[0077] In the technical solution of the present invention, the driver's current driving intention can be predicted based on the vehicle environment information and the driver's state information during vehicle driving, and then the safe driving behavior to achieve the current driving intention can be determined. The safe movement state and safe movement trajectory of the vehicle in the future can be predicted based on the safe driving behavior. Finally, based on the actual movement state and safe movement state of the vehicle, it can be determined whether there is any driving error; if so, an alarm is issued and / or the vehicle is controlled to operate according to the safe movement trajectory. The vehicle environment information can include not only the vehicle's location information, but also the obstacle information around the vehicle. Based on the vehicle environment information, it can be determined whether the vehicle is in a dangerous driving environment. The driver state information can include driving behavior information determined based on the driver's historical driving operation data under the above-mentioned vehicle environment information. This driving behavior information can characterize the driver's driving habits under the above-mentioned vehicle environment information.

[0078] The embodiment of the present invention can accurately predict the driver's current driving intention based on the vehicle environment information and the driver status information during the vehicle's driving process. For example, if the driving behavior information is to brake when there is an obstacle in front of the vehicle, then it can be determined that the driver's driving intention is to slow down when there is an obstacle in front of the vehicle.

[0079] Different driving intentions can correspond to different safe driving behaviors. Based on the predicted current driving intention, the safe driving behavior that should be adopted when controlling the vehicle under this driving intention (the safe driving behavior that realizes the driving intention) can be determined. After the vehicle is controlled by the safe driving behavior, the vehicle can be prevented from losing control, ensuring the safe driving of the vehicle and preventing traffic accidents. For example, if the current driving intention is to decelerate, then the safe driving behavior can be to step on the brake pedal to brake the vehicle. Furthermore, based on the safe driving behavior, the acceleration, deceleration, lateral acceleration, vehicle speed, throttle opening, brake opening, gear position, steering wheel angle and other safe motion states of the vehicle when the vehicle is driving according to the safe driving behavior in the future can be predicted, and the safe motion trajectory of the vehicle when the vehicle is driving according to the safe driving behavior (the vehicle's driving trajectory) can also be obtained.

[0080] Since the above-mentioned safe motion state represents the motion state of the vehicle when it is driving safely, by comparing the actual motion state of the vehicle with the safe motion state, it is possible to accurately determine whether the driver has made a driving error in the vehicle, resulting in the inconsistency between the actual motion state of the vehicle and the safe motion state. If it is determined that there is a driving error, an alarm can be issued and / or the vehicle can be controlled to run according to a safe motion trajectory. The alarm can remind the driver that a driving error has been made, which may cause the vehicle to lose control and cause a traffic accident, so that the driver can promptly confirm whether to correct the erroneous driving operation and control the vehicle to drive safely as soon as possible. In addition, in addition to issuing an alarm, the vehicle can also be directly controlled to run according to the safe motion trajectory to avoid the driver's inability to correct the error in time, resulting in the vehicle losing control and causing a traffic accident. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] The disclosure of the present invention will be more easily understood with reference to the accompanying drawings. Those skilled in the art will readily appreciate that these drawings are for illustrative purposes only and are not intended to limit the scope of protection of the present invention. Furthermore, similar numbers in the drawings represent similar components, wherein:

[0082] Figure 1 This is a flow chart of the main steps of a vehicle monitoring method according to an embodiment of the present invention;

[0083] Figure 2 is a flow chart of main steps of a vehicle driving monitoring method according to another embodiment of the present invention;

[0084] Figure 3 It is a schematic diagram of the main structure block diagram of a vehicle driving monitoring device according to an embodiment of the present invention.

[0085] Reference Signs List :

[0086] 11: Driving intention prediction module; 12: Safe motion state and trajectory prediction module; 13: Driving error operation judgment and control module. DETAILED DESCRIPTION

[0087] Some embodiments of the present invention are described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0088] In the description of the present invention, "module" and "processor" may include hardware, software, or a combination of both. A module may include hardware circuitry, various suitable sensors, communication ports, and memory. It may also include software components, such as program code, or a combination of software and hardware. A processor may be a central processing unit, a microprocessor, a digital signal processor, or any other suitable processor. A processor has data and / or signal processing capabilities. A processor may be implemented in software, hardware, or a combination of both. Non-transitory computer-readable storage media include any suitable medium capable of storing program code, such as magnetic disks, hard disks, optical disks, flash memory, read-only memory, random access memory, and the like. The term "A and / or B" refers to all possible combinations of A and B, such as only A, only B, or both A and B. The terms "at least one of A or B" or "at least one of A and B" have similar meanings to "A and / or B" and may include only A, only B, or both A and B. The singular forms "a" and "the" may also include the plural forms.

[0089] See attached Figure 1 , Figure 1 FIG. 1 is a flow chart showing the main steps of a vehicle monitoring method according to an embodiment of the present invention. Figure 1 As shown, the vehicle driving monitoring method in the embodiment of the present invention mainly includes the following steps S101 to S104.

[0090] Step S101: During vehicle driving, the driver's current driving intention is predicted based on vehicle environment information and driver status information.

[0091] The driver's driving intention refers to the driver's desired driving purpose, for example, the driving intention may be to slow down or accelerate.

[0092] Vehicle environment information includes, but is not limited to, the vehicle's location information and surrounding obstacle information. Obstacle information may include at least static obstacle information and dynamic obstacle information. In a preferred embodiment, vehicle environment information also includes traffic information at the vehicle's current location, such as traffic light information. This traffic information may be obtained from the vehicle's own sensors or from external sources via a communication network.

[0093] Driver state information includes, but is not limited to, driving behavior information determined based on the driver's historical driving operation data under vehicle environment information. This driving behavior information can characterize the driver's driving habits under the aforementioned vehicle environment information. Furthermore, based on the current vehicle environment information during driving and the driver's driving behavior information under this current vehicle environment information, the driver's driving intention can be accurately predicted. For example, if the current vehicle environment information indicates an obstacle 50 meters ahead of the vehicle, and the driver's driving behavior information under this vehicle environment information indicates braking, it can be predicted that the driver's driving intention under the current vehicle environment information is to slow down.

[0094] Step S102: Determine safe driving behavior to achieve the current driving intention, and predict the safe motion state and safe motion trajectory of the vehicle in the future based on the safe driving behavior.

[0095] Safe driving behavior refers to driving behavior that achieves the driving intention. For example, if the current driving intention is to slow down, the safe driving behavior may be to apply the brake pedal to brake the vehicle. In this embodiment, different safe driving behaviors corresponding to different driving intentions can be pre-set. After the current driving intention is predicted, the corresponding safe driving behavior can be directly determined.

[0096] The safe motion state refers to the vehicle's motion state after safe driving behavior is implemented. This includes, but is not limited to, the operating status of various components within the vehicle and the vehicle's speed. For example, the safe motion state may include the vehicle's acceleration, deceleration, lateral acceleration, speed, throttle position, brake position, gear position, and steering wheel angle.

[0097] The safe motion trajectory refers to the vehicle's driving trajectory determined by combining the safe motion status with the vehicle's driving control using safe driving behavior. For example, the safe driving behavior is to step on the brake pedal to brake the vehicle. Based on the vehicle's safe motion status, it can be predicted that the vehicle's driving trajectory is to stop after driving 30 meters along the current road.

[0098] Step S103: Determine whether there is a driving error based on the actual motion state and safe motion state of the vehicle; if so, go to step S104; if not, go to step S101, continue to predict the driver's current driving intention based on the vehicle environment information and driver status information and execute subsequent steps.

[0099] Since the above-mentioned safe motion state represents the motion state of the vehicle when it is driving safely, by comparing the actual motion state of the vehicle with the safe motion state, it is possible to accurately determine whether the driver has made an erroneous driving operation of the vehicle, resulting in the inconsistency between the actual motion state of the vehicle and the safe motion state.

[0100] In one implementation of an embodiment of the present invention, it can be determined whether the state deviation value between the actual motion state and the safe motion state is greater than or equal to a preset deviation threshold; if so, it is determined that a driving error occurs; if not, it is determined that no driving error occurs.

[0101] It should be noted that those skilled in the art can flexibly set the specific value of the preset deviation threshold according to actual needs. For example, the deviation threshold of the steering wheel angle in the actual motion state and the safe motion state can be 30 degrees. When the deviation of the steering wheel angle in the actual motion state and the safe motion state is greater than 30 degrees, it can be determined that a driving error has occurred.

[0102] Step S104: issuing an alarm and / or controlling the vehicle to run along a safe motion trajectory.

[0103] If a driving error is detected, an alarm can be issued and / or the vehicle can be controlled to follow a safe trajectory. This alarm can alert the driver to the possibility of a driving error, which could lead to loss of control and an accident. This allows the driver to promptly correct the incorrect driving operation and control the vehicle to drive safely as quickly as possible. Furthermore, in addition to issuing an alarm, the vehicle can be directly controlled to follow a safe trajectory, preventing the driver from losing control and causing an accident due to the inability to correct the incorrect operation in a timely manner.

[0104] Based on the above steps S101 to S104, it is possible to accurately determine whether the driver has made a driving error in the vehicle based on the vehicle environment information and the driver status information during the vehicle driving process, and after determining that a driving error has occurred, it is possible to promptly issue an alarm and / or directly control the vehicle to operate according to a safe motion trajectory, thereby reducing driving risks and improving driving safety.

[0105] The above steps S101 and S104 are further explained below.

[0106] In one implementation of the above step S101 , the driver's current driving intention may be predicted based on the vehicle environment information and the driver's state information through the following steps 11 to 13 .

[0107] Step 11: Obtain training samples. The training samples may include driver state information of the driver when the vehicle is operating under different vehicle environment information. The sample labels of the training samples may include driving intentions.

[0108] Step 12: Perform driving intention classification training on the preset classification model based on the training samples and their sample labels to obtain a driving intention classification model.

[0109] It should be noted that, in this embodiment, a conventional classification model in the field of machine learning technology can be used to construct a preset classification model. For example, a classification model based on a neural network can be used to construct a preset classification model. At the same time, in this embodiment, a conventional model training method in the field of machine learning technology can also be used to perform model training on the preset classification model. For example, in each iterative training, the model loss value of the classification model is calculated based on the training samples and their sample labels, the gradient of the model parameters of the classification model is calculated based on the model loss value, and the model parameters are updated according to the gradient back propagation. After multiple iterative trainings, if the model loss value is less than the preset loss threshold, the training is stopped, and the classification model obtained after stopping the training is a driving intention classification model that can predict the classification intention based on the vehicle environment information and the driver status information.

[0110] Step 13: Use the driving intention classification model to predict the driver's current driving intention based on the vehicle environment information and the driver's state information.

[0111] In another embodiment of step S101, to improve the accuracy of driving intention prediction, in addition to considering driving behavior information determined from the driver's historical driving operation data under vehicle environment information, the driver's biometric information may also be used. This biometric information may include at least emotional information determined from the driver's facial image and the driver's bioelectrical information. Because biometric information can reflect the driver's physiological and psychological state to a certain extent, different physiological and psychological states may exist under different driving intentions. For example, when decelerating, the driver may be in a relatively tense psychological state, and the facial image may primarily show nervous emotions. In this case, the bioelectrical information changes more significantly than the information in a calm state. Therefore, by analyzing the driver's emotional information and bioelectrical information, the driver's driving intention can also be assisted in predicting the driver's driving intention. In this embodiment, the method described in steps 11 to 13 can also be used to predict the driver's current driving intention. The main difference is that in step 11, the training samples can include both the driver's state information and biometric information when the vehicle is operating under different vehicle environment information. The biometric information may include at least emotional information determined from the driver's facial image and the driver's bioelectrical information.

[0112] In one embodiment of step S104, after determining that a driving error has occurred, the vehicle can be selectively controlled to follow a safe motion trajectory based on whether the vehicle is in a preset critical driving state. If the vehicle is not in a critical driving state, indicating that the driver has been warned to correct the error by an alarm, the vehicle can be controlled to travel safely after the driver corrects the error, and in this case, the vehicle can be controlled to follow the safe motion trajectory. If the vehicle is in a critical driving state, indicating that the driver has been warned to correct the error by an alarm, the vehicle cannot be controlled to travel safely after the driver corrects the error, and the actual motion state of the vehicle needs to be immediately changed to ensure vehicle safety. In this case, the vehicle can be directly controlled to follow the safe motion trajectory. In some embodiments, to further improve driving safety, the vehicle can be controlled to not respond to the driver's driving operations during the process of controlling the vehicle to follow the safe motion trajectory. For example, when controlling the vehicle to decelerate along the safe motion trajectory, if it is detected that the driver has stepped on the gas (or accelerator), no response will be given, and the vehicle will continue to decelerate along the safe motion trajectory. When the state deviation value between the actual motion state and the safe motion state is less than the preset deviation threshold, the vehicle can be stopped from being controlled to decelerate according to the safe motion trajectory and the vehicle can be controlled to respond normally to the driver's driving operation.

[0113] It should be noted that those skilled in the art can pre-determine the critical driving states corresponding to different actual motion states in different vehicle environments. After determining that there is a driving error, they can directly judge whether the vehicle is in a critical driving state corresponding to the actual motion state based on the actual motion state.

[0114] In another embodiment of step S104, when controlling the vehicle to follow a safe motion trajectory, the safe motion state can be used as a reference variable, the actual motion state as a controlled variable, and the state deviation between the safe motion state and the actual motion state as a feedback variable. Closed-loop feedback control of the vehicle's actual motion state is performed until the state deviation falls within a preset deviation range. In other words, negative feedback control is performed on the vehicle's actual motion state based on the reference variable, controlled variable, and feedback variable. This negative feedback control ensures that the actual motion state continuously tracks the safe motion state and operates according to the safe motion trajectory.

[0115] For further information, see the attached Figure 2 In a vehicle driving monitoring method according to another embodiment of the present invention, the vehicle driving monitoring method may include three parts: information perception, prediction and decision-making, and vehicle control.

[0116] 1. Information Perception

[0117] Information perception can be used to obtain vehicle environment information and driver status information during driving. Information perception mainly includes three parts: cloud server information processing, environmental perception, and driver monitoring.

[0118] In the cloud server information processing part, the cloud server can call the high-definition map to obtain the vehicle's location information, and the cloud server can call the driver's historical driving operation data under the vehicle environment information ( Figure 2 The driver's driving behavior information can be determined by analyzing the historical driving operation data.

[0119] In the environmental perception part, different types of environmental information such as obstacle information can be obtained, and all the obtained environmental information can be integrated so that it can be called in the prediction and decision-making part.

[0120] In the driver monitoring part, the driver's emotional information can be determined by obtaining the driver's facial expression information, and the driver's bioelectric information can be detected by the bioelectric sensor.

[0121] The method of the information perception part in the embodiment is similar to the method described in step S101 in the aforementioned method embodiment. For the convenience of explanation, only the part related to the embodiment of the present invention is shown. For specific technical details not disclosed, please refer to the aforementioned method embodiment part.

[0122] 2. Forecasting and Decision-making

[0123] Prediction and decision-making mainly include motion safety boundary prediction and motion safety decision-making.

[0124] Motion safety boundary prediction primarily includes intention prediction, behavior prediction, and trajectory prediction. Intention prediction is similar to the method for predicting driving intention described in step S101 of the aforementioned method embodiment. Behavior prediction is similar to the method for determining safe driving behavior described in step S102 of the aforementioned method embodiment. Trajectory prediction is similar to the method for predicting safe motion trajectories described in step S102 of the aforementioned method embodiment. For ease of illustration, only the portions relevant to the present invention are shown. For specific technical details not disclosed, please refer to the aforementioned method embodiment.

[0125] Motion safety decision-making mainly includes safety limit decision-making and safety trajectory planning.

[0126] Safe trajectory planning is similar to the method for controlling the vehicle to follow a safe trajectory, as described in step S104 of the aforementioned method embodiment. Safety restriction decision-making is similar to the method for controlling the vehicle to not respond to the driver's driving operations in the method for controlling the vehicle to follow a safe trajectory. Specifically, to improve safety when controlling the vehicle to follow a safe trajectory, the vehicle's response to the driver's driving operations can be restricted. For ease of illustration, only the portions relevant to the embodiments of the present invention are shown. For specific technical details not disclosed, please refer to the aforementioned method embodiment.

[0127] 3. Vehicle Control

[0128] Vehicle control primarily involves torque control, braking control, vertical control, and / or steering control of the vehicle during the process of controlling the vehicle to operate along a safe motion trajectory, so as to enable the vehicle to operate along the safe motion trajectory. Simultaneously, during the process of controlling the vehicle to operate along the safe motion trajectory, the vehicle's motion state (actual motion state) is detected in real time and transmitted to the motion safety decision-making system, so that the motion safety decision-making system can continue to control the vehicle to operate along the safe motion trajectory based on the received vehicle motion state.

[0129] It should be pointed out that although the various steps in the above embodiments are described in a specific order, those skilled in the art will understand that in order to achieve the effects of the present invention, different steps do not have to be performed in such an order. They can be performed simultaneously (in parallel) or in other orders. These changes are within the scope of protection of the present invention.

[0130] Furthermore, the present invention also provides a vehicle driving monitoring device.

[0131] See attached Figure 3 Figure 3 FIG. 1 is a main structural block diagram of a vehicle monitoring device according to an embodiment of the present invention. Figure 3As shown, the driving monitoring device in the embodiment of the present invention mainly includes a driving intention prediction module 11, a safe movement state and trajectory prediction module 12 and a driving error operation judgment and control module 13. The driving intention prediction module 11 can be configured to predict the driver's current driving intention based on the vehicle environment information and the driver's state information during the vehicle driving process. The driver's state information can at least include driving behavior information determined based on the driver's historical driving operation data under the vehicle environment information; the safe movement state and trajectory prediction module 12 can be configured to determine the safe driving behavior to achieve the current driving intention, and predict the safe movement state and safe movement trajectory of the vehicle in a period of time in the future based on the safe driving behavior; the driving error operation judgment and control module 13 can be configured to determine whether there is a driving error operation based on the actual movement state and the safe movement state of the vehicle; if so, issue an alarm and / or control the vehicle to operate according to the safe movement trajectory.

[0132] In one embodiment, the driving intention prediction module 11 can be further configured to perform the following operations: obtaining training samples, the training samples may include the driver status information of the driver when the vehicle is operating under different vehicle environment information, and the sample labels of the training samples may include driving intention; performing driving intention classification training on a preset classification model based on the training samples and their sample labels to obtain a driving intention classification model; using the driving intention classification model and predicting the driver's current driving intention based on the vehicle environment information and the driver status information.

[0133] In one embodiment, the driver status information may further include biological information of the driver, and the biological information may at least include emotional information determined based on the driver's facial image and bioelectric information of the driver.

[0134] In one embodiment, the driving error operation judgment and control module 13 may include a driving error operation judgment submodule and / or a driving error operation control submodule. The driving error operation judgment submodule may be configured to judge whether the state deviation value between the actual motion state and the safe motion state is greater than or equal to a preset deviation threshold; if so, it is determined that a driving error operation exists; if not, it is determined that no driving error operation exists; the driving error operation control submodule may be configured to, after determining that a driving error operation exists, judge whether the vehicle is in a preset critical driving state based on the actual motion state and vehicle environment information; if so, issue an alarm and control the vehicle to operate according to a safe motion trajectory; if not, issue an alarm; wherein the preset critical driving state indicates that the actual motion state of the vehicle needs to be changed immediately to ensure the safety of the vehicle.

[0135] In one embodiment, the driving error operation judgment and control module 13 can be configured to control the vehicle to operate along a safe motion trajectory by performing the following operations: using the safe motion state as a reference quantity, the actual motion state as a controlled quantity, and the state deviation value between the safe motion state and the actual motion state as a feedback quantity, performing closed-loop feedback control on the actual motion state of the vehicle until the state deviation value falls within a preset deviation range.

[0136] The above-mentioned vehicle monitoring device is used to perform Figure 1 The driving monitoring method embodiment shown in the figure has similar technical principles, technical problems solved and technical effects produced. Technicians in this technical field can clearly understand that for the convenience and conciseness of description, the specific working process and related instructions of the driving monitoring device can refer to the contents described in the embodiment of the driving monitoring method, and will not be repeated here.

[0137] Those skilled in the art will appreciate that all or part of the processes in the method for implementing the above-mentioned embodiment of the present invention may also be accomplished by instructing the relevant hardware through a computer program. The computer program may be stored in a computer-readable storage medium. When the computer program is executed by a processor, it may implement the steps of each of the above-mentioned method embodiments. The computer program includes computer program code, which may be in source code form, object code form, executable file, or some intermediate form. The computer-readable storage medium may include: any entity or device, medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory, random access memory, electric carrier signal, telecommunication signal, and software distribution medium capable of carrying the computer program code. It should be noted that the content contained in the computer-readable storage medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable storage media do not include electric carrier signals and telecommunication signals.

[0138] Furthermore, the present invention also provides a vehicle monitoring device. In an embodiment of a vehicle monitoring device according to the present invention, the vehicle monitoring device includes a processor and a storage device. The storage device can be configured to store a program for executing the vehicle monitoring method of the above-mentioned method embodiment, and the processor can be configured to execute the program in the storage device, which includes but is not limited to a program for executing the vehicle monitoring method of the above-mentioned method embodiment. For ease of explanation, only the parts related to the embodiment of the present invention are shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present invention. The vehicle monitoring device can be a control device device formed by various electronic devices.

[0139] Furthermore, the present invention also provides a computer-readable storage medium. In a computer-readable storage medium embodiment according to the present invention, the computer-readable storage medium can be configured to store a program for executing the vehicle monitoring method of the above-mentioned method embodiment, and the program can be loaded and run by the processor to implement the above-mentioned vehicle monitoring method. For ease of explanation, only the parts related to the embodiment of the present invention are shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present invention. The computer-readable storage medium can be a storage device formed by various electronic devices. Optionally, the computer-readable storage medium in the embodiment of the present invention is a non-temporary computer-readable storage medium.

[0140] Furthermore, the present invention also provides a vehicle. In one embodiment of a vehicle according to the present invention, the vehicle may include the vehicle monitoring device described in the aforementioned embodiment of the vehicle monitoring device. It should be noted that vehicle types include, but are not limited to, electric vehicles and fuel vehicles.

[0141] Furthermore, it should be understood that since the configuration of each module is merely for the purpose of illustrating the functional units of the apparatus of the present invention, the physical devices corresponding to these modules may be the processor itself, or a portion of the software in the processor, a portion of the hardware, or a combination of software and hardware. Therefore, the number of modules in the figure is merely illustrative.

[0142] Those skilled in the art will appreciate that the various modules in the device can be adaptively split or merged. Such splitting or merging of specific modules does not cause the technical solution to deviate from the principles of the present invention. Therefore, the technical solutions after splitting or merging will fall within the scope of protection of the present invention.

[0143] Thus far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art may make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will fall within the scope of protection of the present invention.

Claims

1. A vehicle driving monitoring method, characterized in that: The method comprises: During vehicle travel, predicting the driver's current driving intention based on vehicle environment information and driver state information, the driver state information including at least driving behavior information determined based on the driver's historical driving operation data under the vehicle environment information; Determine safe driving behaviors to achieve current driving intentions, and predict the safe motion state and safe motion trajectory of the vehicle over a period of time in the future based on the safe driving behaviors; Determining whether there is a driving error based on the actual motion state of the vehicle and the safe motion state; If yes, issue an alarm and / or control the vehicle to move according to the safe motion trajectory; The step of "determining whether there is a driving error according to the actual motion state of the vehicle and the safe motion state" specifically includes: determining whether a state deviation between the actual motion state and the safe motion state is greater than or equal to a preset deviation threshold; if so, determining that there is a driving error; if not, determining that there is no driving error; and / or, After determining that a driving error has occurred, the steps of "issuing an alarm and / or controlling the vehicle to move according to the safe motion trajectory" specifically include: Based on the actual motion state and the vehicle environment information, determine whether the vehicle is in a preset critical driving state; if so, issue an alarm and control the vehicle to run according to the safe motion trajectory; if not, issue an alarm; wherein, the preset critical driving state indicates that the actual motion state of the vehicle needs to be changed immediately to ensure the safety of the vehicle.

2. The vehicle driving monitoring method according to claim 1, characterized in that: The steps of "predicting the driver's current driving intention based on vehicle environment information and driver status information" specifically include: Acquire training samples, wherein the training samples include driver state information of a driver when the vehicle is running under different vehicle environment information, and the sample labels of the training samples include driving intention; Performing driving intention classification training on a preset classification model based on the training samples and their sample labels to obtain a driving intention classification model; The driving intention classification model is used to predict the driver's current driving intention based on the vehicle environment information and the driver status information.

3. The vehicle driving monitoring method according to claim 2, characterized in that: The driver status information also includes biological information of the driver, and the biological information at least includes emotional information determined based on the driver's facial image and bioelectric information of the driver.

4. The vehicle driving monitoring method according to any one of claims 1 to 3, characterized in that: The steps of "controlling the vehicle to run according to the safe motion trajectory" specifically include: Taking the safe motion state as a reference quantity, the actual motion state as a controlled quantity, and the state deviation value between the safe motion state and the actual motion state as a feedback quantity, closed-loop feedback control is performed on the actual motion state of the vehicle until the state deviation value falls within a preset deviation range.

5. A vehicle monitoring device, characterized in that: The device comprises: a driving intention prediction module configured to predict the driver's current driving intention during vehicle driving based on vehicle environment information and driver state information, the driver state information including at least driving behavior information determined based on historical driving operation data of the driver under the vehicle environment information; a safe motion state and trajectory prediction module, configured to determine safe driving behavior to achieve the current driving intention and predict the safe motion state and safe motion trajectory of the vehicle in the future based on the safe driving behavior; a driving error operation judgment and control module, configured to judge whether a driving error operation exists based on the actual motion state of the vehicle and the safe motion state; if so, issue an alarm and / or control the vehicle to operate according to the safe motion trajectory; The driving misoperation judgment and control module includes a driving misoperation judgment submodule and / or a driving misoperation control submodule; The driving error operation judgment submodule is configured to judge whether a state deviation value between the actual motion state and the safe motion state is greater than or equal to a preset deviation threshold; if so, it is judged that a driving error operation exists; if not, it is judged that no driving error operation exists; The driving error operation control submodule is configured to, after determining that a driving error operation exists, determine whether the vehicle is in a preset critical driving state based on the actual motion state and the vehicle environmental information; if so, issue an alarm and control the vehicle to run according to the safe motion trajectory; if not, issue an alarm; wherein the preset critical driving state indicates that the actual motion state of the vehicle needs to be changed immediately to ensure the safety of the vehicle.

6. The vehicle driving monitoring device according to claim 5, characterized in that: The driving intention prediction module is further configured to perform the following operations: Acquire training samples, wherein the training samples include driver state information of a driver when the vehicle is running under different vehicle environment information, and the sample labels of the training samples include driving intention; Performing driving intention classification training on a preset classification model based on the training samples and their sample labels to obtain a driving intention classification model; The driving intention classification model is used to predict the driver's current driving intention based on the vehicle environment information and the driver status information.

7. The vehicle driving monitoring device according to claim 6, characterized in that: The driver status information also includes biological information of the driver, and the biological information at least includes emotional information determined based on the driver's facial image and bioelectric information of the driver.

8. The vehicle driving monitoring device according to any one of claims 5 to 7, characterized in that: The driving error operation judgment and control module is configured to control the vehicle to run according to the safe motion trajectory by performing the following operations: Taking the safe motion state as a reference quantity, the actual motion state as a controlled quantity, and the state deviation value between the safe motion state and the actual motion state as a feedback quantity, closed-loop feedback control is performed on the actual motion state of the vehicle until the state deviation value falls within a preset deviation range.

9. A vehicle monitoring device, comprising a processor and a storage device, wherein the storage device is adapted to store a plurality of program codes, wherein: The program code is suitable for being loaded and run by the processor to execute the vehicle driving monitoring method according to any one of claims 1 to 4.

10. A computer-readable storage medium storing a plurality of program codes, characterized in that: The program code is suitable for being loaded and run by a processor to execute the vehicle driving monitoring method according to any one of claims 1 to 4.

11. A vehicle, characterized in that: The vehicle includes the driving monitoring device according to any one of claims 5 to 9.

Images