Method and device for analyzing vehicle driving state, storage medium and electronic equipment

Abstract

The invention relates to the technical field of automatic driving, and particularly provides a method and device for analyzing a vehicle driving state, a storage medium and electronic equipment, and the method can comprise the steps: analyzing video data, collected in real time, of a driver, and obtaining an emotion deviation angle of the driver; the actual driving direction and the preset driving direction of the vehicle are calculated, and the driving deviation angle is determined; and judging whether the vehicle is in an abnormal driving state or not by performing fusion calculation on the emotion deviation angle and the driving deviation angle so as to conveniently confirm whether an automatic driving system is automatically started or not. According to some embodiments of the invention, accurate determination of the vehicle state can be realized, intervention measures can be taken in time in an abnormal state, and traffic accidents can be prevented.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and more specifically, to a method, device, storage medium, and electronic device for analyzing vehicle driving status. Background Art

[0002] With the increase in the number of cars and the continuous increase in traffic volume, traffic safety issues have become increasingly prominent. Research has found that driver emotional loss of control is a major cause of traffic accidents.

[0003] At present, with the continuous development of autonomous driving technology, in order to reduce the occurrence of traffic accidents, the existing technology determines whether the driver's emotions have been upgraded from abnormal to pathological state according to the duration of the driver's emotions in the abnormal state and the vehicle's driving state. If so, the vehicle is controlled to the autonomous driving mode. It can be seen that the existing technology mainly determines whether to enter the autonomous driving mode based on the driver's emotions, and cannot guarantee the accuracy of vehicle status recognition.

[0004] Therefore, how to provide a technical solution for a method for analyzing vehicle driving status with high accuracy has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] The purpose of some embodiments of the present application is to provide a method, device, storage medium and electronic device for analyzing the driving status of a vehicle. The technical solutions of the embodiments of the present application can improve the accuracy of identifying the driving status of the vehicle, provide effective assistance for autonomous driving, and improve driving safety.

[0006] In a first aspect, some embodiments of the present application provide a method for analyzing a vehicle driving state, comprising: analyzing video data of a driver collected in real time to obtain the driver's emotion deviation angle; calculating the actual driving direction and the preset driving direction of the vehicle to determine the driving deviation angle; and determining whether the vehicle is in an abnormal driving state by fusing the emotion deviation angle and the driving deviation angle, so as to confirm whether to automatically start the automatic driving system.

[0007] Some embodiments of the present application perform a fusion analysis of the driver's emotional deviation angle and the vehicle's driving deviation angle to determine whether the vehicle is in an abnormal driving state, and then confirm whether to start the automatic driving system. Some embodiments of the present application can improve the accuracy of vehicle driving state recognition, provide effective assistance for automatic driving, and improve driving safety.

[0008] In some embodiments, the analyzing the video data of the driver collected in real time to obtain the emotion deviation angle of the driver includes: identifying the key feature points of the driver from the video data, wherein the key feature points are the eyes and eyebrow edge points of the driver, the eyes are the left eye or the right eye, and the eyebrow edge points are the left eyebrow edge points or the right eyebrow edge points; calculating the key feature points to obtain the emotion deviation angle.

[0009] Some embodiments of the present application extract and calculate the key feature points of the driver in the collected video data to obtain the emotion deviation angle, thereby achieving accurate recognition of the driver's emotions.

[0010] In some embodiments, the key feature points are calculated to obtain the emotion deviation angle, including: connecting the corners of the eyes on both sides to obtain an eye corner line segment; connecting the midpoint of the eye corner line segment and the eyebrow edge point to obtain an eyebrow corner line segment; calculating the angle between the eye corner line segment and the eyebrow corner line segment to determine the emotion deviation angle.

[0011] Some embodiments of the present application calculate the emotion deviation angle by calculating the eye corner line segments and eyebrow corner line segments of the driver's eyes. The calculation method is simple and easy to implement.

[0012] In some embodiments, the actual driving direction and the preset driving direction of the vehicle are calculated to determine the driving offset angle, including: calculating the actual driving direction through a sensor inside the vehicle; reading the preset driving direction, or calculating the preset driving direction in real time; and using the difference between the actual driving direction and the preset driving direction as the driving offset angle.

[0013] Some embodiments of the present application calculate the actual driving direction and the preset driving direction of the vehicle to obtain the driving deviation angle, thereby achieving accurate analysis of the vehicle's driving conditions.

[0014] In some embodiments, the determining whether the vehicle is in an abnormal driving state by fusing the emotion offset angle and the driving offset angle comprises: fusing the emotion offset angle and the driving offset angle to determine the fusion angle; determining whether the vehicle is in the abnormal driving state by acquiring the relationship between the fusion angle and the eye corner line segment, wherein the eye corner line segment is a line connecting the two side corners of the driver's eyes, and the eye is a left eye or a right eye.

[0015] Some embodiments of the present application can accurately identify the vehicle status by confirming the relationship between the fusion angle of the emotion offset angle and the driving offset angle and the eye corner line segment, and then determining whether the vehicle is in an abnormal driving state.

[0016] In some embodiments, the fusion calculation of the emotion offset angle and the driving offset angle to determine the fusion angle includes: determining the emotion intensity level of the driver; obtaining an emotion weight factor and a driving weight factor corresponding to the emotion intensity level; and performing weighted summation of the emotion offset angle and the emotion weight factor, and the driving offset angle and the driving weight factor to obtain the fusion angle.

[0017] Some embodiments of the present application determine relevant weight factors by the driver's emotion intensity level, and then calculate the fusion angle to improve the accuracy of data fusion and the accuracy of vehicle status recognition.

[0018] In some embodiments, determining whether the vehicle is in the abnormal driving state is performed by obtaining the relationship between the fusion angle and the canthus line segment, including: if the fusion angle and the canthus line segment are not in a vertical relationship, determining that the vehicle is not in the abnormal driving state; if the fusion angle and the canthus line segment are in the vertical relationship, determining that the vehicle is in the abnormal driving state.

[0019] Some embodiments of the present application determine whether the vehicle is in an abnormal driving state by confirming whether the fusion angle and the eye corner line segment are in a vertical relationship, with high accuracy.

[0020] In some embodiments, after determining that the vehicle is in the abnormal driving state, the method further includes: starting the automatic driving system.

[0021] Some embodiments of the present application ensure timely intervention of the autonomous driving system to ensure driving safety by starting the autonomous driving system under abnormal driving conditions.

[0022] In a second aspect, some embodiments of the present application provide a device for analyzing the driving state of a vehicle, including: an emotion analysis module, used to analyze the video data of the driver collected in real time to obtain the emotion deviation angle of the driver; a driving calculation module, used to calculate the actual driving direction and the preset driving direction of the vehicle to determine the driving deviation angle; a judgment module, used to determine whether the vehicle is in an abnormal driving state by fusing the emotion deviation angle and the driving deviation angle, so as to confirm whether to automatically start the automatic driving system.

[0023] In a third aspect, some embodiments of the present application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can implement the method described in any embodiment of the first aspect.

[0024] In a fourth aspect, some embodiments of the present application provide an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor can implement a method as described in any embodiment of the first aspect when executing the program.

[0025] In a fifth aspect, some embodiments of the present application provide a computer program product, wherein the computer program product comprises a computer program, wherein the computer program, when executed by a processor, can implement the method described in any embodiment of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of some embodiments of the present application, the drawings required for use in some embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0027] Figure 1 A system diagram for analyzing vehicle driving status provided for some embodiments of the present application;

[0028] Figure 2 One of the flow charts of the method for analyzing the driving state of a vehicle provided in some embodiments of the present application;

[0029] Figure 3 A schematic diagram of key point distribution provided for some embodiments of the present application;

[0030] Figure 4 A second flow chart of a method for analyzing a vehicle driving state provided in some embodiments of the present application;

[0031] Figure 5 A block diagram of a device for analyzing a vehicle driving state provided in some embodiments of the present application;

[0032] Figure 6 A schematic diagram of an electronic device is provided for some embodiments of the present application. DETAILED DESCRIPTION

[0033] The technical solutions in some embodiments of the present application will be described below in conjunction with the drawings in some embodiments of the present application.

[0034] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0035] Relevant technologies show that driver's emotional out-of-control is a major cause of traffic accidents. Negative emotions such as anger and anxiety will significantly affect the driver's judgment and reaction speed, thereby increasing the risk of accidents. In addition, driving direction deviation is also an important factor leading to traffic accidents. Driving direction deviation refers to the deviation between the actual driving direction of the vehicle and the expected driving direction, which is usually caused by the driver's inattention or emotional out-of-control. In the current existing technology, the vehicle's driving state is detected either based on the driver's emotions or the vehicle's driving state. However, the driver's emotional changes are more complex, and the vehicle's driving state is also affected by many factors. It is impossible to accurately identify the vehicle's driving state by analyzing only one of the above situations.

[0036] In view of this, some embodiments of the present application provide a method for analyzing the driving state of a vehicle, in which the driver's emotional deviation angle and the vehicle's driving deviation angle are integrated and analyzed to determine whether the vehicle is in an abnormal driving state, and then confirm whether to start the automatic driving system. Some embodiments of the present application can monitor the driver's emotional state in real time and quantify the emotional deviation angle. Comprehensively consider the emotional deviation angle and driving direction deviation to accurately judge abnormal driving behavior. When abnormal driving behavior is detected, the automatic driving system intervenes in time to ensure driving safety.

[0037] The following is combined with Figure 1 The overall composition structure of the system for analyzing the driving status of a vehicle provided by some embodiments of the present application is exemplified.

[0038] like Figure 1 As shown, some embodiments of the present application provide a system for analyzing the driving state of a vehicle, and the system for analyzing the driving state of a vehicle includes: a vehicle 100 and a processing end 200. Among them, the shooting equipment installed in the vehicle 100 can shoot the driver, obtain video data, and send it to the processing end 200. After receiving the video data, the processing end 200 can analyze it to obtain the driver's emotional deviation angle; then the processing end 200 can receive the data sent by the sensor in the vehicle 100, and calculate the driving deviation angle of the actual driving data and the preset driving data. Finally, the emotional deviation angle and the driving deviation angle are fused and analyzed to determine whether the vehicle 100 is in an abnormal driving state, and then confirm whether to start the automatic driving system to intervene, so as to ensure driving safety.

[0039] In some embodiments of the present application, the processing end 200 may be a cloud server remotely connected to the vehicle 100; the processing end 200 may also be deployed inside the vehicle 100 to connect to various types of sensors. Specifically, it can be determined according to actual conditions, and the embodiments of the present application are not specifically limited here.

[0040] The following is combined with Figure 2 The implementation process of analyzing the driving status of a vehicle performed by the processing end 200 provided in some embodiments of the present application is exemplified.

[0041] Please see attached Figure 2 , Figure 2 A flow chart of a method for analyzing a vehicle driving state is provided for some embodiments of the present application. The method for analyzing a vehicle driving state may include:

[0042] S210, analyzing the video data of the driver collected in real time to obtain the emotion deviation angle of the driver.

[0043] For example, in some embodiments of the present application, a high-resolution RGB camera (i.e., a camera) can be used in the vehicle 100 to perform real-time video capture of the vehicle interior image. These cameras can be installed in front of the vehicle 100, such as the reading lights, and can capture video surveillance footage of the driver's position in the cabin (as a specific example of video data). The video surveillance footage is then analyzed to calculate the driver's emotional deviation angle.

[0044] In some embodiments of the present application, S210 may include: identifying the key feature points of the driver from the video data, wherein the key feature points are the eyes and eyebrow edge points of the driver, the eyes are the left eye or the right eye, and the eyebrow edge points are the left eyebrow edge points or the right eyebrow edge points; calculating the key feature points to obtain the emotion deviation angle.

[0045] For example, in some embodiments of the present application, facial key point detection can be used to obtain facial feature point data related to emotions (as a specific example of key feature points) from video surveillance footage in real time, providing basic data for the subsequent calculation of the emotion offset angle. For example, using a facial key point detection algorithm (such as Dlib or OpenCV), the positions of the left and right eye corners and eyebrow edge points of the right eye are obtained in real time, and then the emotion offset angle is obtained by calculation. Figure 3 The schematic diagram of the right eye is shown, wherein A is the left eye corner, B is the right eye corner, C is the midpoint of the line connecting A and B, and D is the eyebrow edge point.

[0046] In some embodiments of the present application, S210 may include: connecting the corners of the eyes on both sides to obtain a corner line segment; connecting the midpoint of the corner line segment and the eyebrow edge point to obtain an eyebrow corner line segment; calculating the angle between the corner line segment and the eyebrow corner line segment to determine the emotion deviation angle.

[0047] For example, in some embodiments of the present application, when the driver is angry, his eyebrows will be significantly raised or locked; when anxious, his eyebrows may droop. These emotional changes can be reflected by the angle of the eyebrow edge points. For example, A and B are connected to obtain the eye corner line segment L, and the position of point C is calculated. Then A and D are connected to obtain the eyebrow corner line segment AD. The angle between AD and L is calculated to obtain the emotional deviation angle M.

[0048] When the driver's emotions are abnormal, the eyebrow edge point D will move out of the normal range, causing the angle M between the line segments AD and L to increase, thereby quantifying the degree of emotional deviation. This is because the center point C of L is relatively fixed on the face and will not change too much due to slight movements of the head, so point C is a stable reference benchmark. By calculating the angle between AD and L, emotional changes can be captured more sensitively. The calculation of the angle can reduce the error caused by distance changes, focus on directional changes, and make the results of emotion recognition more reliable. The data of the entire process is easy to obtain and the calculation is simple.

[0049] S220, calculating the actual driving direction and the preset driving direction of the vehicle to determine the driving deviation angle.

[0050] For example, in some embodiments of the present application, the driving deviation angle reflects the degree of deviation between the actual driving direction of the vehicle and the expected normal driving direction (as a specific example of the preset driving direction).

[0051] In some embodiments of the present application, S220 may include: calculating the actual driving direction through sensors inside the vehicle; reading the preset driving direction, or calculating the preset driving direction in real time; and using the difference between the actual driving direction and the preset driving direction as the driving offset angle.

[0052] For example, in some embodiments of the present application, the actual driving direction of the current vehicle can be calculated by the steering wheel angle sensor, vehicle posture sensor (such as a gyroscope) or GPS data of the vehicle 100. For example, the actual driving direction θ is calculated by the following formula using the steering wheel angle sensor or vehicle posture sensor of the vehicle: actual :θ avtual =atan2(v y ,v x ), where v x and v yare the velocity components of the vehicle on the x-axis and y-axis.

[0053] The expected normal driving direction is usually an ideal driving direction pre-set by a road planning and navigation system, which can be directly read when used. Alternatively, the expected normal driving direction can also be calculated in real time based on a lane detection system (e.g., a camera and an image processing algorithm). For example, the expected normal driving direction θ is calculated by a road planning and navigation system or a lane detection system. normal :θ normal =atan2(d y ,d x ), where d x and d y They are the direction components of the road on the x-axis and y-axis respectively.

[0054] Finally, calculate the actual driving direction θ actual and the expected normal driving direction θ normal The deviation angle is obtained, and the driving deviation angle N = θ actual -θ normal .

[0055] It is known from the prior art that simply relying on one of the vehicle's driving deviation angle or emotional deviation angle to determine whether the driving behavior is abnormal may not be accurate enough. This is because each single indicator may be affected by multiple factors, so combining the two indicators can provide more comprehensive information and improve the accuracy of abnormality detection. Therefore, this application provides the following fusion solution.

[0056] S230, determining whether the vehicle is in an abnormal driving state by fusing the emotion deviation angle and the driving deviation angle, so as to confirm whether to automatically start the automatic driving system.

[0057] For example, in some embodiments of the present application, the emotion deviation angle M and the driving deviation angle N are fused and calculated to determine the driving state of the vehicle.

[0058] In some embodiments of the present application, S230 may include:

[0059] S231, performing a fusion calculation on the emotion deviation angle and the driving deviation angle to determine the fusion angle.

[0060] In order to more accurately fuse the driver's emotional state and the vehicle's driving state, the emotional intensity level is first determined, and then the fusion angle is determined according to the emotional intensity level. Therefore, in some embodiments of the present application, S231 may include: determining the driver's emotional intensity level; obtaining an emotional weight factor and a driving weight factor corresponding to the emotional intensity level; performing weighted summation on the emotional offset angle and the emotional weight factor, and the driving offset angle and the driving weight factor to obtain the fusion angle.

[0061] For example, in some embodiments of the present application, the driver's emotion intensity level E is detected based on the emotion detection model. The emotion weight factor w is determined according to the emotion intensity level E. M and driving weight factor w N . Used for weighting of M and N respectively to get the fusion angle, that is: fusion angle M2 = w M ·M+w N ·N. Among them, w M and w N The specific value of can be adjusted according to actual conditions and is not specifically limited in the embodiments of the present application.

[0062] For example, the emotional intensity level E is divided into 5 levels, from 1 to 5, and the larger the value, the higher the emotional intensity level. Among them, when E=1, it means no abnormal emotion. At this time, w M =0,w N =1. When E=2, the abnormal emotion intensity is low. M =0.2, w N = 0.8. When E = 3, it indicates that the abnormal emotion intensity is moderate. At this time, w M =0.4, w N =0.6. When E=4, it indicates that the abnormal emotion intensity is high. At this time, w M =0.6, w N =0.4. When E=5, it indicates that the abnormal emotion intensity is extremely high. At this time, w M =0.8, w N =0.2.

[0063] In a specific example, the emotional deviation angle M = 30°, the driving deviation angle N = 15°, and the emotional intensity level E = 4. M =0.6, w N =0.4. Fusion angle M2 = 0.6·30° + 0.4·15° = 24°.

[0064] S232, determining whether the vehicle is in the abnormal driving state by obtaining the relationship between the fusion angle and the eye corner line segment, wherein the eye corner line segment is a line connecting the two corners of the driver's eyes, and the eye is the left eye or the right eye.

[0065] For example, in some embodiments of the present application, the relationship between M2 and L is determined to determine whether the vehicle is in an abnormal driving state. Figure 3 The line segment AB in .

[0066] In some embodiments of the present application, S232 may include: if the fusion angle and the eye corner line segment are not in a vertical relationship, determining that the vehicle is not in the abnormal driving state; if the fusion angle and the eye corner line segment are in the vertical relationship, determining that the vehicle is in the abnormal driving state.

[0067] For example, in some embodiments of the present application, under normal circumstances, neither the emotional offset angle nor the driving offset angle will cause M2 to be perpendicular to L. When the emotional offset angle M or the driving offset angle N is too large, the driver's emotional state or driving state is obviously abnormal. If the fused angle M2 is perpendicular to L, this situation is regarded as an indication of dangerous driving behavior. Therefore, by using extreme geometric changes (i.e., whether it is perpendicular or not) as a criterion for determining abnormal driving states, normal driving behavior and abnormal driving behavior can be clearly distinguished.

[0068] In some embodiments of the present application, after executing S232, the method for analyzing the driving status of the vehicle further includes: starting the automatic driving system.

[0069] For example, in some embodiments of the present application, in an abnormal driving state, the autonomous driving program (that is, the autonomous driving system) can be automatically started to take over and prevent accidents from occurring.

[0070] The following is combined with Figure 4 The specific process of analyzing the driving status of a vehicle provided by some embodiments of the present application is exemplified.

[0071] Please see attached Figure 4 , Figure 4 A flow chart of a method for analyzing vehicle driving status is provided for some embodiments of the present application.

[0072] The above process is explained below as an example.

[0073] S410, acquiring video data of the driver in real time.

[0074] S420, identifying key feature points of the driver from the video data, and calculating the key feature points to obtain an emotion deviation angle.

[0075] S430, obtaining an actual driving direction and a preset driving direction.

[0076] S440: Taking the difference between the actual driving direction and the preset driving direction as the driving offset angle.

[0077] S450: Determine the driver's emotion intensity level, and obtain an emotion weight factor and a driving weight factor corresponding to the emotion intensity level.

[0078] S460: Perform weighted summation on the emotion offset angle and the emotion weight factor, and the driving offset angle and the driving weight factor to obtain a fusion angle.

[0079] S470, determine whether the fusion angle and the eye corner line segment are perpendicular, if so, execute S471, otherwise execute S472.

[0080] S471, determine that the vehicle is in an abnormal driving state, and execute S480.

[0081] S472, determining that the vehicle is not in an abnormal driving state.

[0082] S480, start the automatic driving system.

[0083] It can be understood that the specific implementation process of S410 to S480 can refer to the method embodiment provided above, and in order to avoid repetition, the detailed description is appropriately omitted here.

[0084] Through some of the above embodiments of the present application, it can be known that the present application can monitor and quantify the emotional state of the driver in real time. The emotional state of the driver is combined with the driving behavior of the vehicle (including driving direction deviation) to accurately judge abnormal driving behavior. When abnormal driving behavior is detected, timely intervention measures are taken to prevent the occurrence of traffic accidents.

[0085] Please refer to Figure 5 , Figure 5 The block diagram of the device for analyzing the driving state of a vehicle provided in some embodiments of the present application is shown. It should be understood that the device for analyzing the driving state of a vehicle corresponds to the above method embodiment and can execute each step involved in the above method embodiment. The specific functions of the device for analyzing the driving state of a vehicle can be found in the description above. To avoid repetition, the detailed description is appropriately omitted here.

[0086] Figure 5The device for analyzing the driving state of a vehicle includes at least one software function module that can be stored in a memory in the form of software or firmware or solidified in the device for analyzing the driving state of a vehicle. The device for analyzing the driving state of a vehicle includes: an emotion analysis module 510, which is used to analyze the video data of the driver collected in real time to obtain the emotion deviation angle of the driver; a driving calculation module 520, which is used to calculate the actual driving direction and the preset driving direction of the vehicle to determine the driving deviation angle; a determination module 530, which is used to determine whether the vehicle is in an abnormal driving state by fusing the emotion deviation angle and the driving deviation angle, so as to confirm whether to automatically start the automatic driving system.

[0087] In some embodiments of the present application, the emotion analysis module 510 is used to identify the key feature points of the driver from the video data, wherein the key feature points are the eyes and eyebrow edge points of the driver, the eyes are the left eye or the right eye, and the eyebrow edge points are the left eyebrow edge points or the right eyebrow edge points; the key feature points are calculated to obtain the emotion deviation angle.

[0088] In some embodiments of the present application, the emotion analysis module 510 is used to connect the corners of the eyes on both sides to obtain an eye corner line segment; connect the midpoint of the eye corner line segment and the eyebrow edge point to obtain an eyebrow corner line segment; calculate the angle between the eye corner line segment and the eyebrow corner line segment to determine the emotion deviation angle.

[0089] In some embodiments of the present application, the driving calculation module 520 is used to calculate the actual driving direction through sensors inside the vehicle; read the preset driving direction, or calculate the preset driving direction in real time; and use the difference between the actual driving direction and the preset driving direction as the driving offset angle.

[0090] In some embodiments of the present application, the judgment module 530 is used to perform a fusion calculation on the emotion offset angle and the driving offset angle to determine the fusion angle; and determine whether the vehicle is in the abnormal driving state by obtaining the relationship between the fusion angle and the eye corner line segment, wherein the eye corner line segment is a line connecting the two side corners of the driver's eyes, and the eye is the left eye or the right eye.

[0091] In some embodiments of the present application, the judgment module 530 is used to determine the emotion intensity level of the driver; obtain the emotion weight factor and the driving weight factor corresponding to the emotion intensity level; and perform weighted summation of the emotion offset angle and the emotion weight factor, and the driving offset angle and the driving weight factor to obtain the fusion angle.

[0092] In some embodiments of the present application, the judgment module 530 is used to determine that the vehicle is not in the abnormal driving state if the fusion angle and the eye corner line segment are not in a vertical relationship; if the fusion angle and the eye corner line segment are in the vertical relationship, determine that the vehicle is in the abnormal driving state.

[0093] In some embodiments of the present application, the determination module 530 is used to start the automatic driving system after determining that the vehicle is in the abnormal driving state.

[0094] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the device described above can refer to the corresponding process in the aforementioned method, and will not be described in detail here.

[0095] Some embodiments of the present application further provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can implement the operations of the method corresponding to any of the above methods provided in the above embodiments.

[0096] Some embodiments of the present application further provide a computer program product, which includes a computer program, wherein when the computer program is executed by a processor, it can implement the operations corresponding to any of the above methods provided in the above embodiments.

[0097] like Figure 6 As shown, some embodiments of the present application provide an electronic device 600, which includes: a memory 610, a processor 620, and a computer program stored in the memory 610 and executable on the processor 620, wherein the processor 620 can implement a method as described in any of the above embodiments when reading the program from the memory 610 through a bus 630 and executing the program.

[0098] Processor 620 can process digital signals and can include various computing structures, such as complex instruction set computer structure, reduced instruction set computer structure, or a structure that implements a combination of multiple instruction sets. In some examples, processor 620 can be a microprocessor.

[0099] The memory 610 may be used to store instructions executed by the processor 620 or data related to the execution of instructions. These instructions and / or data may include codes for implementing some or all functions of one or more modules described in the embodiments of the present application. The processor 620 of the disclosed embodiment may be used to execute instructions in the memory 610 to implement the method shown above. The memory 610 includes a dynamic random access memory, a static random access memory, a flash memory, an optical memory, or other memory known to those skilled in the art.

[0100] The above description is only an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0101] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0102] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

Claims

1. A method for analyzing vehicle driving status, characterized in that: include: Analyze the video data of the driver collected in real time to obtain the driver's emotion deviation angle; Calculate the actual driving direction and preset driving direction of the vehicle to determine the driving deviation angle; By fusing and calculating the emotion deviation angle and the driving deviation angle, it is determined whether the vehicle is in an abnormal driving state, so as to confirm whether to automatically start the automatic driving system.

2. The method according to claim 1, characterized in that The step of analyzing the real-time collected video data of the driver to obtain the driver's emotion deviation angle includes: Identify key feature points of the driver from the video data, wherein the key feature points are eyes and eyebrow edge points of the driver, the eyes are left eyes or right eyes, and the eyebrow edge points are left eyebrow edge points or right eyebrow edge points; The key feature points are calculated to obtain the emotion deviation angle.

3. The method according to claim 2, characterized in that The calculating the key feature points to obtain the emotion deviation angle includes: Connecting the eye corners on both sides of the eye to obtain eye corner line segments; Connect the midpoint of the eye corner line segment and the eyebrow edge point to obtain an eyebrow corner line segment; The angle between the eye corner line segment and the eyebrow corner line segment is calculated to determine the emotion deviation angle.

4. The method according to any one of claims 1 to 3, characterized in that The calculating the actual driving direction and the preset driving direction of the vehicle to determine the driving deviation angle includes: Calculating the actual driving direction through sensors inside the vehicle; Reading the preset driving direction, or calculating the preset driving direction in real time; The difference between the actual driving direction and the preset driving direction is used as the driving deviation angle.

5. The method according to any one of claims 1 to 3, characterized in that The determining whether the vehicle is in an abnormal driving state by fusing and calculating the emotion deviation angle and the driving deviation angle includes: Performing a fusion calculation on the emotion deviation angle and the driving deviation angle to determine a fusion angle; By acquiring the relationship between the fusion angle and the eye corner line segment, it is determined whether the vehicle is in the abnormal driving state, wherein the eye corner line segment is a line connecting the two corners of the driver's eyes, and the eye is the left eye or the right eye.

6. The method according to claim 5, characterized in that The step of performing a fusion calculation on the emotion deviation angle and the driving deviation angle to determine the fusion angle includes: determining an emotion intensity level of the driver; Obtaining an emotion weight factor and a driving weight factor corresponding to the emotion intensity level; The emotion deviation angle and the emotion weight factor, as well as the driving deviation angle and the driving weight factor are weightedly summed to obtain the fusion angle.

7. The method according to claim 5, characterized in that The determining whether the vehicle is in the abnormal driving state by acquiring the relationship between the fusion angle and the eye canthus line segment includes: If the fusion angle and the eye corner line segment are not in a perpendicular relationship, it is determined that the vehicle is not in the abnormal driving state; If the fusion angle and the eye corner line segment are in the vertical relationship, it is determined that the vehicle is in the abnormal driving state.

8. The method according to claim 7, characterized in that After determining that the vehicle is in the abnormal driving state, the method further includes: starting the automatic driving system.

9. A device for analyzing vehicle driving status, characterized in that: include: An emotion analysis module is used to analyze the video data of the driver collected in real time to obtain the driver's emotion deviation angle; A driving calculation module is used to calculate the actual driving direction and the preset driving direction of the vehicle and determine the driving deviation angle; The determination module is used to determine whether the vehicle is in an abnormal driving state by fusing the emotion deviation angle and the driving deviation angle, so as to confirm whether to automatically start the automatic driving system.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program executes the method according to any one of claims 1 to 8 when executed by a processor.

11. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the computer program executes the method according to any one of claims 1 to 8 when being run by the processor.

12. A computer program product, characterized in that The computer program product comprises a computer program, wherein the computer program executes the method according to any one of claims 1 to 8 when executed by a processor.

Images