Driving behavior detection method and device, electronic equipment and storage medium

Abstract

This application relates to a driving behavior detection method, apparatus, electronic device, and storage medium. The method includes: acquiring the current driving speed of a vehicle and a surround-view image of the vehicle; detecting lane lines in the surround-view image; detecting obstacle information in the surround-view image; calculating lane line correlation statistics based on the lane lines for a predetermined number of recent surround-view images, the lane line correlation statistics including: deviation frequency, mean of center deviation values, and standard deviation of center deviation values; and determining vehicle driving behavior based on the current driving speed, the obstacle information, and the lane line correlation statistics. This method improves the accuracy of vehicle driving behavior detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent driving, in particular to a driving behavior detection method and device, electronic equipment and a storage medium. BACKGROUND

[0002] With the development of intelligent driving technology, driving behavior detection technology based on visual perception system plays an important role in ensuring the driving safety of vehicles and reducing the incidence of traffic accidents. At present, the traditional technology mainly obtains the lane line in front of the vehicle through the front camera or obtains the driver's face information through the camera in the vehicle to realize the judgment of driving behavior. When the vehicle touches the lane line or it is judged that the driver is fatigue driving, an alarm is sent to remind the driver.

[0003] However, for some active steering behaviors in actual driving, the existing driving behavior detection system has a large number of misjudgments. For example, lane line detection error in a frame of image, driver's active steering behavior to avoid obstacles, and driver's behavior of accelerating and changing lanes to overtake, all of which may cause the detection system to issue an alarm. Frequent alarms not only fail to play a safety protection role, but also affect the normal driving of the driver. SUMMARY

[0004] Therefore, it is necessary to provide a driving behavior detection method, device, electronic equipment and storage medium capable of improving accuracy in view of the above technical problems.

[0005] A driving behavior detection method, the method comprising:

[0006] obtaining the current driving speed of the vehicle and the surround view image of the vehicle;

[0007] detecting the lane line in the surround view image;

[0008] detecting the obstacle information in the surround view image;

[0009] calculating the lane line related statistics of the nearest predetermined number of frames of surround view images based on the lane line, the lane line related statistics including: deviation frequency, mean value of center deviation value, and standard deviation of center deviation value;

[0010] determining the driving behavior of the vehicle based on the current driving speed, the obstacle information and the lane line related statistics.

[0011] A driving behavior detection device, the device comprising:

[0012] a speed acquisition module configured to obtain the current driving speed of the vehicle;

[0013] an image acquisition module configured to obtain the surround view image of the vehicle;

[0014] a lane line processing module configured to detect lane lines in the surround view image;

[0015] an obstacle information obtaining module configured to detect obstacle information in the surround view image;

[0016] a statistical quantity determining module configured to calculate lane line related statistical quantities of a predetermined number of most recent frames of surround view images based on the lane lines, the lane line related statistical quantities including: a deviation frequency, a mean value of center deviation values, and a standard deviation of center deviation values;

[0017] a behavior judging module configured to determine a driving behavior of the vehicle based on the current driving speed, the obstacle information, and the lane line related statistical quantities.

[0018] An electronic device includes a memory and a processor, the memory storing a computer program, and the processor implementing the steps of the method as described above when executing the computer program.

[0019] A computer readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the steps of the method as described above.

[0020] The driving behavior detection method, device, electronic device, and storage medium described above, when detecting a driving behavior, obtain a surround view image of a vehicle, detect lane lines based on the surround view image, calculate lane line related statistical quantities of a predetermined number of most recent frames of surround view images in combination with the lane lines, and determine a driving behavior of the vehicle in combination with a current driving speed, obstacle information, and the lane line related statistical quantities. The driving behavior of the vehicle is detected in combination with panoramic surround view images of the vehicle of multiple frames, so that the environmental information around the vehicle is fully utilized, the false positive rate is reduced, false positives caused by single frame detection errors are reduced, and the accuracy of detection of the driving behavior of the vehicle is improved. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 An application environment diagram of the driving behavior detection method in one embodiment;

[0022] Figure 2 A flowchart of the driving behavior detection method in one embodiment;

[0023] Figure 3 A flowchart of determining lane line related statistical quantities in one specific example;

[0024] Figure 4 A flowchart of the driving behavior detection method in one specific example;

[0025] Figure 5A structural block diagram of the driving behavior detection device in one embodiment;

[0026] Figure 6 An internal structural diagram of the electronic device in one embodiment. DETAILED DESCRIPTION

[0027] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application.

[0028] The driving behavior detection method provided by the present application can be applied in an application environment as shown in Figure 1 . In the application environment, in order to ensure the driving safety of the vehicle 10, the driving behavior of the vehicle 10 is detected by the driving behavior detection technology during the driving process of the vehicle 10 on the road, and once a dangerous driving behavior is detected, an alarm is sent to remind the driver. In the traditional driving behavior detection method, the alarm is sent to remind the driver as soon as the vehicle touches the lane line. However, as shown in Figure 1 , when there is an obstacle 20 in front of the vehicle, for example, the obstacle 20 is a stationary obstacle that needs to be bypassed, or the obstacle 20 is a slow-moving vehicle that needs to be overtaken, the driver needs to drive the vehicle 10 to other lanes to bypass or overtake. However, in this process, it is inevitable to touch the lane line, and according to the traditional method, an alarm will be sent as soon as the vehicle touches the lane line, which will cause false positives and affect the safe driving of the driver. In the scheme of the present embodiment, the driving behavior of the vehicle is detected comprehensively in combination with the panoramic surround view images of multiple frames of the vehicle, so as to fully utilize the environmental information around the vehicle, reduce the false positive rate, and reduce the false positives caused by single-frame detection errors.

[0029] In one embodiment, as shown in Figure 2 , a driving behavior detection method is provided, which is applied to the vehicle 10 in Figure 1 as an example, and includes the following steps S201 to S205.

[0030] Step S201: obtaining the current driving speed of the vehicle and the surround view image of the vehicle.

[0031] The current driving speed of the vehicle refers to the speed of the vehicle in the current driving process. The current driving speed can be obtained in various possible ways. In some embodiments of the present application, the current driving speed of the vehicle can be obtained through the CAN (Controller Area Network) bus of the vehicle.

[0032] The surround view image of the vehicle is a 360-degree overhead image of the vehicle body periphery. In the embodiments of the present application, the surround view image can be obtained in combination with a vehicle-mounted surround view system of the vehicle. The vehicle-mounted surround view system is to erect a plurality of wide-angle cameras around the vehicle, generally 4 to 8, which can cover all the field of view ranges of the vehicle periphery, and process the multi-channel videos collected at the same time into a 360-degree overhead view of the vehicle body periphery. Thus, through the overhead view of the vehicle body, it can be clearly seen whether there are obstacles around the vehicle, and the relative direction and distance of the obstacles.

[0033] Step S202: detecting lane lines in the surround view image.

[0034] As described above, in the obtained surround view image, lane lines of the road on which the vehicle travels can be included. The lane lines in the surround view image can be obtained by image processing on the obtained surround view image. The specific way of detecting and obtaining lane lines from the surround view image can be performed by using existing image processing methods, which are not limited in the embodiments of the present application.

[0035] In one embodiment, after detecting the lane lines in the surround view image, the detected lane lines can be fitted to obtain a lane line fitting equation in the vehicle body coordinate system, wherein the lane line fitting equation includes a current lane line fitting equation of the lane lines of the current lane of the vehicle. The specific fitting method is not limited in the embodiments of the present application, for example, the least square method can be used to fit the lane line fitting equation. It can be understood that, since the driving lane generally includes two lane lines on both sides, the current lane line fitting equation includes a first lane line equation and a second lane line equation. In the embodiments of the present application, the first lane line equation and the second lane line equation can both be a monomial quadratic equation.

[0036] When the first lane line equation and the second lane line equation are both monomial quadratic equations, the first lane line equation can be denoted as y1=a1x 2 +b1x+c1, and the second lane line equation can be denoted as y2=a2x 2 +b2x+c2. Wherein a1 is the quadratic term coefficient of the first lane line equation, b1 is the linear term coefficient of the first lane line equation, c1 is the constant term of the first lane line equation, which is also called the first constant term in the embodiments of the present application. a2 is the quadratic term coefficient of the second lane line equation, b2 is the linear term coefficient of the second lane line equation, and c2 is the constant term of the second lane line equation, which is also called the second constant term in the embodiments of the present application.

[0037] Step S203: detecting obstacle information in the surround view image.

[0038] In the field of driving, obstacles generally refer to objects that may affect or obstruct the normal driving of a vehicle, such as potholes, cones, water-filled barriers, motor vehicles, and pedestrians in front of or to the sides of the vehicle. Based on their movement type, obstacles can be divided into stationary obstacles and moving obstacles. Stationary obstacles can include, for example, potholes, cones, and water-filled barriers, while moving obstacles can include, for example, motor vehicles and pedestrians. Obstacle information can be obtained from surround-view images using any existing or future image processing methods, such as using neural networks to process surround-view images to identify obstacles, their types, and locations. This application does not impose specific limitations on these methods.

[0039] Step S204: Based on the lane lines, calculate the lane line correlation statistics for the most recent predetermined number of frame surround view images. The lane line correlation statistics include: deviation frequency, mean of center deviation value, and standard deviation of center deviation value.

[0040] Based on the identified lane lines, relevant statistics based on lane lines are obtained from the most recent predetermined number of surround view images. This predetermined number can be determined based on actual technical needs. Generally, the predetermined number should not be set too small, as too few surround view images will fail to reflect the actual changes; however, it should also not be set too large, as too many surround view images will increase processing load and include information from too early a time period, affecting the accuracy of the final judgment.

[0041] The calculated deviation frequency reflects the number of times the vehicle center changes within a predetermined number of frames; the mean of the center deviation values ​​reflects the average change in the vehicle center within that predetermined number of frames; and the standard deviation of the center deviation values ​​reflects the dispersion of the change in the vehicle center within that predetermined number of frames. Therefore, based on the calculated lane line-related statistics, the changes in the vehicle center during driving can be reasonably and effectively reflected, thereby enabling the evaluation of driving behavior.

[0042] In one embodiment, after fitting the current lane line fitting equation as described above, the lane line correlation statistics of the most recent predetermined number of surround view images are calculated based on the lane line fitting equation. Specifically, this may include: calculating and determining the lane line correlation statistics of the most recent predetermined number of surround view images based on the current lane line fitting equation corresponding to the most recent predetermined number of surround view images.

[0043] In a specific example, refer to Figure 3 As shown, the lane line correlation statistics of the most recent predetermined number of surround view images are calculated and determined based on the fitting equation of the current lane line corresponding to the most recent predetermined number of surround view images, which may include the following steps S2041 to S2043.

[0044] Step S2041: determining a center deviation value, a slope and a curvature of the lane line based on the current lane line fitting equation.

[0045] In the determination of the center deviation value, the slope and the curvature, taking the above-mentioned current lane line fitting equation including a first lane line equation of a monomial quadratic and a second lane line equation of a monomial quadratic as an example, the above-mentioned center deviation value can be the mean value of a first constant term of the first lane line equation and a second constant term of the second lane line equation; the slope can be the mean value of a first order term coefficient of the first lane line equation and a first order term coefficient of the second lane line equation; and the curvature can be the mean value of a second order term coefficient of the first lane line equation and a second order term coefficient of the second lane line equation.

[0046] In one specific example, based on the above-mentioned first lane line equation y1=a1x+b1x+c1 and the second lane line equation y2=a2x+b2x+c2, the center deviation value is denoted as d, the slope is denoted as θ, and the curvature is denoted as ρ, then d=(c1+c2) / 2, θ=(b1+b2) / 2, and ρ=(a1+a2) / 2. 2 2

[0047] Step S2042: determining a deviation frequency according to the center deviation value, the slope and the curvature corresponding to the latest predetermined number of surround view images.

[0048] In the determination of the deviation frequency according to the center deviation value, the slope and the curvature corresponding to the latest predetermined number of surround view images, one embodiment can include the following steps.

[0049] First, a deviation frequency coefficient of the latest predetermined number of surround view images is determined according to the center deviation value, the slope and the curvature corresponding to the latest predetermined number of surround view images. In some specific examples, the deviation frequency coefficient is the mean value of the number of times of direction change of the center deviation value, the slope and the curvature in the latest predetermined number of surround view images. For example, the number of times of direction change of the center deviation value in the latest predetermined number of surround view images is denoted as a first number, the number of times of direction change of the slope in the latest predetermined number of surround view images is denoted as a second number, and the number of times of direction change of the curvature in the latest predetermined number of surround view images is denoted as a third number, then the frequency coefficient can be the mean value of the first number, the second number and the third number.

[0050] In some specific examples, the frequency coefficient can be defined by the formula:

[0051]

[0052] ​​wherein the function g(x1, x2) is a deviation frequency function, defined as follows:

[0053]

[0054] It can be seen that g(x1, x2) is counted when the signs of x1 and x2 change, otherwise it is not counted, thereby embodying that the first number is the number of sign changes of the center deviation value d in N frames, It embodies that the first number is the number of sign changes of the center deviation value d in N frames, It embodies that the second number is the number of sign changes of the slope θ in N frames, It embodies that the third number is the number of sign changes of the curvature p in N frames.

[0055] Then, based on the deviation frequency coefficient and the frame rate of the surround view image, the deviation frequency is determined. In one embodiment, the deviation frequency can be the ratio of the product of the deviation frequency coefficient and the frame rate to the predetermined number.

[0056] Let the deviation frequency be f and the frame rate be s, then it can be expressed by the formula: f = gs / N.

[0057] Step S2043: According to the center deviation values corresponding to the latest predetermined number of frames of surround view images, the mean of the center deviation values and the standard deviation of the center deviation values are calculated and determined.

[0058] The mean of the center deviation values embodies the average situation of the change of the vehicle center in the predetermined number of frames, and therefore, in some specific examples, the average value of the center deviation values corresponding to the latest predetermined number of frames of surround view images can be directly taken as the mean of the center deviation values. The standard deviation of the center deviation values is calculated in combination with the mean of the center deviation values. In one embodiment, the standard deviation is the square root of the ratio of the sum of the absolute values of the center deviation values of each frame of surround view image to the mean of the center deviation values.

[0059] Let the mean of the center deviation values be μ d , and the standard deviation of the center deviation values be δ d , then it can be expressed by the formula:

[0060]

[0061] Step S205: Based on the current driving speed, the obstacle information and the lane line related statistics, the vehicle driving behavior is determined.

[0062] In some specific examples, when determining vehicle driving behavior based on the current driving speed, obstacle information, and lane line related statistics, different comprehensive determination methods can be used, taking into account the actual current driving speed, obstacle information, and lane line related statistics.

[0063] In some embodiments, dangerous driving behavior can be directly determined when the deviation frequency is greater than a deviation frequency threshold. Since the deviation frequency reflects the number of times the vehicle center changes within a predetermined number of frames, if the deviation frequency is greater than the deviation frequency threshold, it means that the vehicle center has changed too many times within the predetermined number of frames, and therefore dangerous driving behavior can be directly determined without considering other relevant information such as obstacle information.

[0064] The deviation frequency threshold can be set according to actual needs. In some embodiments, the deviation frequency threshold can be determined based on the current driving speed. For example, in some specific examples, the deviation frequency threshold can be the ratio of the current driving speed to a first predetermined parameter, denoted as T. f The first predetermined parameter is D f Then, it can be expressed by the formula T. f =v / D f The first predetermined parameter D f It is a constant value that can be set based on experience. For example, in some embodiments, the first predetermined parameter D can be... f Set it to 20.

[0065] In some embodiments, when the deviation frequency is less than or equal to a deviation frequency threshold, the mean of the center deviation values ​​is less than or equal to a mean threshold, and the standard deviation of the center deviation values ​​is less than or equal to a standard deviation threshold, it is determined that the vehicle has not engaged in dangerous driving behavior. Since the mean of the center deviation values ​​reflects the average change in the vehicle's center within a predetermined number of frames, and the standard deviation of the center deviation values ​​reflects the dispersion of the change in the vehicle's center within the predetermined number of frames, when the deviation frequency is less than or equal to a deviation frequency threshold, the mean of the center deviation values ​​is less than or equal to a mean threshold, and the standard deviation of the center deviation values ​​is less than or equal to a standard deviation threshold, it indicates that the change in the vehicle's center deviation values ​​is still within a controllable range, and it can be determined that the vehicle has not engaged in dangerous driving behavior.

[0066] The mean threshold can be determined based on the first constant term of the first lane line equation and the second constant term of the second lane line equation. In some embodiments, the mean threshold can be the ratio of the absolute value of the difference between the first and second constant terms to a second predetermined parameter. Let the mean threshold be T. μ The second predetermined parameter is D. μ, then it can be expressed by the formula The second predetermined parameter D μ is a constant value, which can be set in combination with experience. For example, in some embodiments, the second predetermined parameter D μ may be set to 10.

[0067] The standard deviation threshold is determined according to the first constant term of the first lane line equation and the second constant term of the second lane line equation. In some embodiments, the standard deviation threshold can be the absolute value of the difference between the first constant term and the second constant term, divided by a third predetermined parameter. Let the mean threshold be T σ , and the third predetermined parameter be D σ , then it can be expressed by the formula The third predetermined parameter D σ is a constant value, which can be set in combination with experience. For example, in some embodiments, the third predetermined parameter D σ may be set to 5.

[0068] In some embodiments, when the deviation frequency is less than or equal to the deviation frequency threshold, the mean of the center deviation value is greater than the mean threshold or the standard deviation of the center deviation value is greater than the standard deviation threshold, and there is a vehicle in the adjacent lane in the deviation direction of the vehicle, it is determined that the vehicle has a dangerous driving behavior.

[0069] In some embodiments, when the deviation frequency is less than or equal to the deviation frequency threshold, the mean of the center deviation value is greater than the mean threshold or the standard deviation of the center deviation value is greater than the standard deviation threshold, and there is no vehicle in the adjacent lane in the deviation direction of the vehicle, it is determined that the vehicle has not a dangerous driving behavior.

[0070] In some embodiments, when the deviation frequency is less than or equal to the deviation frequency threshold, the mean of the center deviation value is greater than the mean threshold, the standard deviation of the center deviation value is greater than the standard deviation threshold, and it is determined that there is a bypassing behavior according to the obstacle information, it is determined that the vehicle has not a dangerous driving behavior.

[0071] In some embodiments, it can be determined that there is a bypassing behavior when there is a static obstacle in front of the current lane and the mean of the center deviation value of the obstacle information is greater than the obstacle bypassing width threshold. In some embodiments, the obstacle bypassing width threshold is determined according to the vehicle body width and the obstacle width. For example, in some embodiments, the obstacle bypassing width threshold can be half the sum of the vehicle body width and the obstacle width.

[0072] Let the vehicle body width be w1 and the obstacle width be w2, then the obstacle bypassing width threshold is (w1+w2) / 2.

[0073] In some embodiments, when the deviation frequency is less than or equal to a deviation frequency threshold, the mean of the center deviation values is greater than a mean threshold, the standard deviation of the center deviation values is greater than a standard deviation threshold, and it is determined that there is a passing behavior according to the obstacle information, it is determined that no dangerous driving behavior occurs.

[0074] In some embodiments, the passing behavior can be determined when there is a vehicle in front of the current driving lane and it is determined that there is an accelerating behavior according to the current driving speed. In some embodiments, the presence of a vehicle in front of the current driving lane can be determined in combination with the detected obstacle information. In general, when there is a motor vehicle in front of the current driving lane, it can be determined that there is a vehicle in front of the current driving lane. When determining whether there is an accelerating behavior based on the current driving speed, any possible manner can be used, for example, when the vehicle driving speed continuously increases within a certain time, it can be determined that the vehicle has an accelerating behavior.

[0075] In some embodiments, when the deviation frequency is less than or equal to a deviation frequency threshold, the mean of the center deviation values is greater than a mean threshold, the standard deviation of the center deviation values is greater than a standard deviation threshold, and no detour behavior and passing behavior are detected according to the obstacle information, i.e. neither detour behavior nor passing behavior exists, it is determined that a dangerous driving behavior occurs.

[0076] Based on the above-described embodiments, Figure 4 A flowchart of a driving behavior detection method in a specific example is shown in FIG. 1. As shown in FIG. 1, in the specific driving behavior detection process, the following steps can be included. Figure 4

[0077] First, the current driving speed v of the vehicle is obtained through the CAN bus.

[0078] The surround view image of the vehicle is obtained using the vehicle-mounted surround view system, and the lane lines in the surround view image are detected, and the detected lane lines are fitted to obtain a lane line fitting equation. The lane line fitting equation includes the current lane line fitting equation of the current lane, which specifically includes a first lane line equation and a second lane line equation.

[0079] The surround view image is subjected to all-around obstacle detection to obtain obstacle information in the surround view image. The obtained obstacle information can include information of stationary obstacles such as potholes, cone barrels, and water barriers, and information of moving obstacles such as motor vehicles and pedestrians.

[0080] According to the first lane line equation y1=a1x 2 +b1x+c1and the second lane line equation y2=a2x 2 +b2x+c2, the center deviation value d, the slope θ, and the curvature p of the lane line can be determined. ​

[0081] According to the center deviation value, the slope and the curvature corresponding to the latest predetermined number (N frames) of surround view images, the deviation frequency coefficient g of the latest predetermined number of frames of surround view images is determined, and the frame rate of the vehicle-mounted surround view system is s, then the deviation frequency f = gs / N is calculated and obtained.

[0082] Then, the mean μ of the center deviation values of the latest predetermined number N frames of frames is calculated d and the standard deviation δ d .

[0083] Then, the driving behavior of the vehicle is judged in combination with the vehicle speed, the obstacle information and the lane line related statistics.

[0084] Wherein, if the deviation frequency f is greater than the threshold T f , it is judged that the vehicle has dangerous driving behavior. Otherwise, the next step of judgment is performed according to the mean μ d of the center deviation values, the standard deviation δ d .

[0085] If the mean μ d of the center deviation values and the standard deviation δ d are both not greater than the corresponding threshold T μ , T σ , it is determined that the vehicle does not have dangerous driving behavior.

[0086] If one of the mean μ d of the center deviation values and the standard deviation δ d is greater than the corresponding threshold T μ , T σ , and there is a vehicle in the adjacent lane in the direction of deviation of the vehicle, it is determined that the vehicle has dangerous driving behavior and an alarm is issued. If there is no vehicle in the adjacent lane in the direction of deviation of the vehicle, it is determined that the vehicle does not have dangerous driving behavior and is in a normal driving state.

[0087] If the mean μ d of the center deviation values and the standard deviation δ d are both greater than the corresponding threshold T μ , T σ , and there is an obstacle in front of the current lane, and the mean μ d of the center deviation values is greater than the obstacle detour width threshold, it is determined that the vehicle has detour behavior and does not have dangerous driving behavior.

[0088] If the mean μ d of the center deviation values and the standard deviation δ d are both greater than the corresponding threshold T μ , T σIf the mean value μ of the center deviation value is greater than the corresponding threshold value T μ, and the current driving lane is detected to have a vehicle in front, and the current driving speed collected from the CAN bus determines that the vehicle has an accelerating behavior, it is determined that the vehicle has a passing behavior, and no dangerous driving behavior occurs.

[0089] If the mean value μ of the center deviation value is greater than the corresponding threshold value T μ, and the current driving lane is detected to have a vehicle in front, and the current driving speed collected from the CAN bus determines that the vehicle has an accelerating behavior, it is determined that the vehicle has a passing behavior, and no dangerous driving behavior occurs. d , the standard deviation δ of the center deviation value is greater than the corresponding threshold value T δ, and no passing behavior or passing behavior is detected, it is determined that the vehicle has a dangerous driving behavior, and an alarm is issued. d μ σ

[0090] It should be understood that although each step in the related flowchart involved in the embodiments of the present application is displayed in sequence according to the arrow, these steps are not necessarily executed in the order indicated by the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other orders. Moreover, at least part of the steps in these flowcharts can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be alternately executed with at least part of other steps or steps or stages in other steps.

[0091] In one embodiment, as shown in Figure 5 , a driving behavior detection device is provided, comprising:

[0092] The speed acquisition module 501 is configured to acquire the current driving speed of the vehicle.

[0093] The image acquisition module 502 is configured to acquire the surround view image of the vehicle.

[0094] The lane line processing module 503 is configured to detect the lane line in the surround view image.

[0095] The obstacle information acquisition module 504 is configured to detect the obstacle information in the surround view image.

[0096] The statistical quantity determination module 505 is configured to calculate the lane line related statistical quantity of the nearest predetermined number of frames of surround view images based on the lane line, the lane line related statistical quantity including: deviation frequency, mean value of center deviation value, standard deviation of center deviation value.

[0097] The behavior judgment module 506 is configured to determine the vehicle driving behavior based on the current driving speed, the obstacle information and the lane line related statistical quantity.

[0098] ​​​In some embodiments, the method further comprises: fitting the detected lane line to obtain a lane line fitting equation in the body coordinate system, the lane line fitting equation comprising a current lane line fitting equation of the lane line of the current lane in which the vehicle is driving.

[0099] At this time, the statistical quantity determination module 505 determines lane line related statistical quantities of the most recent predetermined number of surround view images based on current lane line fitting equations corresponding to the most recent predetermined number of surround view images.

[0100] In some embodiments, the statistical quantity determination module 505 comprises:

[0101] a deviation information determination module configured to determine a center deviation value, a slope and a curvature of the lane line based on the current lane line fitting equation;

[0102] a deviation frequency determination module configured to determine a deviation frequency according to the center deviation value, the slope and the curvature corresponding to the most recent predetermined number of surround view images;

[0103] a mean standard deviation determination module configured to determine a mean value of the center deviation value and a standard deviation of the center deviation value according to the center deviation value corresponding to the most recent predetermined number of surround view images.

[0104] In some embodiments, the deviation frequency determination module determines a deviation frequency coefficient of the most recent predetermined number of surround view images according to the center deviation value, the slope and the curvature corresponding to the most recent predetermined number of surround view images; and determines the deviation frequency based on the deviation frequency coefficient and a frame rate of the surround view images.

[0105] In some embodiments, the deviation frequency coefficient is a mean value of a number of times that the center deviation value, the slope and the curvature change direction within the most recent predetermined number of surround view images.

[0106] In some embodiments, the deviation frequency is a ratio of a product of the deviation frequency coefficient and the frame rate to the predetermined number.

[0107] In some embodiments, the current lane line fitting equation comprises a first lane line equation of a quadratic function of one variable and a second lane line equation of a quadratic function of one variable; the center deviation value is a mean value of a first constant term of the first lane line equation and a second constant term of the second lane line equation; the slope is a mean value of a coefficient of a first order term of the first lane line equation and a coefficient of a first order term of the second lane line equation; and the curvature is a mean value of a coefficient of a second order term of the first lane line equation and a coefficient of a second order term of the second lane line equation.

[0108] In some embodiments, the behavior judging module 506 determines that the dangerous driving behavior occurs when the deviation frequency is greater than the deviation frequency threshold.

[0109] In some embodiments, the behavior judging module 506 determines that the dangerous driving behavior does not occur when the deviation frequency is less than or equal to the deviation frequency threshold, the mean of the center deviation values is less than or equal to the mean threshold, and the standard deviation of the center deviation values is less than or equal to the standard deviation threshold.

[0110] In some embodiments, the behavior judging module 506 determines that the dangerous driving behavior occurs when the deviation frequency is less than or equal to the deviation frequency threshold, the mean of the center deviation values is greater than the mean threshold or the standard deviation of the center deviation values is greater than the standard deviation threshold, and there is a vehicle in the adjacent lane in the deviation direction of the vehicle.

[0111] In some embodiments, the behavior judging module 506 determines that the dangerous driving behavior does not occur when the deviation frequency is less than or equal to the deviation frequency threshold, the mean of the center deviation values is greater than the mean threshold or the standard deviation of the center deviation values is greater than the standard deviation threshold, and there is no vehicle in the adjacent lane in the deviation direction of the vehicle.

[0112] In some embodiments, the behavior judging module 506 determines that the dangerous driving behavior does not occur when the deviation frequency is less than or equal to the deviation frequency threshold, the mean of the center deviation values is greater than the mean threshold, the standard deviation of the center deviation values is greater than the standard deviation threshold, and it is determined from the obstacle information that there is a detour behavior.

[0113] In some embodiments, the behavior judging module 506 determines that there is a detour behavior when there is a static obstacle in front of the current driving lane, and the mean of the obstacle information center deviation values is greater than the obstacle detour width threshold.

[0114] In some embodiments, the behavior judging module 506 determines that the dangerous driving behavior does not occur when the deviation frequency is less than or equal to the deviation frequency threshold, the mean of the center deviation values is greater than the mean threshold, the standard deviation of the center deviation values is greater than the standard deviation threshold, and it is determined from the obstacle information that there is a passing behavior.

[0115] In some embodiments, the behavior judging module 506 determines that there is a passing behavior when there is a vehicle in front of the current driving lane, and it is determined from the current driving speed that there is an acceleration behavior.

[0116] In some embodiments, the behavior judging module 506 determines that the dangerous driving behavior occurs when the deviation frequency is less than or equal to a deviation frequency threshold, the mean of the center deviation values is greater than a mean threshold, the standard deviation of the center deviation values is greater than a standard deviation threshold, and neither a detour behavior nor a overtaking behavior is detected according to the obstacle information.

[0117] The specific limitations of the driving behavior detection apparatus can refer to the limitations of the driving behavior detection method described above, which will not be repeated here. Each module in the above driving behavior detection apparatus can be realized by software, hardware and their combination in whole or in part. The above modules can be embedded in or independent of the processor in the electronic device in hardware form, or can be stored in the memory in the electronic device in software form, so that the processor calls and executes the operations corresponding to each of the above modules.

[0118] In one embodiment, an electronic device is provided, which can be a vehicle terminal device installed in a vehicle, and its internal structure diagram can be as shown in Figure 6 The electronic device includes a processor, a memory, a communication interface connected through a system bus, and in some embodiments, can further include a display screen and an input device. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The communication interface of the electronic device is used for wired or wireless communication with external terminals. Wireless communication can be achieved through WIFI, operator network, NFC (Near Field Communication) or other technologies. The computer program is executed by the processor to implement a driving behavior detection method. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen. The input device of the electronic device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the electronic device, or an external keyboard, touchpad or mouse, etc.

[0119] Those skilled in the art can understand that Figure 6 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied. The specific electronic device can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0120] In one embodiment, an electronic device is provided, which includes a memory and a processor, the memory stores a computer program, and the processor executes the computer program to implement the steps of the driving behavior detection method in any of the above embodiments.

[0121] In one embodiment, a computer readable storage medium is provided, having stored thereon a computer program which, when executed by a processor, implements the steps of the driving behavior detection method in any of the above embodiments.

[0122] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware, and the computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, the processes of the above-mentioned embodiments can be included. Any reference to memory, storage, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0123] Any combination of the technical features of the above embodiments can be made. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist, it should be considered as the scope of the present application.

[0124] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be pointed out that for those of ordinary skill in the art, without departing from the concept of the present application, some modifications and improvements can be made, which are all within the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. A driving behavior detection method characterized by, The method comprises: obtaining a current driving speed of the vehicle and a surround view image of the vehicle; detecting lane lines in the surround view image; fitting the detected lane lines to obtain a lane line fitting equation in a vehicle body coordinate system, the lane line fitting equation comprising a current lane line fitting equation of a lane line of a current lane of the vehicle; detecting obstacle information in the surround view image; based on current lane line fitting equations corresponding to a predetermined number of most recent frames of surround view images, calculating and determining lane line related statistics of the predetermined number of most recent frames of surround view images, the lane line related statistics comprising: a deviation frequency, a mean value of a center deviation value, and a standard deviation of the center deviation value; based on the current driving speed, the obstacle information, and the lane line related statistics, determining a vehicle driving behavior; the calculation and determination of the lane line related statistics of the predetermined number of most recent frames of surround view images based on the current lane line fitting equations corresponding to the predetermined number of most recent frames of surround view images comprises: based on the current lane line fitting equation, determining a center deviation value, a slope, and a curvature of the lane line; based on the center deviation value, the slope, and the curvature corresponding to the predetermined number of most recent frames of surround view images, determining a deviation frequency; based on the center deviation value corresponding to the predetermined number of most recent frames of surround view images, calculating and determining a mean value of the center deviation value and a standard deviation of the center deviation value.

2. The method of claim 1, wherein, the determination of the deviation frequency based on the center deviation value, the slope, and the curvature corresponding to the predetermined number of most recent frames of surround view images comprises: based on the center deviation value, the slope, and the curvature corresponding to the predetermined number of most recent frames of surround view images, determining a deviation frequency coefficient of the predetermined number of most recent frames of surround view images; based on the deviation frequency coefficient and a frame rate of the surround view image, determining the deviation frequency.

3. The method of claim 2, wherein, at least one of the following is included: the deviation frequency coefficient is a mean value of a number of times of direction changes of the center deviation value, the slope, and the curvature within the predetermined number of most recent frames of surround view images; the deviation frequency is a ratio of a product of the deviation frequency coefficient and the frame rate to the predetermined number.

4. The method of claim 1, wherein, the current lane line fitting equation comprises a first lane line equation of a quadratic function of one variable and a second lane line equation of a quadratic function of one variable; the center deviation value is a mean value of a first constant term of the first lane line equation and a second constant term of the second lane line equation; the slope is a mean value of a first order term coefficient of the first lane line equation and a first order term coefficient of the second lane line equation; the curvature is a mean value of a second order term coefficient of the first lane line equation and a second order term coefficient of the second lane line equation.

5. The method according to any one of claims 1 to 4, characterized in that, the determination of the vehicle driving behavior based on the current driving speed, the obstacle information, and the lane line related statistics comprises at least one of the following: when the deviation frequency is greater than a deviation frequency threshold, determining that a dangerous driving behavior occurs; when the deviation frequency is less than or equal to a deviation frequency threshold, the mean value of the center deviation value is less than or equal to a mean value threshold, and the standard deviation of the center deviation value is less than or equal to a standard deviation threshold, determining that the vehicle does not have a dangerous driving behavior; determining that the vehicle is in dangerous driving behavior when the frequency of lane deviation is less than or equal to a frequency threshold, a mean value of the center deviation value is greater than a mean threshold value or a standard deviation of the center deviation value is greater than a standard deviation threshold value, and there is a vehicle in the adjacent lane in the direction of lane deviation of the vehicle; determining that the vehicle is not in dangerous driving behavior when the frequency of lane deviation is less than or equal to a frequency threshold, a mean value of the center deviation value is greater than a mean threshold value or a standard deviation of the center deviation value is greater than a standard deviation threshold value, and there is no vehicle in the adjacent lane in the direction of lane deviation of the vehicle; determining that the vehicle is not in dangerous driving behavior when the frequency of lane deviation is less than or equal to a frequency threshold, a mean value of the center deviation value is greater than a mean threshold value, a standard deviation of the center deviation value is greater than a standard deviation threshold value, and it is determined that there is a detour behavior according to the obstacle information; determining that the vehicle is not in dangerous driving behavior when the frequency of lane deviation is less than or equal to a frequency threshold, a mean value of the center deviation value is greater than a mean threshold value, a standard deviation of the center deviation value is greater than a standard deviation threshold value, and it is determined that there is a passing behavior according to the obstacle information; determining that the vehicle is in dangerous driving behavior when the frequency of lane deviation is less than or equal to a frequency threshold, a mean value of the center deviation value is greater than a mean threshold value, a standard deviation of the center deviation value is greater than a standard deviation threshold value, and no detour behavior and passing behavior are detected according to the obstacle information.

6. The method of claim 5, wherein, including at least one of: determining that there is a detour behavior when there is a static obstacle in front of the current driving lane, and a mean value of the center deviation value of the obstacle information is greater than an obstacle detour width threshold value; determining that there is a passing behavior when there is a vehicle in front of the current driving lane, and it is determined that there is an acceleration behavior according to the current driving speed.

7. The method of claim 6, wherein, The current lane line fitting equation includes a first lane line equation of a monomial quadratic and a second lane line equation of a monomial quadratic, and the method further includes at least one of: The frequency threshold is determined according to the current driving speed; The mean threshold value is determined according to a first constant term of the first lane line equation and a second constant term of the second lane line equation; the first lane line equation and the second lane line equation are lane line fitting equations of the current lane in the vehicle body coordinate system; The standard deviation threshold value is determined according to a first constant term of the first lane line equation and a second constant term of the second lane line equation; the first lane line equation and the second lane line equation are lane line fitting equations of the current lane in the vehicle body coordinate system; The obstacle detour width threshold value is determined according to the vehicle body width and the obstacle width.

8. A driving behavior detection device characterized by comprising: The device includes: a speed acquisition module for acquiring the current driving speed of the vehicle; an image acquisition module for acquiring the surround view image of the vehicle; a lane line processing module for detecting lane lines in the surround view image; a lane line fitting module for fitting the detected lane lines to obtain lane line fitting equations in the vehicle body coordinate system, the lane line fitting equations including a current lane line fitting equation of the lane line of the current driving lane of the vehicle; an obstacle information acquisition module for detecting obstacle information in the surround view image; The statistical quantity determination module is configured to determine lane line related statistical quantities of the last predetermined number of surround view image pairs based on current lane line fitting equations corresponding to the last predetermined number of surround view image pairs, wherein the lane line related statistical quantities comprise: a deviation frequency, a mean value of a center deviation value, and a standard deviation of the center deviation value. The behavior determination module is configured to determine a driving behavior of the vehicle based on the current driving speed, the obstacle information, and the lane line related statistical quantities. The statistical quantity determination module is configured to determine a center deviation value, a slope, and a curvature of a lane line based on the current lane line fitting equation, determine a deviation frequency based on the center deviation value, the slope, and the curvature corresponding to the last predetermined number of surround view image pairs, and calculate and determine a mean value of the center deviation value and a standard deviation of the center deviation value based on the center deviation value corresponding to the last predetermined number of surround view image pairs.

9. An electronic device comprising a memory and a processor, the memory storing a computer program, characterized in that, The processor, when executing the computer program, implements the steps of the method in any one of claims 1 to 7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by the processor, implements the steps of the method in any one of claims 1 to 7.

Images