Automobile safe driving auxiliary control system and method based on visual identification

Abstract

The invention discloses an automobile safe driving auxiliary control system and method based on visual identification, and relates to the technical field of automobile safety. The vehicle hidden danger value is determined by analyzing the number of hidden danger vehicles from the traffic video, the weather state of the target vehicle outside the current driving evaluation time window is obtained, the environmental hidden danger value is obtained in combination with the time interval, and finally the risk estimation index is comprehensively calculated. In addition, when the road section state evaluation index of the remaining mileage of the target vehicle is analyzed, the traffic flow data, the congestion mileage and the accident number factor are considered, the driving environment risk is comprehensively evaluated, and the problems that in the prior art, evaluation of the driving environment risk is not comprehensive and careful, and the influence of multiple factors on driving safety cannot be comprehensively considered are solved.

Description

Technical Field

[0001] The present invention relates to the field of automobile safety technology, and in particular to an automobile safety driving assistance control system and method based on visual recognition. Background Art

[0002] With the vigorous development of the automobile industry, the number of cars has exploded, and traffic safety issues have become increasingly prominent. In a complex traffic environment, drivers face many challenges. The frequent occurrence of traffic accidents has posed a huge threat to people's lives and property safety. Therefore, the importance of automobile safety driving assistance control systems has become increasingly prominent.

[0003] However, the existing automobile safety driving assistance control system still has the following deficiencies in actual application:

[0004] Existing systems are not comprehensive and detailed enough in assessing driving environment risks. They fail to comprehensively consider the impact of multiple factors on driving safety, such as weather conditions, time of day, traffic volume, congestion mileage, and accidents, all of which have a significant impact on driving safety.

[0005] Furthermore, existing technologies also have shortcomings in driver status monitoring. Fatigue and distracted driving are significant factors in traffic accidents, but existing systems struggle to accurately monitor driver fatigue and distraction and implement appropriate countermeasures based on these conditions.

[0006] To this end, a car safety driving assistance control system and method based on visual recognition is introduced. Summary of the Invention

[0007] The purpose of the present invention is to solve the problems pointed out in the background technology and to propose a vehicle safety driving assistance control system and method based on visual recognition.

[0008] The purpose of the present invention can be achieved through the following technical solutions: a vehicle safety driving assistance control system based on visual recognition, comprising:

[0009] Visual acquisition module: collects traffic videos of the target vehicle while it is driving; and also captures driving videos of the target driver while he is driving to the destination;

[0010] Driving evaluation module: Based on the preset driving evaluation time window; after reaching the preset driving evaluation time window, the driving video of the target driver during the current driving evaluation time window is extracted and comprehensively evaluated to determine the target driver's driving state evaluation index PLz within the current driving evaluation time window;

[0011] Data processing module: extracts traffic video of the target vehicle driving to the destination during the current driving evaluation time window and performs a comprehensive evaluation to determine the target vehicle's risk estimation index within the current driving evaluation time window;

[0012] Decision control module: receives the target driver's driving state evaluation index PLz and the target vehicle's risk estimation index within the current driving evaluation time window, and performs corresponding operations to control the target vehicle.

[0013] As a preferred embodiment of the present invention, a driving video of the target driver during the current driving evaluation time window is extracted and a comprehensive evaluation is performed, specifically:

[0014] The driving video of the target driver driving to the destination during the current driving evaluation time window is divided into driving image frames; each driving image frame is preprocessed; each driving image frame is analyzed, and if it is detected that the target driver's mouth is open to a degree exceeding a set threshold and the duration exceeds a set time in a frame image during the current driving evaluation time window, it is determined that the target driver yawned during the current driving evaluation time window;

[0015] Counting the number of yawning behaviors of the target driver within the current driving evaluation time window, and extracting the duration corresponding to each yawning behavior and accumulating them to obtain the cumulative yawning duration of the target driver within the current driving evaluation time window;

[0016] Calculate the ratio of the number of yawning behaviors and the cumulative yawning duration to the current driving evaluation time window. This means the target driver's yawn frequency is calculated as the number of yawning behaviors divided by the current driving evaluation time window, and the yawn duration ratio is calculated as the cumulative yawning duration divided by the current driving evaluation time window. Label the yawn frequency and yawn duration ratio as yg and yh, respectively.

[0017] Locate the target driver's eye position during each frame and obtain the aspect ratio of the target driver's eyes in each frame. If the aspect ratio of the target driver's eyes in a certain frame is lower than a set threshold and lasts longer than a set time, it is determined that the target driver dozed off during the current driving assessment time window.

[0018] Counting the number of dozing behaviors of the target driver within the current driving evaluation time window, and extracting the duration corresponding to each dozing behavior and accumulating them to obtain the cumulative drowsiness duration of the target driver within the current driving evaluation time window;

[0019] Calculate the ratios of the number of dozing behaviors and the cumulative dozing duration to the current driving evaluation time window respectively. That is, the target driver's drowsiness frequency is calculated by dividing the number of dozing behaviors by the current driving evaluation time window, and the target driver's drowsiness duration ratio is calculated by dividing the cumulative dozing duration by the current driving evaluation time window. Mark the drowsiness frequency and drowsiness duration ratio as fg and fh respectively.

[0020] As a preferred embodiment of the present invention, the driving state evaluation index of the target driver within the current driving evaluation time window is determined as follows:

[0021] Substitute the target driver's yawn frequency yg, yawn duration ratio yh, sleepiness frequency fg, and sleepiness duration ratio fh in the current driving evaluation time window into the formula Perform weighted calculation to obtain the target driver's driving state evaluation index PLz within the current driving evaluation time window; They represent the yawning threshold frequency, yawning duration threshold ratio, dozing threshold frequency, and dozing duration threshold ratio corresponding to the current driving evaluation time window respectively; They are the influencing weight factors of yawn frequency yg, yawn duration ratio yh, sleepiness frequency fg and sleepiness duration ratio fh.

[0022] As a preferred embodiment of the present invention, traffic video of the target vehicle during its driving to the destination in the current driving evaluation time window is extracted and a comprehensive evaluation is performed, specifically:

[0023] Divide the traffic video of the target vehicle driving towards the destination in the current driving evaluation time window into traffic image frames, and obtain the number of identifiable vehicles other than the target vehicle in each traffic image frame;

[0024] Draw a circle with the target vehicle's position in each frame of traffic image as the center and the set distance as the radius, and identify the number of identifiable vehicles appearing in the circle in each frame of traffic image;

[0025] The average number of identifiable vehicles appearing within the circle in each frame of traffic image is calculated, and the calculated average is rounded off to the nearest integer, which is used as the number of potential vehicles within the current driving evaluation time window for the target vehicle.

[0026] The value ranges of each group of vehicles corresponding to the number of vehicles with hidden dangers are preset, and each group of vehicle value ranges is set to correspond to a vehicle hidden danger value; after matching the number of vehicles with hidden dangers of the target vehicle in the current driving evaluation time window with the value ranges of each group of vehicles, the vehicle hidden danger value of the target vehicle in the current driving evaluation time window is obtained.

[0027] As a preferred embodiment of the present invention, the risk estimation index of the target vehicle within the current driving evaluation time window is determined as follows:

[0028] Obtain the weather conditions of the target vehicle outside the current driving evaluation time window; weather conditions include rain, heavy snow, and fog;

[0029] Set a weather hazard base value corresponding to each weather state; obtain the time interval of the target vehicle corresponding to the current driving evaluation time window, and set each time interval to correspond to an environmental state; the environmental state includes daytime and nighttime;

[0030] Set a basic additional coefficient corresponding to each environmental state;

[0031] Multiplying the basic weather hazard value determined for the target vehicle within the current driving evaluation time window by the basic additional coefficient to obtain the environmental hazard value for the target vehicle within the current driving evaluation time window;

[0032] Set the weight coefficients corresponding to the vehicle hidden danger value and the environmental hidden danger value respectively, multiply the vehicle hidden danger value and the environmental hidden danger value of the target vehicle in the current driving evaluation time window by the corresponding weight coefficients respectively, and then sum them to obtain the risk estimation index of the target vehicle in the current driving evaluation time window.

[0033] As a preferred embodiment of the present invention, the driving state evaluation index PLz of the target driver in the current driving evaluation time window is received, and corresponding operations are performed to control the target vehicle, specifically:

[0034] A threshold index range corresponding to the driving state evaluation index PLz is preset. If the target driver's driving state evaluation index PLz within the current driving evaluation time window is within the preset threshold index range, an alarm signal is sent to the target driver, and a notification is sent to the target driver whether to activate the assisted driving system;

[0035] If the target driver's driving state evaluation index PLz within the current driving evaluation time window is higher than the preset threshold index range, the assisted driving system will be directly turned on, the vehicle's hazard lights will be turned on, and an alarm signal will be sent to the target driver.

[0036] As a preferred embodiment of the present invention, a risk estimation index of a target vehicle is received, and corresponding operations are performed to control the target vehicle, specifically:

[0037] Three groups of index value ranges corresponding to the risk estimation index are set, and the three groups of index value ranges respectively correspond to a vehicle risk level, including a low vehicle risk level, a medium vehicle risk level, and a high vehicle risk level;

[0038] Determine the vehicle risk level of the target vehicle within the current driving assessment time window based on the three sets of index value ranges set;

[0039] If the vehicle risk level is a low vehicle risk level, the target driver will be notified whether to turn on the assisted driving system;

[0040] If the vehicle risk level is medium, it will identify whether the target vehicle is in an assisted driving system. If so, it will inform the target driver of the vehicle risk level and whether to turn off the assisted driving system. It will also automatically adjust the sensitivity of the target vehicle's automatic emergency braking system according to the settings.

[0041] If the vehicle risk level is a higher vehicle risk level, it will identify whether the target vehicle is in the assisted driving system. If it is in the assisted driving system, the assisted driving system will be directly turned off, and the sensitivity of the target vehicle's automatic emergency braking system will be automatically adjusted according to the settings; at the same time, the road section status evaluation index of the remaining mileage of the target vehicle will be analyzed. If the road section status evaluation index of the remaining mileage of the target vehicle is higher than the preset threshold index, the current position of the target vehicle will be used as the starting point and the destination will be used as the end point. Each planned route will be obtained and pushed to the target driver. At the same time, each planned route will be sorted from small to large according to the corresponding road section status evaluation index.

[0042] As a preferred embodiment of the present invention, the road section status evaluation index of the target vehicle's remaining mileage is analyzed, specifically:

[0043] Obtain the traffic flow data, congestion mileage, and number of accidents of the target vehicle's remaining mileage, set the traffic flow value ranges and congestion mileage value ranges corresponding to the traffic flow data and congestion mileage, respectively. Each traffic flow value range corresponds to a traffic flow assessment value, and each congestion mileage group corresponds to a congestion assessment value.

[0044] After determining the traffic flow assessment value and congestion assessment value of the target vehicle's remaining mileage, based on the preset weight coefficients of the traffic flow assessment value and the congestion assessment value, respectively, the traffic flow assessment value and the congestion assessment value of the target vehicle's remaining mileage are multiplied by the corresponding preset weight coefficients, and then multiplied together to obtain the driving experience value of the target vehicle's remaining mileage;

[0045] According to the number of accidents in the remaining mileage of the target vehicle, the impact coefficient is determined, and the driving experience value is multiplied by the set impact coefficient to obtain the road section condition evaluation index of the remaining mileage of the target vehicle.

[0046] As a preferred embodiment of the present invention, the influence coefficient is determined according to the number of accidents of the target vehicle with respect to the remaining mileage, specifically:

[0047] When the number of accidents in the target vehicle's remaining mileage is 0, the initial value of the impact coefficient is set to an integer of 1. If the number of accidents is greater than 0, an impact adjustment value corresponding to the number of accidents is set, and the impact adjustment value is added to the integer of 1 to serve as the impact coefficient multiplied by the driving experience value.

[0048] That is, the value range of each component number corresponding to the number of accidents is set, and each component number value range corresponds to an impact adjustment value.

[0049] The automobile safe driving assistance control method based on visual recognition includes:

[0050] Data collection: Collect traffic videos of the target vehicle while driving and driving videos of the target driver while driving to the destination;

[0051] Driver evaluation: After reaching the preset driving evaluation time window, the driving video of the target driver during the current driving evaluation time window is extracted and comprehensively evaluated to determine the target driver's driving state evaluation index PLz within the current driving evaluation time window;

[0052] Environmental assessment: Extract traffic video of the target vehicle driving to the destination during the current driving assessment time window and conduct a comprehensive assessment to determine the target vehicle's risk estimation index within the current driving assessment time window;

[0053] Decision-making control: The target vehicle is controlled accordingly based on the target driver's driving state evaluation index PLz within the current driving evaluation time window and the target vehicle's risk estimation index.

[0054] Compared with the prior art, the present invention has the following beneficial effects:

[0055] The present invention determines the vehicle hazard value by analyzing the number of vehicles with potential hazards from traffic videos. It also obtains the weather conditions of the target vehicle outside the current driving assessment time window, combines this with the time interval to derive the environmental hazard value, and ultimately calculates a comprehensive risk estimation index. Furthermore, when analyzing the road section status assessment index for the target vehicle's remaining mileage, it considers factors such as traffic flow data, congestion mileage, and the number of accidents to comprehensively assess driving environment risks. This addresses the problem in existing technologies where driving environment risk assessments are not comprehensive and detailed enough and fail to comprehensively consider the impact of multiple factors on driving safety.

[0056] The present invention quantifies the driver's fatigue performance into indicators such as yawn frequency, yawn duration ratio, and doze frequency and doze duration ratio. These data are obtained by analyzing each frame of the driving video and then substituted into a formula for weighted calculation to derive a driving status assessment index, thereby accurately monitoring the driver's fatigue status.

[0057] The present invention intelligently controls the assisted driving system based on the driving state assessment index and risk prediction index. Different actions are performed when the driver state assessment index falls within different ranges, including sending an alarm signal, informing whether to activate the assisted driving system, or directly activating the assisted driving system and turning on the hazard lights. The assisted driving system's activation and deactivation, as well as the sensitivity of the automatic emergency braking system, are also adjusted based on the vehicle's risk level to ensure driving safety.

[0058] The present invention predicts potential dangers in advance based on the risk estimation index and the road section status assessment index. When the risk estimation index shows a higher vehicle risk level and the road section status assessment index of the target vehicle's remaining mileage is higher than a preset threshold index, the target vehicle's current position is used as the starting point and the destination is used as the end point. Each planned route is obtained and pushed to the target driver, and each planned route is sorted from small to large according to the corresponding road section status assessment index, helping the driver to plan a safer driving route in advance and avoid danger. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] To facilitate understanding by those skilled in the art, the present invention is further described below with reference to the accompanying drawings.

[0060] Figure 1 It is a principle block diagram of the present invention;

[0061] Figure 2 Flowchart of the present invention. DETAILED DESCRIPTION

[0062] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0063] Example 1

[0064] See also Figure 1 As shown in the figure, the automobile safety driving assistance control system based on visual recognition includes a visual acquisition module, a driving evaluation module, a data processing module, and a decision control module:

[0065] The visual acquisition module is used to use the on-board visual sensor to collect traffic videos of the target vehicle while it is driving; at the same time, the built-in high-definition camera of the target vehicle is used to capture the driving video of the target driver while driving to the destination;

[0066] The driving evaluation module is used to evaluate the target driver's driving status within a preset driving evaluation time window, which can be set to, for example, every 5 minutes. After reaching the preset driving evaluation time window, the driving video of the target driver during the current driving evaluation time window is extracted and comprehensively evaluated to determine the target driver's driving status evaluation index PLz within the current driving evaluation time window.

[0067] It should be noted that the driving evaluation time window is set based on the estimated driving time of the current target vehicle during its journey to the destination. By presetting the time value ranges corresponding to the estimated driving time, each time value range is set as a driving evaluation time window. When the target driver determines the current destination, the estimated driving time to the destination is obtained, and the driving evaluation time window corresponding to the current destination is determined.

[0068] For example, the following sets of duration value ranges and their corresponding driving evaluation time windows are preset:

[0069] (0-10 minutes) - assessment every 5 minutes; (10-30 minutes) - assessment every 10 minutes; 30-60 minutes - assessment every 15 minutes; more than 60 minutes - assessment every 25 minutes;

[0070] When the target driver decides to go to Destination A, the map navigation system calculates the estimated driving time to Destination A as 45 minutes. Since 45 minutes is within the range of 30-60 minutes, the driving evaluation time window corresponding to Destination A is determined to be every 15 minutes.

[0071] While the vehicle is driving, the system analyzes the driver's driving video every 15 minutes to determine whether the driver is fatigued, distracted, or driving in violation of regulations. If the estimated driving time changes due to road conditions, the driving assessment time window will be updated;

[0072] Determine the target driver's driving status evaluation index within the current driving evaluation time window, specifically:

[0073] The driving video of the target driver driving to the destination during the current driving evaluation time window is divided into driving image frames; each driving image frame is preprocessed; the preprocessing includes denoising and enhancement; each driving image frame is analyzed using a pre-trained VGGNet model. If the target driver's mouth is open to a degree exceeding a set threshold and the duration exceeds a set time period (the set time period can be set to 3.5 seconds) in a frame within the current driving evaluation time window, it is determined that the target driver yawned during the current driving evaluation time window;

[0074] Counting the number of yawning behaviors of the target driver within the current driving evaluation time window, and extracting the duration corresponding to each yawning behavior and accumulating them to obtain the cumulative yawning duration of the target driver within the current driving evaluation time window;

[0075] Calculate the ratio of the number of yawning behaviors and the cumulative yawning duration to the current driving evaluation time window. This means the target driver's yawn frequency is calculated as the number of yawning behaviors divided by the current driving evaluation time window, and the yawn duration ratio is calculated as the cumulative yawning duration divided by the current driving evaluation time window. Label the yawn frequency and yawn duration ratio as yg and yh, respectively.

[0076] Locate the target driver's eye position during each frame and obtain the aspect ratio of the target driver's eyes in each frame. If the aspect ratio of the target driver's eyes in a certain frame is lower than a set threshold and lasts longer than a set time, it is determined that the target driver dozed off during the current driving assessment time window.

[0077] Counting the number of dozing behaviors of the target driver within the current driving evaluation time window, and extracting the duration corresponding to each dozing behavior and accumulating them to obtain the cumulative drowsiness duration of the target driver within the current driving evaluation time window;

[0078] Calculate the ratio of the number of dozing behaviors and the cumulative dozing duration to the current driving evaluation time window. This means the target driver's dozing frequency is calculated as the number of dozing behaviors / the current driving evaluation time window, and the target driver's dozing duration is calculated as the cumulative dozing duration / the current driving evaluation time window. Label the dozing frequency and dozing duration as fg and fh, respectively.

[0079] It should be noted that the driver's fatigue symptoms (yawning, dozing off) are quantified into indicators such as yawn frequency, yawn duration ratio, dozing frequency and dozing duration ratio to form a specific driving status assessment index. These quantitative indicators can more accurately judge the driver's fatigue level.

[0080] Substitute the target driver's yawn frequency yg, yawn duration ratio yh, sleepiness frequency fg, and sleepiness duration ratio fh in the current driving evaluation time window into the formula Perform weighted calculation to obtain the target driver's driving state evaluation index PLz within the current driving evaluation time window; Respectively represent the yawning threshold frequency, yawning duration threshold ratio, dozing threshold frequency, and dozing duration threshold ratio corresponding to the current driving evaluation time window; set according to the specific window duration range of the current driving evaluation time window; are the weight factors of yawn frequency yg, yawn duration ratio yh, doze frequency fg and doze duration ratio fh respectively; and ;

[0081] The data processing module is used to extract traffic videos of the target vehicle during its driving to the destination in the current driving evaluation time window and perform a comprehensive evaluation to determine the risk estimation index of the target vehicle in the current driving evaluation time window;

[0082] Specifically:

[0083] Divide the traffic video of the target vehicle driving towards the destination in the current driving evaluation time window into traffic image frames, and obtain the number of identifiable vehicles other than the target vehicle in each traffic image frame;

[0084] A circle is drawn with the target vehicle's position in each frame of traffic image as the center and the set distance as the radius. The number of identifiable vehicles that appear within the circle in each frame of traffic image is identified. Vehicles that are partially within the circle are still counted as being within the circle.

[0085] The average number of identifiable vehicles appearing within the circle in each frame of traffic image is calculated, and the calculated average is rounded off to the nearest integer, which is used as the number of potential vehicles within the current driving evaluation time window for the target vehicle.

[0086] Preset the value ranges for each group of vehicles corresponding to the number of vehicles with hidden dangers, and set each vehicle value range to correspond to a vehicle hidden danger value; the vehicle hidden danger value range is set to 1-10, and the higher the number of vehicles with hidden dangers, the higher the corresponding matched vehicle hidden danger value; after matching the number of vehicles with hidden dangers in the current driving evaluation time window of the target vehicle with the value ranges of each group of vehicles, the vehicle hidden danger value of the target vehicle in the current driving evaluation time window is obtained;

[0087] Obtain the target vehicle's weather conditions outside the current driving evaluation time window; weather conditions include rain, heavy snow, and fog; weather conditions can be obtained through meteorological data interface calls or vehicle network data interaction;

[0088] Set a weather hazard basic value corresponding to each weather state; the weather hazard basic value range is set to 1-10, where the weather hazard basic value of foggy days is greater than the weather hazard basic value of rainy days and greater than the weather hazard basic value of heavy snow; obtain the time interval of the target vehicle corresponding to the current driving evaluation time window, and set each time interval to correspond to an environmental state; the environmental state includes daytime and nighttime;

[0089] Set a basic additional coefficient for each environmental state; the basic additional coefficients are 0.957 and 1.132, corresponding to daytime and nighttime respectively;

[0090] Multiplying the basic weather hazard value determined for the target vehicle within the current driving evaluation time window by the basic additional coefficient to obtain the environmental hazard value for the target vehicle within the current driving evaluation time window;

[0091] Set the weight coefficients corresponding to the vehicle hidden danger value and the environmental hidden danger value respectively, multiply the vehicle hidden danger value and the environmental hidden danger value of the target vehicle in the current driving evaluation time window by the corresponding weight coefficients, and then sum them to obtain the risk estimation index of the target vehicle in the current driving evaluation time window;

[0092] It should be noted that the number of vehicles with hidden dangers is analyzed from traffic videos, and then the vehicle hidden danger value is determined. At the same time, the environmental hidden danger value is obtained by combining the weather conditions and the time period. Finally, the risk estimation index is comprehensively calculated. This quantitative assessment method can more accurately reflect the risk level of the driving environment than traditional simple judgment.

[0093] The decision control module is used to receive the target driver's driving state evaluation index PLz and the target vehicle's risk estimation index within the current driving evaluation time window, and perform corresponding operations to control the target vehicle;

[0094] Specifically:

[0095] S1: A threshold index range corresponding to a preset driving state evaluation index PLz is set. If the target driver's driving state evaluation index PLz within the preset threshold index range within the current driving evaluation time window is within the preset threshold index range, an alarm signal is sent to the target driver via the vehicle terminal; the alarm signal can be a preset music; and a notification is simultaneously sent to the target driver regarding whether to enable the assisted driving system, which includes adaptive cruise control and lane keeping functions.

[0096] If the target driver's driving state evaluation index PLz within the current driving evaluation time window is higher than the preset threshold index range, the assisted driving system will be directly activated, the vehicle's hazard lights will be turned on, and an alarm signal will be sent to the target driver;

[0097] It should be noted that the preset threshold index range of the driving state evaluation index PLz can accurately judge the driver's state. When the driver state evaluation index is within the threshold range, an alarm signal is sent to the driver in time and the driver is asked whether to turn on the assisted driving system. This can not only attract the driver's attention to his own state, but also give him the right to choose independently; if the evaluation index is higher than the threshold range, the assisted driving system is directly turned on and the double flash is turned on, and an alarm is sounded at the same time to ensure that measures can be taken quickly when the driver is in poor condition to ensure driving safety.

[0098] S2: setting three groups of index value ranges corresponding to the risk estimation index, and the three groups of index value ranges respectively correspond to a vehicle risk level, including a low vehicle risk level, a medium vehicle risk level, and a high vehicle risk level;

[0099] Determine the vehicle risk level of the target vehicle within the current driving assessment time window based on the three sets of index value ranges set;

[0100] S2-201: If the vehicle risk level is a low vehicle risk level, a notification is sent to the target driver regarding whether to enable the assisted driving system;

[0101] S2-202: If the vehicle risk level is medium, identify whether the target vehicle is in an assisted driving system. If so, inform the target driver of the vehicle risk level and whether to disable the assisted driving system. The system also automatically adjusts the sensitivity of the target vehicle's automatic emergency braking system based on the settings. The sensitivity of the automatic emergency braking system of a vehicle corresponding to a medium vehicle risk level is set.

[0102] S2-203: If the vehicle risk level is a high vehicle risk level, identify whether the target vehicle is in an assisted driving system. If so, directly turn off the assisted driving system and automatically adjust the sensitivity of the target vehicle's automatic emergency braking system according to the settings; set the sensitivity of the target vehicle's automatic emergency braking system to correspond to a high vehicle risk level; and simultaneously analyze the target vehicle's remaining mileage road condition assessment index. If the target vehicle's remaining mileage road condition assessment index is higher than a preset threshold index, then use the target vehicle's current position as the starting point and the destination as the end point to obtain each planned route and push it to the target driver. At the same time, each planned route is sorted from small to large according to the corresponding road condition assessment index.

[0103] It's important to note that the entire decision-making and control process is based on real-time risk assessment and road condition assessment indices. This allows for real-time perception of changes in risk in the vehicle's driving environment and prompt adjustments. Whether it's enabling or disabling the assisted driving system, adjusting the sensitivity of the automatic emergency braking system, or rerouting, all actions are dynamically adjusted based on actual risk conditions, ensuring the vehicle maintains a safe driving strategy and effectively responds to the ever-changing traffic environment.

[0104] S2-204: Obtain traffic flow data, congestion mileage, and the number of accidents for the target vehicle's remaining mileage, and set the traffic flow value ranges and congestion mileage value ranges corresponding to the traffic flow data and congestion mileage, respectively. Each traffic flow value range corresponds to a traffic flow assessment value, and each congestion mileage corresponds to a congestion assessment value. The traffic flow assessment value and congestion assessment value are both set in the range of 1-10. A higher traffic flow corresponds to a higher traffic flow assessment value, and a higher congestion mileage corresponds to a higher congestion assessment value.

[0105] After determining the traffic flow assessment value and congestion assessment value of the target vehicle's remaining mileage, based on the preset weight coefficients of the traffic flow assessment value and the congestion assessment value, respectively, the traffic flow assessment value and the congestion assessment value of the target vehicle's remaining mileage are multiplied by the corresponding preset weight coefficients, and then multiplied together to obtain the driving experience value of the target vehicle's remaining mileage;

[0106] Determine the impact coefficient based on the number of accidents with the remaining mileage of the target vehicle, and multiply the driving experience value by the set impact coefficient to obtain the road section condition evaluation index with the remaining mileage of the target vehicle;

[0107] When the number of accidents in the target vehicle's remaining mileage is 0, the initial value of the impact coefficient is set to an integer of 1. If the number of accidents is greater than 0, an impact adjustment value corresponding to the number of accidents is set, and the impact adjustment value is added to the integer of 1 to serve as the impact coefficient multiplied by the driving experience value.

[0108] That is, the value range of each component number corresponding to the number of accidents is set, and each component number value range corresponds to an impact adjustment value; the impact adjustment value range is set to 0.137-0.149, and the greater the number of accidents, the higher the corresponding impact adjustment value;

[0109] Example 2

[0110] See also Figure 2 As shown, based on the visual recognition-based automobile driving safety assistance control system provided in Example 1 of this application, Example 2 of this application proposes a visual recognition-based automobile driving safety assistance control method. Example 2 is merely a preferred embodiment of Example 1, and the implementation of Example 2 will not affect the independent implementation of Example 1.

[0111] Specifically, the difference of the automobile safe driving assistance control method based on visual recognition provided in Example 2 of the present application is that it includes:

[0112] Data collection: Collect traffic videos of the target vehicle while driving and driving videos of the target driver while driving to the destination;

[0113] Driver evaluation: After reaching the preset driving evaluation time window, the driving video of the target driver during the current driving evaluation time window is extracted and comprehensively evaluated to determine the target driver's driving state evaluation index PLz within the current driving evaluation time window;

[0114] Environmental assessment: Extract traffic video of the target vehicle driving to the destination during the current driving assessment time window and conduct a comprehensive assessment to determine the target vehicle's risk estimation index within the current driving assessment time window;

[0115] Decision-making control: The target vehicle is controlled accordingly based on the target driver's driving state evaluation index PLz within the current driving evaluation time window and the target vehicle's risk estimation index.

[0116] The preferred embodiments of the present invention disclosed above are intended only to help illustrate the present invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the present invention to specific embodiments. Obviously, many modifications and variations are possible based on the contents of this specification. These embodiments are selected and described in detail in this specification to better explain the principles and practical applications of the present invention, thereby enabling those skilled in the art to better understand and utilize the present invention. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A vehicle safety driving assistance control system based on visual recognition, characterized in that: include: Visual acquisition module: collects traffic videos of target vehicles while driving; At the same time, the driving video of the target driver is captured while driving to the destination; Driving evaluation module: according to the preset driving evaluation time window; after reaching the preset driving evaluation time window, extract the driving video of the target driver in the process of driving to the destination in the current driving evaluation time window, and conduct a comprehensive evaluation to determine the driving state evaluation index PLz of the target driver in the current driving evaluation time window; Data processing module: extract the traffic video of the target vehicle driving to the destination in the current driving evaluation time window and conduct a comprehensive evaluation to determine the risk estimation index of the target vehicle in the current driving evaluation time window; Decision control module: receives the driving state evaluation index PLz of the target driver within the current driving evaluation time window and the risk estimation index of the target vehicle, and performs corresponding operations to control the target vehicle.

2. The automobile safe driving assistance control system based on visual recognition according to claim 1 is characterized in that: Extract the driving video of the target driver during the current driving evaluation time window and perform a comprehensive evaluation, specifically: Divide the driving video of the target driver in the process of driving to the destination in the current driving evaluation time window into driving image frames; and pre-process each driving image frame; Analyze each frame of driving image. If the target driver's mouth is open to a degree exceeding a set threshold and the duration exceeds a set duration in a certain frame of image within the current driving evaluation time window, it is determined that the target driver yawns within the current driving evaluation time window. Counting the number of yawning behaviors of the target driver within the current driving evaluation time window, and extracting the duration corresponding to each yawning behavior and accumulating them, to obtain the cumulative yawning duration of the target driver within the current driving evaluation time window; The ratios of the number of yawning behaviors and the cumulative yawning duration to the current driving evaluation time window are calculated respectively, that is, the yawning frequency of the target driver is calculated by the number of yawning behaviors / the current driving evaluation time window, and the yawning duration ratio of the target driver is calculated by the cumulative yawning duration / the current driving evaluation time window; the yawning frequency and yawning duration ratio are marked as yg and yh respectively; Locate the target driver's eye position in each frame of image, and obtain the aspect ratio of the target driver's eyes in each frame of image. If the aspect ratio of the target driver's eyes in a certain frame of image is lower than a set threshold and lasts longer than a set time, it is determined that the target driver has dozed off in the current driving evaluation time window; Counting the number of dozing behaviors of the target driver within the current driving evaluation time window, and extracting the duration corresponding to each dozing behavior and accumulating them, to obtain the cumulative drowsiness duration of the target driver within the current driving evaluation time window; Calculate the ratio of the number of dozing behaviors and the cumulative drowsiness duration to the current driving evaluation time window respectively, that is, the drowsiness frequency of the target driver is calculated by the number of dozing behaviors / the current driving evaluation time window, and the drowsiness duration ratio of the target driver is calculated by the cumulative drowsiness duration / the current driving evaluation time window; mark the drowsiness frequency and drowsiness duration ratio as fg and fh respectively.

3. The automobile safe driving assistance control system based on visual recognition according to claim 2 is characterized in that: Determine the driving status evaluation index of the target driver within the current driving evaluation time window, specifically: Substitute the target driver's yawning frequency yg, yawning duration ratio yh, drowsiness frequency fg and drowsiness duration ratio fh in the current driving evaluation time window into the formula A weighted calculation is performed to obtain the target driver's driving state evaluation index PLz within the current driving evaluation time window; wherein They respectively represent the yawning threshold frequency, yawning duration threshold ratio, dozing threshold frequency, and dozing duration threshold ratio corresponding to the current driving evaluation time window; They are the influencing weight factors of yawn frequency yg, yawn duration ratio yh, doze frequency fg and doze duration ratio fh.

4. The automobile safe driving assistance control system based on visual recognition according to claim 3 is characterized in that: Extract the traffic video of the target vehicle driving to the destination in the current driving evaluation time window and conduct a comprehensive evaluation, specifically: Divide the traffic video of the target vehicle driving to the destination in the current driving evaluation time window into frames of traffic images, and obtain the number of identifiable vehicles other than the target vehicle in each frame of traffic image; Draw a circle with the position of the target vehicle in each frame of traffic image as the center and the set distance as the radius, and identify the number of identifiable vehicles appearing in the circle in each frame of traffic image; The average number of identifiable vehicles that appear in the circle in each frame of traffic image is calculated, and the calculated average is rounded off to the nearest integer, which is used as the number of potential vehicles of the target vehicle in the current driving evaluation time window; The value ranges of each group of vehicles corresponding to the number of vehicles with hidden dangers are preset, and each group of vehicle value ranges is set to correspond to a vehicle hidden danger value; after matching the number of vehicles with hidden dangers of the target vehicle in the current driving evaluation time window with the value ranges of each group of vehicles, the vehicle hidden danger value of the target vehicle in the current driving evaluation time window is obtained.

5. The automobile safe driving assistance control system based on visual recognition according to claim 4 is characterized in that: Determine the risk estimation index of the target vehicle within the current driving evaluation time window, specifically: Obtain the weather conditions of the target vehicle outside the current driving evaluation time window; the weather conditions include rain, heavy snow, and fog; Set a weather hazard basic value corresponding to each weather state; obtain the time interval of the target vehicle corresponding to the current driving evaluation time window, and set each time interval corresponding to an environmental state; the environmental state includes daytime and nighttime; Set different environmental states to correspond to a basic additional coefficient; The basic value of weather hazards determined for the target vehicle in the current driving evaluation time window is multiplied by the basic additional coefficient to obtain the environmental hazard value of the target vehicle in the current driving evaluation time window; The weight coefficients corresponding to the vehicle hidden danger value and the environmental hidden danger value are set respectively, and the vehicle hidden danger value and the environmental hidden danger value of the target vehicle in the current driving evaluation time window are multiplied by the corresponding weight coefficients respectively, and then the sum is obtained to obtain the risk estimation index of the target vehicle in the current driving evaluation time window.

6. The automobile safe driving assistance control system based on visual recognition according to claim 5 is characterized in that: Receive the target driver's driving state evaluation index PLz within the current driving evaluation time window, and perform corresponding operations to control the target vehicle, specifically: A threshold index range corresponding to the preset driving state evaluation index PLz is set. If the driving state evaluation index PLz of the target driver in the current driving evaluation time window is within the preset threshold index range, an alarm signal is sent to the target driver, and at the same time, a notification is given to the target driver whether to turn on the auxiliary driving system; If the target driver's driving state evaluation index PLz within the current driving evaluation time window is higher than the preset threshold index range, the assisted driving system will be directly turned on, the vehicle's double flash will be turned on, and an alarm signal will be sent to the target driver.

7. The automobile safe driving assistance control system based on visual recognition according to claim 6 is characterized in that: Receive the risk estimation index of the target vehicle and perform corresponding operations to control the target vehicle, specifically: Three groups of index value ranges corresponding to the risk estimation index are set, and the three groups of index value ranges correspond to a vehicle risk level respectively, including a low vehicle risk level, a medium vehicle risk level, and a high vehicle risk level; According to the three groups of index value ranges set, the vehicle risk level of the target vehicle within the current driving assessment time window is determined; If the vehicle risk level is a lower vehicle risk level, a notification is sent to the target driver asking whether to turn on the assisted driving system; If the vehicle risk level is a medium vehicle risk level, it identifies whether the target vehicle is in an assisted driving system. If so, it pushes the vehicle risk level to the target driver and tells him whether to turn off the assisted driving system. It also automatically adjusts the sensitivity of the target vehicle's automatic emergency braking system according to the settings. If the vehicle risk level is a higher vehicle risk level, it is identified whether the target vehicle is in an assisted driving system. If so, the assisted driving system is directly turned off, and the sensitivity of the target vehicle's automatic emergency braking system is automatically adjusted according to the settings; at the same time, the road section state evaluation index of the target vehicle's remaining mileage is analyzed. If the road section state evaluation index of the target vehicle's remaining mileage is higher than the preset threshold index, the target vehicle's current position is used as the starting point and the destination is used as the end point. Each planned route is obtained and pushed to the target driver. At the same time, each planned route is sorted from small to large according to the corresponding road section state evaluation index.

8. The automobile safe driving assistance control system based on visual recognition according to claim 7 is characterized in that: Analyze the road section status evaluation index of the target vehicle's remaining mileage, specifically: Obtain the traffic flow data, congestion mileage and number of accidents of the target vehicle's remaining mileage, set the traffic flow value ranges and congestion mileage value ranges corresponding to the traffic flow data and congestion mileage, respectively, each group of traffic flow value ranges corresponds to a traffic flow assessment value, and each group of congestion mileage corresponds to a congestion assessment value; After determining the flow evaluation value and congestion evaluation value of the remaining mileage of the target vehicle, based on the weight coefficients preset for the flow evaluation value and the congestion evaluation value, respectively, the flow evaluation value and the congestion evaluation value of the remaining mileage of the target vehicle are multiplied by the corresponding preset weight coefficients, and then multiplied together to obtain the driving experience value of the remaining mileage of the target vehicle; According to the number of accidents in the remaining mileage of the target vehicle, the influence coefficient is determined, and the driving experience value is multiplied by the set influence coefficient to obtain the road section condition evaluation index of the remaining mileage of the target vehicle.

9. The automobile safe driving assistance control system based on visual recognition according to claim 8 is characterized in that: According to the number of accidents of the target vehicle's remaining mileage, the impact coefficient is determined as follows: When the number of accidents in the remaining mileage of the target vehicle is 0, the initial value of the impact coefficient is set to an integer of 1. If the number of accidents is greater than 0, the impact adjustment value corresponding to the number of accidents is set, and the impact adjustment value is added to the integer of 1 to serve as the impact coefficient multiplied by the driving experience value. That is, the value range of each component number corresponding to the number of accidents is set, and each component number value range corresponds to an impact adjustment value.

10. A vehicle safety driving assistance control method based on visual recognition, applied to a vehicle safety driving assistance control system based on visual recognition as claimed in any one of claims 1 to 9, characterized in that: include: Data collection: Collect traffic videos of the target vehicle while driving and driving videos of the target driver while driving to the destination; Driver evaluation: After reaching the preset driving evaluation time window, extract the driving video of the target driver during the current driving evaluation time window to the destination, and conduct a comprehensive evaluation to determine the driving state evaluation index PLz of the target driver in the current driving evaluation time window; Environmental assessment: extract the traffic video of the target vehicle driving to the destination in the current driving assessment time window and conduct a comprehensive assessment to determine the risk estimation index of the target vehicle in the current driving assessment time window; Decision control: According to the target driver's driving state evaluation index PLz within the current driving evaluation time window and the risk estimation index of the target vehicle, the target vehicle is controlled accordingly.

Images