Vehicle-mounted driving assistance method

Abstract

The invention discloses a vehicle-mounted driving assistance method, which relates to the technical field of driving assistance and comprises the following steps: acquiring road image data acquired by a vehicle-mounted camera and vehicle state data acquired by a vehicle-mounted sensor; performing image recognition on the road image data, extracting traffic signs, lane lines and surrounding vehicle position information, and generating road environment data; and judging a current driving risk level according to the road environment data and the vehicle state data, and feeding back early warning information to a driver according to the driving risk level. According to the invention, a comprehensive utilization and real-time feedback mechanism of multi-source data is provided, and the perception and reaction capability of a driver to potential risks is enhanced. Therefore, the system not only can deal with conventional driving situations, but also can deal with complex and dynamically changing traffic situations, so that the driving safety is greatly improved, and the possibility of accidents is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of driving assistance, and in particular to a vehicle-mounted driving assistance method. Background Art

[0002] With the progress of society and the increasing attention paid to special groups, in-vehicle driving assistance technology for special groups such as the deaf has gradually become a research hotspot. In the early days, in-vehicle driving assistance systems mainly focused on providing visual and auditory auxiliary information to ordinary drivers, such as voice broadcasts of traffic conditions and navigation instructions, and less consideration was given to the needs of deaf drivers. However, with the development of technology, some preliminary assistance methods for deaf drivers have begun to appear, such as using flashing lights to prompt vehicle steering, braking and other operations, but these methods are relatively simple in function and can only provide simple driving operation prompts, and cannot fully perceive the road environment and conduct risk assessment and warning.

[0003] In recent years, with the continuous advancement of sensor technology, image recognition technology and data processing technology, the functions and performance of vehicle-mounted driving assistance systems have been greatly improved, and the assistance methods provided for deaf drivers have become more diverse and intelligent. However, the existing vehicle-mounted driving assistance methods for the deaf still have many shortcomings. Although the existing system uses some visual and tactile feedback methods in terms of warning information feedback, it lacks a mechanism for differentiated warnings based on driving risk levels, and cannot provide deaf drivers with clear, clear and hierarchical warning information, making it difficult for deaf drivers to accurately judge the current driving risk situation and take corresponding driving operations, thereby delaying the opportunity to take emergency measures. Summary of the invention

[0004] In view of the problems existing in the existing vehicle-mounted driving assistance method, the present invention is proposed. Therefore, the problem to be solved by the present invention is how to provide a vehicle-mounted driving assistance method.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0006] In a first aspect, the present invention provides a vehicle-mounted driving assistance method, which includes collecting road image data acquired by a vehicle-mounted camera and vehicle status data acquired by a vehicle-mounted sensor;

[0007] Performing image recognition on the road image data, extracting traffic signs, lane lines, and surrounding vehicle position information, and generating road environment data;

[0008] The current driving risk level is determined according to the road environment data and the vehicle status data, and warning information is fed back to the driver according to the driving risk level.

[0009] As a preferred solution of the vehicle-mounted driving assistance method described in the present invention, the road image data includes traffic signs, lane lines, driving paths and surrounding environment data; the vehicle status data includes the vehicle's current speed, acceleration, steering angle, and driving trajectory data.

[0010] As a preferred solution of the vehicle-mounted driving assistance method of the present invention, the image recognition comprises the following steps:

[0011] Preprocess the collected road image data, including denoising, grayscale conversion and edge detection;

[0012] The deep learning algorithm YOLOv8 is used to analyze the preprocessed images, identify various traffic signs, and determine their locations and categories;

[0013] Extract lane lines through image processing technology to determine the boundaries and directions of lanes;

[0014] Use target detection algorithms to identify surrounding vehicles and obtain vehicle location, speed, and direction information;

[0015] Integrate the identified traffic signs, lane lines and surrounding vehicle position information into road environment data.

[0016] As a preferred solution of the vehicle-mounted driving assistance method of the present invention, wherein: the determination of the current driving risk level comprises the following steps:

[0017] Input data into the risk assessment model, and input the traffic sign category, lane line deviation, surrounding vehicle position information in the road environment data, and the vehicle speed, steering angle and acceleration value in the vehicle status data into the preset risk assessment model;

[0018] Identify the speed limit value V corresponding to the speed limit sign from the traffic sign category information in the input road environment data l , using the input current vehicle speed value, calculate the proportion of the current vehicle speed exceeding the value specified by the speed limit sign. When the vehicle speed exceeds the speed limit, calculate the speed risk score:

[0019]

[0020] In the formula, K V is the ratio of the current vehicle speed exceeding the value specified by the speed limit sign, V x is the current vehicle speed, V l is the speed limit value corresponding to the speed limit sign, F s Score the speed risk;

[0021] The lane deviation D is obtained from the input road environment data and vehicle status data. land the steering angle value θ, and the steering angle change rate is obtained by performing differential operation on the continuously collected steering angle values;

[0022] Set the preset threshold D of lane deviation t and the preset range of the steering angle value change rate When D l >D t and When the preset range is exceeded, the lane departure risk score is calculated:

[0023]

[0024] In the formula, K D is the ratio of the current lane line deviation exceeding the preset threshold, D l is the lane deviation of the vehicle, D t is the preset threshold of lane deviation, F c Score lane departure risk;

[0025] Obtain the position information of surrounding vehicles from the input road environment data and vehicle status data v and the acceleration value a of the vehicle, and the pre-set safety distance value D s , set the preset threshold a of the acceleration value t , when D v <D s And a>a t Calculate the collision risk score when:

[0026]

[0027] In the formula, K P is the ratio of the current vehicle position information to the safety distance, D s To set the safety distance, D v is the current vehicle location information, F p score for collision risk;

[0028] The comprehensive risk score of the vehicle is calculated by combining the speed risk score, lane departure risk score and collision risk score to determine the current driving risk level.

[0029] As a preferred solution of the vehicle-mounted driving assistance method of the present invention, the calculation of the vehicle comprehensive risk score includes:

[0030] The weight coefficients of speed risk score, lane departure risk score and collision risk score are set to ω1, ω2 and ω3 respectively, and the weighted sum is calculated:

[0031] F z =ω1*F s +ω2*Fc +ω3*F p

[0032] In the formula, F z is the comprehensive risk score, F s is the speed risk score, F c is the lane departure risk score, F p Score the collision risk.

[0033] As a preferred solution of the vehicle-mounted driving assistance method of the present invention, wherein: the driving risk level includes:

[0034] According to the comprehensive risk score F z Divide the risk level, if the comprehensive risk score F z When the vehicle is in a low-risk state, the road environment data and vehicle status data for n consecutive time periods are recorded and saved. After the recording is completed, the secondary comprehensive risk score calculation for n consecutive time periods is performed. If the comprehensive risk score of the vehicle is F1 in n consecutive time periods, the vehicle is initially judged to be low-risk. When the vehicle is in a low-risk state, the road environment data and vehicle status data for n consecutive time periods are recorded and saved. After the recording is completed, the secondary comprehensive risk score calculation for n consecutive time periods is performed. z When the risk level is below the preset risk threshold F1, the risk level is kept at low risk, and early warning information is fed back according to the low risk level. If the comprehensive risk score of the vehicle is F1 within n consecutive time periods, z When the risk level is above the preset risk threshold F1, the risk level is raised to medium risk, and the road environment data and vehicle status data within m consecutive time periods are immediately obtained, and three comprehensive risk score calculations within m consecutive time periods are performed. If the risk level increases, it is calculated again. If the risk level decreases or remains the same, early warning information feedback is given according to the medium risk level;

[0035] If the preset risk threshold F1 ≤ comprehensive risk score F z When the vehicle is in a medium risk state, the system immediately provides early warning information feedback based on the medium risk level and calculates the secondary comprehensive risk score for m consecutive time periods. If the comprehensive risk score of the vehicle is F2 in m consecutive time periods, the system will calculate the risk level of the vehicle. z If both are above the preset risk threshold F2, the risk level is raised to high risk;

[0036] If the preset risk threshold F2 ≤ the comprehensive risk score F zWhen the vehicle is in a high-risk state, the system immediately performs a secondary comprehensive risk score calculation for q consecutive time periods, and the system immediately provides early warning information feedback based on the medium risk level. At the same time, it performs a secondary comprehensive risk score calculation for q consecutive time periods. If the comprehensive risk score remains or increases, it performs early warning information feedback based on the high risk level. If the comprehensive risk score decreases, it performs a third comprehensive risk score calculation for q consecutive time periods.

[0037] As a preferred solution of the vehicle-mounted driving assistance method of the present invention, the feedback of warning information includes:

[0038] According to the driving risk level, warning information is sent to the driver through the on-board display screen and seat vibrator in different colors of visual prompts and vibrations;

[0039] When the vehicle is in a low-risk state, the display screen presents a normal interface layout, the background color is the normal display color, the text and icons are presented in normal contrast, no color prompts are given, and vehicle information, navigation or other general information are displayed. The seat vibrator remains stationary and no tactile feedback is generated;

[0040] When the vehicle is in a medium-risk state, the background color of the display screen switches to yellow, and the yellow background covers the screen or a specific prompt area, accompanied by a yellow warning icon or graphic. The screen uses a gradient effect or slight flashing, and the vibration mode of the seat vibrator is adjusted to intermittent vibration, with a short vibration at a set interval until the medium-risk state is lifted;

[0041] When the vehicle is in a high-risk state, the background color of the display screen switches to red, the red background covers the screen or a specific prompt area, red warning messages appear in text and icons, the screen display is accompanied by a dynamic flashing effect or rapid color change, and the vibration mode of the seat vibrator is adjusted to continuous vibration until the risk level drops. At the same time, the autonomous driving assistance system is enabled to assist in braking or steering operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0043] Figure 1 The figure is a flow chart of the in-vehicle driving assistance method. DETAILED DESCRIPTION

[0044] In order to make the above-mentioned purposes, features and advantages of the present invention more understandable, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0045] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0046] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor is it a separate or selective embodiment that is mutually exclusive with other embodiments.

[0047] Example 1

[0048] Reference Figure 1 , which is the first embodiment of the present invention, and provides a vehicle-mounted driving assistance method, comprising:

[0049] S1: Collect road image data obtained by the vehicle camera and vehicle status data obtained by the vehicle sensor;

[0050] Specifically, vehicle-mounted cameras: installed in multiple locations such as the front, rear, and side of the vehicle to collect road image data. High-definition cameras and infrared sensors are used to improve image quality in low-light environments. Image data includes information such as traffic signs, lane lines, driving paths, and surrounding environment. Vehicle-mounted sensors: including GPS positioning systems, speed sensors, acceleration sensors, steering wheel angle sensors, etc., are used to obtain vehicle status data. These data help determine the vehicle's current speed, acceleration, steering angle, driving trajectory, and other dynamic information.

[0051] S2: Perform image recognition on the road image data, extract traffic signs, lane lines, and surrounding vehicle position information, and generate road environment data;

[0052] Specifically, S2.1: performing a preprocessing operation on the collected road image data, using a Gaussian filter algorithm to perform image denoising, converting the RGB image into a grayscale image, and using a Canny operator to perform edge detection to obtain a preprocessed road image;

[0053] Gaussian filtering is a commonly used image denoising method. Its core is to use the Gaussian kernel to perform weighted averaging on image pixels. The Gaussian kernel is a two-dimensional Gaussian function with a bell-shaped shape. The weight of the central pixel is the largest, and the weight of the surrounding pixels gradually decreases as the distance from the center increases. By applying the Gaussian kernel to each pixel in the image for convolution, the value of the central pixel is affected by the values ​​of the surrounding pixels, thereby smoothing the image and effectively removing random noise.

[0054] For example, for a Gaussian kernel of size 3×3, its weight coefficient can be calculated according to the Gaussian function, and then the kernel is slid on the image, and the weighted summation is performed on each 3×3 pixel area to obtain the denoised pixel value.

[0055] RGB images contain three color channels: red, green, and blue. Each channel represents the brightness information of the pixel at the corresponding color. Converting an RGB image to a grayscale image is achieved by linearly combining the values ​​of the three color channels according to certain weights. Generally, the human eye is most sensitive to green, followed by red, and the lowest to blue. The grayscale conversion formula is used to convert the color information in a color image into grayscale information, simplifying the image data and facilitating subsequent processing.

[0056] The Canny operator is a multi-level edge detection algorithm. First, the image is smoothed using a Gaussian filter to reduce the interference of noise on edge detection. Then, the gradient amplitude and direction of the image in the horizontal and vertical directions are calculated. The gradient amplitude reflects the intensity of the grayscale change of the image, and the direction indicates the direction of the grayscale change. Next, non-maximum suppression is performed, that is, the pixels are compared in the gradient direction, and only the pixels with the largest local gradient amplitude are retained as edge pixels, and other non-edge pixels are suppressed to make the edges sharper. Finally, double threshold detection is performed by setting high and low thresholds. The high threshold is used to determine the strong edge, and the low threshold is used to determine the weak edge. The weak edge is connected to the strong edge through connectivity analysis, and finally a complete edge image is obtained.

[0057] S2.2: Use deep learning algorithms (such as YOLOv8) to analyze the preprocessed images, identify various traffic signs, such as speed limit signs, no overtaking signs, etc., and determine their locations and categories.

[0058] Furthermore, the YOLOv8 deep learning model is used to analyze the preprocessed road images. The YOLOv8 model outputs the pixel coordinate position of the traffic sign and the probability value of its category based on the pre-trained traffic sign dataset. The traffic sign categories include speed limit signs, prohibition signs, warning signs, and instruction signs.

[0059] The YOLOv8 model is trained based on a pre-trained traffic sign dataset, which contains a large number of image samples with traffic sign categories and location information labeled. When used, the pre-processed road image is input into the YOLOv8 model, and the model automatically extracts feature information from the image.

[0060] The YOLOv8 model extracts features from the input image through a convolutional neural network. The convolutional layer and pooling layer in the network can automatically learn the spatial features and semantic information in the image. Based on the feature extraction, the model uses the anchor box mechanism and bounding box regression algorithm to locate the traffic signs in the image and output the pixel coordinates of the traffic signs. At the same time, the model also classifies each detected traffic sign and outputs the probability value of its category, including speed limit signs, prohibition signs, warning signs, and instruction signs.

[0061] For example, for a detected traffic sign, the model will output the pixel coordinates of its upper left corner and lower right corner (x1, y1, x2, y2), as well as the probability values ​​of belonging to each category (such as the probability of a speed limit sign is 0.8, the probability of a prohibition sign is 0.1, etc.). The category of the traffic sign can be determined based on the size of the probability value.

[0062] S2.3: Extract lane lines through image processing technology to determine the boundaries and directions of lanes and provide a reference for vehicle driving.

[0063] Furthermore, the lane line features are extracted from the preprocessed road image based on the Hough transform algorithm, the lane line equation is obtained through curve fitting, the lane boundary coordinate system is established, and the lane direction angle value is calculated;

[0064] Hough transform is a transformation method that maps curves or straight lines in image space to points in parameter space. For each straight line in the image, it corresponds to a point in the parameter space (such as polar coordinate space). By performing Hough transform on all pixels in the image, the problem of detecting straight lines in the image can be transformed into the problem of finding peak points in the parameter space.

[0065] Specifically, for a pixel point (x, y) in the image, the curve or point corresponding to the pixel point in the parameter space can be calculated based on the general equation of a straight line y=kx+b or the polar coordinate equation ρ=xcosθ+ysinθ.

[0066] First, edge detection is performed on the preprocessed road image to obtain an image containing lane edge information. Then, Hough transform is performed on each edge pixel in the edge image to map it to the parameter space. In the parameter space, the mapping results of all pixels are accumulated to form an accumulation plane. The peak points on the accumulation plane correspond to the straight lines in the image. The peak points are determined by setting a threshold, so that the straight lines in the image can be detected. These straight lines are the candidate lines of the lane lines, and the candidate lines are further analyzed and screened. For example, the final lane lines are determined based on the length, direction, and continuity of the lane lines. The detected lane lines are fitted by a curve fitting algorithm, such as polynomial fitting, to obtain the lane line equation, and then the lane boundary coordinate system is established to calculate the lane direction angle value, providing a basis for subsequent vehicle driving decisions.

[0067] S2.4: Use target detection algorithms to identify surrounding vehicles and obtain information such as their location, speed, and direction of travel in order to assess potential collision risks.

[0068] Furthermore, the object detection algorithm based on RetinaNet is used to identify the surrounding vehicles in the preprocessed road image, extract the vehicle boundary box coordinates, calculate the vehicle motion vector by combining the adjacent frame images, and obtain the vehicle position coordinates, speed value and driving direction angle;

[0069] RetinaNet is a single-stage target detection algorithm based on deep learning. Its core is to use the FocalLoss loss function to solve the problem of class imbalance. In the vehicle recognition task, the RetinaNet model enhances the detection ability of targets of different scales by adding a feature pyramid network (FPN) based on the feature extraction of the convolutional neural network, and can effectively detect surrounding vehicles in the image.

[0070] The preprocessed road image is input into the RetinaNet model, and the model will identify the vehicle in the image and output the bounding box coordinates of the vehicle. The bounding box coordinates are usually expressed in the form of (x1, y1, x2, y2), where (x1, y1) is the pixel coordinate of the upper left corner of the bounding box, and (x2, y2) is the pixel coordinate of the lower right corner of the bounding box. At the same time, the model will also output the category information of the vehicle, such as sedan, truck, bus, etc. In order to obtain the motion parameters of the vehicle, it is necessary to combine adjacent frame images for calculation. By tracking and matching the bounding box coordinates of the same vehicle in adjacent frame images, the change in the pixel coordinates of the vehicle between the two frames is calculated, and the speed value of the vehicle can be obtained by combining the time interval of image acquisition. The driving direction angle of the vehicle can be determined by calculating the angle between the vector of the vehicle's position change in adjacent frame images and the reference direction (such as the horizontal direction of the image).

[0071] S2.5: Integrate the identified traffic signs, lane lines and surrounding vehicle position information into road environment data to provide a basis for subsequent risk level judgment.

[0072] Furthermore, the traffic sign coordinate position and category information, lane boundary coordinates and heading angle values, vehicle position coordinates and motion parameters are fused to construct road environment data containing key elements of the road scene.

[0073] The weighted average method or other data fusion algorithms are used to fuse the traffic sign coordinate position and category information, lane boundary coordinates and direction angle values, vehicle position coordinates and motion parameters.

[0074] First, preprocess each data source and convert the data into a unified coordinate system and time base. Then, determine the corresponding weight coefficient according to the reliability and importance of each data source.

[0075] For example, for traffic sign information, the data weight with higher category probability value can be relatively large; for vehicle motion parameters, the data weight with higher consistency between adjacent frame images can be large. By calculating the weighted average of each data source, the fused road environment data is obtained.

[0076] The fused road environment data structure contains key elements of the road scene, including the location and category of traffic signs, lane boundaries and directions, and the location, speed, and driving direction of surrounding vehicles. This information is stored in a structured form, such as using data structures such as arrays, linked lists, or objects. By building a road environment data structure, it helps vehicles make reasonable driving decisions and improves the safety and reliability of autonomous driving.

[0077] S3: Determine the current driving risk level based on the road environment data and vehicle status data, and provide early warning information feedback to the driver according to the driving risk level.

[0078] Specifically, S3.1 establishes a risk assessment model based on machine learning, which takes road environment data and vehicle status data as input and outputs the driving risk level. The traffic sign category, lane line deviation, surrounding vehicle position information in the road environment data, and the vehicle speed, steering angle and acceleration value in the vehicle status data are input into the preset risk assessment model.

[0079] S3.2 Calculate the speed risk score and identify the speed limit value V corresponding to the speed limit sign from the traffic sign category information in the input road environment data l , calculate the speed excess ratio: use the input current speed value to calculate the current speed exceeding the speed limit sign specified value ratio:

[0080]

[0081] In the formula, K V is the ratio of the current vehicle speed exceeding the value specified by the speed limit sign, V c is the current vehicle speed, V l is the speed limit value corresponding to the speed limit sign;

[0082] When the vehicle speed exceeds the speed limit, the speed risk score increases by 10 points for every 10% exceeding the speed limit. The speed risk score is calculated as follows:

[0083]

[0084] In the formula, F s Score the speed risk;

[0085] S3.3 Calculate the lane departure risk score and obtain the lane deviation degree D from the input road environment data and vehicle status data l and steering angle value θ.

[0086] By performing differential calculation on the continuously collected steering angle values, the steering angle change rate is obtained:

[0087]

[0088] In the formula, θ c is the current steering angle value collected continuously, θ p is the most recent steering angle value collected continuously, and Δt is the time interval between continuous collections;

[0089] Set the preset threshold D of lane deviation t and the preset range of the steering angle value change rate When D l >D t and When the vehicle exceeds the preset range, the lane departure risk score is calculated using the following formula:

[0090]

[0091] Where, L D is the ratio of the current lane line deviation exceeding the preset threshold, D l is the lane deviation of the vehicle, D t is the preset threshold of lane deviation, F c Score lane departure risk;

[0092] S3.4 Calculate the collision risk score and obtain the position information of surrounding vehicles from the input road environment data and vehicle status data D v And the acceleration value a of the vehicle.

[0093] According to vehicle speed, braking performance and other parameters, the safety distance value D is pre-set. s , judge the conditions and determine the collision risk score: set the preset threshold a of the acceleration value t , when D v <D s And a>a t The collision risk score increases by 20 points for every 10% reduction in safety distance:

[0094]

[0095] In the formula, K P is the ratio of the current vehicle position information to the safety distance, D s To set the safety distance, D v is the current vehicle location information, F p score for collision risk;

[0096] S3.5 calculates the comprehensive risk score, sets the weight coefficients of the speed risk score, lane departure risk score, and collision risk score to ω1, ω2, and ω3, respectively, and performs weighted summation:

[0097] F z =ω1*F s +ω2*F c +ω3*F p

[0098] In the formula, F z is the comprehensive risk score;

[0099] S3.6 is mapped to risk levels, based on the comprehensive risk score F z Divide the risk level, if the comprehensive risk score F z When the vehicle is in a low-risk state, the road environment data and vehicle status data for n consecutive time periods are recorded and saved. After the recording is completed, the secondary comprehensive risk score calculation for n consecutive time periods is performed. If the comprehensive risk score of the vehicle is F1 in n consecutive time periods, the vehicle is initially judged to be low-risk. When the vehicle is in a low-risk state, the road environment data and vehicle status data for n consecutive time periods are recorded and saved. After the recording is completed, the secondary comprehensive risk score calculation for n consecutive time periods is performed. z When the risk level is below the preset risk threshold F1, the risk level is kept at low risk, and early warning information is fed back according to the low risk level. If the comprehensive risk score of the vehicle is F1 within n consecutive time periods, z When the risk level is above the preset risk threshold F1, the risk level is raised to medium risk, and the road environment data and vehicle status data within m consecutive time periods are immediately obtained, and three comprehensive risk score calculations within m consecutive time periods are performed. If the risk level increases, it is calculated again. If the risk level decreases or remains the same, early warning information feedback is given according to the medium risk level;

[0100] If the preset risk threshold F1 ≤ comprehensive risk score F z When the vehicle is in a medium risk state, the system immediately provides early warning information feedback based on the medium risk level and calculates the secondary comprehensive risk score for m consecutive time periods. If the comprehensive risk score of the vehicle is F2 in m consecutive time periods, the system will calculate the risk level of the vehicle. z If both are above the preset risk threshold F2, the risk level is raised to high risk;

[0101] If the preset risk threshold F2 ≤ the comprehensive risk score F z When the vehicle is in a high-risk state, the system immediately performs a secondary comprehensive risk score calculation for q consecutive time periods, and the system immediately provides early warning information feedback based on the medium risk level. At the same time, it performs a secondary comprehensive risk score calculation for q consecutive time periods. If the comprehensive risk score remains or increases, it performs early warning information feedback based on the high risk level. If the comprehensive risk score decreases, it performs a third comprehensive risk score calculation for q consecutive time periods.

[0102] Among them, n, m, and q are preset numbers of continuous time periods, which are used to avoid frequent changes in risk levels due to short-term fluctuations in risk scores, n>m>q.

[0103] Driver warning information feedback, based on the driving risk level, warning information is sent to the driver through the on-board display screen and seat vibrator in the form of visual prompts and vibrations in different colors.

[0104] When the vehicle is in a low-risk state, the display screen presents a normal interface layout, the background color is the normal display color, and the text and icons are presented in normal contrast without color prompts, to ensure that the driver maintains normal visual perception in low-risk situations, is not disturbed by additional color prompts, and can focus on road and traffic conditions. The seat vibrator remains stationary and does not produce tactile feedback. The driver cannot feel vibration in the seat, ensuring driving in a stable and comfortable environment, avoiding discomfort or distraction caused by seat vibration.

[0105] When the vehicle is in a medium-risk state, the background color of the display screen switches to a striking yellow, covering the entire screen or a specific prompt area, and the text and icons change to high-contrast fonts, which form a clear contrast with the normal display color to attract the driver's attention. The screen uses a gradient effect or slight flashing to remind the driver of potential risks. The color change is designed to make the driver aware of the current driving risks without distracting too much attention, and to be vigilant and pay attention to road conditions and vehicle conditions. The seat vibrator vibrates in an intermittent manner, with a short vibration (lasting 1 second) every 5 seconds. The vibration feels slight, similar to a slight pulse. The intermittent vibration mode reminds the driver to pay attention to the driving condition without causing excessive tension to the driver, and to adjust the driving behavior appropriately to reduce the risk. The seat vibrator vibrates briefly at regular intervals, with a short vibration duration and a long interval, so that the driver has enough time to perceive the vibration reminder and respond.

[0106] When the vehicle is in a high-risk state, the background color of the display screen changes to a more eye-catching red, and the red background covers the entire screen or a specific prompt area. The text and icons change to eye-catching red warning information. The strong red signal quickly attracts the driver's attention, making him immediately aware of the high-risk state, prompting the driver to quickly take emergency measures, such as slowing down, stopping or avoiding, to ensure driving safety. The vibration mode of the seat vibrator is adjusted to continuous vibration, and the vibration lasts for 3 to 5 seconds, with a high frequency and high intensity. The vibration mode will continue to vibrate until the risk level decreases, ensuring that the driver can respond quickly. Enable the automatic driving assistance system to assist in braking or steering operations, and cooperate with seat vibration and visual warnings to strengthen the driver's response. Continuous and strong vibration quickly causes the driver to be highly alert, making him realize that the degree of danger is high and that emergency avoidance measures need to be taken immediately. The seat vibrator continues to vibrate until the risk level is reduced, ensuring that the driver remains sensitive and vigilant to danger in a high-risk state, and makes correct driving decisions in time to avoid accidents.

[0107] This embodiment also provides a computer device, which is suitable for the case of an in-vehicle driving assistance method, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute computer-executable instructions to implement all or part of the steps of the method described in the embodiment of the present invention as proposed in the above embodiment.

[0108] This embodiment also provides a storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the method in any optional implementation of the above embodiment is executed. Wherein, the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, referred to as EPROM), programmable read-only memory (Programmable Red-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, disk or optical disk.

[0109] The storage medium proposed in this embodiment and the data storage method proposed in the above embodiment belong to the same inventive concept. The technical details not fully described in this embodiment can be found in the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0110] In summary, the present invention provides a comprehensive utilization of multi-source data and a real-time feedback mechanism, which enhances the driver's perception and response to potential risks. This enables the system to not only cope with conventional driving scenarios, but also handle complex and dynamically changing traffic conditions, greatly improving driving safety and reducing the possibility of accidents.

Claims

1. A vehicle-mounted driving assistance method, characterized in that: include, Collect road image data obtained by on-board cameras and vehicle status data obtained by on-board sensors; Performing image recognition on the road image data, extracting traffic signs, lane lines, and surrounding vehicle position information, and generating road environment data; The current driving risk level is determined according to the road environment data and the vehicle status data, and warning information is fed back to the driver according to the driving risk level.

2. The vehicle-mounted driving assistance method according to claim 1, characterized in that: The road image data includes traffic signs, lane lines, driving paths and surrounding environment data; the vehicle status data includes the vehicle's current speed, acceleration, steering angle, and driving trajectory data.

3. The vehicle-mounted driving assistance method according to claim 2, characterized in that: The image recognition comprises the following steps: Preprocess the collected road image data, including denoising, grayscale conversion and edge detection; The deep learning algorithm YOLOv8 is used to analyze the preprocessed images, identify various traffic signs, and determine their locations and categories; Extract lane lines through image processing technology to determine the boundaries and directions of lanes; Use target detection algorithms to identify surrounding vehicles and obtain vehicle location, speed, and direction information; Integrate the identified traffic signs, lane lines and surrounding vehicle position information into road environment data.

4. The vehicle-mounted driving assistance method according to claim 3, characterized in that: Determining the current driving risk level includes the following steps: Input data into the risk assessment model, and input the traffic sign category, lane line deviation, surrounding vehicle position information in the road environment data, and the vehicle speed, steering angle and acceleration value in the vehicle status data into the preset risk assessment model; Identify the speed limit value V corresponding to the speed limit sign from the traffic sign category information in the input road environment data l , using the input current vehicle speed value, calculate the proportion of the current vehicle speed exceeding the value specified by the speed limit sign. When the vehicle speed exceeds the speed limit, calculate the speed risk score: In the formula, K V is the ratio of the current vehicle speed exceeding the value specified by the speed limit sign, V c is the current vehicle speed, V l is the speed limit value corresponding to the speed limit sign, F s Score the speed risk; The lane deviation D is obtained from the input road environment data and vehicle status data. l and the steering angle value θ, and the steering angle change rate is obtained by performing differential operation on the continuously collected steering angle values; Set the preset threshold D of lane deviation t and the preset range of the steering angle value change rate When D l >D t and When the preset range is exceeded, the lane departure risk score is calculated: In the formula, K D is the ratio of the current lane line deviation exceeding the preset threshold, D l is the lane deviation of the vehicle, D t is the preset threshold of lane deviation, F c Score lane departure risk; Obtain the position information of surrounding vehicles from the input road environment data and vehicle status data v and the acceleration value a of the vehicle, and the pre-set safety distance value D s , set the preset threshold a of the acceleration value t , when D v <D s And a>a t Calculate the collision risk score when: In the formula, K P is the ratio of the current vehicle position information to the safety distance, D s To set the safety distance, D v is the current vehicle location information, F p score for collision risk; The comprehensive risk score of the vehicle is calculated by combining the speed risk score, lane departure risk score and collision risk score to determine the current driving risk level.

5. The vehicle-mounted driving assistance method according to claim 4, characterized in that: The vehicle comprehensive risk score calculated includes: The weight coefficients of speed risk score, lane departure risk score and collision risk score are set to ω1, ω2 and ω3 respectively, and the weighted sum is calculated: F z =ω1*F s +ω2*F c +ω3*F p In the formula, F z is the comprehensive risk score, F s is the speed risk score, F c is the lane departure risk score, F p Score the collision risk.

6. The vehicle-mounted driving assistance method according to claim 5, characterized in that: The driving risk levels include: According to the comprehensive risk score F z Divide the risk level, if the comprehensive risk score is F z When the vehicle is in a low-risk state, the road environment data and vehicle status data for n consecutive time periods are recorded and saved. After the recording is completed, the secondary comprehensive risk score calculation for n consecutive time periods is performed. If the comprehensive risk score of the vehicle is F1 in n consecutive time periods, the vehicle is initially judged to be low-risk. When the vehicle is in a low-risk state, the road environment data and vehicle status data for n consecutive time periods are recorded and saved. After the recording is completed, the secondary comprehensive risk score calculation for n consecutive time periods is performed. z When the risk level is below the preset risk threshold F1, the risk level is kept at low risk, and early warning information is fed back according to the low risk level. If the comprehensive risk score of the vehicle is F1 within n consecutive time periods, z When the risk level is above the preset risk threshold F1, the risk level is raised to medium risk, and the road environment data and vehicle status data within m consecutive time periods are immediately obtained, and three comprehensive risk score calculations within m consecutive time periods are performed. If the risk level increases, it is calculated again. If the risk level decreases or remains the same, early warning information feedback is given according to the medium risk level; If the preset risk threshold F1 ≤ comprehensive risk score F z When the vehicle is in a medium risk state, the system immediately provides early warning information feedback based on the medium risk level and calculates the secondary comprehensive risk score for m consecutive time periods. If the comprehensive risk score of the vehicle is F2 in m consecutive time periods, the system will calculate the risk level of the vehicle. z If both are above the preset risk threshold F2, the risk level is raised to high risk; If the preset risk threshold F2 ≤ the comprehensive risk score F z When the vehicle is in a high-risk state, the system immediately performs a secondary comprehensive risk score calculation for q consecutive time periods, and the system immediately feeds back warning information based on the medium risk level. At the same time, it performs a secondary comprehensive risk score calculation for q consecutive time periods. If the comprehensive risk score remains or increases, it feeds back warning information based on the high risk level. If the comprehensive risk score decreases, it performs a third comprehensive risk score calculation for q consecutive time periods.

7. The vehicle-mounted driving assistance method according to claim 6, characterized in that: The feedback of early warning information includes: According to the driving risk level, warning information is sent to the driver through the on-board display screen and seat vibrator in different colors of visual prompts and vibrations; When the vehicle is in a low-risk state, the display screen presents a normal interface layout, the background color is the normal display color, the text and icons are presented in normal contrast, no color prompts are given, and vehicle information, navigation or other general information are displayed. The seat vibrator remains stationary and no tactile feedback is generated; When the vehicle is in a medium-risk state, the background color of the display screen switches to yellow, and the yellow background covers the screen or a specific prompt area, accompanied by a yellow warning icon or graphic. The screen uses a gradient effect or slight flashing, and the vibration mode of the seat vibrator is adjusted to intermittent vibration, with a short vibration at a set interval until the medium-risk state is lifted; When the vehicle is in a high-risk state, the background color of the display screen switches to red, the red background covers the screen or a specific prompt area, red warning messages appear in text and icons, the screen display is accompanied by a dynamic flashing effect or rapid color change, and the vibration mode of the seat vibrator is adjusted to continuous vibration until the risk level drops. At the same time, the autonomous driving assistance system is enabled to assist in braking or steering operations.

Images