Safety early warning method and system based on brake lamp recognition and vehicle

Abstract

The invention discloses a safety early warning method and system based on brake lamp recognition and a vehicle, and belongs to the technical field of vehicle safety early warning. The method comprises the steps that an image sequence of a front vehicle is obtained, dynamic feature modeling is conducted on the image sequence, and the braking state of a braking lamp of the front vehicle is determined; distance identification is performed based on the image sequence, and a first vehicle distance is determined; a second vehicle distance is obtained, the first vehicle distance and the second vehicle distance are adaptively fused in combination with weather conditions, and the front vehicle distance is determined; the collision time is determined according to the front vehicle distance and the driving speed of the vehicle; and the response state of the driver is determined, and the safety early warning level is determined according to the braking state, the collision time and the response state. An early warning state can be comprehensively identified and judged by combining a plurality of signals such as a braking light state and a front vehicle distance, and safety early warning is performed before an accident occurs; the problems that existing safety early warning is not timely and the response time is too short are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle safety warning, and particularly to a safety warning method, system and vehicle based on brake light recognition. Background Art

[0002] The statements in this section only mention the background art related to the present invention and do not necessarily constitute prior art.

[0003] With the improvement of the intelligence level of automobiles and the increasing demand for vehicle use, the public's demand for vehicle use is getting higher and higher; the increase in road vehicles and the uneven driving experience of drivers have made the safety awareness of drivers driving vehicles increasingly important.

[0004] During the process of following a vehicle, the driver sometimes cannot fully recognize the braking intention of the vehicle in front, or due to personal distraction or other operations, it affects the driver's judgment and cannot timely recognize the braking intention of the vehicle in front and control the vehicle to decelerate or stop. Vehicle safety accidents caused by similar reasons are becoming more and more frequent.

[0005] In response to such phenomena, the existing technical solution is emergency avoidance before an accident is about to occur. When it is detected that the distance from the vehicle in front is too close and a collision may be triggered, the vehicle will automatically identify and judge to control the vehicle to stop or avoid obstacles. However, the existing technical solution is an emergency avoidance treatment solution based on the imminent occurrence of an accident and has the following deficiencies:

[0006] (1) Lack of active predictability, unable to predict risks earlier and provide warning information to the driver; only actively perform emergency braking or avoidance on the vehicle in the first few seconds before an accident occurs, with a high risk of losing control, and sudden vehicle intervention will cause the driver to make misjudgments, resulting in incorrect driving operations and a high safety risk; the handling method is urgent, prone to cause misoperations by the driver, and has a high failure risk.

[0007] (2) Only simply rely on the comparison result of the data collected by in-vehicle sensors and a preset threshold for warning, and the warning reliability needs to be improved; and the subjective driving experience of the driver is not considered, and there may be an overreaction phenomenon. Summary of the Invention

[0008] In order to solve the deficiencies of the prior art, the present invention provides a safety warning method, device, system and vehicle based on brake light recognition, which combines the brake light state of the vehicle in front with the information of the own vehicle such as the driver's intention for safety warning, prompting the driver to drive with concentrated attention and maintain a safe vehicle distance.

[0009] In the first aspect, the present invention provides a safety warning method based on brake light recognition;

[0010] A safety warning method based on brake light recognition, comprising:

[0011] Obtain an image sequence of the vehicle ahead, perform dynamic feature modeling on the image sequence, and determine the braking state of the brake lights of the vehicle ahead;

[0012] Perform distance recognition based on the image sequence to determine the first vehicle distance; obtain the second vehicle distance, and adaptively fuse the first vehicle distance and the second vehicle distance in combination with the weather conditions to determine the distance of the vehicle ahead;

[0013] Use the distance of the vehicle ahead and the driving speed of the own vehicle to determine the collision time;

[0014] Determine the response state of the driver, and determine the safety warning level according to the braking state, collision time and response state, so as to perform warning actions according to the safety warning level.

[0015] In some embodiments, the performing dynamic feature modeling on the image sequence and determining the braking state of the brake lights of the vehicle ahead includes:

[0016] Calculate the pixel brightness mean value of the brake light area in the image sequence and judge the on / off state; perform sliding window fast Fourier transform analysis on the on / off state of the brake light to determine the brake light flashing frequency; perform feature point tracking on the image sequence to determine the brake light position change amount;

[0017] Determine the lighting level of the brake lights of the vehicle ahead according to the on / off state and flashing frequency of the brake lights.

[0018] In some embodiments, the performing distance recognition based on the image sequence and determining the first vehicle distance includes:

[0019] Process the image of the vehicle ahead at the nearest moment in the image sequence through a trained yolo object detection model, and extract the pixel coordinates of the midpoint of the brake light in the image;

[0020] Obtain the camera internal parameters and external parameters, and calculate the first vehicle distance according to the camera internal parameters, camera external parameters and the pixel coordinates of the midpoint of the brake light in the image.

[0021] In some embodiments, the second vehicle distance is collected by a millimeter wave radar provided on the vehicle.

[0022] In some embodiments, the adaptively fusing the first vehicle distance and the second vehicle distance in combination with the weather conditions to determine the distance of the vehicle ahead includes:

[0023] Obtain real-time weather data, and calculate the reliability scores corresponding to the camera and the millimeter wave radar according to the real-time weather data;

[0024] Calculate the confidence weights corresponding to the camera and millimeter-wave radar based on their respective reliability scores, and fuse the first vehicle distance and the second vehicle distance to obtain the distance to the vehicle ahead.

[0025] In some embodiments, the determination of the driver's response status is specifically as follows:

[0026] If the throttle pedal signal is greater than a preset first opening threshold and no brake pedal signal is detected, the driver's response status is a non-response risk;

[0027] If the brake pedal signal is greater than a preset second opening threshold, the driver's response status is a response risk.

[0028] In some embodiments, the determination of the safety warning level based on the braking state, vehicle distance, and response status includes:

[0029] If the brake light of the vehicle ahead is a normal brake and the driver's response status is a non-response risk, the safety warning level is a primary warning;

[0030] If the brake light of the vehicle ahead is a normal brake, the collision time is less than a preset first time threshold, and the response status is a non-response risk, or the brake light of the vehicle ahead is an emergency brake and the collision time is less than a preset first time threshold, the safety warning level is an intermediate warning;

[0031] If the brake light of the vehicle ahead is a normal brake and the collision time is less than a preset second time threshold, or the distance to the vehicle ahead is less than the dynamic safety distance, the safety warning level is an emergency warning.

[0032] In a second aspect, the present invention provides a safety warning device based on brake light recognition;

[0033] A safety warning device based on brake light recognition includes:

[0034] A braking state determination module configured to: obtain an image sequence of the vehicle ahead, perform dynamic feature modeling on the image sequence, and determine the braking state of the brake light of the vehicle ahead;

[0035] A distance recognition module configured to: perform distance recognition based on the image sequence to determine a first vehicle distance; obtain a second vehicle distance, adaptively fuse the first vehicle distance and the second vehicle distance in combination with the weather conditions to determine the distance to the vehicle ahead; and use the distance to the vehicle ahead and the driving speed of the own vehicle to determine the collision time;

[0036] A warning module configured to: determine the driver's response status, determine the safety warning level based on the braking state, collision time, and response status, and perform a warning action according to the safety warning level.

[0037] In a third aspect, the present invention provides a safety warning system based on brake light recognition;

[0038] A safety warning system based on brake light recognition, comprising:

[0039] An identification module, configured to collect an image sequence and point cloud data of the vehicle ahead;

[0040] An operation and processing module, configured to obtain the image sequence of the vehicle ahead, perform dynamic feature modeling on the image sequence to determine the braking state of the brake light of the vehicle ahead; determine the point cloud data of the vehicle ahead, perform clustering processing on the point cloud data, use a stereo matching technology to process the image sequence and perform spatio-temporal alignment with the processed point cloud data, combine the weather condition to perform distance recognition to determine the vehicle distance; determine the driving intention of the driver, and determine the safety warning level according to the braking state, vehicle distance and driving intention;

[0041] An execution module, configured to execute a warning action according to the safety warning level.

[0042] In a fourth aspect, the present invention provides a vehicle;

[0043] A vehicle, comprising the above safety warning system based on brake light recognition.

[0044] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0045] 1. The technical solution provided by the present invention combines the brake light state, the distance of the vehicle ahead and the response state of the driver when facing risks to perform hierarchical safety warnings in advance when a collision risk may occur, which can remind the driver to pay attention to the safe vehicle distance earlier and avoid accidents; fully consider the driving intention of the driver during the driving process, while improving the driver's driving attention, ensuring the driving experience.

[0046] 2. The technical solution provided by the present invention combines the weather condition to evaluate the reliability of the data collected by the millimeter wave radar and the data collected by the camera. While fusing multi-source data, adaptively set the confidence weight according to the weather condition, effectively reducing the influence of external condition changes on the recognition accuracy, and improving the accuracy and reliability of vehicle distance recognition; at the same time, the two data are redundantly backed up to prevent the loss of functions due to hardware damage.

[0047] 3. The technical solution provided by the present invention simultaneously recognizes the state of the brake light and the vehicle distance based on the image sequence of the vehicle ahead. The recognition targets and features are fixed, and the influence caused by the change of a single system is small, and the recognition accuracy is higher. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings forming a part of this specification are used to provide a further understanding of the present invention. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation to the present invention.

[0049] Figure 1 A flowchart of the safety warning method based on brake light recognition provided by an embodiment of the present invention;

[0050] Figure 2 An architecture diagram of the safety warning system based on brake light recognition provided by an embodiment of the present invention. Detailed implementation manners

[0051] It should be noted that the following detailed descriptions are all exemplary and are intended to provide further descriptions of the present invention. Unless otherwise specified, all technical and scientific terms used in the present invention have the same meanings as those commonly understood by those of ordinary skill in the technical field to which the present invention belongs.

[0052] It should be noted that the terms used herein are only for describing specific implementation manners and are not intended to limit the exemplary implementation manners according to the present invention. As used herein, unless otherwise clearly specified in the context, the singular form is also intended to include the plural form. In addition, it should be understood that the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those clearly listed steps or units, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0053] In the case of no conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other.

[0054] Embodiment 1

[0055] The prior art lacks a reliable pre-warning of the risk of collision with the vehicle in front. Therefore, the present application provides a safety warning method based on brake light recognition, which is conducive to grading prediction of the distance between the vehicle and the vehicle in front and the driver's response state, so as to assist the driver in giving an early warning and giving the driver a warning and sufficient operation time.

[0056] Next, in combination with Figure 1 , a safety warning method based on brake light recognition disclosed in this embodiment will be described in detail. The safety warning method based on brake light recognition includes the following steps:

[0057] S1. Obtain an image sequence of the vehicle in front, perform dynamic feature modeling on the image sequence, and determine the braking state of the brake light of the vehicle in front.

[0058] Considering that during the vehicle driving process, the braking light state of the vehicle in front may change dynamically. Therefore, in this embodiment, a timing state matrix of the braking light of the vehicle in front is constructed through dynamic feature modeling, and the flashing frequency, average brightness, and position change amount of the braking light of the vehicle in front are detected in real time. The braking state of the braking light of the vehicle in front is determined by combining the flashing frequency and the average brightness.

[0059] Here, the image sequence is collected by a camera installed on the vehicle, and the image sequence is a time series of images collected in real time during the vehicle driving process.

[0060] Furthermore, it is judged whether the braking light is within the effective working area through the position change amount of the braking light.

[0061] Here, the effective working area refers to whether the camera is in the original installation position and whether the position is relatively unchanged. If the vehicle lamp falls off or the camera is loose, it may cause the position change of the braking light, which may lead to misjudgment of the braking light. This situation belongs to not being in the effective working area, and function degradation and a prompt for a fault request for inspection are required.

[0062] As an implementation manner, S1 specifically includes:

[0063] S101. Preprocess the image sequence.

[0064] In order to remove noises such as the reflection of the taillight, each front vehicle image in the image sequence is preprocessed in this step. Specifically, first, the front vehicle image is converted to the HSV space, and the red area is extracted based on a preset red threshold range and a mask is generated to obtain an intermediate image; then, morphological filtering is performed on the intermediate image; finally, the area, circumscribed rectangle, and centroid of each connected region are calculated, and the qualified connected regions are extracted and screened based on a preset editing threshold and aspect ratio; the horizontal distance and vertical alignment degree between the screened connected regions are calculated, and it is judged whether they conform to the symmetry feature of the braking light. If so, they are screened as the braking light area.

[0065] Exemplarily, the horizontal distance can be calculated as the difference in the x-axis coordinates of the centroid coordinates of two red connected regions. If the horizontal distance is within the allowable horizontal distance error, it is judged as symmetric; the vertical alignment degree can be calculated as the difference in the y-axis coordinates of the centroid coordinates of two red connected regions. If the vertical alignment degree is less than a preset threshold (such as 5 pixels), it is determined as vertically aligned.

[0066] S102. Calculate the pixel brightness mean value of the braking light area, and judge the on / off state according to the pixel brightness mean value.

[0067] Specifically, in the V channel of the HSV color space, Gaussian weighted averaging is performed on all pixels within the brake light area to obtain the average pixel brightness. If the average pixel brightness is greater than a preset brightness threshold, it is in the bright state; otherwise, it is in the dark state.

[0068] Furthermore, the duration of the brake light state can be determined based on the brake light states corresponding to the front vehicle images in adjacent time frames.

[0069] S103. Perform sliding window fast Fourier transform analysis on the image sequence to determine the brake light flashing frequency.

[0070] Arrange the average pixel brightness values of the pixels within the brake light area corresponding to the time frames within the sliding time window in chronological order as the brightness time series signal of the brake light. Perform fast Fourier transform analysis on the brightness time series signal, calculate the frequency domain energy distribution, and extract the main frequency as the brake light flashing frequency of the current window.

[0071] Here, the sliding time window is 300 ms.

[0072] S104. Through ORB feature point tracking, calculate the Euclidean distance of the centroid offset of the same brake light between adjacent frames to determine the position change amount of the brake light.

[0073] S105. Determine the lighting level of the front vehicle's brake light based on the on / off state and flashing frequency of the brake light.

[0074] Specifically, if the brake light is constantly on for ≥500 ms or the flashing frequency is 1.2 Hz ± 0.3 Hz, it is a normal brake; if the flashing frequency is 3.5 Hz ± 0.5 Hz, it is an emergency brake; if the flashing frequency is 0.8 Hz ± 0.2 Hz, it is a malfunction light.

[0075] S2. Perform distance recognition based on the image sequence to determine the first vehicle distance. Specifically, it includes:

[0076] S201. Process the front vehicle image at the nearest moment in the image sequence through a trained yolo object detection model, and extract the pixel coordinates (u, v) of the midpoint of the brake light in the image.

[0077] In this embodiment, the yolo object detection model uses YOLOv8. The input is the front vehicle image, and the output is the pixel coordinates of the midpoint at the bottom of the license plate in the image. YOLOv8 is trained using a large number of vehicle images annotated with the above pixel coordinates.

[0078] S202. Obtain the camera internal parameters (focal length f, optical center coordinates (u0, v0)) and external parameters (installation height H, pitch angle θ). According to the camera internal parameters, camera external parameters, and the pixel coordinates (u, v) of the midpoint of the brake light in the image, calculate the first vehicle distance.

[0079] Exemplarily, the first vehicle distance D1 is expressed as:

[0080]

[0081] Using the status of the brake lights for vehicle distance recognition, the recognition target and features are fixed, and the impact caused by the change of a single system is small, and the recognition accuracy is higher.

[0082] S3. Obtain the second vehicle distance, adaptively fuse the first vehicle distance and the second vehicle distance in combination with the weather condition, and determine the distance of the vehicle ahead.

[0083] In this embodiment, the second vehicle distance is directly collected by a millimeter-wave radar installed on the host vehicle.

[0084] Since the hardware recognition accuracy is related to multiple systems, such as on rainy days, considering that there may be misjudgments in distance recognition based on a single data, therefore, in this embodiment, fusing the first vehicle distance obtained by image recognition and the second vehicle distance measured by the millimeter-wave radar can effectively reduce the impact of external condition changes on the recognition accuracy, so as to improve the accuracy of the vehicle distance recognition of the vehicle ahead.

[0085] At the same time, the two data are redundantly backed up to prevent the loss of functions due to hardware damage.

[0086] Moreover, considering that the confidence levels of the data collected by the camera and the millimeter-wave radar are different under different weather conditions, in order to further fit the actual weather during vehicle driving to improve the data authenticity, the confidence level weights are adaptively set according to the weather condition.

[0087] As an implementation manner, S3 includes:

[0088] S301. Obtain real-time weather data and quantify it.

[0089] Specifically, first, collect the rainfall, visibility, and light intensity through an in-vehicle meteorological sensor; then, determine the rainfall intensity quantization value according to the rainfall; determine the light quantization value according to the light intensity and the maximum light intensity of the local season in the current season; set the visibility quantization value according to the visibility.

[0090] Exemplarily, the rainfall intensity quantization value R is expressed as:

[0091]

[0092] In the formula, m represents the rainfall, and M represents the quantization threshold, such as 10 mm / h.

[0093] The light quantization value L is expressed as:

[0094]

[0095] In the formula, l represents the light intensity, lmax Represents the maximum light intensity.

[0096] The visibility quantization value W is expressed as:

[0097]

[0098] Where w represents visibility.

[0099] S302. Calculate the reliability scores corresponding to the first vehicle distance and the second vehicle distance respectively according to the rainfall intensity quantization value, the light quantization value, and the visibility quantization value.

[0100] Exemplarily, the reliability score s1 corresponding to the first vehicle distance is expressed as:

[0101] s1 = w1·W + w2·(1 - R) + w3·C detect ;

[0102] Where w1, w2, w3 represent weight coefficients, and C detect Represents the detection confidence, which can be output by the yolo object detection model in S201.

[0103] The reliability score s2 corresponding to the second vehicle distance is expressed as:

[0104] s2 = α·W + β·(1 - R) + γ·ρ;

[0105] Where α, β, γ represent adjustment coefficients, which can be 0.5, 0.3, 0.2 respectively; ρ represents the tracking stability.

[0106] S303. Determine the confidence weights w1, w2 corresponding to the first vehicle distance d1 and the second vehicle distance d2 according to the reliability scores corresponding to the first vehicle distance d1 and the second vehicle distance d2, and fuse the first vehicle distance d1 and the second vehicle distance d2 to obtain the leading vehicle distance d.

[0107] Exemplarily, the leading vehicle distance d is expressed as:

[0108] d = w1·d1 + w2·d2;

[0109]

[0110] Where a, b represent weighting coefficients.

[0111] S4. Use the leading vehicle distance and the driving speed of the own vehicle to determine the time to collision TTC and the dynamic safety distance.

[0112] TTC = leading vehicle distance / (own vehicle speed - leading vehicle speed);

[0113] Here, if the leading vehicle speed is unknown, assume the leading vehicle speed is 80% of the own vehicle speed.

[0114] Safety distance threshold = own vehicle speed (m / s) * 1.5 s (reserved reaction time).

[0115] S5. Determine the driver's response status, and determine the safety warning level according to the braking status, collision time and response status, so as to execute warning actions according to the safety warning level.

[0116] Exemplarily, the safety warning levels and corresponding triggering conditions and warning actions are shown in the following table.

[0117]

[0118] Furthermore, the above warning conditions already include the following typical scenario examples:

[0119] a. High-speed following (80 km / h)

[0120] At this time, the brake light of the vehicle in front is on, TTC = 3 s, and the driver does not brake, triggering a primary warning. If TTC drops to 1.5 s after 2 s, it is upgraded to an emergency warning.

[0121] b. Urban congestion (20 km / h)

[0122] The brake light of the vehicle in front is not on, but the radar distance suddenly shortens to 5 m (safety distance = 20 × 1.5 / 3.6 ≈ 8.3 m), triggering a conventional distance warning.

[0123] c. False trigger suppression

[0124] The driver has stepped on the brake (brake pedal > 30%), and all warnings are silenced.

[0125] Embodiment 2

[0126] This embodiment discloses a safety warning device based on brake light recognition, including:

[0127] A braking state determination module, configured to: obtain an image sequence of the vehicle in front, perform dynamic feature modeling on the image sequence, and determine the braking state of the brake light of the vehicle in front;

[0128] A distance recognition module, configured to: perform distance recognition based on the image sequence to determine the first vehicle distance; obtain the second vehicle distance, adaptively fuse the first vehicle distance and the second vehicle distance in combination with the weather condition to determine the distance of the vehicle in front; use the distance of the vehicle in front and the driving speed of the own vehicle to determine the collision time;

[0129] A warning module, configured to: determine the driver's response status, and determine the safety warning level according to the braking status, collision time and response status, so as to execute warning actions according to the safety warning level.

[0130] It should be noted here that the above-mentioned braking state determination module, distance recognition module, and warning module correspond to the steps in the first embodiment. The examples and application scenarios implemented by the above modules and the corresponding steps are the same, but are not limited to the content disclosed in the first embodiment. It should be noted that the above modules, as part of the system, can be executed in a computer system such as a set of computer-executable instructions.

[0131] Embodiment Three

[0132] Based on the safety warning method based on brake light recognition described in Embodiment One, this embodiment provides a safety warning system based on brake light recognition, including an identification module, an operation processing module, a signal input module, and an execution module; the identification module is used to collect the image sequence and point cloud data of the vehicle in front, the signal input module is used to input the image sequence and point cloud data of the vehicle in front into the operation processing module, and the operation processing module is used to obtain the image sequence of the vehicle in front, perform dynamic feature modeling on the image sequence, and determine the braking state of the brake light of the vehicle in front; perform distance recognition based on the image sequence to determine the first vehicle distance; obtain the second vehicle distance, adaptively fuse the first vehicle distance and the second vehicle distance in combination with the weather condition to determine the distance of the vehicle in front; use the distance of the vehicle in front and the driving speed of the own vehicle to determine the collision time; determine the response state of the driver, and determine the safety warning level according to the braking state, collision time, and response state; the execution module is used to perform warning actions according to the safety warning level.

[0133] Specifically, the identification module includes a camera and a millimeter-wave radar installed at the front of the vehicle. The camera is used to collect the image sequence, and the millimeter-wave radar is used to collect the second vehicle distance; the execution module includes the vehicle's built-in voice system, buzzer, and seat belt. During primary warning, the voice system gives a prompt and the buzzer gives a short beep. During intermediate warning, the voice system gives a prompt and the buzzer gives a long beep. During emergency warning, the voice system gives a prompt and the seat belt is prepared to tighten; the specific implementation process of the operation processing module and the control of the execution module are the same as those in Embodiment One, and will not be elaborated in this embodiment.

[0134] Furthermore, it also includes a signal input module, which is used to collect the throttle pedal signal and brake pedal signal of the current vehicle, so as to facilitate the operation processing module to judge the driving intention of the driver.

[0135] Embodiment Four

[0136] Based on the above safety warning system based on brake light recognition, this embodiment also provides a vehicle, in which the above-mentioned safety warning system based on brake light recognition described in the above embodiment is provided. Since the above safety warning system based on brake light recognition has the above technical effects, the technical effects of the vehicle adopting the above safety warning system based on brake light recognition can refer to the above embodiment.

[0137] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in one Figure 1 flow or multiple flows and / or blocks Figure 1 or multiple blocks.

[0138] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means that implement the functions specified in one Figure 1 flow or multiple flows and / or blocks Figure 1 or multiple blocks.

[0139] These computer program instructions can also be loaded onto a computer or other programmable data processing device, and a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one Figure 1 flow or multiple flows and / or blocks Figure 1 or multiple blocks.

[0140] In the above embodiments, the descriptions of the respective embodiments have their own focuses. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0141] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A safety warning method based on brake light recognition, characterized in that, Including: Obtain an image sequence of the vehicle ahead, perform dynamic feature modeling on the image sequence, and determine the braking state of the brake light of the vehicle ahead; Based on the image sequence, perform distance recognition to determine the first vehicle distance; Obtain the second vehicle distance, adaptively fuse the first vehicle distance and the second vehicle distance in combination with the weather condition to determine the vehicle distance ahead; Utilize the vehicle distance ahead and the driving speed of the own vehicle to determine the collision time; Determine the response state of the driver, and determine the safety warning level according to the braking state, collision time, and response state, so as to execute a warning action according to the safety warning level.

2. The safety warning method based on brake light recognition according to claim 1, wherein, The performing dynamic feature modeling on the image sequence and determining the braking state of the brake light of the vehicle ahead includes: Calculate the average pixel brightness of the brake light area in the image sequence and judge the on / off state; perform sliding window fast Fourier transform analysis on the on / off state of the brake light to determine the brake light flashing frequency; perform feature point tracking on the image sequence to determine the change amount of the brake light position; According to the on / off state and flashing frequency of the brake light, determine the lighting level of the brake light of the vehicle ahead.

3. The safety warning method based on brake light recognition according to claim 1, wherein, The performing distance recognition based on the image sequence and determining the first vehicle distance includes: Process the image of the vehicle ahead at the nearest moment in the image sequence through a trained YOLO object detection model, and extract the pixel coordinates of the midpoint of the brake light in the image; Obtain the internal parameters and external parameters of the camera, and calculate the first vehicle distance according to the internal parameters of the camera, the external parameters of the camera, and the pixel coordinates of the midpoint of the brake light in the image.

4. The safety warning method based on brake light recognition according to claim 1, characterized in that, The second vehicle distance is collected by a millimeter wave radar provided on the vehicle.

5. The safety warning method based on brake light recognition according to claim 1, wherein, The adaptively fusing the first vehicle distance and the second vehicle distance in combination with the weather condition to determine the vehicle distance ahead includes: Obtain real-time weather data, and calculate the reliability scores corresponding to the camera and the millimeter wave radar according to the real-time weather data; According to the reliability scores corresponding to the camera and the millimeter wave radar, calculate the confidence weights corresponding to the camera and the millimeter wave radar and fuse the first vehicle distance and the second vehicle distance to obtain the vehicle distance ahead.

6. The safety warning method based on brake light recognition according to claim 1, characterized in that, The determining the response state of the driver specifically is: If the throttle pedal signal is greater than a preset first opening threshold and no brake pedal signal is detected, the response state of the driver is the unresponsive risk; If the brake pedal signal is greater than a preset second opening threshold, the response state of the driver is the responsive risk.

7. The safety warning method based on brake light recognition according to claim 1, characterized in that, The determining the safety warning level according to the braking state, vehicle distance, and response state includes: If the brake light of the vehicle ahead is in normal braking and the response state of the driver is the unresponsive risk, the safety warning level is the primary warning; If the brake light of the vehicle ahead is in normal braking, the collision time is less than a preset first time threshold and the response state is the unresponsive risk, or the brake light of the vehicle ahead is in emergency braking and the collision time is less than a preset first time threshold, the safety warning level is the intermediate warning; If the brake light of the vehicle ahead is in normal braking and the collision time is less than a preset second time threshold, or the vehicle distance ahead is less than the dynamic safety distance, the safety warning level is the emergency warning.

8. The safety warning device based on brake light recognition is characterized in that Including: A braking state determination module, configured to: obtain an image sequence of the vehicle ahead, perform dynamic feature modeling on the image sequence, and determine the braking state of the brake light of the vehicle ahead; The distance recognition module is configured to: perform distance recognition based on an image sequence to determine a first vehicle distance; obtain a second vehicle distance, adaptively fuse the first vehicle distance and the second vehicle distance in combination with the weather condition to determine the distance to the vehicle ahead; and use the distance to the vehicle ahead and the driving speed of the host vehicle to determine the time to collision. The warning module is configured to: determine the response state of the driver, determine the safety warning level according to the braking state, the time to collision, and the response state, so as to perform a warning action according to the safety warning level.

9. A safety warning system based on brake light recognition, characterized in that, Comprising: The recognition module is used for collecting the image sequence and point cloud data of the vehicle ahead. The operation and processing module is used for obtaining the image sequence of the vehicle ahead, performing dynamic feature modeling on the image sequence to determine the braking state of the brake light of the vehicle ahead; performing distance recognition based on the image sequence to determine a first vehicle distance; obtaining a second vehicle distance, adaptively fusing the first vehicle distance and the second vehicle distance in combination with the weather condition to determine the distance to the vehicle ahead; and using the distance to the vehicle ahead and the driving speed of the host vehicle to determine the time to collision. Determine the response state of the driver, and determine the safety warning level according to the braking state, the time to collision, and the response state. The execution module is used for performing a warning action according to the safety warning level.

10. A vehicle, characterized in that, Including the safety warning system based on brake light recognition described in claim 9.

Images