Vehicle early warning method, device and equipment and storage medium

Abstract

Embodiments of the present application provide a vehicle early warning method, device, equipment and storage medium. The method comprises: acquiring a road surface image of a road ahead of a vehicle; using a deep learning model, acquiring a target lane line in the road surface image, a relative distance and a relative speed of an obstacle ahead of the vehicle and the vehicle; correcting the relative distance according to the width of the target lane line near the vehicle and the width of the target lane line near the obstacle in the road surface image; correcting the relative distance according to the size of the bounding box of the obstacle ahead in multiple frames of road surface images; judging whether the obstacle ahead and the vehicle exist a collision risk according to the corrected relative distance and the relative speed; and if yes, outputting a collision risk warning information. The method only needs to install a camera on the vehicle, needs a small number of sensors, has a low system cost, is easy to maintain, and has a high collision risk prediction accuracy.

Description

Technical Field

[0001] This application relates to the field of driver assistance technology, and in particular to a vehicle warning method, device, equipment and storage medium. Background Technology

[0002] Because heavy vehicles operate in complex environments and have variable operating conditions, and because of their heavy weight, they are difficult to decelerate in emergencies and have a high risk of overturning, heavy vehicles often cause great injury and loss to drivers and other road users if they encounter an emergency collision.

[0003] To achieve forward collision warning, the current mainstream approach is to integrate data from both cameras and millimeter-wave radar to accurately monitor the lane ahead and targets. This approach requires a large number of sensors, resulting in higher system costs and greater maintenance difficulties. Summary of the Invention

[0004] This application provides a vehicle warning method, device, equipment, and storage medium to address the problems of existing vehicle warning methods that rely on the fusion of data from both cameras and millimeter-wave radar, resulting in a large number of sensors, high system costs, and difficult maintenance.

[0005] In a first aspect, this application provides a vehicle collision warning method, the method comprising:

[0006] Acquire the first road surface image of the road ahead of the vehicle;

[0007] Using a deep learning model, the target lane line in the first road surface image, the first relative distance between the obstacle in front of the vehicle and the vehicle, and the first relative speed are obtained;

[0008] Based on the first width of the target lane line at the first position and the second width at the second position in the first road surface image, the second relative distance between the obstacle ahead and the vehicle is calculated; the first position is the position near the vehicle, and the second position is the position near the obstacle ahead;

[0009] Based on the first road surface image and the size of the bounding box of the obstacle in at least one frame of the second road surface image acquired before the first road surface image, the second relative speed between the obstacle and the vehicle is calculated.

[0010] The first relative distance is corrected based on the second relative distance to obtain a third relative distance, and the first relative velocity is corrected based on the second relative velocity to obtain a third relative velocity;

[0011] Based on the third relative distance and the third relative speed, determine whether there is a risk of collision between the obstacle ahead and the vehicle;

[0012] If so, output a collision risk warning message.

[0013] In one possible design, the step of determining the first width of the target lane line at a first location and the second width at a second location in the first road surface image includes:

[0014] Detect a first number of pixels in the width direction of the target lane line at the first position, and a second number of pixels in the width direction of the target lane line at the second position;

[0015] The second relative distance is calculated based on the difference between the first quantity and the second quantity.

[0016] In one possible design, the step of correcting the first relative distance based on the second relative distance to obtain a third relative distance, and correcting the first relative velocity based on the second relative velocity to obtain a third relative velocity, includes:

[0017] The first relative distance and the second relative distance are input into the calibration deep learning model, and the calibration deep learning model outputs the third relative distance.

[0018] The first relative velocity and the second relative velocity are input into the correction deep learning model, and the correction deep learning model outputs the third relative velocity.

[0019] In one possible design, obtaining the target lane line in the first road surface image using a deep learning model includes:

[0020] Based on the first road surface image, identify the current road conditions of the road in question;

[0021] Based on the current road conditions, determine the recognition strategy corresponding to the current road conditions;

[0022] According to the recognition strategy, the first road surface image is adjusted to determine the third road surface image;

[0023] The third road surface image is input into a recognition deep learning model, which then identifies the target lane line based on the third road surface image.

[0024] In one possible design, after obtaining the target lane line in the first road surface image using a deep learning model, the method further includes:

[0025] Based on the vehicle's trajectory data within a future preset time range and the target lane line information, the actual lateral distance between the vehicle and the target lane line within the future preset time range is determined.

[0026] Based on the actual lateral distance, determine whether the vehicle is at risk of lane departure;

[0027] If so, then output a lane departure warning message.

[0028] In one possible design, determining the actual lateral distance between the vehicle and the target lane line within the future preset time range, based on the vehicle's trajectory data within that range and the target lane line information, includes:

[0029] Based on the vehicle's trajectory data within a preset time range and the target lane information, a first lateral distance between the vehicle and the target lane is determined.

[0030] Based on the lateral acceleration signal and longitudinal acceleration signal of the vehicle sent by the vehicle, the lateral displacement value of the vehicle relative to the target lane line within the future preset time range is determined;

[0031] The first lateral distance is corrected based on the lateral displacement value to determine the actual lateral distance.

[0032] In one possible design, acquiring a first road surface image of the road ahead of the vehicle includes:

[0033] The light intensity of the environment in which the vehicle is located is detected, and it is determined whether the light intensity is lower than a preset light intensity threshold.

[0034] If so, the settings parameters of the vehicle camera are adjusted according to the light intensity of the environment in which the vehicle is located.

[0035] Control the camera to acquire the first road surface image.

[0036] Secondly, this application provides a vehicle warning device, the device comprising:

[0037] The visual detection module is used to: acquire the first road surface image of the road ahead of the vehicle;

[0038] The recognition module is used to: use a deep learning model to obtain the target lane line in the first road surface image, the first relative distance between the obstacle in front of the vehicle and the vehicle, and the first relative speed;

[0039] The calculation module is configured to: calculate a second relative distance between the obstacle ahead and the vehicle based on a first width of the target lane line at a first position and a second width at a second position in the first road surface image; the first position is a position near the vehicle, and the second position is a position near the obstacle ahead;

[0040] Based on the first road surface image and the size of the bounding box of the obstacle in at least one frame of the second road surface image acquired before the first road surface image, the second relative speed between the obstacle and the vehicle is calculated.

[0041] The correction module is configured to: correct the first relative distance based on the second relative distance to obtain a third relative distance, and correct the first relative velocity based on the second relative velocity to obtain a third relative velocity;

[0042] Collision risk warning module: Based on the third relative distance and the third relative speed, determine whether there is a risk of collision between the obstacle ahead and the vehicle;

[0043] If so, output a collision risk warning message.

[0044] Thirdly, this application provides an electronic device, including: a memory and a processor;

[0045] The memory stores computer-executed instructions;

[0046] The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in the first aspect.

[0047] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method described in the first aspect.

[0048] The vehicle warning method, apparatus, device, and storage medium provided in this application embodiment complete the collision risk warning only through images captured by a camera. It only requires installing a camera on the vehicle, necessitating a small number of sensors, resulting in low system cost and easy maintenance. Since it only uses image data and lacks data from other sensors, this method corrects the relative distance between the obstacle and the vehicle identified by the deep learning model by using the width of lane lines in the road image, and corrects the relative speed between the obstacle and the vehicle identified by the deep learning model by using changes in the size of the bounding box of the obstacle in multiple frames of images. This improves the accuracy of identifying the relative distance and relative speed between the obstacle and the vehicle. Attached Figure Description

[0049] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0050] Figure 1 Flowchart of the vehicle early warning method provided in the embodiments of this application Figure 1 ;

[0051] Figure 2 A schematic diagram of a vehicle warning system provided in an embodiment of this application;

[0052] Figure 3 This is a schematic diagram of the interaction between the instrument panel and camera controller assembly of the vehicle warning system provided in an embodiment of this application;

[0053] Figure 4 Flowchart of the vehicle early warning method provided in the embodiments of this application Figure 2 ;

[0054] Figure 5 A schematic diagram of the vehicle warning device provided in this application;

[0055] Figure 6 A schematic diagram of the structure of the electronic device provided in this application.

[0056] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0057] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0058] First, let me explain the terms used in this application:

[0059] Tenengrad gradient algorithm: The Tenengrad gradient algorithm is a method for evaluating image sharpness. It determines image sharpness based on gradient information. Specifically, the algorithm calculates the gradient of each pixel in the image (usually using the Sobel operator), and then evaluates the image sharpness by statistically analyzing the magnitude of these gradients. The larger the gradient, the more detail the image contains, and the higher the sharpness.

[0060] Bounding box: A bounding box is a commonly used concept in computer vision, referring to a rectangular box used to mark the location of a target object in an image. A bounding box is typically defined by four parameters: the coordinates of the top-left corner (x, y), and the width and height (width, height). In object detection tasks, bounding boxes are used to identify and locate target objects in an image.

[0061] Regression Box: A regression box is a bounding box predicted by a regression algorithm. In object detection tasks, regression boxes are used to accurately locate target objects. The regression model predicts the bounding box parameters (such as center point coordinates, width, and height) of the target object based on the features of the input image, thereby obtaining a more accurate target location.

[0062] CAN Network: The CAN network (Controller Area Network) of a vehicle is a standardized network protocol used for communication between electronic control units (ECUs) within a car.

[0063] Model prediction algorithms: Model prediction algorithms refer to the process of using trained machine learning or deep learning models to predict new data. The specific algorithm depends on the type of model and the application scenario. For example, in classification tasks, model prediction algorithms output class labels based on the input data; in regression tasks, they output continuous values. In object detection tasks, they output the class and location (bounding box) of the target object.

[0064] Existing vehicle warning methods, which integrate data from both cameras and millimeter-wave radar, require a large number of sensors, resulting in high system costs and maintenance difficulties. With the rapid advancement of camera perception technology, camera data alone can accurately determine lane and forward target conditions, making radar data fusion less beneficial. To address these issues, this application proposes the following technical concept: acquiring first road surface image data of the current road location, inputting this image into a deep learning model to identify lane lines and the relative distance and speed between the vehicle and obstacles in front. The relative distance identified by the deep learning model is then corrected based on the width of lane lines near the vehicle and obstacles in the first road surface image, and the relative speed is corrected based on changes in the bounding box size of obstacles in multiple frames of the first road surface image. Based on the corrected relative speed and distance between the vehicle and obstacles, if a collision risk is identified, a collision risk warning is issued to the driver.

[0065] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0066] Example 1

[0067] Figure 1 Flowchart of the vehicle early warning method provided in the embodiments of this application Figure 1 This method can be applied to vehicle warning systems, such as... Figure 1 As shown, the method includes:

[0068] S101. Obtain the first road surface image of the road ahead of the vehicle;

[0069] Specifically, this step controls the camera mounted in front of the vehicle to acquire a first image of the road surface. During the preprocessing stage of this first road surface image, the Tenengrad gradient algorithm can be used to calculate the image change gradients in both the horizontal and vertical directions. Based on different gradient values, edge detection and sharpness assessment of the image can be performed, resulting in more accurate target features and thus enabling more accurate and rapid target identification.

[0070] The preprocessed first road surface image is input into a deep learning model for recognition.

[0071] S102. Using a deep learning model, obtain the target lane line, the first relative distance between the vehicle and the obstacle in front of the vehicle, and the first relative speed in the first road surface image;

[0072] Since this method only uses image information captured by a camera and lacks other dimensions of information for auxiliary judgment, deep learning models including convolutional neural networks (CNNs) and recurrent neural networks (RNNs) can be used. CNNs are used for image feature extraction, while simultaneously identifying and analyzing the motion posture of targets ahead (vehicles, pedestrians, etc.). RNNs are used to consider various information from previous targets when processing each target in the time series and to utilize this information when processing subsequent target inputs, thus efficiently predicting the accurate type, position, and posture changes of the targets. Specifically, the image frames of the preprocessed first road surface image are input into the CNN, which outputs feature vectors. The sequence of feature vectors from the CNN is then input into the RNN, which outputs the target lane lines, the first relative distance between the vehicle and obstacles in front of it, and the first relative speed. By combining CNNs and RNNs, more accurate information about the type, speed, and distance of obstacles ahead can be obtained.

[0073] S103. Calculate the second relative distance between the obstacle ahead and the vehicle based on the first width of the target lane line at the first position and the second width at the second position in the first road surface image;

[0074] Specifically, the first position is the location near the vehicle, and the second position is the location near the obstacle in front.

[0075] Since lane widths are consistent across the same road, in the first road image captured by the vehicle's camera, the lane widths near the vehicle and near the obstacle will inevitably differ due to the distance between the vehicle and the obstacle. By detecting this difference and performing corresponding calculations, the second relative distance between the vehicle and the obstacle can be calculated.

[0076] S104. Based on the first road surface image and the size of the bounding box of the obstacle in front in at least one frame of the second road surface image acquired before the first road surface image, calculate the second relative speed between the obstacle in front and the vehicle.

[0077] Specifically, by comparing the changes in the size of the bounding box of the obstacle in front in multiple frames of images, the second relative speed between the obstacle and the vehicle can be calculated.

[0078] S105. Correct the first relative distance based on the second relative distance to obtain the third relative distance, and correct the first relative velocity based on the second relative velocity to obtain the third relative velocity;

[0079] Since the first relative distance and first relative velocity identified solely by a deep learning model may contain errors, this step can utilize data fusion techniques to perform a weighted average of the first and second relative distances to obtain a more accurate third relative distance. Similarly, a weighted average of the first and second relative velocities can be performed to obtain the third relative velocity. Alternatively, a Kalman filter can be used to correct for the distance and velocity.

[0080] S106. Based on the third relative distance and the third relative speed, determine whether there is a risk of collision between the obstacle ahead and the vehicle;

[0081] In this step, the third relative distance and the third relative speed can be used to calculate the collision time between the vehicle and the obstacle in front, and the existence of a collision risk can be determined based on the collision time.

[0082] If yes, then execute S107; otherwise, execute S101.

[0083] S107, Output collision risk warning information.

[0084] Specifically, a collision risk warning can be output through an alarm installed in the vehicle's driver's compartment, and the warning information can include sound, text, and images.

[0085] The technical advantages of this embodiment are: the method completes collision risk warning only through images captured by a camera, requiring only a camera to be installed on the vehicle, resulting in a small number of sensors, low system cost, and easy maintenance. Since it only uses image data and lacks data from other sensors, this method corrects the relative distance between the obstacle and the vehicle identified by the deep learning model by using the width of lane lines in the road image, and corrects the relative speed between the obstacle and the vehicle identified by the deep learning model by using changes in the size of the bounding box of the obstacle in multiple frames of images, thereby improving the accuracy of identifying the relative distance and relative speed between the obstacle and the vehicle.

[0086] The solution of this application will be described below with a specific embodiment.

[0087] Example 2

[0088] Figure 2 This is a schematic diagram of a vehicle warning system provided in an embodiment of this application, such as... Figure 2 As shown, the system includes a camera controller assembly, an instrument cluster central control screen, and a driver's cabin alarm. The camera controller assembly integrates a camera sensor, image processor, and function controller, enabling high-precision image acquisition and high-speed processing to extract target information. Simultaneously, utilizing the functional logic of the function controller, it issues accurate and timely warnings for risky situations. The camera controller assembly should be installed in the center of the windshield near the dashboard within the driver's cabin. This position does not obstruct the driver's view and provides a wide and symmetrical field of view, facilitating target monitoring and identification. The camera controller assembly connects to the vehicle via a CAN network, exchanging information such as vehicle speed, steering, and gear position to aid in inferring the target's status and as a condition for warning function judgment. Simultaneously, the camera can also transmit warning signals via the CAN network to the instrument cluster central control screen and the driver's cabin alarm to alert the driver to collision risks or lane departure risks.

[0089] The vehicle's instrument cluster and central control screen are connected to the camera controller assembly via the in-vehicle CAN network to display warning information. At the same time, the screen can accurately transmit the driver's operating intentions (such as switching the system on and off, setting sensitivity, etc.) to the camera controller assembly, improving the flexibility of the system's human-machine interaction.

[0090] The alarms in the driver's cab are installed on the left and right sides of the cab. They communicate with the camera controller assembly via the vehicle's CAN network to receive warning messages and convert them into audible and visual signals to alert the driver to potential risks.

[0091] When the vehicle is started and powered on, the camera controller assembly activates its target monitoring capability. Simultaneously, the camera activates the corresponding warning function when the vehicle speed reaches a set value. If there is a risk of collision ahead, the camera will report a warning message to the instrument cluster screen and alarm via the CAN network to remind the driver of the risk of collision ahead. If the vehicle deviates from the left or right lane lines, the camera will detect the change in the lane line position and determine the lane line that the vehicle has deviated from. At this time, the camera will also report a lane departure warning signal to the instrument cluster screen and alarm via the CAN network to remind the driver that the vehicle is currently deviating from the main lane and there is a risk of collision with vehicles in adjacent lanes. Specifically, after recognizing road conditions ahead, the camera controller assembly integrates the road condition information into a virtual vehicle coordinate system based on the vehicle's body information (front overhang, wheelbase, width, tire distance, etc.) and the camera controller assembly's installation location. This allows it to calculate the relative position of the target ahead to the vehicle. Simultaneously, it combines vehicle information such as speed, steering, and gear position obtained from the vehicle's CAN network, and uses a built-in dual-warning algorithm to calculate potential forward collision and lane departure risks. For example, it determines there is no collision risk when the vehicle is turning or in reverse. Once the risk reaches a set threshold, the camera controller assembly sends a risk warning signal via CAN message.

[0092] When the vehicle is started and powered on, the instrument cluster and central control screen will also activate and perform a self-check. If the instrument cluster and central control screen do not detect any messages from the camera at this time, it determines that the camera system is faulty, and the instrument cluster and central control screen will display the corresponding warning symbol to remind the driver that the lane departure warning or forward collision warning functions are currently unavailable. If the instrument cluster and central control screen detects messages from the camera, but the messages indicate a fault, the instrument cluster and central control screen will display the corresponding warning symbol to remind the driver that the lane departure warning or forward collision warning functions are currently unavailable.

[0093] The instrument cluster's central control screen can also be configured to switch on / off the forward collision warning system and the lane departure warning system, allowing the driver to manually turn the system off or on. When the driver manually turns off the system, the instrument cluster's central control screen converts the corresponding signal into a message and sends it to the camera controller assembly via the CAN network. Upon receiving the switch control message from the instrument cluster, the camera controller assembly can accordingly turn the lane departure warning or forward collision warning function on or off. Simultaneously, the lane departure warning or forward collision warning functions, while meeting regulatory requirements, allow the driver to manually adjust the warning sensitivity or warning range to suit the individual needs of different users. The instrument cluster's central control screen can be used to set the corresponding warning sensitivity level (low, medium, high) and priority warning targets (pedestrians, bicycles, cars, trucks, etc.). The instrument cluster sends the user's requirements to the camera controller assembly via specially configured messages. After receiving this external parameter configuration information, the camera controller assembly adjusts the relevant priority parameters of the strategy to meet the user's needs. In addition, after receiving the warning messages sent by the camera, the instrument panel will also parse and generate corresponding prompt icons to alert the user of the risks. Furthermore, the instrument panel can be configured with different reminder methods based on the different severity levels of the warning messages sent by the camera, such as changing the color of the icon, changing the flashing frequency of the icon, and changing the icon's style, in order to convey the risks as accurately as possible.

[0094] Once the vehicle is started and powered on, the alarm will also start working. The alarm can receive warning messages sent by the camera, and after parsing the warning messages, it will indicate the collision risk through sound and light signals to remind the driver to pay attention to the collision risk.

[0095] Figure 3 This is a schematic diagram illustrating the interaction between the instrument cluster and camera controller assembly of a vehicle warning system provided in an embodiment of this application. Figure 3As shown, the driver can access the system settings via the instrument cluster's central control screen. In these settings, there are options to toggle the forward collision warning system and lane departure warning system, adjust the warning system sensitivity, select target priority, etc. When the driver clicks to disable the forward collision warning system, the instrument cluster's central control screen converts this control command into a CAN communication message and sends it to the camera controller assembly via the vehicle's CAN network. Upon receiving the control command, the camera controller assembly disables the forward collision warning function, and the system no longer sends forward collision warning signals. To reactivate the forward collision warning system, the driver can access the system settings again and click the forward collision warning system switch button again. This time, the instrument cluster's central control screen will again convert the reactivation command into a CAN communication message and send it to the camera controller assembly via the vehicle's CAN network. Upon receiving the control command, the camera controller assembly reactivates the forward collision warning function, and the system returns to normal. Similarly, the driver can also control the lane departure warning system to be turned on or off via the instrument cluster's central control screen.

[0096] The driver can adjust the system's warning sensitivity using the system sensitivity button on the instrument cluster's central control screen. The instrument cluster's sensitivity can be further subdivided into adjustments to the timing of risk warnings, and the detection and warning thresholds for small targets (such as animals, small stones, children, etc.) and static targets (such as guardrails, cones, trees, etc.). After the driver adjusts the sensitivity, the instrument cluster sends the sensitivity change request to the camera controller assembly via the CAN network. Upon receiving the information, the camera controller assembly modifies the information to reflect the corresponding external parameter changes, which in turn affects the operation of the internal recognition and functional logic algorithms, ensuring the system meets the driver's required warning sensitivity.

[0097] Figure 4 Flowchart of the vehicle early warning method provided in the embodiments of this application Figure 2 This method can be applied to the aforementioned vehicle warning system, and the method includes:

[0098] S401. Obtain the first road surface image of the road ahead of the vehicle;

[0099] Optionally, the first road surface image can be obtained through the following steps:

[0100] Step 1: Detect the light intensity of the environment in which the vehicle is located, and determine whether the light intensity is lower than the preset light intensity threshold;

[0101] If so, adjust the settings of the vehicle camera according to the light intensity of the environment in which the vehicle is located;

[0102] Control the camera to acquire the first road surface image.

[0103] It should be noted that the camera can be equipped with an infrared transceiver module, which can greatly increase the camera's working range when the vehicle is driving in a low-light environment. When the light intensity of the environment in which the vehicle is located is lower than a preset light intensity threshold, i.e., when the ambient light is weak, the camera's image contrast, exposure, brightness, and other settings can be adjusted appropriately to maximize the image quality perceived by the camera and facilitate basic target recognition.

[0104] Furthermore, the first road surface image can be preprocessed using the Tenengrad gradient algorithm described in Embodiment 1 to detect the feature contours of obstacles ahead for target recognition; this will not be elaborated upon in this embodiment. In addition, since changes in the vehicle's attitude can cause deviations in the target feature contours detected by the camera, the target feature contours must be adjusted based on the vehicle's speed and yaw rate to obtain accurate target information, thereby further improving the accuracy of target recognition.

[0105] Furthermore, since improper installation during actual camera installation may lead to angular deviations, the following measures are required: First, after the camera is installed on the vehicle during production, it must be precisely calibrated using a target. The vehicle undergoes target calibration at a standard workstation. Using the target's position as a reference, the camera performs reverse calculations on the identified target image to obtain the installation angle deviation in various directions. The camera can then write these angle deviations into its internal logic. During actual vehicle operation, the camera will calibrate the identified image based on the pre-stored angle deviations, ensuring that the target identified by the camera matches the actual target. Second, during vehicle operation, the camera will also continuously adjust based on fixed road markings, such as guardrails, curbs, and trees, to adjust the offset angle of the identified image. By adjusting the camera angle deviation, the offset angle for target identification can be adjusted, thereby ensuring more accurate target recognition.

[0106] S402. Using a deep learning model, obtain the target lane line, the first relative distance between the vehicle and the obstacle in front of the vehicle, and the first relative speed in the first road surface image;

[0107] In this step, model prediction algorithms can be combined to track and predict the trajectory of targets such as vehicles, pedestrians, and electric vehicles approaching from a distance, thereby accurately predicting the location of the target.

[0108] Furthermore, vehicles encounter various environments during operation, such as: crossing zebra crossings at intersections, temporarily lost lane markings at traffic light intersections, drastic changes in light intensity when passing through tunnels, intersections with complex lane markings, roads covered by rain or snow, and blurred or discontinuous lane markings. Therefore, lane marking recognition strategies need to be adjusted for different road conditions. Optionally, a deep learning model can be used to obtain the target lane markings from the first road surface image, including the following steps:

[0109] Step 1: Based on the first road surface image, identify the current road conditions of the road you are on;

[0110] Step 2: Determine the recognition strategy corresponding to the current road conditions;

[0111] Step 3: Adjust the first road surface image according to the recognition strategy to determine the third road surface image;

[0112] Step 4: Input the third road surface image into the recognition deep learning model, which will then identify the target lane line based on the third road surface image.

[0113] Specifically, in step two, determining the corresponding recognition strategy based on the current road conditions can include: When passing through a tunnel, the drastic change in light at the tunnel entrance and exit can easily reach the sensor's recognition limit, causing the entire image recognition system to briefly malfunction. However, if the tunnel entrance is detected in advance, and lane line position information is estimated when the vehicle arrives at the tunnel entrance, this can prevent the system from becoming temporarily unavailable. In another environment where roads are covered by rain or snow, the reflection of sunlight by rain or snow may affect the accurate recognition of lane lines. In this case, it is necessary to reduce the overall contrast of the image and minimize lane line misrecognition caused by reflections. When lane lines are blurred, the image recognition algorithm will first apply preprocessing techniques such as edge detection and color segmentation to separate the lane lines from the background. Subsequently, classical methods such as Hough transform or more advanced feature extraction algorithms, such as the Sobel operator and the Canny edge detector, are used to further accurately depict the outline of the lane lines.

[0114] S403. Calculate the second relative distance between the obstacle ahead and the vehicle based on the first width of the target lane line at the first position and the second width at the second position in the first road surface image;

[0115] Specifically, the first position is the location near the vehicle, and the second position is the location near the obstacle in front. The second relative distance can be calculated using the following steps:

[0116] Step 1: Detect the first number of pixels in the width direction of the target lane line at the first position, and the second number of pixels in the width direction of the target lane line at the second position.

[0117] Step 2: Calculate the second relative distance based on the difference between the first and second quantities.

[0118] Since the lane width is consistent on the same road, by detecting the number of pixels occupied by the lane width next to the obstacle (usually a vehicle) and the number of pixels occupied by the lane width near the vehicle, and by comparing the difference between the two pixels, i.e. the difference between the first number and the second number, and referring to a pre-calibrated distance and pixel reference table, the second relative distance between the obstacle and the vehicle can be calculated.

[0119] S404. Based on the first road surface image and the size of the bounding box of the obstacle in front in at least one frame of the second road surface image acquired before the first road surface image, calculate the second relative speed between the obstacle in front and the vehicle.

[0120] In this step, the bounding box of the obstacle ahead can be obtained using a regression method. The accurate relative speed between the obstacle and the vehicle is calculated by comparing the changes in the size of the regressed bounding box across multiple frames. Taking a target vehicle ahead as an example, after obtaining the regressed bounding box of the target vehicle, the left-right changes in the bounding box indicate that the target vehicle may intend to change lanes. By combining the left-right changes in the target vehicle's bounding box with the yaw rate detected by the vehicle's yaw rate sensor, it can be determined whether the vehicle is deviating left or right, and thus inferring whether the target vehicle actually intends to change lanes. If the target vehicle is changing from an adjacent lane to the current lane, it requires close monitoring and collision risk detection to prevent a collision. If the target vehicle is changing from the current lane to an adjacent lane, collision risk detection is not performed.

[0121] S405. Correct the first relative distance based on the second relative distance to obtain the third relative distance, and correct the first relative velocity based on the second relative velocity to obtain the third relative velocity;

[0122] Specifically, the first relative distance and the second relative distance can be input into the calibration deep learning model, and the calibration deep learning model can output the third relative distance. Similarly, the first relative velocity and the second relative velocity can be input into the calibration deep learning model, and the calibration deep learning model can output the third relative velocity.

[0123] S406. Based on the third relative distance and the third relative speed, determine whether there is a risk of collision between the obstacle ahead and the vehicle;

[0124] If yes, then execute S407; otherwise, execute S401.

[0125] S407, Output collision risk warning information;

[0126] Following step S401, the method further includes:

[0127] S408. Based on the vehicle's trajectory data within a future preset time range and the target lane line information, determine the actual lateral distance between the vehicle and the target lane line within the future preset time range.

[0128] The vehicle's trajectory data within a preset future time frame can be predicted based on the vehicle's detected lateral and longitudinal accelerations using the vehicle's IMU (Inertial Measurement Unit), or it can be predicted by combining satellite navigation positioning data and environmental feature matching positioning data. Optionally, based on the vehicle's trajectory data within a preset future time frame and the target lane line information, the actual lateral distance between the vehicle and the target lane line within the preset future time frame is determined, including the following steps:

[0129] Step 1: Based on the vehicle's trajectory data within a preset time range and the target lane information, determine the first lateral distance between the vehicle and the target lane.

[0130] Step 2: Based on the lateral acceleration signal and longitudinal acceleration signal transmitted by the vehicle, determine the lateral displacement value of the vehicle relative to the target lane line within a preset time range in the future;

[0131] Step 3: Correct the first lateral distance based on the lateral displacement value to determine the actual lateral distance.

[0132] Specifically, the lateral and longitudinal acceleration signals of the vehicle are actually emitted by the driver. When the driver operates the vehicle, these signals are sent to the CAN network. The camera controller assembly receives these signals and calculates the vehicle's lateral displacement relative to the target lane line within a preset future timeframe. Since these signals represent the driver's intentions, they further represent future changes in the vehicle's lateral and longitudinal acceleration, which inevitably lead to changes in the vehicle's lateral distance relative to the target lane line. The aforementioned trajectory data within the preset future timeframe is derived from the vehicle's current lateral and longitudinal acceleration. By incorporating changes in the vehicle's future lateral and longitudinal acceleration, the accuracy of the actual lateral distance between the vehicle and the target lane line within the preset future timeframe can be improved.

[0133] S409. Based on the actual lateral distance, determine whether the vehicle is at risk of lane departure;

[0134] If yes, then execute S410; otherwise, execute S401.

[0135] S411, output lane departure warning information.

[0136] In addition, cameras can detect the distance between the vehicle and the left and right lane lines. If the difference between the distance between the vehicle and the left and right lane lines is too large, a lane departure warning message can be issued to the driver.

[0137] Figure 5 A schematic diagram of the vehicle warning device provided in this application is shown below. Figure 5 As shown, the device 50 includes:

[0138] The visual detection module 501 is used to: acquire a first road surface image of the road ahead of the vehicle;

[0139] The recognition module 502 is used to: use a deep learning model to obtain the target lane line, the first relative distance between the obstacle in front of the vehicle and the vehicle, and the first relative speed in the first road surface image;

[0140] The calculation module 503 is used to: calculate the second relative distance between the obstacle ahead and the vehicle based on the first width of the target lane line in the first road surface image at the first position and the second width at the second position; the first position is the position near the vehicle, and the second position is the position near the obstacle ahead;

[0141] The second relative speed between the obstacle and the vehicle is calculated based on the first road surface image and the size of the bounding box of the obstacle in at least one frame of the second road surface image acquired before the first road surface image.

[0142] The correction module 504 is used to: correct the first relative distance according to the second relative distance to obtain a third relative distance, and correct the first relative velocity according to the second relative velocity to obtain a third relative velocity;

[0143] Collision risk warning module 505: Based on the third relative distance and the third relative speed, it determines whether there is a risk of collision between the vehicle and the obstacle ahead;

[0144] If so, output a collision risk warning message.

[0145] The vehicle warning device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0146] Figure 6 A schematic diagram of the structure of the electronic device provided in this application. Figure 6As shown, the electronic device 60 provided in this embodiment includes at least one processor 601 and a memory 602. Optionally, the device 60 further includes a communication component 603. The processor 601, memory 602, and communication component 603 are connected via a bus 604.

[0147] In a specific implementation, at least one processor 601 executes computer execution instructions stored in memory 602, causing at least one processor 601 to perform the above-described method.

[0148] The specific implementation process of processor 601 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0149] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0150] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0151] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0152] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0153] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0154] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0155] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0156] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0157] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0158] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0159] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0160] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0161] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A vehicle collision warning method, characterized in that, The method includes: Acquire the first road surface image of the road ahead of the vehicle; Using a deep learning model, the target lane line in the first road surface image, the first relative distance between the obstacle in front of the vehicle and the vehicle, and the first relative speed are obtained; The step of using a deep learning model to obtain the target lane lines in the first road surface image includes: Based on the first road surface image, identify the current road conditions of the road in question; Based on the current road conditions, determine the recognition strategy corresponding to the current road conditions; According to the recognition strategy, the first road surface image is adjusted to determine the third road surface image; The third road surface image is input into a recognition deep learning model, which then identifies the target lane line based on the third road surface image. Based on the first width of the target lane line at the first position and the second width at the second position in the first road surface image, the second relative distance between the obstacle ahead and the vehicle is calculated; the first position is the position near the vehicle, and the second position is the position near the obstacle ahead; Based on the first road surface image and the size of the bounding box of the obstacle in at least one frame of the second road surface image acquired before the first road surface image, the second relative speed between the obstacle and the vehicle is calculated. The first relative distance is corrected based on the second relative distance to obtain a third relative distance, and the first relative velocity is corrected based on the second relative velocity to obtain a third relative velocity; Based on the third relative distance and the third relative speed, determine whether there is a risk of collision between the obstacle ahead and the vehicle; If so, output a collision risk warning message.

2. The method according to claim 1, characterized in that, The step of determining the first width of the target lane line at a first position and the second width at a second position in the first road surface image includes: Detect a first number of pixels in the width direction of the target lane line at the first position, and a second number of pixels in the width direction of the target lane line at the second position; The second relative distance is calculated based on the difference between the first quantity and the second quantity.

3. The method according to claim 1 or 2, characterized in that, The step of correcting the first relative distance based on the second relative distance to obtain a third relative distance, and correcting the first relative velocity based on the second relative velocity to obtain a third relative velocity, includes: The first relative distance and the second relative distance are input into the calibration deep learning model, and the calibration deep learning model outputs the third relative distance. The first relative velocity and the second relative velocity are input into the correction deep learning model, and the correction deep learning model outputs the third relative velocity.

4. The method according to claim 1, characterized in that, After obtaining the target lane line in the first road surface image using a deep learning model, the method further includes: Based on the vehicle's trajectory data within a future preset time range and the target lane line information, the actual lateral distance between the vehicle and the target lane line within the future preset time range is determined. Based on the actual lateral distance, determine whether the vehicle is at risk of lane departure; If so, then output a lane departure warning message.

5. The method according to claim 4, characterized in that, Determining the actual lateral distance between the vehicle and the target lane line within the future preset time range, based on the vehicle's trajectory data within that range and the target lane line information, includes: Based on the vehicle's trajectory data within a preset time range and the target lane information, a first lateral distance between the vehicle and the target lane is determined. Based on the lateral acceleration signal and longitudinal acceleration signal of the vehicle sent by the vehicle, the lateral displacement value of the vehicle relative to the target lane line within the future preset time range is determined; The first lateral distance is corrected based on the lateral displacement value to determine the actual lateral distance.

6. The method according to claim 1, characterized in that, The acquisition of the first road surface image ahead of the vehicle includes: The light intensity of the environment in which the vehicle is located is detected, and it is determined whether the light intensity is lower than a preset light intensity threshold. If so, the settings parameters of the vehicle camera are adjusted according to the light intensity of the environment in which the vehicle is located. Control the camera to acquire the first road surface image.

7. A vehicle warning device, characterized in that, The device includes: The visual detection module is used to: acquire the first road surface image of the road ahead of the vehicle; The recognition module is used to: use a deep learning model to obtain the target lane line in the first road surface image, the first relative distance between the obstacle in front of the vehicle and the vehicle, and the first relative speed; The step of using a deep learning model to obtain the target lane lines in the first road surface image includes: Based on the first road surface image, identify the current road conditions of the road in question; Based on the current road conditions, determine the recognition strategy corresponding to the current road conditions; According to the recognition strategy, the first road surface image is adjusted to determine the third road surface image; The third road surface image is input into a recognition deep learning model, which then identifies the target lane line based on the third road surface image. The calculation module is configured to: calculate a second relative distance between the obstacle ahead and the vehicle based on a first width of the target lane line at a first position and a second width at a second position in the first road surface image; the first position is a position near the vehicle, and the second position is a position near the obstacle ahead; Based on the first road surface image and the size of the bounding box of the obstacle in at least one frame of the second road surface image acquired before the first road surface image, the second relative speed between the obstacle and the vehicle is calculated. The correction module is configured to: correct the first relative distance based on the second relative distance to obtain a third relative distance, and correct the first relative velocity based on the second relative velocity to obtain a third relative velocity; Collision risk warning module: Based on the third relative distance and the third relative speed, determine whether there is a risk of collision between the obstacle ahead and the vehicle; If so, output a collision risk warning message.

8. An electronic device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-6.

Images