Vehicle early warning method, device and equipment and storage medium

Abstract

The embodiment of the invention provides a vehicle early warning method and device, equipment and a storage medium. The method comprises the following steps: acquiring a pavement image of a road ahead of a vehicle; obtaining a target lane line in the road surface image, and a relative distance and a relative speed between an obstacle in front of the vehicle and the vehicle by using a deep learning model; correcting the relative distance according to the width of the target lane line near the vehicle and the width of the target lane line near the front obstacle in the pavement image; correcting the relative distance according to the size of a front obstacle surrounding frame in the multi-frame road surface image; according to the corrected relative distance and relative speed, whether the front obstacle has a collision risk with the vehicle or not is judged; and if yes, outputting collision risk warning information. According to the method, only the camera needs to be installed on the vehicle, the number of needed sensors is small, the system cost is low, maintenance is easy, and meanwhile the collision risk prediction precision is high.

Description

Technical Field

[0001] The present application relates to the field of assisted driving technology, and in particular to a vehicle early warning method, device, equipment and storage medium. Background Art

[0002] Due to the complex working environment of heavy vehicles and the changeable vehicle operating conditions, as well as the heavy mass of heavy vehicles, emergency deceleration is difficult and there is a great risk of overturning. Therefore, if heavy vehicles encounter the risk of emergency collision, they often cause great harm and losses to the driver and other road users.

[0003] In order to achieve vehicle forward collision warning, the current mainstream solution is to integrate the data of two sensors, camera and millimeter wave radar, to accurately monitor the lane and target ahead. This solution requires a large number of sensors, the system cost is high and maintenance is difficult. Summary of the invention

[0004] The embodiments of the present application provide a vehicle warning method, device, equipment and storage medium to solve the problems of existing vehicle warning methods that integrate the data of two sensors, a camera and a millimeter-wave radar, require a large number of sensors, have high system costs, and are difficult to maintain.

[0005] In a first aspect, the present application provides a vehicle collision warning method, the method comprising:

[0006] Acquire a first road surface image of a road ahead of the vehicle;

[0007] Using a deep learning model, obtaining a target lane line in the first road surface image, a first relative distance between the obstacle in front of the vehicle and the vehicle, and a first relative speed;

[0008] Calculating a second relative distance between the front obstacle and the vehicle according to a first width of the target lane line at a first position and a second width of the target lane line at a second position in the first road surface image; the first position is a position near the vehicle, and the second position is a position near the front obstacle;

[0009] Calculating a second relative speed between the front obstacle and the vehicle according to the first road surface image and a size of a bounding box of the front obstacle in at least one frame of a second road surface image acquired before the first road surface image;

[0010] Correcting the first relative distance according to the second relative distance to obtain a third relative distance, and correcting the first relative speed according to the second relative speed to obtain a third relative speed;

[0011] determining, according to the third relative distance and the third relative speed, whether there is a collision risk between the front obstacle and the vehicle;

[0012] If yes, a collision risk warning message is output.

[0013] In a possible design, the first width of the target lane line at the first position and the second width of the target lane line at the second position in the first road surface image include:

[0014] Detecting a first number of pixels in a width direction of the target lane line at the first position, and a second number of pixels in a width direction of the target lane line at the second position;

[0015] The second relative distance is calculated according to a difference between the first number and the second number.

[0016] In a possible design, the correcting the first relative distance according to the second relative distance to obtain a third relative distance, and the correcting the first relative speed according to the second relative speed to obtain a third relative speed, include:

[0017] Inputting the first relative distance and the second relative distance into a correction deep learning model, and outputting the third relative distance by the correction deep learning model;

[0018] The first relative speed and the second relative speed are input into the corrected deep learning model, and the corrected deep learning model outputs the third relative speed.

[0019] In a possible design, the using a deep learning model to obtain a target lane line in the first road surface image includes:

[0020] Identifying the current road condition of the road according to the first road surface image;

[0021] Determining, according to the current road condition, an identification strategy corresponding to the current road condition;

[0022] According to the recognition strategy, adjusting the first road surface image to determine a third road surface image;

[0023] The third road surface image is input into a recognition deep learning model, and the recognition deep learning model recognizes the target lane line according to the third road surface image.

[0024] In a possible design, after acquiring the target lane line in the first road surface image by using the deep learning model, the method further includes:

[0025] Determine the actual lateral distance between the vehicle and the target lane line within the future preset time range according to the motion trajectory data of the vehicle within the future preset time range and the target lane line information;

[0026] determining, according to the actual lateral distance, whether the vehicle has a lane departure risk;

[0027] If so, a lane departure warning message is output.

[0028] In a possible design, determining the actual lateral distance between the vehicle and the target lane line within the future preset time range according to the motion trajectory data of the vehicle within the future preset time range and the target lane line information includes:

[0029] Determine a first lateral distance between the vehicle and the target lane line according to the motion trajectory data of the vehicle within a preset time range in the future and the target lane line information;

[0030] Determine, according to the lateral acceleration signal and the longitudinal acceleration signal of the vehicle sent by the vehicle, a lateral displacement value of the vehicle relative to the target lane line within the future preset time range;

[0031] The first lateral distance is corrected according to the lateral displacement value to determine the actual lateral distance.

[0032] In a possible design, the step of acquiring a first road surface image of a road ahead of the vehicle includes:

[0033] Detecting the light intensity of the environment in which the vehicle is located, and determining whether the light intensity is lower than a preset light intensity threshold;

[0034] If yes, adjusting the setting parameters of the vehicle camera according to the light intensity of the environment in which the vehicle is located;

[0035] The camera is controlled to acquire the first road surface image.

[0036] In a second aspect, the present application provides a vehicle warning device, the device comprising:

[0037] A visual detection module is used to: obtain a first road surface image of a road ahead of the vehicle;

[0038] A recognition module, configured to: use a deep learning model to obtain a target lane line in the first road surface image, a first relative distance between the obstacle in front of the vehicle and the vehicle, and a first relative speed;

[0039] a calculation module, configured to calculate a second relative distance between the front obstacle and the vehicle according to a first width of the target lane line at a first position and a second width of the target lane line at a second position in the first road surface image; the first position is a position near the vehicle, and the second position is a position near the front obstacle;

[0040] Calculating a second relative speed between the front obstacle and the vehicle according to the first road surface image and a size of a bounding box of the front obstacle in at least one frame of a second road surface image acquired before the first road surface image;

[0041] a correction module, configured to: correct the first relative distance according to the second relative distance to obtain a third relative distance, and correct the first relative speed according to the second relative speed to obtain a third relative speed;

[0042] A collision risk warning module: judging whether there is a collision risk between the front obstacle and the vehicle according to the third relative distance and the third relative speed;

[0043] If yes, a collision risk warning message is output.

[0044] In a third aspect, the present application provides an electronic device, including: a memory, a processor;

[0045] The memory stores computer-executable instructions;

[0046] The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to the first aspect.

[0047] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method described in the first aspect.

[0048] The vehicle warning method, device, equipment and storage medium provided by the embodiment of the present application complete the collision risk warning only through the image captured by the camera, only need to install the camera on the vehicle, the number of sensors required is small, the system cost is low, and the maintenance is easy. Since only image data is used and data from other sensors is lacking, the method corrects the relative distance between the front obstacle and the vehicle identified by the deep learning model through the width of the lane line in the road image, and corrects the relative speed between the front obstacle and the vehicle identified by the deep learning model through the size change of the front obstacle bounding box in multiple frames of images, thereby improving the accuracy of the relative distance and relative speed recognition between the front obstacle and the vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0050] Figure 1 Schematic diagram of the vehicle warning method process provided in the embodiment of the present application Figure 1 ;

[0051] Figure 2 A schematic diagram of a vehicle warning system provided in an embodiment of the present application;

[0052] Figure 3 A schematic diagram of the interaction between the vehicle warning system instrument and the camera controller assembly provided in an embodiment of the present application;

[0053] Figure 4 Schematic diagram of the vehicle warning method process provided in the embodiment of the present application Figure 2 ;

[0054] Figure 5 A schematic diagram of the structure of the vehicle warning device provided for this application;

[0055] Figure 6 A schematic diagram of the structure of the electronic device provided in this application.

[0056] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0057] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0058] First, the terms involved in this application are explained:

[0059] Tenengrad gradient algorithm: The Tenengrad gradient algorithm is a method for image clarity assessment. It judges the clarity of an image based on the gradient information of the image. Specifically, the algorithm calculates the gradient of each pixel in the image (usually using the Sobel operator), and then evaluates the clarity of the image by counting the magnitude of these gradients. The larger the gradient, the more details the image has and the higher the clarity.

[0060] Bounding box: Bounding box is a commonly used concept in computer vision, which refers to the use of a rectangular box to mark the location of the target object in the image. The bounding box is usually defined by four parameters: the coordinates of the upper left corner (x, y) and the width and height (width, height). In the object detection task, the bounding box is used to identify and locate the target object in the image.

[0061] Regression box: The regression box refers to the bounding box predicted by the regression algorithm. In the target detection task, the regression box is used to accurately locate the target object. The regression model predicts the bounding box parameters of the target object (such as the center point coordinates, width, and height) based on the input image features, thereby obtaining a more accurate target location.

[0062] CAN network: The vehicle's CAN network (Controller Area Network) is a standardized network protocol used for communication between electronic control units (ECUs) inside the car.

[0063] Model prediction algorithm: A model prediction algorithm is the process of using a trained machine learning or deep learning model to make predictions about new data. The specific algorithm depends on the type of model and the application scenario. For example, in a classification task, the model prediction algorithm outputs a class label based on the input data; in a regression task, the model prediction algorithm outputs a continuous value. In an object detection task, the model prediction algorithm outputs the class and location (bounding box) of the target object.

[0064] The existing vehicle warning method integrates the data of the two sensors, camera and millimeter wave radar, which requires a large number of sensors, has a high system cost and is difficult to maintain. With the rapid improvement of camera perception technology, the lane and the front target situation can be more accurately judged by relying on the perception data of the camera. At this time, the fusion of radar data adds little benefit to the system. In view of the problems of the prior art, the technical concept of this application is to obtain the first road image data of the road where the vehicle is currently located, input the first road image into the deep learning model, identify the lane line of the road, and the relative distance and relative speed between the obstacle in front of the vehicle and the vehicle. Then, according to the width of the lane line near the vehicle in the first road image and the width of the lane line near the obstacle in front, the relative distance identified by the deep learning model is corrected, and according to the change in the size of the bounding box of the obstacle in front of the multiple frames of the first road image, the relative speed identified by the deep learning model is corrected. According to the relative speed and relative distance of the obstacle in front of the vehicle after correction, if it is determined that there is a risk of collision between the vehicle and the obstacle in front, a collision risk prompt is issued to the vehicle driver.

[0065] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0066] Embodiment 1

[0067] Figure 1 Schematic diagram of the vehicle warning method process provided in the embodiment of the present application Figure 1 ,This method can be applied to vehicle warning systems, such as Figure 1 As shown, the method includes:

[0068] S101, acquiring a first road surface image of a road ahead of a vehicle;

[0069] Specifically, this step controls the camera installed in front of the vehicle to obtain a first road surface image. In the preprocessing stage of the first road surface image, the Tenengrad gradient algorithm can be used to calculate the image change gradients in the horizontal and vertical directions respectively, and the edge of the image can be detected and the clarity can be evaluated according to different gradient values, so that more accurate target features can be obtained, and the target can be identified more accurately and quickly.

[0070] The preprocessed first road surface image is input into the deep learning model for recognition.

[0071] S102, using a deep learning model to obtain a target lane line in a first road surface image, a first relative distance between an obstacle in front of the vehicle and the vehicle, and a first relative speed;

[0072] Since this method only uses the image information collected by the camera and lacks information of other dimensions as auxiliary judgment, deep learning models including convolutional neural networks and recurrent neural networks can be used. Convolutional neural networks (CNNs) are used to perform image feature extraction tasks, while identifying the motion posture of the front target (vehicles, pedestrians, etc.) and analyzing its posture. Recurrent neural networks (RNNs) are used to consider various information of previous targets when processing each target in the time series, and to use previous target information when processing subsequent target inputs in the time series, so as to efficiently predict the accurate type, position and posture change of the target. Specifically, the image frame of the preprocessed first road image is input into the convolutional neural network, and the convolutional neural network outputs a feature vector; the feature vector sequence from the convolutional neural network is input into the recurrent neural network, and the target lane line, the first relative distance between the obstacle in front of the vehicle and the vehicle and the first relative speed are output. By combining convolutional neural networks and recurrent neural networks, more accurate types, speeds, and distances of obstacles in front can be obtained.

[0073] S103, calculating a second relative distance between the front obstacle and the vehicle according to a first width of the target lane line at the first position and a second width of the target lane line at the second position in the first road surface image;

[0074] Specifically, the first position is a position near the vehicle, and the second position is a position near a front obstacle.

[0075] Since the lane widths on the same road are consistent, in the first road image captured by the vehicle camera, the vehicle is at a distance from the obstacle ahead, so there must be a difference between the lane widths near the vehicle and the lane widths near the obstacle ahead. By detecting this difference and performing corresponding calculations, the second relative distance between the vehicle and the obstacle ahead can be calculated.

[0076] S104, calculating a second relative speed between the front obstacle and the vehicle according to the first road surface image and the size of the bounding box of the front obstacle in at least one frame of the second road surface image collected before the first road surface image;

[0077] Specifically, by comparing the size changes of the bounding boxes of the front obstacle in multiple frames of images, the second relative speed between the front obstacle and the vehicle can be calculated.

[0078] S105, correcting the first relative distance according to the second relative distance to obtain a third relative distance, and correcting the first relative speed according to the second relative speed to obtain a third relative speed;

[0079] Since the first relative distance and the first relative speed identified by the deep learning model alone may have errors, in this step, data fusion technology can be used to weight the first relative distance and the second relative distance to obtain a more accurate third relative distance. Similarly, the first relative speed and the second relative speed are weighted averaged to obtain the third relative speed. Alternatively, a Kalman filter can be used to correct the distance and speed.

[0080] S106. Determine whether there is a collision risk between the front obstacle and the vehicle according to the third relative distance and the third relative speed;

[0081] In this step, the third relative distance and the third relative speed may be used to calculate the collision time between the vehicle and the obstacle ahead, and whether there is a collision risk may be determined based on the collision time.

[0082] If yes, execute S107; if no, execute S101;

[0083] S107: Output collision risk warning information.

[0084] Specifically, the collision risk warning information output by the alarm installed in the vehicle cab may be sound information, text information, image information, etc.

[0085] The technical effect of this embodiment is that the method completes the collision risk warning only through the images taken by the camera, only needs to install the camera on the vehicle, requires a small number of sensors, has a low system cost, and is easy to maintain. Since only image data is used and data from other sensors is missing, the method corrects the relative distance between the front obstacle and the vehicle identified by the deep learning model through the width of the lane line in the road image, and corrects the relative speed between the front obstacle and the vehicle identified by the deep learning model through the size change of the front obstacle bounding box in multiple frames of images, thereby improving the accuracy of the relative distance and relative speed recognition between the front obstacle and the vehicle.

[0086] The solution of the present application is described below with a specific embodiment.

[0087] Embodiment 2

[0088] Figure 2 A schematic diagram of a vehicle warning system provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the system includes: a camera controller assembly, a central control screen of the instrument, and an alarm in the cab. The camera controller assembly integrates a camera sensor, an image processor, and a function controller, which can achieve the purpose of high-precision image acquisition and high-speed processing to extract target information. At the same time, the function logic of the function controller is used to issue accurate and timely warning information for risk situations. The camera controller assembly should be installed in the center of the windshield in the cab, close to the driving platform. This position does not block the driver's line of sight. At the same time, the field of view is large and symmetrical, which is convenient for the camera to monitor and identify targets; the camera controller assembly is connected to the vehicle through the CAN network and interacts with the vehicle. For example, the camera obtains information such as the speed, steering, gear position, etc. of the vehicle to assist in inferring the state of the target and the conditions for judging the function of the warning. At the same time, the camera can also transmit the warning signal to the central control screen of the instrument and the alarm in the cab through the CAN network to remind the driver to pay attention to the risk of collision or lane departure.

[0089] The vehicle instrument panel is connected to the camera controller assembly through the in-vehicle CAN network to display warning information. At the same time, it can accurately transmit the driver's operating intentions (such as switching the system on and off, setting sensitivity, etc.) to the camera controller assembly, improving the human-machine interaction flexibility of the system.

[0090] The in-cab alarm is installed on the left and right sides of the cab respectively, and communicates with the camera controller assembly through the in-vehicle CAN network. It is used to receive early warning messages and convert them into sound and light signals to remind the driver of risks.

[0091] When the vehicle is ignited and powered on, the camera controller assembly will activate the target monitoring capability. At the same time, the camera will activate the corresponding warning function after the vehicle speed reaches the set value. If there is a risk of collision ahead at this time, the camera will report the warning message to the instrument center control screen and alarm through the CAN network to remind the driver to pay attention to the risk of collision ahead; if the vehicle deviates from the left or right lane lines at this time, the camera will monitor the changes in the lane line position and determine the lane line that the vehicle has deviated from at this time. At this time, the camera will also report the lane departure warning signal to the instrument center control screen and alarm through the CAN network to remind the driver that the vehicle is currently deviating from the main lane and there is a risk of collision with vehicles in the adjacent lane. Specifically, after the camera controller assembly recognizes the road condition information ahead, it will integrate the road condition information into the virtual vehicle coordinate system based on the vehicle body information (front overhang, wheelbase, vehicle width, tire distance, etc.) and the installation location information of the camera controller assembly, thereby calculating the relative position relationship between the target ahead and the vehicle. At the same time, combined with the vehicle information such as speed, steering, gear position, etc. obtained from the vehicle CAN network, the built-in dual warning function algorithm is used to calculate the possible forward collision risk and lane departure risk. For example, the vehicle is judged to have no collision risk when turning or reversing. After judging that the risk reaches the set threshold, the camera controller assembly will send a risk warning signal through a CAN message.

[0092] When the vehicle is powered on, the instrument panel will also be activated and perform a self-check. If the instrument panel does not detect the message sent by the camera at this time, it is judged that there is a fault in the camera system, and the instrument panel will display the corresponding warning symbol to remind the driver that the lane departure warning function or forward collision warning function is not available; if the instrument panel detects the message sent by the camera, but the message indicates a fault, the instrument panel will display the corresponding warning symbol to remind the driver that the lane departure warning function or forward collision warning function is not available.

[0093] The instrument panel can also be used to set the forward collision warning system switch and lane departure warning system switch for the driver to manually shut down or restart the system. When the driver manually shuts down the system, the instrument panel can convert the corresponding signal into a corresponding message and send it to the camera controller assembly through the CAN network. After the camera controller assembly receives the switch control message sent by the instrument, it can turn off or turn on the lane departure warning function or forward collision warning function accordingly. At the same time, the lane departure warning function or forward collision warning function can allow the driver to manually adjust the warning sensitivity or warning range on the basis of meeting regulatory requirements, so as to meet the personalized needs of different users; the corresponding warning sensitivity level (low, medium, high), priority warning target (pedestrians, bicycles, cars, trucks, etc.) can be set through the instrument panel. The instrument sends the user's needs to the camera controller assembly through a specially configured message. After receiving these external parameter configuration information, the camera controller assembly adjusts the relevant priority parameters of the strategy to meet the user's needs. In addition, after receiving the early warning message sent by the camera, the central control screen of the instrument will parse and generate corresponding prompt icons to remind the user of the existing risks. Furthermore, the instrument can configure different reminder methods according to the early warning messages of different severity levels sent by the camera, such as changing the color of the icon, changing the flashing frequency of the icon, changing the icon, etc., to achieve the purpose of conveying risks as accurately as possible.

[0094] When the vehicle is ignited and powered on, the alarm also starts working. The alarm can receive the warning message sent by the camera, parse the warning message, and indicate the collision risk through sound and light signals to remind the driver to pay attention to the collision risk.

[0095] Figure 3 This is a schematic diagram of the interaction between the vehicle warning system instrument and the camera controller assembly provided in the embodiment of the present application. Figure 3As shown, the driver can call up the system settings through the instrument panel, in which the switch of the forward collision warning system, the switch of the lane departure warning system, the warning system sensitivity adjustment menu, the target priority selection, etc. are set; if the driver clicks to turn off the forward collision warning system, the instrument panel will convert this control command into a CAN communication message, and send it to the camera controller assembly through the vehicle CAN network. After the camera controller assembly receives the control command, it turns off the forward collision warning function, and the system no longer sends a forward collision warning signal; when the driver wants to reactivate the forward collision warning system, he can call up the system settings again and click the forward collision warning system switch button again. At this time, the instrument panel will convert the reactivation command into a CAN communication message again, and send it to the camera controller assembly through the vehicle CAN network. After the camera controller assembly receives the control command, it turns on the forward collision warning function again, and the system returns to normal again. Similarly, the driver can also control the lane departure warning system to be turned off or on through the instrument panel.

[0096] The driver can adjust the system's warning sensitivity through the system sensitivity button on the instrument panel. The sensitivity of the instrument panel can be further subdivided into the timing adjustment of risk warnings, the system's detection and warning thresholds for small targets (such as animals, small stones, children, etc.) and static targets (such as guardrails, cones, trees, etc.). When the driver adjusts the corresponding sensitivity, the instrument panel sends the sensitivity change request to the camera controller assembly through the CAN network. After receiving the information, the camera controller assembly will change the information to the corresponding external parameter change value, which can then affect the operation of the camera controller assembly's internal recognition algorithm and functional logic algorithm, so that the system can meet the driver's requirements for warning sensitivity.

[0097] Figure 4 Schematic diagram of the vehicle warning method process provided in the embodiment of the present application Figure 2 , the method can be applied to the above-mentioned vehicle warning system, the method comprising:

[0098] S401, acquiring a first road surface image of a road ahead of the vehicle;

[0099] Optionally, the first road surface image may be acquired by the following steps:

[0100] Step 1: Detect the light intensity of the environment in which the vehicle is located, and determine whether the light intensity is lower than a preset light intensity threshold;

[0101] If so, adjusting the setting parameters of the vehicle camera according to the light intensity of the vehicle's environment;

[0102] The camera is controlled to acquire a first road surface image.

[0103] It should be noted that the camera can be equipped with an infrared transceiver module, which can greatly improve the camera's light working range when the vehicle is driving in a dark environment. When the light intensity of the vehicle's environment is detected to be lower than the preset light intensity threshold, that is, when the ambient light is weak, the camera's image contrast, exposure, brightness and other setting parameters can be appropriately adjusted to maximize the image quality perceived by the camera and facilitate basic target recognition.

[0104] Furthermore, the first road surface image can be preprocessed in combination with the Tenengrad gradient algorithm in the first embodiment to detect the characteristic contour of the obstacle in front of the image for target recognition, which is not described in detail in this embodiment. In addition, since the change of the vehicle posture will cause the characteristic contour of the target recognized by the camera to deviate, it is necessary to adjust the characteristic contour of the target in combination with the vehicle speed and yaw rate to obtain accurate target information, thereby further improving the accuracy of target recognition.

[0105] Furthermore, since the camera may have an angle deviation caused by improper installation in actual installation, it is first required that the vehicle must use a target for accurate calibration after installing the camera during the production process. The vehicle performs target calibration of the camera at a standard workstation. The camera uses the position of the target as a reference and reversely calculates the recognized target image to obtain the installation angle deviation of the camera in all directions. The camera can then write these angle deviations into the internal logic, that is, when the vehicle is actually running, the camera will calibrate the recognized image according to the pre-stored angle deviation to ensure that the target recognized by the camera is consistent with the actual one. Secondly, during the operation of the vehicle, the camera will also be adjusted according to the fixed signs on the road at all times. For example, the offset angle of the recognized image can be reversely adjusted by recognizing targets such as guardrails, curbs, trees, etc. By adjusting the camera angle deviation, the offset angle of target recognition can be adjusted, thereby ensuring more accurate target recognition.

[0106] S402, using a deep learning model to obtain a target lane line in a first road surface image, a first relative distance between an obstacle in front of the vehicle and the vehicle, and a first relative speed;

[0107] In this step, the model prediction algorithm can be combined to track and predict the trajectory of targets such as vehicles, pedestrians, electric vehicles, etc. approaching from a distance, so that the location of the target can be accurately predicted.

[0108] In addition, during operation, the vehicle will pass through a variety of environments, such as: passing through zebra crossings at intersections, temporary loss of lane lines at traffic light intersections, drastic changes in light and darkness when passing through tunnels, passing through intersections with complex road markings, passing through roads covered with rain and snow, and blurred or intermittent lane lines. Therefore, it is necessary to adjust the lane line recognition strategy according to different road conditions. Optionally, using a deep learning model to obtain the target lane line in the first road surface image includes the following steps:

[0109] Step 1: Identify the current road condition of the road according to the first road surface image;

[0110] Step 2: Determine the recognition strategy corresponding to the current road condition according to the current road condition;

[0111] Step 3: According to the recognition strategy, the first road surface image is adjusted to determine a third road surface image;

[0112] Step 4: Input the third road surface image into the recognition deep learning model, and the recognition deep learning model recognizes the target lane line based on the third road surface image.

[0113] Specifically, in step 2, according to the current road conditions, the recognition strategy corresponding to the current road conditions is determined, which may include: when passing through a tunnel, the light will change dramatically when entering and exiting the tunnel, and it is very easy to reach the recognition limit of the sensor, causing the entire image recognition system to temporarily shut down. However, if the position of the tunnel entrance is detected in advance when entering or exiting the tunnel, the lane line position information is estimated in advance when the vehicle arrives at the tunnel entrance, thereby avoiding the system from being temporarily unavailable; in another environment where the road is covered with rain or snow, the accurate recognition of the lane line may be affected due to the reflection of sunlight by rain or snow. At this time, it is necessary to reduce the overall contrast of the image and reduce the problems of misidentification of lane lines caused by reflection; when the lane line is blurred, the image recognition algorithm will first apply preprocessing techniques such as edge detection and color segmentation to separate the lane line in the image from the background. Subsequently, the outline of the lane line is further accurately depicted through classic methods such as Hough transform or more advanced feature extraction algorithms such as Sobel operator and Canny edge detector.

[0114] S403, calculating a second relative distance between the front obstacle and the vehicle according to a first width of the target lane line at the first position and a second width of the target lane line at the second position in the first road surface image;

[0115] Specifically, the first position is a position near the vehicle, and the second position is a position near the front obstacle. The second relative distance can be calculated by the following steps:

[0116] Step 1: Detect a first number of pixels in a width direction of a target lane line at a first position, and a second number of pixels in a width direction of a target lane line at a second position;

[0117] Step 2: Calculate the second relative distance according to the difference between the first quantity and the second quantity.

[0118] Since the lane line width is consistent on the same road, by detecting the pixel points occupied by the lane line width next to the obstacle in front (usually a vehicle) and the pixel points occupied by the lane line width near the vehicle, by comparing the difference between the two pixel points, that is, the difference between the first number and the second number, and comparing with the pre-calibrated distance and pixel point comparison table, the second relative distance between the obstacle in front and the vehicle can be calculated.

[0119] S404, calculating a second relative speed between the front obstacle and the vehicle according to the first road surface image and the size of the bounding box of the front obstacle in at least one frame of the second road surface image collected before the first road surface image;

[0120] In this step, the bounding box of the obstacle in front can be obtained by the regression method, and the accurate relative speed between the obstacle in front and the vehicle can be calculated by comparing the size change of the regression box in multiple frames. Taking the target vehicle in front of the vehicle as an example, after obtaining the regression box of the target vehicle in front, the left and right changes of the regression box of the target vehicle in front indicate that the target vehicle in front may have the intention to change lanes. It can be combined with the left and right changes of the regression box of the target vehicle in front and the yaw rate of the vehicle detected by the yaw rate sensor of the vehicle to determine whether the vehicle is offset to the left and right, and then infer whether the target vehicle in front really has the intention to change lanes. If the target vehicle in front changes from the adjacent lane to the lane, it is necessary to focus on and perform collision risk detection to avoid collision between the vehicle and the vehicle in front. If the target vehicle in front changes from the lane to the adjacent lane, no collision risk detection is performed.

[0121] S405, correcting the first relative distance according to the second relative distance to obtain a third relative distance, and correcting the first relative speed according to the second relative speed to obtain a third relative speed;

[0122] Specifically, the first relative distance and the second relative distance can be input into the correction deep learning model, and the third relative distance can be output by the correction deep learning model. Also, the first relative speed and the second relative speed can be input into the correction deep learning model, and the third relative speed can be output by the correction deep learning model.

[0123] S406: judging whether there is a collision risk between the front obstacle and the vehicle according to the third relative distance and the third relative speed;

[0124] If yes, execute S407; if no, execute S401.

[0125] S407, outputting collision risk warning information;

[0126] After step S401, the method further includes:

[0127] S408, determining the actual lateral distance between the vehicle and the target lane line within the future preset time range according to the motion trajectory data of the vehicle within the future preset time range and the target lane line information;

[0128] The motion trajectory data of the vehicle within the preset time range in the future can be predicted based on the lateral and longitudinal accelerations of the vehicle detected by the IMU inertial measurement unit mounted on the vehicle, or can be predicted based on satellite navigation positioning data and environmental feature matching positioning data. Optionally, the actual lateral distance between the vehicle and the target lane line within the preset time range in the future is determined based on the motion trajectory data of the vehicle within the preset time range in the future and the target lane line information, including the following steps:

[0129] Step 1: Determine a first lateral distance between the vehicle and the target lane line based on the vehicle's future motion trajectory data within a preset time range and the target lane line information;

[0130] Step 2: determining the lateral displacement value of the vehicle relative to the target lane line within a preset time range in the future according to the lateral acceleration signal and the longitudinal acceleration signal of the vehicle sent by the vehicle;

[0131] Step three: correct the first lateral distance according to the lateral displacement value to determine the actual lateral distance.

[0132] Specifically, the lateral acceleration signal and longitudinal acceleration signal of the vehicle are actually sent by the vehicle driver. When the vehicle driver operates, the vehicle sends the lateral acceleration signal and longitudinal acceleration signal to the CAN network. The camera controller assembly obtains these signals to calculate the lateral displacement value of the vehicle relative to the target lane line within the future preset time range. Since these signals represent the driver's operating intention, they further represent the future changes in the lateral and longitudinal acceleration of the vehicle, and the future changes in the lateral and longitudinal acceleration of the vehicle will inevitably cause changes in the lateral distance of the vehicle relative to the target lane line. The aforementioned motion trajectory data of the vehicle within the future preset time range is obtained based on the lateral and longitudinal acceleration of the vehicle at the current moment. By combining the changes in the future lateral and longitudinal acceleration of the vehicle, the accuracy of the actual lateral distance between the vehicle and the target lane line within the future preset time range can be improved.

[0133] S409, judging whether the vehicle has a lane departure risk according to the actual lateral distance;

[0134] If yes, execute S410; if no, execute S401.

[0135] S411. Output lane departure warning information.

[0136] In addition, the camera can also detect the distance between the vehicle and the left and right lane lines. If the distance difference between the vehicle and the left and right lane lines is too large, a lane departure warning message can be sent to the driver.

[0137] Figure 5 The structural diagram of the vehicle warning device provided in this application is as follows: Figure 5 As shown, the device 50 comprises:

[0138] The visual detection module 501 is used to: obtain a first road surface image of the road ahead of the vehicle;

[0139] The recognition module 502 is used to: use a deep learning model to obtain a target lane line in the first road surface image, a first relative distance between the obstacle in front of the vehicle and the vehicle, and a first relative speed;

[0140] The calculation module 503 is used to calculate a second relative distance between the front obstacle and the vehicle according to a first width of the target lane line at a first position and a second width of the target lane line at a second position in the first road surface image; the first position is a position near the vehicle, and the second position is a position near the front obstacle;

[0141] Calculating a second relative speed between the front obstacle and the vehicle according to the first road surface image and the size of a bounding box of the front obstacle in at least one frame of the second road surface image acquired before the first road surface image;

[0142] The correction module 504 is used to: correct the first relative distance according to the second relative distance to obtain a third relative distance, and correct the first relative speed according to the second relative speed to obtain a third relative speed;

[0143] The collision risk warning module 505 determines whether there is a collision risk between the front obstacle and the vehicle according to the third relative distance and the third relative speed;

[0144] If yes, a collision risk warning message is output.

[0145] The vehicle warning device provided in this embodiment can execute the method provided in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be described in detail here.

[0146] Figure 6 This is a schematic diagram of the structure of the electronic device provided in this application. Figure 6As shown, the electronic device 60 provided in this embodiment includes: at least one processor 601 and a memory 602. Optionally, the device 60 also includes a communication component 603. The processor 601, the memory 602 and the communication component 603 are connected via a bus 604.

[0147] In a specific implementation process, at least one processor 601 executes the computer execution instructions stored in the memory 602, so that at least one processor 601 executes the above method.

[0148] The specific implementation process of the processor 601 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0149] In the above embodiments, it should be understood that the processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the invention can be directly implemented as a hardware processor, or can be implemented by a combination of hardware and software modules in the processor.

[0150] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (NVM), such as at least one disk storage.

[0151] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of this application is not limited to only one bus or one type of bus.

[0152] The present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0153] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.

[0154] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0155] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0156] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0157] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0158] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0159] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0160] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0161] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the art not disclosed by the present invention, are not limited to the precise structure described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A vehicle collision warning method, characterized in that: The method comprises: Acquire a first road surface image of a road ahead of the vehicle; Using a deep learning model, obtaining a target lane line in the first road surface image, a first relative distance between the obstacle in front of the vehicle and the vehicle, and a first relative speed; Calculating a second relative distance between the front obstacle and the vehicle according to a first width of the target lane line at a first position and a second width of the target lane line at a second position in the first road surface image; the first position is a position near the vehicle, and the second position is a position near the front obstacle; Calculating a second relative speed between the front obstacle and the vehicle according to the first road surface image and a size of a bounding box of the front obstacle in at least one frame of a second road surface image acquired before the first road surface image; Correcting the first relative distance according to the second relative distance to obtain a third relative distance, and correcting the first relative speed according to the second relative speed to obtain a third relative speed; determining, according to the third relative distance and the third relative speed, whether there is a collision risk between the front obstacle and the vehicle; If so, a collision risk warning message is output.

2. The method according to claim 1, characterized in that The first width of the target lane line at the first position and the second width of the target lane line at the second position in the first road surface image include: Detecting a first number of pixels in a width direction of the target lane line at the first position, and a second number of pixels in a width direction of the target lane line at the second position; The second relative distance is calculated according to a difference between the first number and the second number.

3. The method according to claim 1 or 2, characterized in that: The step of correcting the first relative distance according to the second relative distance to obtain a third relative distance, and correcting the first relative speed according to the second relative speed to obtain a third relative speed, comprises: Inputting the first relative distance and the second relative distance into a correction deep learning model, and outputting the third relative distance by the correction deep learning model; The first relative speed and the second relative speed are input into the corrected deep learning model, and the corrected deep learning model outputs the third relative speed.

4. The method according to claim 1, characterized in that: The using the deep learning model to obtain the target lane line in the first road surface image includes: Identifying the current road condition of the road according to the first road surface image; Determining, according to the current road condition, an identification strategy corresponding to the current road condition; According to the recognition strategy, adjusting the first road surface image to determine a third road surface image; The third road surface image is input into a recognition deep learning model, and the recognition deep learning model recognizes the target lane line according to the third road surface image.

5. The method according to claim 1, characterized in that After acquiring the target lane line in the first road surface image by using the deep learning model, the method further includes: Determine the actual lateral distance between the vehicle and the target lane line within the future preset time range according to the motion trajectory data of the vehicle within the future preset time range and the target lane line information; determining, according to the actual lateral distance, whether the vehicle has a lane departure risk; If so, a lane departure warning message is output.

6. The method according to claim 5, characterized in that The determining, based on the motion trajectory data of the vehicle within a future preset time range and the target lane line information, an actual lateral distance between the vehicle and the target lane line within the future preset time range includes: Determine a first lateral distance between the vehicle and the target lane line according to the motion trajectory data of the vehicle within a preset time range in the future and the target lane line information; Determine, according to the lateral acceleration signal and the longitudinal acceleration signal of the vehicle sent by the vehicle, a lateral displacement value of the vehicle relative to the target lane line within the future preset time range; The first lateral distance is corrected according to the lateral displacement value to determine the actual lateral distance.

7. The method according to claim 1, characterized in that The step of acquiring a first road surface image of a road ahead of the vehicle includes: Detecting the light intensity of the environment in which the vehicle is located, and determining whether the light intensity is lower than a preset light intensity threshold; If yes, adjusting the setting parameters of the vehicle camera according to the light intensity of the environment in which the vehicle is located; The camera is controlled to acquire the first road surface image.

8. A vehicle warning device, characterized in that: The device comprises: A visual detection module is used to: obtain a first road surface image of a road ahead of the vehicle; A recognition module, configured to: use a deep learning model to obtain a target lane line in the first road surface image, a first relative distance between the obstacle in front of the vehicle and the vehicle, and a first relative speed; a calculation module, configured to calculate a second relative distance between the front obstacle and the vehicle according to a first width of the target lane line at a first position and a second width of the target lane line at a second position in the first road surface image; the first position is a position near the vehicle, and the second position is a position near the front obstacle; Calculating a second relative speed between the front obstacle and the vehicle according to the first road surface image and a size of a bounding box of the front obstacle in at least one frame of a second road surface image acquired before the first road surface image; a correction module, configured to: correct the first relative distance according to the second relative distance to obtain a third relative distance, and correct the first relative speed according to the second relative speed to obtain a third relative speed; A collision risk warning module: judging whether there is a collision risk between the front obstacle and the vehicle according to the third relative distance and the third relative speed; If so, a collision risk warning message is output.

9. An electronic device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.

Images