Intelligent lane departure recognition method, system and device based on computer vision

Abstract

The application discloses a computer vision-based intelligent lane departure identification method, system and device, comprising: obtaining the position of a target vehicle relative to a target lane according to a road surface monitoring image; judging whether there is a following vehicle and / or a target obstacle according to an environment monitoring image; when the following vehicle exists, performing lane departure prediction according to the trajectory of the following vehicle; when the target obstacle exists, performing lane departure prediction according to the position of the target obstacle; and performing lane departure early warning according to the result of lane departure prediction and the position of the target vehicle relative to the target lane. At least the problem that the existing lane departure early warning system only judges whether prompting is needed according to whether a turn signal is turned on, but in actual driving, when an emergency needs to be avoided, the driver is easy to trigger the warning of the lane departure early warning system, and if prompting is performed at this time, the judgment of the driver may be affected.

Description

Technical Field

[0001] This application relates to the field of Internet big data technology, and in particular to intelligent lane departure recognition methods, systems and devices based on computer vision. Background Technology

[0002] Lane departure warning systems are designed to assist drivers in reducing traffic accidents caused by lane departure by issuing warnings. A lane departure warning system consists of an image processing chip, controller, and sensors. Primarily composed of a head-up display (HUD), camera, controller, and sensors, the system works as follows: When activated, the camera continuously captures the lane markings and processes the image to determine the vehicle's position within the lane. When lane departure is detected, the sensors collect vehicle data and driver input, and the controller issues a warning signal. This entire process is completed in a very short time, giving the driver more reaction time. However, if the driver uses their turn signal and changes lanes normally, the lane departure warning system will not provide any warning. When the vehicle deviates from its lane, it can alert the driver through an audible warning, steering wheel vibration, or automatic steering adjustment.

[0003] Currently, existing lane departure warning systems only determine whether a warning is needed based on whether the turn signal is activated. However, in actual driving, when an emergency occurs and evasive maneuvers are required, the driver is likely to trigger the lane departure warning system's warning. If the warning is given through an alarm sound, steering wheel vibration, or automatic steering change at this time, it may affect the driver's judgment. Summary of the Invention

[0004] This invention provides an intelligent lane departure recognition method, system, and device based on computer vision. It provides an intelligent lane departure recognition method that at least solves the problem that existing lane departure warning systems only determine whether a warning is needed based on whether the turn signal is on. However, in actual driving, when an emergency occurs and avoidance is required, the driver can easily trigger the warning of the lane departure warning system. If the warning is given by alarm sound, steering wheel vibration, or automatic steering change at this time, it may affect the driver's judgment.

[0005] Intelligent lane departure recognition methods based on computer vision include:

[0006] Based on road surveillance images, the position of the target vehicle relative to the target lane is obtained;

[0007] Based on the environmental monitoring images, it is determined whether there are following vehicles and / or target obstacles. The following vehicles are configured as vehicles in the monitoring images that are in the same lane as the target vehicle for more than a preset time. The target obstacles are configured as obstacles in the monitoring images that are in the same lane as the target vehicle.

[0008] When the following vehicle is present, lane departure prediction is performed based on the trajectory of the following vehicle;

[0009] When the target obstacle is present, lane departure prediction is performed based on the position of the target obstacle;

[0010] Lane departure warning is issued based on the lane departure prediction results and the position of the target vehicle relative to the target lane.

[0011] Optionally, obtaining the position of the target vehicle relative to the target lane based on the road monitoring image includes:

[0012] Based on the road monitoring image, a lane line detection algorithm is used to determine whether the road monitoring image includes lane lines;

[0013] When the road monitoring image includes lane lines, spatial data of the camera relative to the lane lines is obtained based on the road monitoring image and the camera of the target vehicle; the position of the target vehicle relative to the target lane is obtained based on the spatial data of the camera relative to the lane lines and the spatial data relative to the target vehicle.

[0014] When the road monitoring image does not include lane line data, the following steps are taken: First, the road width is obtained from the road monitoring image. Second, lane planning data is obtained from the road width. Third, spatial data of the virtual lane line relative to the road surface is obtained from the lane planning data. Fourth, spatial data of the camera relative to the road surface is obtained from the road monitoring image and the camera of the target vehicle. Fifth, spatial data of the target vehicle relative to the road surface is obtained from the spatial data of the camera relative to the road surface and the spatial data of the camera relative to the target vehicle. Sixth, the position of the target vehicle relative to the target lane is obtained from the spatial data of the target vehicle relative to the road surface and the spatial data of the virtual lane line relative to the road surface.

[0015] The road monitoring image is configured to be captured by the camera of the target vehicle, and the target lane refers to the lane in which the target vehicle is traveling or should be traveling.

[0016] Optionally, determining whether a following vehicle and / or target obstacle exists based on the environmental monitoring image includes:

[0017] Based on the environmental monitoring images, an environmental monitoring image sequence is obtained using the preset duration as a sliding time window. The environmental monitoring image sequence is configured to include at least the environmental monitoring images within the latest sliding time window.

[0018] Based on the environmental monitoring image sequence, the environmental monitoring image sequence is identified to determine whether the first frame and the last frame of the environmental monitoring image sequence contain the same vehicle;

[0019] If the same vehicle exists in the first and last frames of the environmental monitoring image sequence, the proportion of the vehicle that is in the same lane as the target vehicle in the environmental monitoring image sequence is obtained.

[0020] When the proportion of the vehicle in the same lane as the target vehicle in the environmental monitoring image sequence meets a preset condition, the vehicle is identified as a following vehicle.

[0021] Optionally, determining whether a following vehicle and / or target obstacle exists based on the environmental monitoring image includes:

[0022] Based on the environmental monitoring image, the target lane where the target vehicle is located is obtained by analyzing the environmental monitoring image.

[0023] Based on the target lane where the target vehicle is located, the image corresponding to the target lane is identified to obtain the lane identification result;

[0024] When the lane recognition result includes abnormal objects and / or abnormal road conditions, the abnormal objects and / or abnormal road conditions are identified as target obstacles.

[0025] Optionally, when the following vehicle is present, the step of predicting lane departure based on the trajectory of the following vehicle includes:

[0026] When the following vehicle is present, the position of the following vehicle relative to the target lane is obtained;

[0027] The trajectory of the following vehicle is obtained based on its position relative to the target lane;

[0028] Based on the trajectory of the following vehicle, determine whether the following vehicle has crossed the line.

[0029] When the following vehicle crosses the lane, lane departure prediction is performed based on the trajectory of the target vehicle.

[0030] Optionally, when the following vehicle crosses lanes, predicting lane departure based on the trajectory of the target vehicle includes:

[0031] When the following vehicle crosses the lane, the behavior data of the following vehicle is used to determine whether the crossing behavior is a lane change or an evasive action.

[0032] When the following vehicle's crossing behavior is an avoidance behavior, obtain the crossing trajectory of the following vehicle when it crosses the line;

[0033] Lane departure prediction is performed based on the cross-line trajectory and the distance between the target vehicle and the following vehicle.

[0034] Optionally, the step of predicting lane departure based on the cross-line trajectory and the distance between the target vehicle and the following vehicle includes:

[0035] Based on the cross-line trajectory, the maximum value of the left and right displacement of the following vehicle is obtained;

[0036] Lane departure prediction is obtained based on the maximum left-right displacement of the following vehicle and the distance between the target vehicle and the following vehicle.

[0037] Optionally, the step of providing lane departure warning based on the lane departure prediction result and the position of the target vehicle relative to the target lane includes:

[0038] The trajectory of the target vehicle is obtained based on the position of the target vehicle relative to the target lane and the speed of the target vehicle;

[0039] When the trajectory of the target vehicle deviates from its lane, the similarity between the trajectory of the target vehicle and the lane deviation prediction result is determined.

[0040] Lane departure warning is issued when the similarity does not meet the preset conditions.

[0041] On the other hand, a computer vision-based intelligent lane departure recognition system includes an onboard monitoring module and a lane departure recognition module;

[0042] The vehicle-mounted monitoring module is configured as follows:

[0043] Based on road surveillance images, the position of the target vehicle relative to the target lane is obtained;

[0044] Based on the environmental monitoring images, it is determined whether there are following vehicles and / or target obstacles. The following vehicles are configured as vehicles in the monitoring images that are in the same lane as the target vehicle for more than a preset time. The target obstacles are configured as obstacles in the monitoring images that are in the same lane as the target vehicle.

[0045] The lane departure detection is configured as follows:

[0046] When the following vehicle is present, lane departure prediction is performed based on the trajectory of the following vehicle;

[0047] When the target obstacle is present, lane departure prediction is performed based on the position of the target obstacle;

[0048] Lane departure warning is issued based on the lane departure prediction results and the position of the target vehicle relative to the target lane.

[0049] On the other hand, a computer device includes a memory and a processor, wherein the memory stores a computer program and the processor executes the computer program to implement the above-described method.

[0050] On the other hand, a computer storage medium storing a computer program, wherein a processor executes the computer program to implement the above-described method.

[0051] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0052] This invention discloses an intelligent lane departure recognition method, system, and device based on computer vision, comprising: obtaining the position of a target vehicle relative to a target lane based on a road monitoring image; determining whether a following vehicle and / or a target obstacle exists based on an environmental monitoring image, wherein the following vehicle is configured as a vehicle in the monitoring image that has been in the same lane as the target vehicle for more than a preset time, and the target obstacle is configured as an obstacle in the monitoring image that is in the same lane as the target vehicle; when the following vehicle exists, performing lane departure prediction based on the trajectory of the following vehicle; when the target obstacle exists, performing lane departure prediction based on the position of the target obstacle; and providing a lane departure warning based on the lane departure prediction result and the position of the target vehicle relative to the target lane. This invention at least solves the problem that existing lane departure warning systems rely solely on whether the turn signal is activated to determine whether a warning is needed. However, in actual driving, when an emergency occurs requiring evasive maneuvers, the driver can easily trigger the lane departure warning system's warning. If the warning is given through an audible alarm, steering wheel vibration, or automatic steering change, it may affect the driver's judgment. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0054] Figure 1This is a flowchart illustrating an intelligent lane departure recognition method based on computer vision.

[0055] Figure 2 This is a schematic diagram of a computer device.

[0056] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0057] To enable those skilled in the art to better understand the present disclosure, the technical solutions of the present disclosure will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present disclosure, and not all embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present disclosure.

[0058] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0059] Example 1

[0060] like Figure 1 As shown, an intelligent lane departure recognition method based on computer vision, executed by the target vehicle, includes:

[0061] S1. Obtain the position of the target vehicle relative to the target lane based on the road monitoring image.

[0062] Optionally, road monitoring images are obtained through onboard cameras located at the front and / or sides of the target vehicle. Lane lines in the road monitoring images are extracted using image recognition algorithms. The position of the target vehicle relative to the target lane is then obtained based on the coordinates of the lane lines in the road monitoring images and the position of the onboard cameras.

[0063] S2. Based on the environmental monitoring images, determine whether there are following vehicles and / or target obstacles.

[0064] The following vehicle is configured to be a vehicle in the same lane as the target vehicle in the monitoring image for more than a preset time, and the target obstacle is configured to be an obstacle in the same lane as the target vehicle in the monitoring image.

[0065] Optionally, the target obstacle can be a bump or pothole on the road surface of the target lane, or an unusual object on the target lane, including cargo left behind by the vehicle in front, traffic cones or other signs, or other obstacles or unusual items that may appear on the road surface.

[0066] Optionally, environmental monitoring images can be obtained using an onboard camera located on the front of the target vehicle.

[0067] S3. When there is a following vehicle, predict lane departure based on the trajectory of the following vehicle.

[0068] Optionally, when a following vehicle is present, the lane departure prediction result is configured to be obtained based on the trajectory of the following vehicle.

[0069] S4. When a target obstacle exists, lane departure prediction is performed based on the position of the target obstacle.

[0070] Optionally, when a target obstacle is present, the lane departure prediction results are configured to obtain the possible trajectory of the target vehicle to avoid the target obstacle.

[0071] S5 issues lane departure warnings based on the lane departure prediction results and the position of the target vehicle relative to the target lane.

[0072] Optionally, based on the target vehicle's position data relative to the target lane, the target vehicle's trajectory is obtained. The trajectory is then compared with the lane departure prediction result to determine whether a lane departure warning is needed. This approach allows for lane departure prediction based on the surrounding environment, such as road obstacles and the trajectories of vehicles ahead. The determination of lane departure warning is then based on the degree of match between the driver's own trajectory and the predicted lane departure. By predicting lane departure based on the surrounding environment, warnings from the lane departure warning system can be avoided in emergency situations requiring evasive maneuvers. This addresses the problem that existing lane departure warning systems rely solely on turn signal activation, which can easily trigger warnings in emergency situations. If warnings are issued through audible alarms, steering wheel vibrations, or automatic steering changes, it can impair the driver's judgment.

[0073] Example 2

[0074] This embodiment, based on Embodiment 1, provides an intelligent lane departure recognition method based on computer vision, executed by the target vehicle, including:

[0075] S1. Obtain the position of the target vehicle relative to the target lane based on the road monitoring image.

[0076] Optionally, road monitoring images are obtained through onboard cameras located at the front and / or sides of the target vehicle. Lane lines in the road monitoring images are extracted using image recognition algorithms. The position of the target vehicle relative to the target lane is then obtained based on the coordinates of the lane lines in the road monitoring images and the position of the onboard cameras.

[0077] Optionally, based on road surveillance images, the position of the target vehicle relative to the target lane is obtained, including:

[0078] Based on the road monitoring image, a lane line detection algorithm is used to determine whether the road monitoring image includes lane lines;

[0079] When the road monitoring image includes lane lines, spatial data of the camera relative to the lane lines is obtained based on the road monitoring image and the camera of the target vehicle.

[0080] Based on the spatial data of the camera relative to the lane lines and the spatial data of the camera relative to the target vehicle, the position of the target vehicle relative to the target lane is obtained.

[0081] Optionally, based on road surveillance images, the position of the target vehicle relative to the target lane is obtained, including:

[0082] Obtain road surveillance images using the vehicle's onboard camera;

[0083] The road monitoring images are preprocessed to obtain preprocessed road monitoring images; specifically, image preprocessing may include methods such as grayscale conversion, noise reduction, contrast enhancement, and perspective transformation.

[0084] Based on the preprocessed road monitoring images, lane line data is extracted from the road monitoring images using AI detection algorithms. Specifically, AI detection algorithms can use Canny edge detection algorithm, ROI detection algorithm, RANSAC lane line fitting method, or pre-trained deep learning algorithm.

[0085] Based on the lane line data, the coordinates of the lane lines of the target lane in the road monitoring image are extracted. Specifically, the coordinates of the lane lines in the road monitoring image are obtained by performing a second or third polynomial fitting on the detected lane line pixels.

[0086] Based on the calibration parameters of the vehicle-mounted camera and the coordinates of the lane lines in the road monitoring image, the position of the target vehicle relative to the target lane is obtained. The calibration parameters of the vehicle-mounted camera include focal length, installation height, pitch angle, etc. Based on these parameters, the pixel difference can be converted into actual distance.

[0087] Optionally, based on road surveillance images, the position of the target vehicle relative to the target lane is obtained, including:

[0088] Based on the road monitoring image, a lane line detection algorithm is used to determine whether the road monitoring image includes lane lines;

[0089] When the road monitoring image does not include lane line data, the width of the road surface is obtained based on the road monitoring image;

[0090] Based on the width of the road surface, obtain lane planning data;

[0091] Based on lane planning data, obtain spatial data of virtual lane lines relative to the road surface;

[0092] Based on road surveillance images and the target vehicle's camera, spatial data of the camera relative to the road surface is obtained;

[0093] Based on the spatial data of the camera relative to the road surface and the spatial data of the camera relative to the target vehicle, obtain the spatial data of the target vehicle relative to the road surface;

[0094] Based on the spatial data of the target vehicle relative to the road surface and the spatial data of the virtual lane lines relative to the road surface, the position of the target vehicle relative to the target lane is obtained;

[0095] The road surveillance images are configured to be captured by the target vehicle's camera, and the target lane refers to the lane in which the target vehicle is traveling or should be traveling.

[0096] Optionally, the width of the road surface can be obtained from the road monitoring image, including:

[0097] Based on the camera that captured the road monitoring images, obtain the camera calibration parameters;

[0098] Based on the camera calibration parameters, perform an image bird's-eye view transformation of the road monitoring image to obtain a road bird's-eye view map;

[0099] Obtain road surface pixel data based on the aerial view of the road surface;

[0100] Based on the camera calibration parameters, obtain the actual length corresponding to each pixel;

[0101] The width of the road surface is obtained based on the road surface pixel data and the actual length corresponding to each pixel.

[0102] Optionally, lane planning data can be obtained based on the road width, including:

[0103] The one-way road width is obtained based on the road width and traffic conditions, such as two-way lanes or one-way streets.

[0104] One-way lane planning is obtained based on the one-way road width and the minimum lane width requirement;

[0105] Based on the one-way lane planning, obtain the road lane planning;

[0106] Based on the road lane planning and road width, lane planning data is obtained, which includes the number of lanes and the direction and width of each lane.

[0107] S2. Based on the environmental monitoring images, determine whether there are following vehicles and / or target obstacles.

[0108] The following vehicle is configured to be a vehicle in the same lane as the target vehicle in the monitoring image for more than a preset time, and the target obstacle is configured to be an obstacle in the same lane as the target vehicle in the monitoring image.

[0109] Optionally, the target obstacle can be a bump or pothole on the road surface of the target lane, or an unusual object on the target lane, including cargo left behind by the vehicle in front, traffic cones or other signs, or other obstacles or unusual items that may appear on the road surface.

[0110] Optionally, environmental monitoring images can be obtained using an onboard camera located on the front of the target vehicle.

[0111] Optionally, based on environmental surveillance images, determine whether there are following vehicles and / or target obstacles, including:

[0112] Based on the environmental monitoring images, an environmental monitoring image sequence is obtained using a preset duration as a sliding time window. The environmental monitoring image sequence is configured to include at least the environmental monitoring images within the latest sliding time window.

[0113] Based on the environmental monitoring image sequence, identify the environmental monitoring image sequence and determine whether the first and last frames of the environmental monitoring image sequence contain the same vehicle;

[0114] If the same vehicle exists in the first and last frames of an environmental monitoring image sequence, obtain the proportion of vehicles in the same lane as the target vehicle in the environmental monitoring image sequence.

[0115] When the proportion of vehicles in the same lane as the target vehicle in an environmental monitoring image sequence meets preset conditions, the vehicle is identified as a following vehicle.

[0116] Optionally, the preset duration is generally set to 3-10 seconds, with a typical characteristic value of 6 seconds, calculated based on the high-speed vehicle distance requirement of 200 meters and the speed limit of 120 km / h.

[0117] Optionally, based on environmental surveillance images, determine whether there are following vehicles and / or target obstacles, including:

[0118] Based on environmental surveillance images, analyze the environmental surveillance images to obtain the target lane where the target vehicle is located;

[0119] Based on the target lane where the target vehicle is located, the image corresponding to the target lane is identified to obtain the lane identification result;

[0120] When the lane recognition results include abnormal objects and / or abnormal road conditions, the abnormal objects and / or abnormal road conditions are identified as target obstacles.

[0121] Optionally, the detection of following vehicles and / or target obstacles can be accomplished using a vision-based solution, or a solution combining vision with LiDAR or millimeter-wave radar.

[0122] S3. When there is a following vehicle, predict lane departure based on the trajectory of the following vehicle.

[0123] Optionally, when a following vehicle is present, the lane departure prediction result is configured to be obtained based on the trajectory of the following vehicle.

[0124] Optionally, when a following vehicle is present, lane departure prediction is performed based on the trajectory of the following vehicle, including:

[0125] When a following vehicle is present, obtain the position of the following vehicle relative to the target lane;

[0126] The trajectory of the following vehicle is obtained based on its position relative to the target lane;

[0127] Based on the trajectory of the following vehicle, determine whether the following vehicle has crossed the line;

[0128] When a following vehicle crosses lanes, lane departure prediction is performed based on the target vehicle's trajectory.

[0129] Optionally, when the following vehicle crosses lanes, lane departure prediction is performed based on the target vehicle's trajectory, including:

[0130] When a following vehicle crosses a lane, the behavior data of the following vehicle is used to determine whether the crossing behavior is a lane change or an evasive maneuver.

[0131] When the following vehicle's crossing behavior is an avoidance behavior, obtain the crossing trajectory of the following vehicle when it crosses the line;

[0132] Lane departure prediction is performed based on the cross-line trajectory and the distance between the target vehicle and the following vehicle.

[0133] Optionally, when a following vehicle crosses lanes, the method for determining whether the lane-crossing behavior of the following vehicle is a lane-changing behavior or an evasive behavior based on the behavior data of the following vehicle includes at least one of the following methods:

[0134] If a following vehicle crosses a lane after activating its turn signal, and fails to return to the target lane after crossing, then the following vehicle's lane-crossing behavior constitutes a lane change.

[0135] If the following vehicle returns to the target lane within a certain period of time after crossing the lane, then the following vehicle's crossing of the lane is considered an evasive action.

[0136] If the following vehicle crosses the lane but does not fully enter the lane adjacent to the target lane, then the following vehicle's crossing of the lane is considered an evasive maneuver.

[0137] Optionally, lane departure prediction is performed based on the cross-line trajectory and the distance between the target vehicle and the following vehicle, including:

[0138] Based on the cross-line trajectory, obtain the maximum value of the left and right displacement of the following vehicle; the left and right displacement refers to the displacement on the horizontal plane perpendicular to the direction of travel of the following vehicle to the left or right.

[0139] Lane departure prediction is obtained based on the maximum left and right displacement of the following vehicle and the distance between the target vehicle and the following vehicle.

[0140] Optionally, the lane departure prediction content is configured as the range of left and right displacement of the target vehicle within a preset departure time.

[0141] Specifically, lane departure prediction also includes the range of angles at which the target vehicle will deviate from its lane within a preset time period.

[0142] Specifically, methods for predicting lane departure based on the maximum lateral displacement of the following vehicle and the distance between the target vehicle and the following vehicle include:

[0143] Based on the distance between the target vehicle and the following vehicle and the speed of the target vehicle, the offset preset time is obtained. Optionally, the offset prediction time is a period of time after the target vehicle crosses the lane. This period of time can be calculated from the distance between the target vehicle and the following vehicle and the speed of the target vehicle, or it can be calculated from the distance between the target vehicle and the following vehicle and the speed of the target vehicle plus a certain delay time.

[0144] Based on the left-right displacement of the following vehicle, the range of left-right displacement of the target vehicle is obtained. The range of left-right displacement of the target vehicle is slightly larger than that of the following vehicle. For example, if the following vehicle moves 3 meters to the right, the range of left-right displacement of the target vehicle could be 0-3.5 meters to the right, or 1-3.5 meters, etc.

[0145] Based on the range of left and right displacement of the target vehicle and the distance between the target vehicle and the following vehicles, the angular range of lane departure of the target vehicle can be obtained.

[0146] Optionally, crossing behavior refers to the act of a vehicle driving over or crossing lane lines and virtual lane lines.

[0147] S4. When a target obstacle exists, lane departure prediction is performed based on the position of the target obstacle.

[0148] Optionally, when a target obstacle is present, the lane departure prediction results are configured to obtain the possible trajectory of the target vehicle to avoid the target obstacle.

[0149] Optionally, methods for obtaining the possible trajectory of the target vehicle avoiding the target obstacle include:

[0150] Based on environmental monitoring images, the coordinates of the target obstacle are obtained. The coordinates of the target obstacle are configured to include the distance of the target obstacle relative to the target vehicle and the coordinates of the target obstacle relative to the target lane in a direction perpendicular to the extension of the target lane.

[0151] Based on the coordinates of the target obstacle, plan the predicted trajectory for the target vehicle to avoid the target obstacle.

[0152] Specifically, based on the coordinates of the target obstacle, a predicted trajectory for the target vehicle to avoid the target obstacle is planned, including...

[0153] Based on the distance between the target obstacle and the target vehicle and the speed of the target vehicle, the offset preset time is obtained. Optionally, the offset prediction time is a period of time after the target vehicle detects the target obstacle. This period of time can be calculated from the distance between the target vehicle and the target obstacle and the speed of the target vehicle, or it can be calculated from the distance between the target vehicle and the target obstacle and the speed of the target vehicle plus a certain delay time.

[0154] Based on the coordinates of the target obstacle relative to the target lane in a direction perpendicular to the extension of the target lane, the range of left and right displacement of the target vehicle is obtained;

[0155] Based on the range of left and right displacement of the target vehicle and the distance between the target vehicle and the following vehicles, the angular range of lane departure of the target vehicle can be obtained.

[0156] S5 issues lane departure warnings based on the lane departure prediction results and the position of the target vehicle relative to the target lane.

[0157] Optionally, based on the lane departure prediction results and the position of the target vehicle relative to the target lane, lane departure warning may be issued, including:

[0158] The trajectory of the target vehicle is obtained based on its position relative to the target lane and its speed.

[0159] When the target vehicle's trajectory deviates from its lane, the similarity between the target vehicle's trajectory and the lane deviation prediction result is determined.

[0160] Lane departure warning is issued when the similarity does not meet the preset conditions.

[0161] Specifically, the results of lane departure prediction are obtained from one or more of the following parameters: displacement in the direction of travel, left / right displacement perpendicular to the direction of travel, vehicle speed, or other vehicle parameters.

[0162] Specifically, the lane departure prediction result is a lane departure prediction trajectory generated based on the displacement in the driving direction, the left / right displacement perpendicular to the driving direction, the speed of the target vehicle, and the driving habits of the target vehicle driver.

[0163] Specifically, when a target vehicle's trajectory deviates from its lane, the similarity between the target vehicle's trajectory and the lane deviation prediction result is determined, including:

[0164] Based on the lane departure prediction trajectory and the target vehicle's trajectory, the similarity between the lane departure prediction trajectory and the target vehicle's trajectory is calculated using Frechet distance or dynamic time warping algorithms.

[0165] Example 3

[0166] This embodiment discloses an intelligent lane departure recognition system based on computer vision, including an on-board monitoring module and a lane departure recognition module;

[0167] The vehicle monitoring module is configured as follows:

[0168] Based on road surveillance images, the position of the target vehicle relative to the target lane is obtained;

[0169] Based on the environmental monitoring images, determine whether there are following vehicles and / or target obstacles. Following vehicles are defined as vehicles in the monitoring images that are in the same lane as the target vehicle for more than a preset time. Target obstacles are defined as obstacles in the monitoring images that are in the same lane as the target vehicle.

[0170] Lane departure detection is configured as follows:

[0171] When there is a following vehicle, lane departure prediction is performed based on the trajectory of the following vehicle;

[0172] When a target obstacle is present, lane departure prediction is performed based on the location of the target obstacle;

[0173] Lane departure warnings are issued based on the lane departure prediction results and the position of the target vehicle relative to the target lane.

[0174] Optionally, based on road surveillance images, the position of the target vehicle relative to the target lane is obtained, including:

[0175] Based on the road monitoring image, a lane line detection algorithm is used to determine whether the road monitoring image includes lane lines;

[0176] When the road monitoring image includes lane lines, spatial data of the camera relative to the lane lines is obtained based on the road monitoring image and the camera of the target vehicle; the position of the target vehicle relative to the target lane is obtained based on the spatial data of the camera relative to the lane lines and the spatial data of the camera relative to the target vehicle.

[0177] When the road monitoring image does not include lane line data, the following steps are taken: First, obtain the road width from the road monitoring image. Second, obtain lane planning data from the road width. Third, obtain the spatial data of the virtual lane lines relative to the road surface from the lane planning data. Fourth, obtain the spatial data of the camera relative to the road surface from the road monitoring image and the target vehicle's camera. Fifth, obtain the spatial data of the target vehicle relative to the road surface from the camera's spatial data relative to the road surface and its spatial data relative to the target vehicle. Sixth, obtain the position of the target vehicle relative to the target lane from the target vehicle's spatial data relative to the road surface and the spatial data of the virtual lane lines relative to the road surface.

[0178] The road surveillance images are configured to be captured by the target vehicle's camera, and the target lane refers to the lane in which the target vehicle is traveling or should be traveling.

[0179] Optionally, based on environmental surveillance images, determine whether there are following vehicles and / or target obstacles, including:

[0180] Based on the environmental monitoring images, an environmental monitoring image sequence is obtained using a preset duration as a sliding time window. The environmental monitoring image sequence is configured to include at least the environmental monitoring images within the latest sliding time window.

[0181] Based on the environmental monitoring image sequence, identify the environmental monitoring image sequence and determine whether the first and last frames of the environmental monitoring image sequence contain the same vehicle;

[0182] If the same vehicle exists in the first and last frames of an environmental monitoring image sequence, obtain the proportion of vehicles in the same lane as the target vehicle in the environmental monitoring image sequence.

[0183] When the proportion of vehicles in the same lane as the target vehicle in an environmental monitoring image sequence meets preset conditions, the vehicle is identified as a following vehicle.

[0184] Optionally, based on environmental surveillance images, determine whether there are following vehicles and / or target obstacles, including:

[0185] Based on environmental surveillance images, analyze the environmental surveillance images to obtain the target lane where the target vehicle is located;

[0186] Based on the target lane where the target vehicle is located, the image corresponding to the target lane is identified to obtain the lane identification result;

[0187] When the lane recognition results include abnormal objects and / or abnormal road conditions, the abnormal objects and / or abnormal road conditions are identified as target obstacles.

[0188] Optionally, when a following vehicle is present, lane departure prediction is performed based on the trajectory of the following vehicle, including:

[0189] When a following vehicle is present, obtain the position of the following vehicle relative to the target lane;

[0190] The trajectory of the following vehicle is obtained based on its position relative to the target lane;

[0191] Based on the trajectory of the following vehicle, determine whether the following vehicle has crossed the line;

[0192] When a following vehicle crosses lanes, lane departure prediction is performed based on the target vehicle's trajectory.

[0193] Optionally, when the following vehicle crosses lanes, lane departure prediction is performed based on the target vehicle's trajectory, including:

[0194] When a following vehicle crosses a lane, the behavior data of the following vehicle is used to determine whether the crossing behavior is a lane change or an evasive maneuver.

[0195] When the following vehicle's crossing behavior is an avoidance behavior, obtain the crossing trajectory of the following vehicle when it crosses the line;

[0196] Lane departure prediction is performed based on the cross-line trajectory and the distance between the target vehicle and the following vehicle.

[0197] Optionally, lane departure prediction is performed based on the cross-line trajectory and the distance between the target vehicle and the following vehicle, including:

[0198] Based on the cross-line trajectory, obtain the maximum value of the left and right displacement of the following vehicle;

[0199] Lane departure prediction is obtained based on the maximum left and right displacement of the following vehicle and the distance between the target vehicle and the following vehicle.

[0200] Optionally, based on the lane departure prediction results and the position of the target vehicle relative to the target lane, lane departure warning may be issued, including:

[0201] The trajectory of the target vehicle is obtained based on its position relative to the target lane and its speed.

[0202] When the target vehicle's trajectory deviates from its lane, the similarity between the target vehicle's trajectory and the lane deviation prediction result is determined.

[0203] Lane departure warning is issued when the similarity does not meet the preset conditions.

[0204] Example 4

[0205] This embodiment provides a computer device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement any of the methods described above.

[0206] Optional, such as Figure 2 As shown, Figure 2 This is a schematic diagram of a computer device, which is an electronic device. The electronic device may include: a processor 101, such as a central processing unit (CPU), a communication bus 102, a user interface 104, a network interface 103, and a memory 105. The communication bus 102 is used to enable communication between these components. The user interface 104 may include a display screen and an input unit such as a keyboard. Optionally, the user interface 104 may also include a standard wired interface or a wireless interface. The network interface 103 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 105 may be a storage device independent of the aforementioned processor 101. The memory 105 may be a high-speed random access memory (RAM) or a stable non-volatile memory (NVM), such as at least one disk storage device. The processor 101 may be a general-purpose processor, including a central processing unit, a network processor, etc., or it may be a digital signal processor, an application-specific integrated circuit, a field-programmable gate array or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component.

[0207] Those skilled in the art will understand that Figure 2 The structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0208] like Figure 2 As shown, the memory 105, which serves as a storage medium, may include an operating system, a network communication module, a user interface module, and an application program for implementing a computer vision-based intelligent lane departure recognition method.

[0209] exist Figure 2 In the computer device shown, the network interface 103 is mainly used for data communication with the network server; the user interface 104 is mainly used for data interaction with the user; the processor 101 and the memory 105 in this application can be set in the electronic device, and the electronic device calls the application program stored in the memory 105 through the processor 101 to implement the intelligent lane departure recognition method based on computer vision to implement the above method.

[0210] Example 5

[0211] This embodiment provides a computer storage medium on which a computer program is stored. A processor executes the computer program to implement any of the methods described above.

[0212] In some embodiments, the computer storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a device including one or any combination of the above-mentioned memories. The computer may be a variety of computing devices, including smart terminals and servers.

[0213] In the above embodiments of this disclosure, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0214] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between units or modules may be electrical or other forms.

[0215] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0216] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0217] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable non-volatile storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a non-volatile storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this disclosure. The aforementioned non-volatile storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0218] The above are merely preferred embodiments of this disclosure. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this disclosure, and these improvements and modifications should also be considered within the scope of protection of this disclosure.

Claims

1. An intelligent lane departure recognition method based on computer vision, characterized in that, include: Based on road surveillance images, the position of the target vehicle relative to the target lane is obtained; Based on the environmental monitoring images, it is determined whether there are following vehicles and / or target obstacles. The following vehicles are configured as vehicles in the monitoring images that are in the same lane as the target vehicle for more than a preset time. The target obstacles are configured as obstacles in the monitoring images that are in the same lane as the target vehicle. When the following vehicle is present, lane departure prediction is performed based on the trajectory of the following vehicle; When the target obstacle is present, lane departure prediction is performed based on the position of the target obstacle; Based on the lane departure prediction results and the position of the target vehicle relative to the target lane, a lane departure warning is issued; When the following vehicle is present, the process of predicting lane departure based on the trajectory of the following vehicle includes: When the following vehicle is present, the position of the following vehicle relative to the target lane is obtained; The trajectory of the following vehicle is obtained based on its position relative to the target lane; Based on the trajectory of the following vehicle, determine whether the following vehicle has crossed the line. When the following vehicle crosses the lane, the behavior data of the following vehicle is used to determine whether the crossing behavior is a lane change or an evasive action. When the following vehicle's crossing behavior is an avoidance behavior, obtain the crossing trajectory of the following vehicle when it crosses the line; Lane departure prediction is performed based on the cross-line trajectory and the distance between the target vehicle and the following vehicle.

2. The intelligent lane departure recognition method based on computer vision according to claim 1, characterized in that, The step of obtaining the position of the target vehicle relative to the target lane based on the road monitoring image includes: Based on the road monitoring image, a lane line detection algorithm is used to determine whether the road monitoring image includes lane lines; When the road monitoring image includes lane lines, spatial data of the camera relative to the lane lines is obtained based on the road monitoring image and the camera of the target vehicle; the position of the target vehicle relative to the target lane is obtained based on the spatial data of the camera relative to the lane lines and the spatial data relative to the target vehicle. When the road monitoring image does not include lane line data, the following steps are taken: First, the road width is obtained from the road monitoring image. Second, lane planning data is obtained from the road width. Third, spatial data of the virtual lane line relative to the road surface is obtained from the lane planning data. Fourth, spatial data of the camera relative to the road surface is obtained from the road monitoring image and the camera of the target vehicle. Fifth, spatial data of the target vehicle relative to the road surface is obtained from the spatial data of the camera relative to the road surface and the spatial data of the camera relative to the target vehicle. Sixth, the position of the target vehicle relative to the target lane is obtained from the spatial data of the target vehicle relative to the road surface and the spatial data of the virtual lane line relative to the road surface. The road monitoring image is configured to be captured by the camera of the target vehicle, and the target lane refers to the lane in which the target vehicle is traveling or should be traveling.

3. The intelligent lane departure recognition method based on computer vision according to claim 1, characterized in that, The step of determining whether there are following vehicles and / or target obstacles based on environmental monitoring images includes: Based on the environmental monitoring images, an environmental monitoring image sequence is obtained using the preset duration as a sliding time window. The environmental monitoring image sequence is configured to include at least the environmental monitoring images within the latest sliding time window. Based on the environmental monitoring image sequence, the environmental monitoring image sequence is identified to determine whether the first frame and the last frame of the environmental monitoring image sequence contain the same vehicle; If the same vehicle exists in the first and last frames of the environmental monitoring image sequence, the proportion of the vehicle that is in the same lane as the target vehicle in the environmental monitoring image sequence is obtained. When the proportion of the vehicle in the same lane as the target vehicle in the environmental monitoring image sequence meets a preset condition, the vehicle is identified as a following vehicle.

4. The intelligent lane departure recognition method based on computer vision according to claim 1, characterized in that, The step of determining whether there are following vehicles and / or target obstacles based on environmental monitoring images includes: Based on the environmental monitoring image, the target lane where the target vehicle is located is obtained by analyzing the environmental monitoring image. Based on the target lane where the target vehicle is located, the image corresponding to the target lane is identified to obtain the lane identification result; When the lane recognition result includes abnormal objects and / or abnormal road conditions, the abnormal objects and / or abnormal road conditions are identified as target obstacles.

5. The intelligent lane departure recognition method based on computer vision according to claim 1, characterized in that, The step of predicting lane departure based on the cross-line trajectory and the distance between the target vehicle and the following vehicle includes: Based on the cross-line trajectory, the maximum value of the left and right displacement of the following vehicle is obtained; Lane departure prediction is obtained based on the maximum left-right displacement of the following vehicle and the distance between the target vehicle and the following vehicle.

6. The intelligent lane departure recognition method based on computer vision according to claim 1, characterized in that, The process of issuing a lane departure warning based on the lane departure prediction result and the position of the target vehicle relative to the target lane includes: The trajectory of the target vehicle is obtained based on the position of the target vehicle relative to the target lane and the speed of the target vehicle; When the trajectory of the target vehicle deviates from its lane, the similarity between the trajectory of the target vehicle and the lane deviation prediction result is determined. Lane departure warning is issued when the similarity does not meet the preset conditions.

7. An intelligent lane departure recognition system based on computer vision, characterized in that, Includes an onboard monitoring module and a lane departure detection module; The vehicle-mounted monitoring module is configured as follows: Based on road surveillance images, the position of the target vehicle relative to the target lane is obtained; Based on the environmental monitoring images, it is determined whether there are following vehicles and / or target obstacles. The following vehicles are configured as vehicles in the monitoring images that are in the same lane as the target vehicle for more than a preset time. The target obstacles are configured as obstacles in the monitoring images that are in the same lane as the target vehicle. The lane departure detection is configured as follows: When the following vehicle is present, lane departure prediction is performed based on the trajectory of the following vehicle; When the target obstacle is present, lane departure prediction is performed based on the position of the target obstacle; Based on the lane departure prediction results and the position of the target vehicle relative to the target lane, a lane departure warning is issued; When the following vehicle is present, the process of predicting lane departure based on the trajectory of the following vehicle includes: When the following vehicle is present, the position of the following vehicle relative to the target lane is obtained; The trajectory of the following vehicle is obtained based on its position relative to the target lane; Based on the trajectory of the following vehicle, determine whether the following vehicle has crossed the line. When the following vehicle crosses the lane, the behavior data of the following vehicle is used to determine whether the crossing behavior is a lane change or an evasive action. When the following vehicle's crossing behavior is an avoidance behavior, obtain the crossing trajectory of the following vehicle when it crosses the line; Lane departure prediction is performed based on the cross-line trajectory and the distance between the target vehicle and the following vehicle.

8. A computer device, characterized in that, The computer device includes a memory and a processor, wherein the memory stores a computer program and the processor executes the computer program to implement the method as described in any one of claims 1-6.

Images