Lane line pressing early warning method and device, electronic equipment and storage medium

Abstract

The invention discloses a lane line pressing early warning method and device, electronic equipment and a storage medium. The method comprises the following steps: determining a first road image; performing lane line detection on the first road image to obtain a local lane classification result; matching a second line pressing detection mode from the at least one first line pressing detection mode according to the local lane classification result; and detecting the line pressing condition of the vehicle according to the second line pressing detection mode. According to the method, the lane lines in the first road image are detected and classified, the second line pressing detection mode is matched according to the local lane classification result, and the line pressing condition of the vehicle is detected according to the second line pressing detection mode, so that the problems that the lane line types cannot be accurately distinguished and the vehicle line pressing judgment is easily misjudged and missed can be effectively solved; the accuracy of vehicle line pressing detection is improved, and meanwhile the driving safety of the vehicle can be guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of lane line detection technology, and in particular to a lane line crossing warning method, device, electronic device and storage medium. Background Art

[0002] Lane crossing detection, a foundational technology for autonomous driving systems, is crucial for ensuring safe lane keeping. It monitors the relative position of the vehicle and lane markings in real time. When a vehicle crosses a lane marking, it issues an alert or takes appropriate control measures to prevent the vehicle from straying from its lane and causing a collision.

[0003] Existing technologies have different methods for lane line crossing warning, but the general process is basically similar: first, based on the image data collected by the on-board camera, the lane line is detected through traditional image processing or deep learning technology, and then projection transformation or spatial fitting is used to determine whether the vehicle crosses the line. However, such methods are more dependent on the installation position of the camera. If the camera installation position or installation direction deviates from the vehicle's center axis, it will cause deviation in the detection of vehicle crossing the line. Summary of the Invention

[0004] The present invention provides a lane line crossing warning method, device, electronic device and storage medium to solve the problem of lane line detection deviation.

[0005] According to one aspect of the present invention, a lane line crossing warning method is provided, comprising:

[0006] Determine a first road image; the first road image is an image of lane lines obtained by photographing the road ahead of the vehicle;

[0007] Performing lane line detection on the first road image to obtain a local lane classification result; the local lane classification result is used to characterize the type of at least one lane line contained in the first road image;

[0008] matching a second line cross detection mode from at least one first line cross detection mode according to the local lane classification result; different line cross detection modes in the at least one first line cross detection mode corresponding to different detection logics;

[0009] The line-crossing condition of the vehicle is detected according to the second line-crossing detection mode.

[0010] According to another aspect of the present invention, a lane line crossing warning device is provided, comprising:

[0011] A first road image determination module is configured to determine a first road image; the first road image is an image of lane lines obtained by photographing the road ahead of the vehicle;

[0012] a local lane classification result determination module, configured to perform lane line detection on the first road image to obtain a local lane classification result; the local lane classification result is used to characterize the type of at least one lane line contained in the first road image;

[0013] a second line cross detection mode determination module, configured to match a second line cross detection mode from at least one first line cross detection mode according to the local lane classification result; different line cross detection modes in the at least one first line cross detection mode corresponding to different detection logics;

[0014] The detection module is used to detect the line-crossing situation of the vehicle according to the second line-crossing detection mode.

[0015] According to another aspect of the present invention, an electronic device is provided, comprising:

[0016] at least one processor; and

[0017] a memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the lane line crossing warning method described in any embodiment of the present invention.

[0019] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the lane line crossing warning method described in any embodiment of the present invention when executed.

[0020] The technical solution of the embodiment of the present invention is to determine a first road image; perform lane line detection on the first road image to obtain a local lane classification result, which can accurately identify the lane line type and provide a basis for determining the lane line detection mode; match a second lane line detection mode from at least one first lane line detection mode according to the local lane classification result, so that the obtained second lane line detection mode can be more adapted to the corresponding lane line type, thereby improving the accuracy of lane line detection; and detect the lane line situation of the vehicle according to the second lane line detection mode. This method detects and classifies the lane lines in the first road image, matches the second lane line detection mode according to the local lane classification result, and detects the lane line situation of the vehicle according to the second lane line detection mode. It can effectively overcome the problem of being unable to accurately distinguish the lane line type and the problem of easy misjudgment or omission of vehicle lane line judgment, improve the accuracy of vehicle lane line detection, and ensure the driving safety of the vehicle.

[0021] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 A flowchart of a lane line crossing warning method provided by an embodiment of the present invention;

[0024] Figure 2 A schematic diagram of a reference vehicle coordinate system provided by an embodiment of the present invention;

[0025] Figure 3 A schematic diagram of a virtual extension line of a vehicle body provided by an embodiment of the present invention;

[0026] Figure 4 A schematic diagram of a lane warning area provided by an embodiment of the present invention;

[0027] Figure 5 A schematic structural diagram of a lane line crossing warning device provided by an embodiment of the present invention;

[0028] Figure 6 A schematic diagram of the structure of an electronic device for implementing the lane line crossing warning method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0031] Figure 1 This is a flow chart of a lane line crossing warning method provided by an embodiment of the present invention. This embodiment is applicable to the case of detecting lane lines. The method can be executed by a lane line crossing warning device. The lane line crossing warning device can be implemented in the form of hardware and / or software. The lane line crossing warning device can be configured in any electronic device with network communication function. Figure 1 As shown, the method includes:

[0032] S110: Determine a first road image. The first road image is an image of lane lines obtained by photographing the road ahead of the vehicle.

[0033] Lane markings are lines on the road that mark different lanes. The virtual and real lane markings can be used to determine whether a vehicle can drive over the lane.

[0034] Exemplarily, the lane markings include at least one of the following: a solid line, a dashed line, a dashed-solid line, a fishbone line, and a zebra crossing.

[0035] Specifically, the road ahead of the vehicle is photographed by a photographing device, and the photographed image is converted into a vehicle reference coordinate system to obtain a first road image.

[0036] The vehicle reference coordinate system is as follows: Figure 2 As shown, the direction in front of the vehicle is the x direction; the left side of the vehicle is the y direction; and the direction perpendicular to the ground is the z direction.

[0037] The shooting device is pre-calibrated before shooting, that is, the homogeneous transformation matrix for converting the shooting device coordinate system to the vehicle reference coordinate system is determined.

[0038] Furthermore, the calibration method uses the Zhang Zhengyou camera calibration method. The specific steps of the Zhang Zhengyou camera calibration method are as follows: photographing the calibration object using a camera to obtain an image of the calibration object. Corner point detection is performed on the captured calibration object image to extract the pixel coordinates of the corner points of the calibration object in the image coordinate system. A correlation is established between the coordinates of the calibration object corner points in the world coordinate system and the pixel coordinates. This correlation is solved using the least squares method to obtain the optimal estimates of the intrinsic parameter matrix and extrinsic parameters. The calibration object can be a checkerboard calibration plate.

[0039] Among them, the relationship between the coordinates of the corner points of the calibration object in the world coordinate system and the pixel coordinates can be expressed as:

[0040]

[0041] Among them, (u, v) is the pixel coordinate of the corner point of the calibration object; (X, Y, 0) is the coordinate of the corner point of the calibration object in the world coordinate system; K is the intrinsic parameter matrix; R is the rotation matrix; t is the translation vector; and s is the scale factor.

[0042] Furthermore, after determining the intrinsic parameter matrix, a conversion relationship between the world coordinate system and the reference vehicle coordinate system is constructed, and a homogeneous transformation matrix for converting the camera coordinate system to the vehicle reference coordinate system is determined based on the conversion relationship between the world coordinate system and the reference vehicle coordinate system.

[0043] S120: Perform lane line detection on the first road image to obtain a local lane classification result. The local lane classification result is used to characterize the type of at least one lane line contained in the first road image.

[0044] The local lane classification results are divided into single lane markings and mixed lane markings. A single lane marking indicates that the first road image contains only one type of lane marking. A mixed lane marking indicates that the first road image contains at least two types of lane markings.

[0045] Specifically, the first road image is grayscaled and then segmented into regions of interest (ROIs) to obtain a second road image. Edge features are extracted from the second road image, and the lane marking type is determined based on the extracted edge features. The local lane marking classification result is determined based on the lane marking type. The ROI is the area containing the lane markings.

[0046] Furthermore, the step of determining the type of lane line based on edge features is as follows: the bottom area of the second road image and the top area of the road image of the previous frame of the second road image are cut out according to a preset range, and edge contour information within the cutout area is obtained, the obtained edge contour information is spliced, and the spliced lane line information is compared with the preset feature information, and the lane line type is determined according to the comparison result.

[0047] S130: Match a second line cross detection mode from at least one first line cross detection mode according to the local lane classification result. Different line cross detection modes in the at least one first line cross detection mode correspond to different detection logics.

[0048] The at least one first line pressing detection mode includes: a first candidate line pressing detection mode and a second candidate line pressing detection mode.

[0049] The first candidate lane crossing detection mode is used to process a single lane line. The first candidate lane crossing detection mode includes two processing methods: the processing method when the lane line is a dotted line and the processing method when the lane line is a solid line.

[0050] Furthermore, when the lane marking is a dotted line, the driver directly determines that the lane crossing result is not crossed. When the lane marking is a solid line, the driver obtains the lane marking anchor point and determines whether the anchor point falls within the lane marking warning area. The lane marking warning area is an extended area in front of the vehicle constructed based on the vehicle body width and the reference vehicle coordinate system.

[0051] The second candidate cross-line detection mode is used to process mixed lane markings. Compared to the first candidate cross-line detection mode, the second candidate cross-line detection mode includes an additional processing step: capturing the second road image and the previous frame of the second road image within a preset range, and further determining whether the lane marking within the captured area is a dotted line.

[0052] In the above steps, the second candidate line-crossing detection mode has one more processing step than the first candidate line-crossing detection mode because the mixed lane lines only affect the vehicle's line-crossing situation within a preset range in front of the vehicle, while the farther lane lines do not affect the vehicle's line-crossing situation at the current moment. Therefore, they need to be intercepted to avoid taking all of them into consideration and causing misjudgment of line crossing.

[0053] Furthermore, the second candidate lane crossing detection mode also includes: a zebra crossing processing mode, that is, if the lane line within the preset range is a zebra crossing, it is directly determined that the vehicle has not crossed the line.

[0054] The above steps include zebra crossing processing because zebra crossings are arranged as parallel solid lines on the road. Therefore, in order to avoid misjudgment, a separate judgment is required.

[0055] Specifically, if the local lane classification result is a single lane line, the first candidate line pressing detection mode is used as the second line pressing detection mode; if the local lane classification result is a mixed lane line, the second candidate line pressing detection mode is used as the second line pressing detection mode.

[0056] S140: Detect the line-crossing condition of the vehicle according to the second line-crossing detection mode.

[0057] Specifically, if the second lane line pressing detection mode is the first candidate lane line pressing detection mode, it is determined whether the lane line is a dotted line. If it is, the lane line pressing result is directly determined to be no lane line pressing; if not, the anchor point of the lane line is obtained, and it is determined whether the anchor point falls within the lane line warning area. If so, it is considered that the vehicle has crossed the line; if not, it is considered that the vehicle has not crossed the line.

[0058] Furthermore, if the second lane pressing detection mode is the second candidate lane pressing detection mode, the second road image and the previous frame image of the second road image are intercepted according to the preset range, and it is determined whether the lane line in the intercepted area is a dotted line. If it is, the lane line pressing result is not crossing the line; if not, the anchor point of the lane line is obtained, and it is determined whether the anchor point falls within the lane line warning area. If so, it is considered that the vehicle has crossed the line; if not, it is considered that the vehicle has not crossed the line.

[0059] Optionally, lane line detection is performed on the first road image to obtain local lane classification, including steps A1-A4:

[0060] Step A1: Determine a region of interest based on a first road image. The region of interest is an image region containing lane lines.

[0061] Specifically, grayscale processing is performed on the first road image, and an area where lane lines are located in the first road image is selected as a region of interest.

[0062] Step A2: cropping the first road image according to the region of interest to obtain a second road image.

[0063] Specifically, the first road image is cropped according to the region of interest, and the lane line image obtained by cropping is used as the second road image.

[0064] Step A3: extract edge features from the second road image to obtain first edge feature information.

[0065] The first edge feature information is used to characterize the shape and change of the lane line. Furthermore, the first edge feature information includes at least one of the following: lane line width, lane line length, lane line continuity, lane line outline, and lane line direction.

[0066] Specifically, edge feature extraction is performed on the second road image to obtain lane edge feature points, which are then connected to obtain closed shapes. The first edge feature information is determined based on changes in the size and width of the closed shapes, as well as changes between different closed shapes.

[0067] Furthermore, the edge feature extraction step is: calculating the difference between the grayscale value of each point in the second road image and the grayscale value of its adjacent points; if the difference is greater than or equal to a preset difference, the point is considered to be an edge point of the lane line; if the difference is less than the preset difference, the point is considered not to be an edge point of the lane line.

[0068] Furthermore, the first edge feature information is determined by connecting and closing the edge feature points to obtain a closed figure. The continuity of the lane line is determined based on the changes in continuity between the closed figures. That is, if there are multiple closed figures with equal spacing between them, the lane line is considered discontinuous; if there is only one closed figure, the lane line is continuous. The width and length of the lane line are determined based on the size of the closed figure. The directionality of the lane line is determined based on changes in its shape. For example, if the lane line becomes narrower as it moves in one direction, that direction is considered to be the extension direction of the lane line, i.e., the direction of travel of the vehicle.

[0069] Furthermore, the directionality of the lane line is determined based on the shape change of the closed figure because: when the camera captures the road in front of the vehicle during driving, the scenery in front of the vehicle will appear larger when near and smaller when far. That is, the lane line closer to the vehicle is closer to the actual width, and the lane line farther away from the vehicle is visually narrower. Therefore, the extension direction of the lane line can be obtained based on the shape change of the closed figure.

[0070] Step A4: Determine a local lane classification result based on the first edge feature information.

[0071] The local lane classification results are divided into single lane markings and mixed lane markings. A single lane marking indicates that the first road image contains only one type of lane marking. A mixed lane marking indicates that the first road image contains at least two types of lane markings.

[0072] Specifically, the first edge feature information is compared with the preset feature information, and the local lane classification result is determined according to the comparison result.

[0073] Furthermore, if the comparison result shows that the first edge feature information can represent at least two types of lane line information, the local lane classification result is a mixed lane line. If the comparison result shows that the first edge feature information can only represent one type of lane line information, the local lane classification result is a single lane line.

[0074] Optionally, determining a local lane classification result based on the first edge feature information includes steps B1-B3:

[0075] Step B1: Determine a first lane line type based on first edge feature information and preset feature information. The preset feature information is predetermined edge information corresponding to different lane lines.

[0076] Among them, the preset feature information is saved using a mapping relationship, that is, lane line type-lane line feature information.

[0077] Specifically, the first edge feature information is compared with the preset feature information. If the corresponding first edge feature information is matched from the preset feature information, the lane line type corresponding to the preset feature information is used as the first lane line type.

[0078] Step B2: Acquire second edge contour information from the second road image within a preset range, and determine the second lane line type based on the second edge contour information. The preset range includes a first preset range and a second preset range. The first preset range is a range selected from the bottom area of the road image with a preset size. The second preset range is a range selected from the top area of the road image with a preset size.

[0079] Specifically, the bottom area of the second road image is captured according to a first preset range, and edge contour information within the captured area is obtained. The top area of the road image in the frame preceding the second road image is captured according to a second preset range, and edge contour information within the captured area is obtained. The acquired edge contour information of the bottom area of the second road image and the acquired edge contour information of the top area of the road image in the frame preceding the second road image are concatenated to obtain second edge contour information. The second edge contour information is compared with the preset feature information, and the second lane line type is determined based on the comparison result.

[0080] Step B3: Determine a local lane classification result based on the first lane line type and the second lane line type.

[0081] The local lane classification results are divided into single lane markings and mixed lane markings. A single lane marking indicates that the first road image contains only one type of lane marking. A mixed lane marking indicates that the first road image contains at least two types of lane markings.

[0082] Specifically, if the first lane line type and the second lane line type are the same, the local lane classification result is a single lane line. If the first lane line type and the second lane line type are different, the local lane classification result is a mixed lane line.

[0083] The above steps and the method for determining the local lane line classification results can ensure that when a vehicle travels to an area where the lane line type changes, there will be no misjudgment of the lane line detection due to the vehicle blocking part of the lane line.

[0084] Optionally, obtaining second edge contour information from the second road image according to a preset range, and determining the second lane line type according to the second edge contour information, includes steps C1-C3:

[0085] Step C1: extract edge contour information of the second road image according to a first preset range to obtain third edge contour information.

[0086] Specifically, the area at the bottom of the second road image is cut out according to the first preset range, and edge contour information in the cut out area is extracted, and the extracted edge contour is used as the third edge contour information.

[0087] Step C2: extracting edge contour information of a previous frame of the second road image according to a second preset range to obtain fourth edge contour information.

[0088] Specifically, the previous frame image of the second road image is obtained, the top image of the previous frame image of the second road image is cut out according to the second preset range, and the edge contour information in the cutout area is extracted, and the extracted edge contour is used as the fourth edge contour information.

[0089] Step C3: splice the third edge contour information and the fourth edge contour information to obtain second edge contour information.

[0090] Specifically, the edge contour information is segment-extended starting from the tail anchor point of the third edge contour information in the reverse direction of the longitudinal axis, and the fourth contour information is spliced with the third edge contour information according to the position of the extended segment to obtain the second edge contour information.

[0091] Furthermore, the line segment is extended according to the reverse extension trend of the lane line in the second road image.

[0092] Step C4: Compare the second edge contour information with the preset feature information to obtain a second lane line type.

[0093] Specifically, the acquired second edge contour information is compared with the preset feature information, and the second lane line type is determined according to the comparison result.

[0094] The above steps, by considering the types of lane lines in the second road image and the frame image before the second road image, can avoid the situation where, when the vehicle travels to the intersection where the lane line type changes, the lane line in the second road image is a dotted line, but the lane line type of the part blocked by the vehicle is a solid line, and the vehicle is detected as having crossed the dotted line, and is directly judged as not having crossed the line, when in fact the vehicle has crossed the line. This improves the accuracy of lane crossing detection.

[0095] Optionally, detecting the line-crossing condition of the vehicle according to the second line-crossing detection mode includes steps D1-D3:

[0096] Step D1: If the local lane classification result is a single lane line, determine whether the lane line is a dotted line.

[0097] The single lane line indicates that the first road image contains only one type of lane line.

[0098] Specifically, if the local lane classification result is a single lane line, the type of the lane line is judged to determine whether the lane line is a dotted line.

[0099] Step D2: If it is a dotted line, the vehicle's crossing condition is that the vehicle does not cross the line.

[0100] Specifically, if the lane line is a dotted line, and the dotted line can be run over by the vehicle, the vehicle's line-crossing situation is that the vehicle does not cross the line.

[0101] Step D3: If the lane is not a dotted line or the partial lane classification result is a mixed lane line, determine the vehicle's lane crossing situation based on the lane line anchor point and the lane line warning area.

[0102] Among them, the lane line warning area is an extended area located in front of the vehicle constructed based on the vehicle body width and the reference vehicle coordinate system.

[0103] Specifically, if the lane marking is not a dotted line, indicating that the vehicle cannot cross the lane marking, the lane marking edge contour information is obtained from the second road image. The lane marking anchor points are determined based on the edge contour information. The lane marking anchor points are sorted along the vertical axis and projected into the first road image. The system then determines whether the projected anchor points fall within the lane marking warning area. If so, the vehicle has crossed the lane marking. If not, the vehicle has not crossed the lane marking.

[0104] Furthermore, if the local lane classification result indicates mixed lane markings, the lane marking warning area is expanded along the vertical axis. The second road image is then intercepted using this expanded area to obtain lane marking edge contour information within the expanded area. Based on this obtained edge contour information, the lane marking anchor points are determined. These lane marking anchor points are sorted along the vertical axis and projected onto the first road image. A determination is made as to whether the projected anchor points fall within the lane marking warning area. If so, the vehicle has crossed the lane marking. If not, the vehicle has not crossed the lane marking.

[0105] In the above steps, when the local lane classification result is a mixed lane line, the second road image is intercepted. This is because mixed lane lines include at least two types of lane lines, but the area where the vehicle can be proven to have crossed the line is only the part in front of the vehicle. Therefore, lane line interception is required to prevent other types of vehicle lines from affecting the crossing situation.

[0106] Optionally, determining the vehicle's lane crossing situation based on the lane anchor point and the lane warning area includes steps E1-E4:

[0107] Step E1: Sort the lane line anchor points in reverse order along the vertical axis to obtain a lane line anchor point set.

[0108] Specifically, the lane line anchor points are sorted in reverse order along the vertical axis, and the sorted lane line anchor points are projected into the first road image to obtain a lane line anchor point set.

[0109] Step E2: traverse each anchor point in the lane line anchor point set along the positive direction of the longitudinal axis, and determine whether each anchor point falls within the lane line warning area.

[0110] Specifically, each anchor point in the lane line anchor point set is traversed along the positive direction of the vertical axis, and the coordinate value of each anchor point is compared with the coordinate value corresponding to the boundary line of the lane line warning area. Based on the comparison result, it is determined whether each anchor point falls within the lane line warning area.

[0111] Step E3: If the vehicle falls into the lane line warning area, the vehicle crosses the lane line.

[0112] Specifically, if each anchor point falls within the lane line warning area, it indicates that the vehicle has crossed the line. The vehicle's line crossing situation is adjusted to vehicle line crossing, and a line crossing warning is issued.

[0113] Step E4: If the vehicle does not fall into the lane line warning area, the vehicle's lane crossing condition is that the vehicle does not cross the lane line.

[0114] Specifically, if each anchor point falls within the lane line warning area, it indicates that the vehicle has not crossed the line, and the vehicle's crossing status is adjusted to the vehicle has not crossed the line.

[0115] Furthermore, after detecting that the vehicle has not crossed the lane line, the system determines the distance between each anchor point on the lane line and the boundary of the lane line warning area, and issues a warning based on the distance. The vehicle's movement trend is determined by whether the vehicle is close to the lane line and is at risk of crossing the lane line.

[0116] The above steps determine the vehicle's lane crossing situation based on the coordinates of the anchor point and the lane line warning area. This can not only determine the accuracy of vehicle lane crossing detection, but also issue an early warning of impending lane crossing.

[0117] Optionally, at least one method for determining the lane warning area includes steps F1-F4:

[0118] Step F1: Measure the vehicle's body width.

[0119] Specifically, the width of the vehicle is measured to obtain the vehicle body width.

[0120] Furthermore, the vehicle width is measured before the vehicle leaves the factory and is stored in the vehicle's electronic control unit (ECU). When needed, it can be directly retrieved.

[0121] Step F2: Starting from the wheels on both sides of the vehicle and extending toward the front of the vehicle, a virtual extension line of the vehicle body is obtained.

[0122] The front of the vehicle is the positive direction of the horizontal axis.

[0123] Specifically, according to the vehicle body width, the coordinate points of the wheels on both sides of the vehicle are determined with the ground point at the center of the vehicle's rear axle as the origin, and the two obtained coordinate points are extended toward the front of the vehicle as the starting point to obtain the virtual extension line of the vehicle body.

[0124] For example, Figure 3 As shown in the figure, the virtual extension line of the vehicle body can be expressed in the reference vehicle coordinate system as follows:

[0125]

[0126] Among them, W v is the vehicle body width. y1 is the virtual extension line of the right side of the vehicle body. y2 is the virtual extension line of the left side of the vehicle body.

[0127] Step F3: In the reference vehicle coordinate system, determine the lane warning area based on the virtual extension line of the vehicle body.

[0128] Specifically, in the reference vehicle coordinate system, the virtual extension line of the vehicle body is intercepted according to a preset length, and the intercepted virtual extension line of the vehicle body is depicted to obtain the extension line, and the extension line area is projected into the first road image to obtain the lane line warning area.

[0129] Among them, the preset length is set according to actual needs, and it must not block the vehicle's front view while also being able to serve as a warning.

[0130] Furthermore, the obtained lane warning area is displayed on a display interface configured on the ECU. For example, Figure 4 As shown, the lane line warning area is the red area in the figure.

[0131] The above steps of determining the lane line warning area can improve the accuracy of lane crossing detection while also instructing the vehicle to travel along the normal driving route.

[0132] The technical solution of this embodiment is to determine a first road image; perform lane line detection on the first road image to obtain a local lane classification result, which can accurately identify the lane line type and provide a basis for determining the lane line detection mode; match a second lane line detection mode from at least one first lane line detection mode according to the local lane classification result, so that the obtained second lane line detection mode can be more adapted to the corresponding lane line type, thereby improving the accuracy of lane line detection; and detect the lane line situation of the vehicle according to the second lane line detection mode. This method detects and classifies the lane lines in the first road image, matches the second lane line detection mode according to the local lane classification result, and detects the lane line situation of the vehicle according to the second lane line detection mode. It can effectively overcome the problem of being unable to accurately distinguish the lane line type and the problem of easy misjudgment or omission of vehicle lane line judgment, improve the accuracy of vehicle lane line detection, and ensure the driving safety of the vehicle.

[0133] Figure 5 This is a schematic diagram of the structure of a lane line crossing warning device provided by an embodiment of the present invention. This embodiment is applicable to the case of detecting lane lines. The lane line crossing warning device can be implemented in the form of hardware and / or software. The lane line crossing warning device can be configured in any electronic device with network communication function. Figure 5 As shown, the device includes: a first road image determination module 210, a local lane classification result determination module 220, a second line pressure detection mode determination module 230 and a detection module 240, wherein:

[0134] The first road image determining module 210 is used to determine a first road image; the first road image is an image of lane lines obtained by photographing the road ahead of the vehicle;

[0135] The local lane classification result determination module 220 is configured to perform lane line detection on the first road image to obtain a local lane classification result; the local lane classification result is used to characterize the type of at least one lane line contained in the first road image;

[0136] A second line cross detection mode determination module 230 is configured to match a second line cross detection mode from at least one first line cross detection mode according to the local lane classification result; different line cross detection modes in the at least one first line cross detection mode correspond to different detection logics;

[0137] Detection module 240: used to detect the line-crossing situation of the vehicle according to the second line-crossing detection mode.

[0138] Optionally, the local lane classification result determination module 220 includes:

[0139] A region of interest determining unit is configured to determine a region of interest based on the first road image; the region of interest is an image region including lane lines;

[0140] A second road image determining unit is configured to perform image cropping on the first road image according to the region of interest to obtain a second road image;

[0141] A first edge feature information determining unit is configured to extract edge features from the second road image to obtain first edge feature information;

[0142] A local lane classification result determination unit is used to determine a local lane classification result based on the first edge feature information.

[0143] Optionally, a local lane classification result determination unit includes:

[0144] A first lane line type determination subunit is configured to determine the first lane line type based on the first edge feature information and preset feature information; the preset feature information is edge information corresponding to predetermined different lane lines;

[0145] A second lane line type determination subunit is configured to obtain second edge contour information from the second road image according to a preset range, and determine the second lane line type according to the second edge contour information; the preset range includes: a first preset range and a second preset range; the first preset range is a range selected according to a preset size in the bottom area of the road image; the second preset range is a range selected according to a preset size in the top area of the road image;

[0146] The local lane classification result determination subunit is used to determine the local lane classification result according to the first lane line type and the second lane line type.

[0147] Optionally, the second lane line type determination subunit is specifically configured to:

[0148] extracting edge contour information of the second road image according to the first preset range to obtain third edge contour information;

[0149] Extracting edge contour information of a previous frame of the second road image according to a second preset range to obtain fourth edge contour information;

[0150] splicing the third edge contour information and the fourth edge contour information to obtain second edge contour information;

[0151] The second edge contour information is compared with the preset feature information to obtain the local lane classification result.

[0152] Optionally, the detection module 240 includes:

[0153] Dashed line judgment unit: used to judge whether the lane line is a dashed line if the local lane classification result is a single lane line;

[0154] Vehicle not crossing the line determination unit: used for determining that the vehicle has not crossed the line if the line is a dotted line;

[0155] Lane crossing determination unit: used to determine the lane crossing situation of the vehicle based on the lane line anchor point and lane line warning area if the lane line is not a dotted line or the partial lane classification result is a mixed lane line.

[0156] Optionally, a line pressure determination unit may include:

[0157] Lane anchor point set determination subunit: used to sort the lane anchor points in reverse order along the vertical axis to obtain a lane anchor point set;

[0158] Anchor point judgment subunit: used to traverse each anchor point in the lane line anchor point set along the positive direction of the longitudinal axis and determine whether each anchor point falls within the lane line warning area;

[0159] Lane crossing situation determination subunit: used for determining the lane crossing situation of the vehicle as lane crossing if it falls into the lane warning area;

[0160] The lane crossing situation determination subunit is used to determine the lane crossing situation of the vehicle as not crossing the lane if the vehicle does not fall into the lane line warning area.

[0161] Optionally, a line pressure determination unit may include:

[0162] Body width determination subunit: used to measure the body width of the vehicle;

[0163] The vehicle body virtual extension line determination subunit is used to extend the vehicle body virtual extension line from the wheels on both sides of the vehicle to the front of the vehicle;

[0164] Lane warning area determination subunit: used to determine the lane warning area based on the virtual extension line of the vehicle body in the reference vehicle coordinate system.

[0165] The lane line crossing warning device provided in the embodiment of the present invention can execute the lane line crossing warning method provided in any embodiment of the present invention mentioned above, and has the corresponding functions and beneficial effects of executing the lane line crossing warning method. For detailed process, please refer to the relevant operations of the lane line crossing warning method in the above embodiment.

[0166] Figure 6A schematic diagram of the structure of an electronic device for implementing the lane line crossing warning method according to an embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0167] like Figure 6 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11. The memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. Various programs and data required for the operation of the electronic device 10 can also be stored in the RAM 13. The processor 11, ROM 12, and RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0168] Multiple components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0169] The processor 11 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any other suitable processor, controller, or microcontroller. The processor 11 executes the various methods and processes described above, such as the lane crossing warning method.

[0170] In some embodiments, the lane line crossing warning method can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the lane line crossing warning method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to execute the lane line crossing warning method by any other appropriate means (for example, by means of firmware).

[0171] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system comprising at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0172] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0173] In the context of the present invention, computer-readable storage media can be tangible media that can contain or store a computer program for use with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Computer-readable storage media can include but are not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, computer-readable storage media can be machine-readable signal media. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0174] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0175] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0176] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0177] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0178] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A lane line crossing warning method, characterized in that: include: Determine a first road image; the first road image is an image of lane lines obtained by photographing the road ahead of the vehicle; Performing lane line detection on the first road image to obtain a local lane classification result; The local lane classification result is used to characterize the type of at least one lane line included in the first road image; matching a second line cross detection mode from at least one first line cross detection mode according to the local lane classification result; different line cross detection modes in the at least one first line cross detection mode corresponding to different detection logics; The line-crossing condition of the vehicle is detected according to the second line-crossing detection mode.

2. The method according to claim 1, characterized in that The performing lane line detection on the first road image to obtain local lane classification includes: Determine a region of interest based on the first road image; the region of interest is an image region including lane lines; performing image cropping on the first road image according to the region of interest to obtain a second road image; performing edge feature extraction on the second road image to obtain first edge feature information; A local lane classification result is determined based on the first edge feature information.

3. The method according to claim 2, characterized in that Determining a local lane classification result according to the first edge feature information includes: Determining a first lane line type based on the first edge feature information and preset feature information; the preset feature information is edge information corresponding to predetermined different lane lines; Acquire second edge contour information from the second road image according to a preset range, and determine a second lane line type according to the second edge contour information; the preset range includes: a first preset range and a second preset range; the first preset range is a range selected according to a preset size in the bottom area of the road image; the second preset range is a range selected according to a preset size in the top area of the road image; A first lane line type is determined according to the first lane line type and the second lane line type.

4. The method according to claim 3, characterized in that The acquiring second edge contour information from the second road image according to the preset range, and determining the second lane line type according to the second edge contour information, includes: extracting edge contour information of the second road image according to the first preset range to obtain third edge contour information; extracting edge contour information of a previous frame of the second road image according to the second preset range to obtain fourth edge contour information; splicing the third edge contour information and the fourth edge contour information to obtain second edge contour information; The second edge contour information is compared with the preset feature information to obtain the lane line type.

5. The method according to claim 1, wherein The detecting the line-crossing condition of the vehicle according to the second line-crossing detection mode includes: If the local lane classification result is a single lane line, determine whether the lane line is a dotted line; If it is a dotted line, the vehicle's crossing condition is that the vehicle does not cross the line; If it is not a dotted line or the partial lane classification result is a mixed lane line, the vehicle's lane crossing situation is determined based on the lane line anchor point and lane line warning area.

6. The method according to claim 5, characterized in that Determining the vehicle's lane crossing situation based on the lane anchor point and the lane warning area includes: Sort the lane line anchor points in reverse order along the vertical axis to obtain a lane line anchor point set; Traverse each anchor point in the lane line anchor point set along the positive direction of the vertical axis and determine whether each anchor point falls within the lane line warning area; If the vehicle falls into the lane line warning area, the vehicle crosses the line; If the vehicle does not fall into the lane line warning area, the vehicle's lane crossing situation is that the vehicle does not cross the lane line.

7. The method according to claim 5, characterized in that At least one method for determining the lane warning area includes: Measure the vehicle's body width; Starting from the wheels on both sides of the vehicle and extending toward the front of the vehicle, a virtual extension line of the vehicle body is obtained; In the reference vehicle coordinate system, the lane line warning area is determined according to the virtual extension line of the vehicle body.

8. A lane line crossing warning device, characterized in that: include: A first road image determination module is configured to determine a first road image; the first road image is an image of lane lines obtained by photographing the road ahead of the vehicle; a local lane classification result determination module, configured to perform lane line detection on the first road image to obtain a local lane classification result; The local lane classification result is used to characterize the type of at least one lane line included in the first road image; a second line cross detection mode determination module, configured to match a second line cross detection mode from at least one first line cross detection mode according to the local lane classification result; different line cross detection modes in the at least one first line cross detection mode corresponding to different detection logics; The detection module is used to detect the line-crossing situation of the vehicle according to the second line-crossing detection mode.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the lane line crossing warning method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the lane line crossing warning method according to any one of claims 1 to 7 when executed.

Images