Vehicle lane marking recognition device

Abstract

To provide a lane marking recognition device for a vehicle capable of properly recognizing a lane marking regardless of gray color of the lane marking relative to a road surface.SOLUTION: A lane marking recognition device 4 for a vehicle includes: edge detection means for detecting an edge point P based on luminance change in a horizontal direction within a detection area of a lane marking set in an image obtained by imaging an own vehicle traveling environment; and approximate line calculation means for calculating approximate lines Ll and Lr of a point group composed of edge points. The lane marking recognition device for a vehicle has a first mode for mainly searching for a standard edge candidate point PS with high luminance relative to a road surface and a second mode for mainly searching for a reverse edge candidate point PO with low luminance relative to the road surface, and selectively switches between the first mode and the second mode depending on the number of detected reverse edge candidate points PO.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present invention relates to a lane marking recognition device for a vehicle that recognizes lane markings based on images captured by an on-board camera. [Background technology]

[0002] In recent years, various driving support devices have been developed to support the driver's driving operation. In general, in order to realize a lane departure prevention function, the driving support devices recognize white lines based on a captured image of the area in front of the vehicle. Lane The lane in which the vehicle is traveling is estimated based on the dividing lines.

[0003] Regarding such white line recognition technology, for example, Patent Document 1 discloses a white line recognition device for vehicles that can perform appropriate white line recognition by eliminating the influence of light images projected onto the road surface. This conventional white line recognition device for vehicles detects edge points on a search line where the brightness changes in a predetermined manner, and extracts edge points where the brightness changes from dark to light as white line start points and edge points where the brightness changes from light to dark as white line end points to recognize white lines. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2019-79470 A Summary of the Invention [Problem to be solved by the invention]

[0005] In countries other than Japan, lane markings with various color schemes are used. For example, in the United States, lane markings are made up of black lines between dashed white lines in the direction of travel.

[0006] However, conventional lane marking recognition technology uses lighter colored lane markings than the road surface. LaneOnly lane markings can be recognized, and the color is darker than the road surface, such as black. Lane Since it is not possible to recognize lane markings, if the white lines are worn, Lane This reduces the reliability of lane marking recognition, which can lead to the cancellation of steering control. Lane In the case of only dividing lines, Lane There was an issue with not being able to recognize the dividing lines at all.

[0007] In view of the above circumstances, the present invention provides a method for detecting a road surface. Lane Suitable for all shades of markings Lane The object of the present invention is to provide a vehicle lane marking recognition device capable of recognizing marking lines. [Means for solving the problem]

[0008] A lane marking recognition device for a vehicle according to one aspect of the present invention includes an edge point detection means for detecting edge points based on a horizontal luminance change within a detection area of ​​a lane marking set in an image captured of an environment in which the vehicle is traveling, and an approximation line calculation means for calculating an approximation line of a point cloud made up of the edge points, In a first predetermined area of ​​the image, Standard edge candidate points with high brightness relative to the road surface and the reverse edge candidate points with low brightness relative to the road surface A first mode of searching for In a second predetermined range of the image that is wider than the first predetermined range, the standard edge candidate points and the and a second mode for searching for reverse edge candidate points, and selectively switching between the first mode and the second mode depending on the number of detected reverse edge candidate points. Effect of the Invention

[0009] According to the present invention, Lane It is possible to provide a vehicle lane marking recognition device that can recognize marking lines regardless of the shade of the color. [Brief description of the drawings]

[0010] [Figure 1] Schematic diagram of a vehicle driving assistance device [Diagram 2] FIG. 1 is an explanatory diagram illustrating an example of a captured image of an environment outside a vehicle; [Diagram 3]FIG. 13 is an explanatory diagram showing segments set for each mode and a point cloud based on the starting points of lane markings detected from an image. [Figure 4] FIG. 13 is a diagram showing an example of the transition of luminance and luminance differential values ​​at the start and end points of the division line of an odd-numbered search line; [Diagram 5] FIG. 13 is a diagram showing an example of the transition of the luminance and the luminance differential value at the start and end points of the division line of the even-numbered search line; [Figure 6] Flowchart showing lane marking recognition routine in standard edge search mode [Figure 7] FIG. 13 is an explanatory diagram showing a first segment set in the standard edge search mode and showing a point cloud of the starting points of the lane markings detected from the image; [Figure 8] Flowchart showing a lane marking recognition routine in the inverse edge search mode [Figure 9] FIG. 13 is an explanatory diagram showing a second segment set in a reverse edge search mode and showing a point cloud based on the starting points of the lane markings detected from an image; [Figure 10] Flowchart showing cancellation decision of reverse edge candidate point [Figure 11] FIG. 1 is an explanatory diagram illustrating an example of a captured image of an external environment of a vehicle in which tire marks are present near lane markings; [Figure 12] FIG. 12 is a diagram showing an example of the transition of the luminance and the differential value of the luminance of the lane markings of the odd-numbered search lines in FIG. 11. [Figure 13] FIG. 12 is a diagram showing an example of the transition of the luminance and the differential value of the luminance of the lane markings of the even-numbered search lines in FIG. 11. [Figure 14] FIG. 1 is an explanatory diagram illustrating an example of a captured image of an external environment of a vehicle in which dark lines such as black border lines are present at the edges of lane markings that are lighter in color than the road surface; [Figure 15] A graph showing an example of the transition of the luminance and the differential value of the luminance of the lane markings in FIG. 14 . [Figure 16] FIG. 1 is an explanatory diagram illustrating an example of a captured image of an external environment of a vehicle in which a dark line such as a black line runs through the center of lane markings that are lighter in color than the road surface, such as a carpool lane; [Figure 17] 17 is a graph showing an example of the transition of the luminance and the differential value of the luminance of the lane markings in FIG. 16 . DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0012] In Fig. 1, reference numeral 1 denotes a vehicle (host vehicle) such as an automobile, and this vehicle 1 is equipped with a driving support device 2. The driving support device 2 is configured with, for example, a stereo camera 3, a stereo image recognition device 4, a control unit 5, and the like.

[0013] The vehicle 1 is also provided with a vehicle speed sensor 11 which detects the vehicle speed, a yaw rate sensor 12 which detects the yaw rate, a main switch 13 which switches each function of the driving assistance control ON / OFF, a steering angle sensor 14 which is disposed opposite a steering shaft connected to the steering wheel and detects the steering angle, and an accelerator opening sensor 15 which detects the amount of accelerator pedal depression (accelerator opening) by the driver.

[0014] The stereo camera 3 is composed of a pair of cameras using solid-state imaging elements such as charge-coupled devices (CCD or CMOS) as a stereo optical system. The left and right cameras are attached at a certain distance from each other on the front ceiling of the vehicle interior, and capture stereo images of objects outside the vehicle from different viewpoints and output image data to the stereo image recognition device 4.

[0015] In the following description, one of the stereoscopically captured images (for example, the right image) is referred to as a reference image, and the other image (for example, the left image) is referred to as a comparison image.

[0016] The stereo image recognition device 4 first divides the reference image into small regions of, for example, 4×4 pixels, compares the brightness or color pattern of each small region with the comparison image to find corresponding regions, and obtains the distance distribution over the entire reference image.

[0017] Furthermore, the stereo image recognition device 4 checks the brightness difference between each pixel on the reference image and adjacent pixels, extracts those pixels whose brightness difference exceeds a threshold as edges, and adds distance information to the extracted pixels (edges) to generate a distribution image of edges (distance image) equipped with distance information.

[0018] Then, based on the generated distance image, the stereo image recognition device 4 recognizes the lane markings LL, LR (see Figure 2), road edges, side walls, three-dimensional objects, etc. ahead of the vehicle, assigns a different ID to each recognized data, and continuously monitors these for each ID between frames.

[0019] Here, in this embodiment, lane markings LL, LR (see FIG. 2) are a general term for lines that extend on the road and divide the lane into which the vehicle is traveling, such as a single line or a multiple line (such as a double line) with a visual guidance line added inside the lane markings LL, LR. The form of each line may be a solid line, a dashed line, or the like, and further includes white lines or yellow lines that are lighter in color than the road surface, or black lines that are darker in color than the road surface, as well as colored lines of various colors.

[0020] In addition, in the recognition of the lane markings LL and LR in this embodiment, even if the actual lines on the road are double lines, they are recognized as being approximately a single straight line or curved line on the left and right.

[0021] When recognizing such lane markings LL, LR, the stereo image recognition device 4 detects one lane marking start point P for each search line Jn based on the brightness changes on multiple search lines Jn set in the horizontal direction (vehicle width direction) within the lane marking search areas AL, AR set on the image based on the previous processing (see Figure 3).

[0022] That is, the stereo image recognition device 4 detects the lane line starting point P as the edge point of the lane line, for example, by examining the change in brightness value of each pixel on each search line Sn from the inside to the outside in the vehicle width direction within each of the left and right lane line search areas AL, AR set on the reference image.

[0023] In addition, when recognizing lane markings LL, LR, the stereo image recognition device 4 sets two segments R1, R2 of different predetermined areas on the reference image depending on the edge search mode, and alternately detects reverse edge candidate points PO and standard edge candidate points PS in the predetermined Jth row of the search line Jn in these two segments R1, R2.

[0024] Of these two segments, in a standard edge searching mode, which will be described later, of the two edge searching modes, a first segment R1 having a range of, for example, 4 to 5 segments (approximately 16 to 20 [m]) is set.

[0025] In the reverse edge search mode, which will be described later, of the two edge search modes, a second segment R2 is set that is wider than the first segment R1, for example, a range of 2 to 12 segments (approximately 8 to 48 m). Note that the second segment R2 here is the entire area on the reference image for detecting edge candidate points of the lane markings LL and LR.

[0026] The stereo image recognition device 4 also calculates approximation lines for each point cloud consisting of the lane line start points P recognized in each of the left and right lane line search areas AL, AR. Furthermore, the stereo image recognition device 4 sets new lane line search areas AL, AR to be used for recognizing the lane line LL, LR in the next frame, based on the calculated lane line LL, LR.

[0027] The left and right lane line search areas AL, AR are set to a range of 2 to 12 segments (approximately 8 to 48 m) ahead of the vehicle 1 in the image data of the stereo images captured by the stereo camera 3. In other words, the range for detecting edge candidate points of the lane lines LL, LR is within the left and right lane line search areas AL, AR in a range of approximately 8 to 48 m ahead of the vehicle 1.

[0028] Thus, in this embodiment, the stereo image recognition device 4 has the function of a lane marking recognition device for a vehicle (host vehicle 1), and realizes the functions of an edge point detection (search) means, an approximation line calculation means, a marking line search means, a marking line calculation means, and a detection area setting means.

[0029] The control unit 5 receives information about the driving environment in front of the vehicle 1 recognized by the stereo image recognition device 4. Furthermore, the control unit 5 receives information about the driving environment of the vehicle 1, such as the vehicle speed from a vehicle speed sensor 11 and the yaw rate from a yaw rate sensor 12, as well as information about the driver's operation, such as an operation signal from a main switch 13, a steering angle from a steering angle sensor 14, and an accelerator opening from an accelerator opening sensor 15.

[0030] Then, for example, when the driver operates the main switch 13 to instruct the execution of an ACC (Adaptive Cruise Control) function, which is one of the driving assistance control functions, the control unit 5 reads the direction of the preceding vehicle recognized by the stereo image recognition device 4, and identifies whether or not the preceding vehicle to be followed is traveling on the vehicle's driving path.

[0031] As a result, if no preceding vehicle to be followed is detected, constant speed driving control is executed to maintain the vehicle speed of the vehicle 1 at the set vehicle speed set by the driver through opening / closing control of the throttle valve 16 (engine output control).

[0032] On the other hand, when a preceding vehicle to be followed is detected and the vehicle speed of the preceding vehicle is equal to or lower than a set vehicle speed, a following driving control is executed to follow the preceding vehicle while converging the inter-vehicle distance to a target inter-vehicle distance.

[0033] During this following cruise control, the control unit 5 basically causes the distance from the preceding vehicle to converge to a target distance through opening / closing control (engine output control) of the throttle valve 16. Furthermore, if it is determined that sufficient deceleration cannot be obtained by controlling the throttle valve 16 alone due to a sudden deceleration of the preceding vehicle, the control unit 5 also uses control of the output hydraulic pressure from the active booster 17 (automatic brake intervention control) to cause the distance to converge to the target distance.

[0034] In addition, when the driver operates the main switch 13 to instruct the execution of the lane departure prevention function, which is one of the driving assistance control functions, the control unit 5 sets an alarm judgment line based on the left and right lane dividing lines that define the vehicle's driving lane, and estimates the vehicle's travel path based on the vehicle speed and yaw rate of the vehicle 1.

[0035] Then, when the control unit 5 determines that the vehicle's travel path crosses either the left or right warning judgment line within a set distance (e.g., 10 to 16 m) ahead of the vehicle, it determines that there is a high possibility that the vehicle 1 will deviate from the current lane in which the vehicle is traveling, and issues a lane departure warning.

[0036] Furthermore, when the driver operates the main switch 13 to instruct execution of an Active Lane Keep Centering (ALKC) function, which is one of the driving assistance control functions, the control unit 5 sets the host vehicle target travel path, for example, in the center of the left and right lane markings LL and LR that define the host vehicle travel lane. Then, the control unit 5 performs travel control of the vehicle 1 to follow the host vehicle target travel path.

[0037] Here, the contents executed by the stereo image recognition device 4 when detecting and recognizing the lane markings LL and LR will be described.

[0038] First, the stereo image recognition device 4 reads the left and right lane line search areas AL and AR set in the previous frame. Then, the stereo image recognition device 4: LaneWhen detecting the edge of, for example, a search line Jn1 (see Figure 2) where the marking lines LL, LR are a lighter color (white, yellow, etc.) than the road surface, the marking line starting point P is detected within the left and right marking line search areas AL, AR, for example, from the inside to the outside in the vehicle width direction based on the image center line on the reference image (or the direction of vehicle travel estimated from the steering angle, etc.).

[0039] Specifically, the stereo image recognition device 4 detects a search line J In the search from the inside to the outside in the vehicle width direction on n1, as shown in FIG. 4, the first point where the brightness of the pixel on the outside in the vehicle width direction is relatively high compared to the brightness of the pixel on the inside and the differential value of the brightness indicating the amount of change is equal to or greater than a preset threshold value (brightness threshold value) on the positive side is detected as a standard edge candidate point PS as a positive edge point, and the edge candidate Point PS is recognized as the starting point P of the lane marking.

[0040] By such processing, for example, the lane markings LL and LR to be recognized are recognized as light-colored lines such as white lines or yellow lines that are lighter in color than the road surface. In other words, if the differential value of the luminance is not equal to or greater than the set threshold value on the positive side, the stereo image recognition device 4 excludes the lane markings LL and LR from the standard edge candidate points PS and does not recognize the marking line starting point P.

[0041] The stereo image recognition device 4 calculates the distance L between the standard edge candidate point PS, which is the edge start point where the brightness differential value is on the positive side, and the edge end point where the brightness differential value is on the negative side. b If the difference between the line width and the lane markings LL and LR is less than a preset distance (line width threshold), for example, 7 cm, the lane markings LL and LR are excluded from recognition.

[0042] In addition, the stereo image recognition device 4 LaneWhen detecting edges on search line Jn2 (see Figure 2), for example, where the marking lines LL and LR are darker (such as black) than the road surface, in a search from the inside to the outside in the vehicle width direction, the first point where the brightness of the pixels on the outside in the vehicle width direction is relatively lower than the brightness of the pixels on the inside, and where the differential value of the brightness indicating the amount of change is below a predetermined threshold value (brightness threshold) on the negative side, is detected as a candidate point PO for the reverse edge as a negative edge point, and this edge point PO is recognized as the starting point P of the marking line.

[0043] By such processing, for example, the lane markings LL and LR to be recognized are recognized as lines with a darker color such as a black line than the road surface. In other words, if the differential value of the luminance is not equal to or less than the set threshold value on the negative side, the stereo image recognition device 4 excludes the lane markings LL and LR from the reverse edge candidate points PO and excludes the lane marking start point P from recognition.

[0044] Furthermore, the stereo image recognition device 4 excludes the recognition of lane markings LL and LR when the distance Lb between the standard edge candidate point PO at the edge start point where the brightness derivative value is on the negative side and the edge end point where the brightness derivative value is on the negative side is less than a predetermined distance (line width threshold) set in advance, for example, 7 cm.

[0045] Then, the stereo image recognition device 4 integrates the point clouds of the multiple marking line start points P of the standard edge candidate points PS and the inverse edge candidate points PO, and performs lane marking line calculation based on the selected point cloud. Through the lane marking line calculation, the stereo image recognition device 4 recognizes, for example, an approximation line obtained by a Hough transform of the selected point cloud as approximation lines Ll, ​​Lr representing the lane marking lines LL, LR.

[0046] Then, the stereo image recognition device 4 sets the lane line search areas AL, AR to be used in the next frame based on the calculated lane lines LL, LR (approximation lines Ll, ​​Lr), and similarly detects the lane lines LL, LR for the next frame.

[0047] In this way, the stereo image recognition device 4 alternates between detecting lane markings LL, LR which are light-colored lines such as white lines or yellow lines, and detecting lane markings LL, LR which are dark-colored lines such as black lines, by detecting multiple search lines Jn of the first segment R1 or the second segment R2 according to the mode.

[0048] Here, the control routine in the standard edge search mode for recognizing the lane markings LL and LR by the above-mentioned stereo image recognition device 4 will be described with reference to the flowchart in Fig. 6. Note that this routine is repeatedly executed each time a new image is captured when the standard edge search mode is selected.

[0049] In the standard edge search mode, the stereo image recognition device 4 first determines whether or not the search has been performed on all the search lines Jn (S1). If the search for all search lines Jn has not been performed, The stereo image recognition device 4 selects a search line Jn (S2).

[0050] When the standard edge search mode is started, the stereo image recognition device 4 selects the first search line J1 (see FIG. 7) in the vicinity of the vehicle 1. In this step S2, the stereo image recognition device 4 selects the next search line Jn+1 in sequence from the near side to the far side in the routine after searching for edge candidate points of the Jth search line Jn.

[0051] That is, the selection of this search line Jn involves, for example, sequentially selecting a pair of search lines Jn (search lines extending horizontally) existing in each lane line search area AL, AR from the bottom on the image, and the stereo image recognition device 4 selects the search line Jn+1 that is one line above each search line Jn on which candidate points were previously detected.

[0052] Next, the stereo image recognition device 4 judges whether or not the currently selected search line Jn is present in the area of ​​the first segment R1 set on the image (S3). If the search line Jn in the area of ​​the first segment R1 is selected, the stereo image recognition device 4 judges whether or not the search line Jn is an even row (S4).

[0053] In step S4, the stereo image recognition device 4 detects the reverse edge candidate point PO when the search line Jn in the area of ​​the first segment R1 is an even-numbered line, for example, the search lines J14, J16, and J18 shown in Fig. 7 (S5). Note that the number of even-numbered search lines Jn in the first segment R1 here is just an example.

[0054] As described above, this reverse edge candidate point PO is detected by recognizing the reverse edge candidate point PO whose negative brightness differential value is below a threshold value as the starting point P of the lane markings LL, LR, which are dark lines on the road surface such as black lines.

[0055] On the other hand, in step S3, if the search line Jn in the area of ​​the first segment R1 is an odd-numbered line, not an even-numbered line, for example, the search lines J13, J15, and J17 shown in Fig. 7, the stereo image recognition device 4 detects the standard edge candidate point PS (S6), exits the routine, and returns to step S1. Note that the number of odd-numbered search lines Jn in the first segment R1 here is just an example.

[0056] As described above, the standard edge candidate points PS are detected by recognizing the standard edge candidate points PS whose positive brightness differential value is greater than or equal to a threshold value as the starting points P of the lane markings LL and LR, which are light-colored lines on the road surface such as white lines or yellow lines.

[0057] Next, the stereo image recognition device 4 judges whether or not the reverse edge candidate point PO is detected (S7), and if the reverse edge candidate point PO is detected, executes a cancellation judgment of the reverse edge candidate point PO, which is a subroutine described later (S8). Note that if the reverse edge candidate point PO is not detected, the stereo image recognition device 4 leaves the routine and returns to step S1.

[0058] In step S8, if the stereo image recognition device 4 recognizes a reverse edge candidate point PO, it counts the reverse edge candidate point PO, and if it cancels the reverse edge candidate point PO, it executes a process of not counting the reverse edge candidate point PO (S9), exits the routine, and returns to step S1.

[0059] After searching all search lines Jn as determined in step S1, the stereo image recognition device 4 determines whether the number of reverse edge candidate points PO counted in step S9 is equal to or greater than a predetermined first set number (S10).

[0060] When the stereo image recognition device 4 determines that the number of reverse edge candidate points PO is equal to or greater than a first set number, it increments a counter C (C←C+1) (S11) and determines whether the counter C is equal to or greater than a first threshold value Cth1 of preset counter threshold values ​​(S12).

[0061] If the stereo image recognition device 4 determines that the counter C is greater than or equal to the first threshold value, it switches the recognition mode of the lane markings LL, LR to a reverse edge search mode (S13), calculates approximation lines Ll, ​​Lr based on the detected lane marking start point P (S14), sets lane marking search areas AL, AR for the next frame based on the calculated approximation lines Ll, ​​Lr (S15), exits the routine, and returns to step S1.

[0062] If the stereo image recognition device 4 determines that the counter C is less than the first threshold value Cth1, it calculates the approximation lines Ll, ​​Lr in step S14, sets the lane line search areas AL, AR in step S15, exits the routine, and returns to step S1.

[0063] Next, regarding the recognition of the lane markings LL and LR by the stereo image recognition device 4, a control routine in the reverse edge search mode will be described with reference to the flowchart in Fig. 8. Note that this routine is repeatedly executed every time a new image is captured when the reverse edge search mode is selected.

[0064] In the reverse edge search mode, the stereo image recognition device 4 determines whether or not all search lines Jn have been searched (S21), as in the standard edge search mode, and selects a search line Jn (S22).

[0065] In addition, even when starting the reverse edge search mode, the stereo image recognition device 4 selects the first search line J1 (see Figure 9) near the vehicle 1, and selects the search line Jn+1 one line above each search line Jn on which candidate points were detected last time.

[0066] Next, the stereo image recognition device 4 judges whether or not the currently selected search line Jn is present in the area of ​​the second segment R2 set on the image (S23). If the search line Jn in the area of ​​the second segment R2 is selected, the stereo image recognition device 4 judges whether or not the search line Jn is an even-numbered row (S24).

[0067] In step S24, the stereo image recognition device 4 detects the reverse edge candidate point PO when the search line Jn in the area of ​​the second segment R2 is an even-numbered line, for example, the search lines J2, J4, . . . J14, J16, J18, . . . J28 shown in Fig. 9 (S25). Note that the number of even-numbered search lines Jn in the second segment R2 here is just an example.

[0068] As described above, this reverse edge candidate point PO is detected by recognizing the reverse edge candidate point PO whose negative brightness differential value is below a threshold value as the starting point P of the lane markings LL, LR, which are dark lines on the road surface such as black lines.

[0069] On the other hand, if the search line Jn in the area of ​​the second segment R2 is an odd-numbered line, not an even-numbered line, for example, the search lines J1, J3, . . . J13, J15, J17, . . . J27 shown in Fig. 9, the stereo image recognition device 4 detects the standard edge candidate point PS (S26), exits the routine, and returns to step S21. Note that the number of odd-numbered search lines Jn in the second segment R2 here is just an example.

[0070] As described above, the detection of the standard edge candidate point PS recognizes the standard edge candidate point PS whose positive brightness differential value is equal to or greater than a threshold value as the starting point P of the lane marking lines LL, LR, which are light-colored lines on the road surface such as white lines or yellow lines.

[0071] Next, the stereo image recognition device 4 judges whether or not the reverse edge candidate point PO is detected (S27), and if the reverse edge candidate point PO is detected, executes a cancellation judgment of the reverse edge candidate point PO, which is a subroutine described later (S28). Note that if the reverse edge candidate point PO is not detected, the stereo image recognition device 4 exits the routine and returns to step S21.

[0072] If the stereo image recognition device 4 recognizes a reverse edge candidate point PO in step S28, it counts the reverse edge candidate point PO, and if it cancels the reverse edge candidate point PO, it executes a process of not counting the reverse edge candidate point PO (S29), exits the routine, and returns to step S21.

[0073] After searching all search lines Jn as determined in step S21, the stereo image recognition device 4 determines whether the number of reverse edge candidate points PO counted in step S29 is equal to or less than a predetermined second set number (S30).

[0074] When the stereo image recognition device 4 determines that the number of reverse edge candidate points PO is equal to or less than the second set number, it decrements the counter C (C←C-1) (S31) and determines whether the counter C is equal to or less than a second threshold value Cth2, which is a preset counter threshold value (S32).

[0075] If the stereo image recognition device 4 determines that the counter C is less than or equal to the second threshold Cth2, it switches the recognition mode of the lane markings LL, LR to the standard edge search mode (S33), calculates approximation lines Ll, ​​Lr based on the detected marking start point P (S34), sets the marking line search areas AL, AR for the next frame based on the calculated approximation lines Ll, ​​Lr (S35), exits the routine, and returns to step S21.

[0076] If the stereo image recognition device 4 determines that the counter C is greater than (larger than) the second threshold Cth2, it calculates the approximation lines Ll, ​​Lr in step S34, sets the lane line search areas AL, AR in step S35, exits the routine, and returns to step S21.

[0077] Next, a control example of the subroutine executed by the stereo image recognition device 4 in step S8 of the standard edge search mode or step S28 of the reverse edge search mode for determining whether to cancel the reverse edge candidate point will be described with reference to the flowchart in Fig. 10. As described above, this subroutine is executed only when the search line Jn is an even-numbered row, and is not executed when searching for the standard edge candidate point PS of an odd-numbered row.

[0078] In determining whether or not to cancel a reverse edge candidate point, the stereo image recognition device 4 first determines whether or not the differential value of the minus side luminance of the detected reverse edge candidate point PO is equal to or smaller than a specified value (S41).

[0079] When the negative brightness differential value of the detected reverse edge candidate point PO is equal to or less than a specified value (brightness threshold), the stereo image recognition device 4 proceeds to step S48 and excludes (cancels) the recognition of the reverse edge candidate point PO. Therefore, the stereo image recognition device 4 does not count the reverse edge candidate points PO in the process of step S9 in the standard edge search mode and step S29 in the reverse edge search mode.

[0080] An example of a situation in which a point is not counted as a reverse edge candidate point PO is when tire marks TM are present near lane markings LL, LR and the tire marks TM are detected in lane marking search areas AL, AR, as shown in Fig. 11. Although tire marks TM are shown as an example here, this includes all detected objects that are darker in color than the road surface, such as dirt and stains on the road surface.

[0081] For example, when a search line Jn1 on lane markings LL and LR, which are light-colored lines on a road surface such as a white line or a yellow line, is an even-numbered line, a start point PO1 of a reverse edge point of a tire mark TM is detected. Because the differential value of the luminance of this start point PO1 is a positive value less than a specified value, the stereo image recognition device 4 excludes (cancels) recognition as a reverse edge candidate point PO (S48). As a result, all detected objects, such as tire marks TM, that are darker than the road surface and are not lane markings LL and LR are excluded.

[0082] Furthermore, if the negative brightness differential value of the start point PO1 of the detected reverse edge point is not equal to or less than a specified value (brightness threshold), the stereo image recognition device 4 determines whether the difference between the average brightness of the reverse edge candidate points PO and the brightness value immediately after the end point of the reverse edge point is small (S42).

[0083] In step S42, if the difference between the average brightness of the reverse edge candidate point PO and the brightness value immediately after the end point of the reverse edge is small, the stereo image recognition device 4 proceeds to step S48 and excludes (cancels) the start point PO1 of the reverse edge point from being recognized as the reverse edge candidate point PO.

[0084] As an example of a situation in which the reverse edge candidate point PO is not recognized here, when the search line Jn2 on the lane markings LL, LR, which are dark lines against the road surface such as a black line, is an even row as shown in FIG. 11, the tire mark TM and multiple, here two, reverse edge points PO1, PO2 of the lane marking line LL are detected as shown in FIG. 13.

[0085] Here again, since the differential brightness value of the starting point PO1 of the reverse edge point of the tire mark TM is a value on the positive side of the specified value, the stereo image recognition device 4 proceeds to step S48 and excludes (cancels) the starting point PO1 of the reverse edge point from the recognition of the reverse edge candidate point PO.

[0086] On the other hand, in step S42 of Fig. 10, if the difference between the average luminance of the reverse edge candidate point PO and the luminance value immediately after the positive edge point that is the end point is not small (large), the stereo image recognition device 4 judges whether the difference between the luminance (road surface luminance) immediately before the start point PO1 of the reverse edge point and the luminance immediately after the end point PS1 of the reverse edge point is large (S43). Here, for example, in the case of lane markings LL and LR that have dark black border lines BE on both sides of a white line as shown in Fig. 14, the luminance of the road surface and the luminance of the lane markings LL and LR are different, so the difference in luminance between the start point PO1 of the reverse edge point and the luminance value at the end point PS1 of the reverse edge point is large.

[0087] Therefore, if the difference in luminance value between the start point PO1 of the reverse edge point and the end point PS1 of the reverse edge point is large, the stereo image recognition device 4 proceeds to step S48 and excludes (cancels) the start point PO1 of the reverse edge point from the recognition of the reverse edge candidate point PO.

[0088] When the difference in luminance between the start point PO1 of the reverse edge point of the reverse edge and the end point PS1 of the reverse edge point is not large (small), the stereo image recognition device 4 determines the distance between the start point PO1 of the reverse edge point and the end point PS1 of the reverse edge point. Lb is a preset value (line width) For example, it is determined whether the distance is 7 cm or less (S44).

[0089] 10, if the distance between the start point PO1 of the reverse edge point and the end point PS1 of the reverse edge point is equal to or less than a predetermined value, the stereo image recognition device 4 proceeds to step S48 and excludes (cancels) the start point PO1 of the reverse edge point from the recognition of the reverse edge candidate point PO. WidthBlack border lines BE and the like are excluded.

[0090] That is, in the judgment in step S43 of FIG. 10, if there is a large difference in luminance between just before the start point PO1 of the reverse edge point and just after the end point PS1 of the reverse edge point, or in the judgment in step S44, if the distance (line width) between the start point PO1 of the reverse edge point and the end point PS1 of the reverse edge point is shorter (thinner) than a specified value, the stereo image recognition device 4 recognizes the lane markings LL, LR as having black border lines BE on both sides.

[0091] On the other hand, in the judgment of step S44 of FIG. 10, if the judgment of step S43 determines that the distance between the start point PO1 of the reverse edge point and the end point PS1 of the reverse edge point is larger (longer) than a predetermined specified value set in advance, the stereo image recognition device 4 determines whether the difference in luminance immediately before the start point PO1 of the reverse edge point and the luminance immediately after the end point PS2 of the reverse edge point is smaller than a predetermined value (S45).

[0092] If the difference in brightness between the brightness immediately before the start point PO1 of the reverse edge point and the brightness immediately after the end point PS2 of the reverse edge point is not small (large) as specified, the stereo image recognition device 4 proceeds to step S49 and recognizes the start point PO1 of the reverse edge point as a reverse edge candidate point PO.

[0093] On the other hand, in the judgment of step S45 of FIG. 10, if the luminance difference between the luminance immediately before the very start point PO1 of the reverse edge point and the luminance immediately after the very end point PS2 of the reverse edge point is small as specified, the stereo image recognition device 4 judges whether the luminance immediately before the start point PO1 of the reverse edge point and the luminance immediately after the end point PS2 of the reverse edge point are both larger than the road surface luminance by a luminance difference equal to or greater than a specified value (luminance difference threshold) (S46).

[0094] In the judgment of step S46 in FIG. 10, if the luminance immediately before the start point PO1 of the reverse edge point and the luminance immediately after the end point PS2 of the reverse edge point are both greater than the road surface luminance by a luminance difference equal to or greater than a predetermined specified value, the stereo image recognition device 4 proceeds to step S48 and excludes (cancels) the start point PO1 of the reverse edge point from being recognized as a reverse edge candidate point PO.

[0095] In the judgment of step S46 in FIG. 10, if the luminance just before the start point PO1 of the reverse edge point and the luminance just after the end point PS2 of the reverse edge point are both not larger (smaller) than the road surface luminance by a luminance difference equal to or greater than a predetermined specified value, the stereo image recognition device 4 judges whether the distance between the start point PO1 of the reverse edge point and the end point PS2 of the reverse edge point is equal to or greater than a predetermined specified value (S47).

[0096] 10, if the distance between the start point PO1 of the reverse edge point and the end point PS2 of the reverse edge point is equal to or greater than a predetermined specified value, the stereo image recognition device 4 excludes (cancels) the recognition of the start point PO1 of the reverse edge point as a reverse edge candidate point PO (S48).On the other hand, if the distance between the start point PO1 of the reverse edge point and the end point PS2 of the reverse edge point is not equal to or greater than a predetermined specified value, the stereo image recognition device 4 proceeds to step S49 and recognizes the start point PO1 of the reverse edge point as a reverse edge candidate point PO.

[0097] The routine of steps S45 to S47 is performed to prevent a black line BL from being recognized as a lane marking LL, LR when a black line BL is provided in the center of a white line, as seen in the lane markings LL, LR that divide a carpool lane, as shown in FIG. 16.

[0098] In such a situation, if the difference in luminance between just before the start point PO1 of the reverse edge and just after the end point PS2 of the reverse edge point is small and the difference between these luminances is greater than a specified value compared to the road surface luminance, the stereo image recognition device 4 will not recognize the start point PO1 of the reverse edge point as a reverse edge candidate point PO, but will instead recognize it as a black line BL in the center of the carpool lane.

[0099] Alternatively, in the lane markings LL and LR, which are white lines with a black line BL or the like in the center, as shown in FIG. 17, the edge point PS1 is the end point, and the edge point PO2 is the end point, and the distance between the start point PO1 of the first reverse edge point and the end point PS2 of the intermediate reverse edge point (line Width)Lb is equal to or greater than a predetermined value, the start point PO1 of the reverse edge point is not recognized as a reverse edge candidate point PO, but as a black line BL in the center of the carpool lane.

[0100] This prevents the black line BL in the center of the white line of a carpool lane, etc. from being erroneously recognized as lane markings LL, LR. In the case of lane markings LL, LR having a black line BL or the like in the center of the white line, the stereo image recognition device 4 first determines in step S41 not to recognize the start point PO1 of the reverse edge point as a reverse edge candidate point PO if the distance between the start point PO1 of the intermediate reverse edge point and the end point PS2 of the reverse edge point is less than a predetermined specified value.

[0101] In this way, the stereo image recognition device 4 can determine whether or not to cancel the recognition of the reverse edge candidate points PO for various special shapes of the lane markings LL and LR.

[0102] As described above, the stereo image recognition device 4, which is a lane marking recognition device for a vehicle according to this embodiment, recognizes light-colored road surfaces more easily than bright road surfaces. Lane The darker color than the lighter road surfaces such as black other than the dividing lines LL and LR Lane It is also capable of recognizing lane markings LL and LR.

[0103] Therefore, vehicle 1 is Lane The markings LL and LR are dark lines such as black, or light lines such as white and yellow, and appear to have been repaired with dark lines such as black. Lane Even if the lane markings are LL or LR, the lane departure prevention function and lane centering function are not canceled, and steering control can be continued.

[0104] Therefore, the stereo image recognition device 4, which is a vehicle lane marking recognition device, is Lane Suitable for all shades of LL and LR demarcation lines Lane It is possible to recognize lane markings LL and LR.

[0105] In addition, Lane The lane markings LL and LR include only dark color lines such as black. In the above embodiment, an example has been described in which the lane marking start points are used as the lane marking edge points, but the present invention is not limited to this, and for example, it is also possible to use lane marking end points, the luminance of which changes from light to dark, instead of the lane marking start points.

[0106] Furthermore, in the above-described embodiment, an example of performing lane marking recognition based on a pair of stereoscopically captured images has been described, but the present invention is not limited to this, and it goes without saying that lane marking recognition may also be performed based on images captured by a monocular camera or the like.

[0107] Moreover, the stereo image recognition device 4 and the control unit 5 each have a processor including a central processing unit (CPU), and storage devices such as a ROM and a RAM. Furthermore, all or part of the configuration of the multiple circuits of the processor may be implemented by software. For example, the CPU may read and execute various programs corresponding to the respective functions stored in the ROM.

[0108] Furthermore, all or part of the functions of the processor may be configured using logic circuits or analog circuits, and the processing of various programs may be realized by electronic circuits such as FPGAs.

[0109] The invention described in the above embodiments is not limited to those embodiments, and various modifications can be made in the implementation stage without departing from the gist of the invention. Furthermore, the above includes inventions at various stages, and various inventions can be extracted by appropriate combinations of the disclosed constituent elements.

[0110] For example, if the stated problem can be solved and the stated effect can be obtained even if some of the constituent elements are deleted from all the disclosed constituent elements, the configuration from which these constituent elements are deleted can be extracted as an invention. [Explanation of symbols]

[0111] 1...Vehicle (own vehicle) 2. Driving support device 3. Stereo camera 4...Stereo image recognition device 5…Control unit 11...Vehicle speed sensor 12...Yaw rate sensor 13…Main switch 14...Steering angle sensor 15...Accelerator opening sensor 16...Throttle valve 17…Active booster AL,AR: Lane line search area BE: Black border line BL…black line C: Counter Cth1: First threshold Cth2: Second threshold Jn…Search line LL: Lane markings LL,LR...Lane markings Ll,Lr…Approximate line P…Start point of lot line PO: Reverse edge candidate point PS: Standard edge candidate points R1…First segment R2: Second segment TM…Tire marks

Claims

1. an edge point detection means for detecting edge points based on a horizontal luminance change within a detection area of ​​a lane marking set in an image captured of an environment in which the vehicle is traveling; an approximation line calculation means for calculating an approximation line of the point group consisting of the edge points; Equipped with a first mode for searching for standard edge candidate points having a high brightness relative to a road surface and reverse edge candidate points having a low brightness relative to the road surface in a first predetermined range of the image; a second mode for searching for the standard edge candidate points and the inverse edge candidate points in a second predetermined range that is wider than the first predetermined range of the image; having A lane marking recognition device for a vehicle, comprising: a first mode and a second mode, the first mode and the second mode being selectively switched depending on the number of detected reverse edge candidate points.

2. during selection of the first mode, a first segment is set for alternately searching for the standard edge candidate points on a first search line in an odd row and the reverse edge candidate points on a second search line in an even row in the first predetermined range, the number of the detected reverse edge candidate points is counted, and when a counter value becomes equal to or greater than a preset first threshold value, the first mode is switched to the second mode; during selection of the second mode, a second segment is set for alternately searching for the standard edge candidate points on the first search lines in odd rows and the reverse edge candidate points on the second search lines in even rows in the second predetermined range, the number of the detected reverse edge candidate points is counted, and when the counter value becomes equal to or less than a preset second threshold value, the second mode is switched to the first mode; 2. The lane marking recognition device for a vehicle according to claim 1.

3. 3. The lane marking recognition device for a vehicle according to claim 2, wherein when a luminance value of the detected reverse edge candidate point is equal to or less than a preset luminance threshold value, the reverse edge candidate point is not counted.

4. A vehicle lane marking recognition device as described in claim 2, characterized in that when the distance from the start point to the end point of the detected reverse edge candidate point is less than a predetermined line width threshold, the reverse edge candidate point is not counted.

5. A lane marking recognition device for a vehicle as described in Claim 2, characterized in that when the standard edge candidate point is a start point and the reverse edge candidate point is an end point, the reverse edge candidate point is not counted when the distance from the start point to the end point is greater than or equal to a predetermined specified value.

Images