Vehicle lane demarcation line recognition device

Abstract

To provide a vehicle lane demarcation line recognition device that can accurately supplement lane demarcation line information in an area behind an own vehicle by using the lane demarcation line information in an area in front of the own vehicle.SOLUTION: An image recognition ECU 13 calculates relative coordinates of a tentative completion point Pt from lane demarcation line information recognized every set cycle based on a driving environment image in front of an own vehicle, calculates a first demarcation line completion point Pc1 based on the relative coordinates of the tentative completion point Pt and driving information of an own vehicle O, and calculates a second demarcation line completion point Pc2 based on the relative coordinates of the tentative completion point Pt and absolute position information of the own vehicle O. The image recognition ECU13 acquires standard deviations σx and σy as evaluation values related to the detection accuracy of the driving information of the own vehicle O and standard deviation σz as evaluation values related to the detection accuracy of the absolute position information of the own vehicle O, and integrates the first demarcation line completion point Pc1 and the second demarcation line completion point Pc2 by using a confidence level calculated based on the standard deviations σx and σy and standard deviation σz.SELECTED DRAWING: Figure 21

Description

[Technical field]

[0001] The present invention relates to a lane marking recognition device for a vehicle that recognizes lane markings based on information acquired by an on-board sensor. [Background technology]

[0002] In recent years, many vehicles such as automobiles are equipped with driving assistance devices. The driving assistance devices realize driving assistance control by appropriately combining, for example, adaptive cruise control (ACC), active lane keep centering control (ALKC), and emergency lane departure prevention control (ELKA).

[0003] In such driving assistance control, it is important to recognize lane markings that divide the lane in which the vehicle is traveling and adjacent lanes in real time. It is preferable that lane markings be recognized not only in the area ahead of the vehicle, but also in an expanded area behind the vehicle. In response to this, for example, Patent Document 1 discloses a technology that complements lane markings in the area behind the vehicle by using a history of candidate points (marking line candidate points) for lane markings ahead of the vehicle detected using an on-board camera, the yaw rate of the vehicle, the vehicle speed, and the like. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2012-164287 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, the yaw rate detected by the yaw rate sensor is subject to detection errors due to the performance of the yaw rate sensor itself, the traveling state of the vehicle, etc. Such yaw rate detection errors affect the lane marking candidate points complemented by the above-mentioned Patent Document 1. In particular, the influence of the yaw rate detection errors increases as the complemented lane marking candidate points move relatively farther away from the vehicle.

[0006] The present invention aims to provide a lane marking recognition device for a vehicle that can accurately supplement lane marking information in the rear area of ​​the vehicle using lane marking candidate points in the forward area of ​​the vehicle. [Means for solving the problem]

[0007] A lane marking recognition device for a vehicle according to one aspect of the present invention includes a lane marking recognition means for recognizing lane marking information ahead of the host vehicle at each set cycle based on sensing information for the driving environment ahead of the host vehicle, a relative coordinate calculation means for extracting a tentative completion point from the lane marking information at each set cycle and calculating a relative coordinate of the tentative completion point with respect to the host vehicle as a reference, a first completion point calculation means for updating the relative coordinate of the tentative completion point up to the present based on driving information of the host vehicle at each set cycle and calculating a first lane marking completion point for completing the lane marking information up to the rear of the host vehicle, a first evaluation value acquisition means for acquiring a first evaluation value relating to the detection accuracy of the driving information at each set cycle, and a first evaluation value acquisition means for acquiring a first evaluation value relating to the detection accuracy of the driving information at each set cycle from a plurality of positioning satellites. the vehicle's absolute position information is calculated from the relative coordinates of the temporary completion point based on absolute position information of the vehicle measured at each set period using a signal from the vehicle's position sensor, and a second lane line completion point is calculated based on the absolute position information of the temporary completion point up to the present and the current absolute position information of the vehicle, for completing the lane line information up to the rear of the vehicle; a second evaluation value acquisition means is provided for acquiring a second evaluation value related to the detection accuracy of the absolute position information of the vehicle at each set period; and a completion point integrating means is provided for integrating the corresponding first lane line completion point and second lane line completion point using a reliability calculated based on the first evaluation value and the second evaluation value.

[0008] A lane marking recognition device for a vehicle according to another aspect of the present invention includes a processor, the processor recognizes lane marking information ahead of the host vehicle at each set period based on sensing information for the driving environment ahead of the host vehicle, extracts a tentative completion point from the lane marking information at each set period, calculates relative coordinates of the tentative completion point with respect to the host vehicle as a reference, updates the relative coordinates of the tentative completion point up to the present based on driving information of the host vehicle at each set period, calculates a first lane marking completion point for completing the lane marking information up to the rear of the host vehicle, obtains a first evaluation value related to detection accuracy of the driving information at each set period, and The absolute position information of the temporary completion point is calculated from the relative coordinates of the temporary completion point based on the absolute position information of the vehicle measured at each set period using a signal from a positioning satellite, a second lane line completion point for completing the lane line information up to the rear of the vehicle is calculated based on the absolute position information of the temporary completion point up to the present and the current absolute position information of the vehicle, a second evaluation value related to the detection accuracy of the absolute position information of the vehicle is obtained at each set period, and the corresponding first lane line completion point and second lane line completion point are integrated using a reliability calculated based on the first evaluation value and the second evaluation value. Effect of the Invention

[0009] According to the lane marking recognition device for a vehicle of the present invention, lane marking information in the area behind the host vehicle can be accurately supplemented by lane marking information in the area ahead of the host vehicle. [Brief description of the drawings]

[0010] [Figure 1] Schematic diagram of a driving support device [Diagram 2] FIG. 1 is an explanatory diagram showing the management areas of a stereo camera and a radar. [Diagram 3] FIG. 13 is an explanatory diagram showing lane marking candidate points detected from a driving environment image; [Figure 4] FIG. 13 is an explanatory diagram showing an example of a change in luminance on a search line; [Diagram 5] FIG. 1 is an explanatory diagram showing an example of a distribution of lane line candidate points in coordinates in real space and a lane line approximation curve; [Figure 6] FIG. 11 is an explanatory diagram showing provisional complement points extracted from each lane line approximation curve. [Figure 7] An explanatory diagram showing the relative coordinate system of the previous frame and the current relative coordinate system [Figure 8] A diagram showing the travel distance of the vehicle from the relative coordinate system of the previous frame to the present. [Figure 9] An explanatory diagram showing the relative coordinate system for two frames before and the relative coordinate system for one frame before [Figure 10] An explanatory diagram showing the travel distance of the host vehicle up to the previous frame as viewed from the relative coordinate system of two frames before. [Figure 11] A diagram showing the provisional interpolation point n frames before, axis-transformed to the same axis direction as the relative coordinate system of the current frame. [Figure 12] FIG. 13 is an explanatory diagram showing coordinates indicating the distance to the vehicle position as viewed from a relative coordinate system n frames before. [Figure 13] FIG. 13 is an explanatory diagram showing coordinates representing the distance from a tentative candidate point to the vehicle position as viewed from a relative coordinate system n frames before. [Figure 14] FIG. 13 is an explanatory diagram showing coordinates representing the distance from the vehicle position to a tentative candidate point n frames before. [Figure 15] FIG. 2 is an explanatory diagram showing a relative coordinate system in which the front / rear / left / right directions of the vehicle are the axial directions, and a relative coordinate system in which the east / west / north / south directions are the axial directions; [Figure 16] FIG. 11 is an explanatory diagram showing the coordinates of a provisionally completed point obtained by converting the coordinates of a relative coordinate system in which the front, rear, left, and right directions of the vehicle are the axial directions into the coordinates of a relative coordinate system in which the east, west, north, south directions are the axial directions; [Figure 17] FIG. 13 is an explanatory diagram showing the relationship between absolute position information of the vehicle and absolute position information of a provisional complement point; [Figure 18] FIG. 1 is an explanatory diagram showing the absolute position of a vehicle and the absolute positions of temporary complement points in each frame; [Figure 19] FIG. 13 is an explanatory diagram showing a second lane line completion point calculated from absolute position information of a temporary completion point. [Figure 20] FIG. 1 is an explanatory diagram showing a lane line completion point calculated by integrating a first lane line completion point and a second lane line completion point; [Figure 21]Flowchart showing lane marking recognition routine [Figure 22] A flowchart showing a first lane line completion point calculation subroutine. [Diagram 23] A flowchart showing a second lane line completion point calculation subroutine. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described with reference to the drawings. The drawings relate to one embodiment of the present invention, and Fig. 1 is a schematic diagram showing a vehicle driving support device.

[0012] 1, the driving assistance device 1 is configured to include a camera unit 10. The camera unit 10 is fixed to the center of the upper front part of the interior of a vehicle (host vehicle) O, for example.

[0013] The camera unit 10 includes a stereo camera 11 as an imaging means, an image processing unit (IPU) 12, an image recognition unit (image recognition_ECU) 13, and a driving control unit (driving_ECU) 14.

[0014] The stereo camera 11 has a main camera 11a and a sub camera 11b as sensors. The main camera 11a and the sub camera 11b are configured with imaging elements such as CMOS. The main camera 11a and the sub camera 11b are arranged, for example, at symmetrical positions with respect to the center in the vehicle width direction. As a result, the main camera 11a and the sub camera 11b take stereo images of the driving environment in the outside front area Af (see FIG. 2) from different viewpoints at a predetermined imaging period synchronized with each other.

[0015] The IPU 12 performs predetermined image processing on the driving environment images captured by the stereo camera 11. As a result, the IPU 12 detects edges of various objects such as three-dimensional objects and lane markings on the road surface displayed on the images. The IPU 12 also obtains distance information from the positional deviation amount of corresponding edges on the left and right images. As a result, the IPU 12 generates image information including distance information (distance image information).

[0016] The image recognition_ECU 13 recognizes lane markings that divide lanes on a road based on distance image information received from the IPU 12, etc. For example, the image recognition_ECU 13 calculates the curvature [1 / m] of the left and right lane markings that divide each lane on the road, and the width between the left and right markings (lane width). Furthermore, the image recognition_ECU 13 calculates the lane width from the difference between the curvatures of the left and right markings. Through these lane marking recognition processes, the image recognition_ECU 13 recognizes lanes on the road, including the lane on which the host vehicle O is traveling (host vehicle travel lane). The details of lane marking recognition will be described later.

[0017] Furthermore, the image recognition_ECU 13 performs predetermined pattern matching on the distance image information. As a result, the image recognition_ECU 13 recognizes three-dimensional objects such as guardrails and curbs extending along the road, and surrounding vehicles traveling on the road. Here, the image recognition_ECU 13 recognizes, for example, the type of the three-dimensional object, the distance to the three-dimensional object, the speed of the three-dimensional object, and the relative speed between the three-dimensional object and the vehicle O.

[0018] Various pieces of information recognized by the image recognition_ECU 13 are output to a traveling_ECU (traveling_ECU) 14 as traveling environment information.

[0019] The driving_ECU 14 is a control unit for overall control of the driving assistance device 1.

[0020] The travel_ECU 14 is connected with various sensors, such as a locator unit 17, a left front side sensor 18lf, a right front side sensor 18rf, a left rear side sensor 18lr, and a right rear side sensor 18rr.

[0021] In addition, various control units, such as an engine control unit (E / G_ECU) 22, a transmission control unit (T / M_ECU) 23, a brake control unit (BK_ECU) 24, and a power steering control unit (PS_ECU) 25, are connected to the driving_ECU 14 via an in-vehicle communication line such as a CAN (Controller Area Network).

[0022] The left front side sensor 18lf and the right front side sensor 18rf are, for example, constituted by a millimeter wave radar. The left front side sensor 18lf and the right front side sensor 18rf are, for example, disposed on the left and right sides of a front bumper, respectively. The left front side sensor 18lf and the right front side sensor 18rf detect, as driving environment information, three-dimensional objects existing in areas Alf, Arf (see FIG. 2) diagonally forward and to the left and right of the vehicle O, which are difficult to recognize in the image of the stereo camera 11.

[0023] The left rear side sensor 18lr and the right rear side sensor 18rr are, for example, constituted by millimeter wave radars. The left rear side sensor 18lr and the right rear side sensor 18rr are, for example, disposed on the left and right sides of the rear bumper, respectively. The left rear side sensor 18lr and the right rear side sensor 18rr detect, as driving environment information, three-dimensional objects existing in areas Alr, Arr (see FIG. 2) diagonally to the left and right and rear of the host vehicle O that are difficult to recognize by the left front side sensor 18lf and the right front side sensor 18rf.

[0024] Here, when each radar 18 is composed of a millimeter wave radar, the millimeter wave radar mainly detects three-dimensional objects such as a vehicle traveling alongside or following the vehicle by analyzing the reflected waves from the object in response to the outputted radio wave. Specifically, each radar detects information about the three-dimensional object, such as the width of the three-dimensional object, the position of the representative point of the three-dimensional object (the relative position with respect to the vehicle O), and the speed.

[0025] Thus, in this embodiment, the left front side sensor 18lf, the right front side sensor 18rf, the left rear side sensor 18lr, and the right rear side sensor 18rr correspond to a specific example of a driving environment recognition means (driving environment recognition unit) that recognizes driving environment information outside the vehicle.

[0026] Locator unit 17 includes a GNSS receiver 17a and a high-precision road map database (road map DB) 17b.

[0027] The GNSS receiver 20a receives positioning signals transmitted from a plurality of positioning satellites, thereby allowing the GNSS receiver 17a to measure the position (latitude, longitude, azimuth angle, etc.) of the host vehicle O.

[0028] The road map DB 17b is a large-capacity storage medium such as an HDD. High-precision road map information (dynamic map) is stored in this road map DB 20b. The road map information includes, for example, lane data required for autonomous driving, such as lane width data, lane center position coordinate data, lane travel azimuth data, and speed limit data. The lane data is stored at intervals of several meters for each lane on the road map. Furthermore, the road map DB 20b stores, for example, traffic light data and the like as data associated with the lane data.

[0029] For example, based on a request signal from the driving_ECU 14, the road map DB 20b outputs road map information of a set range based on the vehicle position measured by the GNSS receiver 20a to the driving_ECU 14 as driving environment information.

[0030] Thus, in this embodiment, locator unit 17 corresponds to a specific example of a driving environment recognition means (driving environment recognition unit).

[0031] A throttle actuator 32 for an electronically controlled throttle and the like are connected to an output side of the E / G_ECU 22. In addition, various sensors such as an accelerator sensor (not shown) are connected to an input side of the E / G_ECU 22.

[0032] The E / G_ECU 22 performs drive control for the throttle actuator 32 etc. based on control signals from the travel_ECU 14 or detection signals from various sensors. In this way, the E / G_ECU 22 adjusts the amount of intake air for the engine to generate a desired engine output. The E / G_ECU 22 also outputs signals such as the accelerator opening degree detected by the various sensors to the travel_ECU 14.

[0033] The output side of the T / M_ECU 23 is connected to a hydraulic control circuit 33. Furthermore, various sensors such as a shift position sensor (not shown) are connected to the input side of the T / M_ECU 23. The T / M_ECU 23 performs drive control of the hydraulic control circuit 33 and the like based on an engine torque signal estimated by the E / G_ECU 22 and detection signals from various sensors. As a result, the T / M_ECU 23 operates friction engagement elements, pulleys, and the like provided in the automatic transmission to shift the engine output at a desired gear ratio. Furthermore, the T / M_ECU 23 outputs signals such as the shift position detected by the various sensors to the travel_ECU 14.

[0034] A brake actuator 34 for adjusting brake fluid pressures output to brake wheel cylinders provided on the respective wheels is connected to the output side of the BK_ECU 24. A vehicle speed sensor 37 as a vehicle speed detection means and a yaw rate sensor 38 as a yaw rate detection means are connected to the input side of the BK_ECU 24. Furthermore, various sensors such as a brake pedal sensor and a longitudinal acceleration sensor (not shown) are connected to the input side of the BK_ECU 24. The vehicle speed sensor 37 detects the vehicle speed V of the host vehicle O. The yaw rate sensor 37 detects the yaw rate ω acting on the host vehicle O.

[0035] The BK_ECU 24 performs drive control for the brake actuator 34 and the like based on control signals from the travel_ECU 14 or detection signals from various sensors. As a result, the BK_ECU 24 appropriately generates braking force on each wheel for performing forced braking control, yaw rate control, etc. on the host vehicle O. In addition, the BK_ECU 24 outputs signals of the brake operation state, yaw rate, longitudinal acceleration, host vehicle speed, etc. detected by the various sensors to the travel_ECU 14.

[0036] An electric power steering motor 35 that applies a steering torque to the steering mechanism by the rotational force of the motor is connected to the output side of the PS_ECU 25. In addition, various sensors such as a steering torque sensor and a steering angle sensor are connected to the input side of the PS_ECU 25.

[0037] The PS_ECU 25 performs drive control for the electric power steering motor 35 and the like based on control signals from the travel_ECU 14 or detection signals from various sensors. As a result, the PS_ECU 25 generates a steering torque for the steering mechanism. The PS_ECU 25 also outputs signals of the steering torque, steering angle, etc. detected by the various sensors to the travel_ECU 14.

[0038] The traveling_ECU 14 performs driving assistance control by, for example, outputting various control signals to the E / G_ECU 22, the T / M_ECU 23, the BK_ECU 24, and the PS_ECU 25.

[0039] This driving assistance control is realized by appropriately combining adaptive cruise control (ACC), active lane keep centering control (ALKC), emergency lane keep assist (ELKA), and auto lane changing control (ALC), etc.

[0040] The following vehicle distance control is realized by selectively executing following travel control and constant speed travel control. For example, when a preceding vehicle is registered ahead of the host vehicle O based on the travel environment information, the travel_ECU 14 performs following travel control. In this following travel control, the travel_ECU 14 performs acceleration / deceleration control to maintain a target vehicle distance according to the vehicle speed of the preceding vehicle. On the other hand, when a preceding vehicle L is not registered ahead of the host vehicle O, the travel_ECU 14 performs constant speed travel control. In this constant speed travel control, the travel_ECU 14 performs acceleration / deceleration control for the host vehicle O, using the set vehicle speed Vset input by the driver as the target vehicle speed.

[0041] The lane center keeping control and lane departure suppression control are performed based on lane marking information and the like included in the driving environment information. That is, the driving_ECU 14 sets, for example, a target travel path along the left and right lane markings in the center of the host vehicle's travel lane. Then, the driving_ECU 14 performs feedforward control and feedback control for steering based on the target travel path. In this way, the driving_ECU 14 keeps the host vehicle O in the center of the lane. Here, the driving_ECU 14 prohibits the lane departure suppression control when, for example, the sensors 18rl, 18rr detect a following vehicle approaching the host vehicle O in the host vehicle's travel lane.

[0042] The automated lane change control is performed based on lane marking information and the like included in the driving environment information. That is, the traveling_ECU 14 sets a target lateral position in a lane adjacent to the host vehicle's driving lane. Furthermore, the traveling_ECU 14 sets a target route from the target route of the host vehicle O to the target lateral position. Then, the traveling_ECU 14 performs feedforward control and feedback control for steering along the target route. In this way, the traveling_ECU 14 changes the lane to the adjacent lane. Here, the traveling_ECU 14 prohibits the automated lane change control when the sensors 18fl, 18fr, 18rl, and 18rr detect a vehicle traveling alongside or a following vehicle in the adjacent lane.

[0043] Next, the lane marking recognition process executed by the image recognition_ECU 13 will be described in detail.

[0044] In the lane marking recognition process of this embodiment, the image recognition_ECU 13 recognizes the lane markings ahead of the vehicle using the driving environment image captured in front of the vehicle as sensing information. Furthermore, the image recognition_ECU 13 also recognizes the lane markings behind the vehicle in a complementary manner based on the information extracted from the lane markings recognized ahead of the vehicle.

[0045] Specifically, the image recognition_ECU 13 sets a search area As for lane markings in the driving environment image captured by the main camera 11a (see FIG. 3). For example, an area of ​​a predetermined width is set as the search area As along the lane markings recognized in the previous frame.

[0046] The image recognition_ECU 13 also checks the luminance change in the search area Aa from the inside to the outside in the vehicle width direction of the host vehicle O for each horizontal search line l set in the driving environment image. The image recognition_ECU 13 then detects the first edge point on the search line l in each search area As where the luminance changes from low to high by more than a set value as the lane line candidate point Pd. More specifically, the image recognition_ECU 13 detects the first edge point where the luminance differential value is more than a set threshold as the lane line candidate point Pd (see FIG. 4).

[0047] The image recognition_ECU 13 also uses the distance image information to locate each of the lane marking candidate points Pd at coordinates in real space (see FIG. 5). The image recognition_ECU 13 then calculates a lane marking approximation curve W that approximates the lane marking, based on each point group of the lane marking candidate points Pd. Note that in FIG. 5, four lane marking approximation curves W are calculated based on the point group of the four sets of lane marking candidate points Pd. L ,W LL ,W R ,W RR An example of the calculation is shown.

[0048] Through these processes, the image recognition_ECU 13 recognizes the lane markings W ahead of the vehicle.

[0049] In addition, in order to complement the lane markings W to the rear of the vehicle, the image recognition_ECU 13 extracts a point on each lane marking that is currently being recognized as a provisional complement point Pt. For example, the image recognition_ECU 13 extracts the coordinates of a point that is the shortest distance from the vehicle O to each lane marking approximation line W as the provisional complement point Pt (see FIG. 6).

[0050] In addition, the image recognition_ECU 13 calculates the relative coordinates (x L ,0), (x LL ,0), (x R ,0), (x RR , 0). That is, for each temporary complement point Pt, the image recognition_ECU 13 calculates the coordinates of a relative coordinate system [xy] in which the vehicle position is the origin, the vehicle width direction of the vehicle O is the x-axis, and the front-rear direction of the vehicle O is the y-axis. Then, the image recognition_ECU 13 stores the relative coordinates of each temporary complement point Pt. The image recognition_ECU 13 holds each saved temporary complement point Pt until a set period t (set frame time t: a set number of imaging periods) has elapsed.

[0051] The image recognition_ECU 13 also calculates a first lane line completion point Pc1 and a second lane line completion point Pc2 using each temporary completion point Pt. The calculation method of the first lane line completion point Pc1 and the second lane line completion point Pc2 based on each temporary completion point Pt is the same for the host vehicle's lane and each adjacent lane. Therefore, in the following, in order to simplify the explanation, the calculation process of the first and second lane line completion points Pc1 and Pc2 will be explained while illustrating only the host vehicle's lane. In addition, in order to facilitate understanding in the following explanation, when each point such as the temporary completion point for the lane line on the left and right sides of the host vehicle O is displayed in a distinguished manner, "L" and "R" are appropriately added to the beginning of the reference numerals.

[0052] The first lane line completion point Pc1 is obtained by updating the relative coordinates of the tentative completion point Pt extracted in each frame for each imaging period based on the traveling information of the host vehicle O. In this case, the traveling information of the host vehicle O includes, for example, the host vehicle speed V and the yaw rate ω acting on the host vehicle O.

[0053] For example, as shown in Figures 7 and 8, the movement distance x1 in the x-axis direction and the movement distance y1 in the y-axis direction from the relative coordinate system [xy] of one frame before to the present can be calculated using the following equations (1) and (2).

[0054]

number

[0055] The movement distances x1, y1 as viewed from the relative coordinate system [xy] of the previous frame can be converted to movement distances X1, Y1 based on the current relative coordinate system [XY] using the following equations (3) and (4).

[0056]

number

[0057]

number

[0058] In addition, by using the following equations (7) and (8), it is possible to calculate the travel distances X2, Y2 of the host vehicle O from two frames before to the present, based on the current relative coordinate system [XY].

[0059]

number

[0060]

number

[0061]

number

[0062] That is, for example, as shown in FIG. 12, the coordinates representing the distance from the relative coordinate system [xy] of n frames before to the current vehicle position (the origin of the relative coordinate system [XY]) are expressed as (X n ,Y n ) as shown in FIG. 13. For example, the coordinates representing the distance from the tentative candidate point RPtn to the current vehicle position as viewed from the relative coordinate system [xy] of the nth frame before are defined as (X tn ,Y tn Based on these relationships, the coordinates that represent the distance from the tentative candidate point RPtn to the current vehicle position (the origin of the relative coordinate system [XY]) are (X n -X tn ,Y n -Y tn ) is obtained. For example, as shown in FIG. 14, the required coordinates are the distance from the current vehicle position to the tentative candidate point RPtn n frames before. Therefore, the coordinates representing the distance to the tentative candidate point RPtn n frames before with the current relative coordinate system as the origin are (-X n +X tn ,-Y n +Y tn )

[0063]

number

[0064] In addition, the image recognition_ECU 13 calculates a standard deviation σ x and standard deviation σ y We obtain the standard deviation σ x and standard deviation σ y The standard deviation σ is obtained for each frame. xand standard deviation σ y When acquiring the standard deviation σ, the image recognition_ECU 13 estimates, for example, the detection accuracy of the yaw rate ω by the yaw rate sensor 38 and the detection accuracy of the vehicle speed V by the vehicle speed sensor 37. The estimation of these detection accuracies is performed, for example, by taking into account external factors such as the shape of the road along which the vehicle is traveling, in addition to the inherent detection performance of the yaw rate sensor 38 and the inherent detection performance of the vehicle speed sensor 37. Then, the image recognition_ECU 13 calculates the standard deviation σ based on at least one of the detection accuracy of the yaw rate ω by the yaw rate sensor 38 and the detection accuracy of the vehicle speed V by the vehicle speed sensor 37, using a preset map or the like. x and standard deviation σ y Calculate.

[0065] The second lane line completion point Pc2 is obtained by calculating absolute position information of the temporary completion point Pt from the relative coordinates of the temporary completion point Pt extracted in each frame. In this case, the absolute position information may be, for example, the latitude and longitude of the temporary completion point Pt.

[0066] In order to calculate the latitude and longitude of the temporary complement point Pt, for example, as shown in FIGS. 15 and 16, the image recognition_ECU 13 converts the coordinates of the temporary complement point Pt in the current frame into coordinates (x t ,y t ) to the coordinate (x t ',y t ').

[0067] Here, the relative coordinate system [xy] is a coordinate system with the host vehicle position as the origin, the x-axis representing the width direction of the host vehicle O, and the y-axis representing the front-rear direction of the host vehicle O. The relative coordinate system [x'-y'] is a coordinate system with the host vehicle position as the origin, the x'-axis representing the east-west direction, and the y'-axis representing the north-south direction.

[0068] In addition, the image recognition_ECU 13 uses the absolute position information (latitude φ1 and longitude λ1) of the vehicle O received by the GNSS receiver 17a to calculate the absolute position information (latitude φ2 and longitude λ2) of the temporary complement point Pt based on the following equations (15) to (18) (see Figure 17).

[0069]

number

[0070] The image recognition_ECU 13 calculates the absolute position information of the temporary complement point Pt every time the temporary complement point Pt is extracted (for each frame). The image recognition_ECU 13 then holds the calculated absolute position information of the temporary complement point Pt until a set period t (frame time t: a set number of imaging periods) has elapsed. As a result, the image recognition_ECU 13 holds the absolute position information of the temporary complement point Pt including the absolute position information, for example, as shown in FIG. n Obtain the sequence of points.

[0071] Then, the image recognition_ECU 13 calculates each provisional completion point (Pt) n Absolute position information (latitude φ2 n and longitude λ2 n ) based on the above equations (15) to (18), the coordinates (x tn ',y tn ') is calculated.

[0072] Furthermore, the image recognition_ECU 13 calculates each provisional completion point Pt n Coordinates (x tn ',y tn ') to find the second plot line complement point Pc2 n That is, the image recognition_ECU 13 calculates each provisional complement point Pt n The coordinates (x tn ',y tn ') to the current relative coordinate system [XY] coordinates (x tn ,y tn ) respectively. The transformed coordinates (x tn ,ytn ) is the second parcel line complement point Pc2 n Coordinates (X Pc2_n ,Y Pc2_n )

[0073] In addition, the image recognition_ECU 13 calculates a standard deviation σ z Obtain the standard deviation σ z The standard deviation σ is obtained for each frame. z When acquiring the standard deviation σ, the image recognition_ECU 13 acquires, for example, the number of positioning satellites from which the GNSS receiver 17a has received positioning signals and the reception level (signal strength) of each positioning signal from the GNSS receiver 17a. Then, the image recognition_ECU 13 calculates the standard deviation σ based on at least one of the number of positioning satellites and the reception level of each positioning signal, for example, by using a preset map or the like. z Calculate the standard deviation σ z The calculation of the standard deviation σ can be performed in the GNSS receiver 17a. In this case, the image recognition_ECU 13 uses the standard deviation σ z is obtained for each frame.

[0074] Furthermore, the image recognition_ECU 13 detects the first detection accuracy σ x ,σ y and the second detection accuracy σ z The reliability K is calculated based on the tentative completion point Pt in each frame. This reliability K is used as a coefficient for fusing the first lane line completion point Pc1 and the second lane line completion point Pc2, which are calculated based on the tentative completion point Pt in each frame, by a Kalman filter. For this reason, the reliability K is calculated for each frame in which the tentative completion point Pt is calculated. In this embodiment, the reliability K is calculated, for example, as the relative reliability of the first lane line completion point Pc1 with respect to the second lane line completion point Pc2. Furthermore, each reliability K is calculated with respect to the X-axis direction and the Y-axis direction of the relative coordinate system [XY] in the current frame. Hereinafter, the reliability K of the tentative completion point Pc1 and the second lane line completion point Pc2 from n frames ago will be referred to as a coefficient for fusing the tentative completion point Pc1 and the second lane line completion point Pc2 from n frames ago. n The first lane line completion point Pc1 calculated based on n The reliability of (kx ) n ,(K y ) n Let us assume that.

[0075] These reliability (K x ) n ,(K y ) n Prior to the calculation of the first lane marking completion point Pc1 n frames before as viewed from the current vehicle position, the image recognition_ECU 13 uses the following equations (19) and (20): n The variance of (σ x 2 ) n and the variance (σ y 2 ) n Calculate.

[0076]

number

[0077]

number

[0078] Furthermore, the image recognition_ECU 13 recognizes the first lane marking completion point Pc1 n frames before. n The variance of (σ x 2 ) n and the variance (σ y 2 ) n and the second partition line complement point Pc2 n frames before n The variance of (σ z 2 ) nBased on the above, the first lane line supplement point Pc1 is calculated using the following equations (22) and (23). n The reliability of (K x ) n ,(K y ) n Calculate.

[0079]

number

[0080]

number

[0081] Thus, in this embodiment, the image recognition_ECU13 corresponds to a specific example of a lane marking recognition means, a relative coordinate calculation means, a first complementary point calculation means, a first evaluation value acquisition means, a second complementary point calculation means, a second evaluation value acquisition means, and a complementary point integration means.

[0082] Next, a lane marking recognition process will be described with reference to a flowchart of a lane marking recognition routine shown in Fig. 21. This routine is executed in the image recognition_ECU 13 at set time intervals (at each image capture period of the stereo camera 11), for example.

[0083] When the routine starts, in step S101, the image recognition_ECU 13 extracts lane marking candidate points Pd ahead of the vehicle as lane marking information based on the driving environment image (see FIG. 3).

[0084] In the following step S102, the image recognition_ECU 13 calculates a lane marking approximation line W ahead of the vehicle as lane marking information based on the point group of the extracted lane marking candidate points Pd (see FIG. 5).

[0085] In the following step S103, the image recognition_ECU 13 sets a search area As for the next frame based on the calculated lane marking approximation line W.

[0086] In the next step S104, the image recognition_ECU 13 extracts temporary complement points Pt on the sides of the vehicle based on the lane marking information. For example, the image recognition_ECU 13 extracts the intersections of the x-axis of a relative coordinate system [xy] based on the vehicle position and each lane marking approximation line W as temporary complement points Pt (see FIG. 6). Then, the image recognition_ECU 13 calculates the coordinates of the relative coordinate system [xy] for each of the extracted temporary complement points Pt.

[0087] In the following step S105, the image recognition_ECU 13 stores the coordinates of each of the extracted temporary complement points Pt.

[0088] In the next step S105, the image recognition_ECU 13 calculates a first lane line completion point Pc1 based on each temporary completion point Pt extracted in each frame. The calculation of the first lane line completion point Pc1 is performed, for example, according to a flowchart of a first lane line completion point calculation subroutine shown in FIG.

[0089] When the subroutine starts, in step S201, the image recognition_ECU 13 The travel information of the host vehicle O in the current frame (the yaw rate ω, the host vehicle speed V, etc.) is stored.

[0090] In the following step S202, the image recognition_ECU 13 reads out the coordinates of the tentative complement point Pt of the previously set frame and the travel information.

[0091] In the next step S203, the image recognition_ECU 13 calculates the provisional complement point Pt n From each coordinate, the first parcel line complement point Pc1 n After calculating the coordinates, the subroutine exits.

[0092] In the main routine shown in Fig. 21, when the process proceeds from step S105 to step S106, the image recognition_ECU 13 calculates a second lane line completion point Pc2 based on each temporary completion point Pt extracted in each frame. The calculation of the second lane line completion point Pc2 is performed, for example, according to a flowchart of a second lane line completion point calculation subroutine shown in Fig. 23.

[0093] When the subroutine starts, the image recognition_ECU 13 acquires positioning information from a positioning satellite in step S301. That is, the image recognition_ECU 13 acquires absolute position information (latitude, longitude, azimuth angle, etc.) of the host vehicle O.

[0094] In the following step S302, the image recognition_ECU13 converts the coordinates of the provisional complement point Pt in the current frame from coordinates in a relative coordinate system [xy] whose axial directions are in the front-rear, back-rear, left-right directions of the vehicle O to coordinates in a relative coordinate system [x'-y'] whose axial directions are in the east-west, north-south directions.

[0095] In the next step S303, the image recognition_ECU 13 calculates absolute position information (latitude and longitude) of the temporary complement point Pt from the coordinates of the temporary complement point Pt in the current frame. That is, the image recognition_ECU 13 calculates the absolute position information of the temporary complement point Pt based on the absolute position information of the host vehicle O, using the above-mentioned equations (15) to (18).

[0096] In the following step S304, the image recognition_ECU 13 stores the absolute position information of the tentative complement point Pt.

[0097] In the next step S305, the image recognition_ECU 13 deletes the absolute position information of the temporary complement point Pt that is located a set number of frames back from the absolute position information of the temporary complement point Pt currently stored.

[0098] In the following step S306, the image recognition_ECU 13 reads out the absolute position information of the tentative complement point Pt of the previously set frame.

[0099] In the next step S307, the image recognition_ECU 13 calculates the coordinates of the temporary complement points Pt in the relative coordinate system [x'-y'] from the absolute position information of the temporary complement points Pt in each frame. That is, the image recognition_ECU 13 calculates the coordinates of each temporary complement point Pt in the relative coordinate system [x'-y'] based on the absolute position information of the host vehicle O, using the above-mentioned equations (15) to (18).

[0100] In the following step S308, the image recognition_ECU13 converts the coordinates of each temporary completion point Pt from the coordinates of the relative coordinate system [x'-y'] whose axial directions are set in the east-west, north-south, and east-west directions to the coordinates of the relative coordinate system [xy] whose axial directions are set in the front-rear, back-left, and right directions of the vehicle O, and then exits the subroutine.

[0101] When the process proceeds from step S107 to step S108 in the main routine of FIG. 21, the image recognition_ECU 13 calculates the standard deviation σ x , standard deviation σ y , and the standard deviation σ for the current absolute position information z is obtained and stored.

[0102] In the next step S109, the image recognition_ECU 13 calculates the standard deviation σ x , standard deviation σ y , and standard deviation σ z The reliability (K x ),(K y ) is calculated.

[0103] In the next step S110, the image recognition_ECU 13 calculates the fusion value (u x ,u y That is, the image recognition_ECU 13 calculates the coordinates (X Pc1 ,Y Pc1 ) and the coordinates of the second parcel line complement point Pc2 (X Pc2 ,Y Pc2 ) fusion value (u x ,u y ) is calculated.

[0104] In the next step S111, the image recognition_ECU 13 calculates a lane line approximation line Wc behind the vehicle based on the group of calculated lane line completion points Pc, and then exits the routine.

[0105] According to this embodiment, the image recognition_ECU 13 recognizes lane marking information based on an image of the driving environment ahead of the host vehicle at each set period (each frame period), and calculates the relative coordinates of a tentative completion point Pt with respect to the host vehicle O. The image recognition_ECU 13 also updates the relative coordinates of the current tentative completion point Pt based on the driving information (yaw rate ω and vehicle speed V) of the host vehicle O at each set period, and calculates a first lane marking completion point Pc1 for completing the lane marking information up to the rear of the host vehicle. The image recognition_ECU 13 also calculates a standard deviation σ as an evaluation value related to the detection accuracy of the driving information of the host vehicle O. x ,σ y The image recognition_ECU 13 also calculates absolute position information of the tentative complement point Pt based on absolute position information (latitude, longitude, azimuth, etc.) of the host vehicle O measured using positioning signals from multiple positioning satellites, and calculates a second lane marking complement point for complementing the lane marking information up to the rear of the host vehicle based on the absolute position information of the tentative complement point Pt up to the present and the current absolute position information of the host vehicle O. The image recognition_ECU 13 also calculates a standard deviation σ z is acquired at every set period. Then, the image recognition_ECU 13 acquires the standard deviation σ x ,σ y and standard deviation σ z The first and second lane line completion points Pc1 and Pc2 are integrated using the reliability calculated based on the above, to calculate the final lane line completion point Pc. This makes it possible to accurately complement the lane line information in the rear area of ​​the host vehicle using the lane line information in the area ahead of the host vehicle.

[0106] That is, the image recognition_ECU 13 uses the travel information of the host vehicle O detected by an on-board sensor and the absolute position information of the host vehicle O measured by a positioning satellite to calculate a first lane line completion point Pc1 and a second lane line completion point Pc2 from the relative coordinates of the tentative completion point Pt extracted from the lane line information ahead of the host vehicle. Then, the image recognition_ECU 13 calculates an evaluation value (standard deviation σ x ,σ y ) and the evaluation value (standard deviation σz ) and calculates the lane line candidate point PC by integrating the first lane line completion point Pc1 and the second lane line completion point Pc2 with a contribution degree according to the reliability. As a result, for example, if there is a large detection error in the driving information of the host vehicle O, the decrease in the calculation accuracy of the first lane line completion point Pc1 can be compensated for by the second lane line completion point Pc2. Similarly, for example, if there is a large detection error in the absolute position information of the host vehicle O, the decrease in the calculation accuracy of the second lane line completion point Pc2 can be compensated for by the first lane line completion point Pc1. As a result, the lane line information in the rear area of ​​the host vehicle O can be accurately complemented.

[0107] By accurately supplementing the lane marking information in the rear area of ​​the vehicle O in this manner, it is possible to accurately determine the lane in which vehicles, etc., behind the vehicle are traveling when performing emergency lane departure prevention control and automated lane change control, etc.

[0108] In the above-mentioned embodiment, the image recognition_ECU 13, the driving_ECU 14, the E / G_ECU 22, the T / M_ECU 23, the BK_ECU 24, and the PS_ECU 25 are configured with a well-known microcomputer equipped with a CPU, RAM, ROM, a non-volatile storage unit, etc., and with peripheral devices thereof, and the ROM stores programs to be executed by the CPU and fixed data such as data tables in advance. Note that all or part of the functions of the processor may be configured with a logic circuit or an analog circuit, and the processing of various programs may be realized by an electronic circuit such as an FPGA.

[0109] The invention described in the above embodiments is not limited to those embodiments, and various modifications can be made in the implementation stage without departing from the gist of the invention. Furthermore, the above embodiments include inventions at various stages, and various inventions can be extracted by appropriate combinations of the disclosed constituent elements.

[0110] For example, if the problem described can be solved and the effect described can be obtained even if some of the constituent elements are deleted from all the constituent elements shown in the above form, the configuration from which these constituent elements are deleted can be extracted as an invention. [Explanation of symbols]

[0111] 1. Driving assistance devices 10 … Camera unit 11 … Stereo camera 11a ... Main camera 11b … Sub camera 13...Image Recognition_ECU 14 … Driving_ECU 17 … Locator unit 17a … GNSS receiver 17b … Road map DB 18lf ... Left front side sensor 18lr ... Left rear side sensor 18rf … Right front side sensor 18rr … Right rear side sensor 22 … E / G_ECU 23 … T / M_ECU 24 … BK_ECU 25 … PS_ECU 32 … Throttle actuator 33 ... Hydraulic control circuit 34 ... Brake actuator 35 ... Electric power steering motor 37 ... Vehicle speed sensor 38 … Yaw rate sensor

Claims

1. lane marking recognition means for recognizing lane marking information ahead of the vehicle at set intervals based on sensing information of a driving environment ahead of the vehicle; a relative coordinate calculation means for extracting a temporary complement point from the lane marking information at each of the set cycles and calculating a relative coordinate of the temporary complement point with respect to the vehicle; a first complement point calculation means for updating the relative coordinates of the temporary complement points up to the present based on the running information of the host vehicle for each set period and calculating a first lane line complement point for complementing the lane line information up to the rear of the host vehicle; a first evaluation value acquisition means for acquiring a first evaluation value related to detection accuracy of the driving information at each set period; a second complement point calculation means for calculating absolute position information of the temporary complement point from the relative coordinates of the temporary complement point based on absolute position information of the host vehicle measured at each set period using signals from a plurality of positioning satellites, and calculating a second lane line complement point for complementing the lane line information up to the rear of the host vehicle based on the absolute position information of the temporary complement point up to the present and the current absolute position information of the host vehicle; a second evaluation value acquisition means for acquiring a second evaluation value related to detection accuracy of the absolute position information of the vehicle at each set period; a complement point integration means for integrating the corresponding first and second lane line complement points using a reliability calculated based on the first evaluation value and the second evaluation value.

2. a yaw rate detection means for detecting a yaw rate acting on the host vehicle as the travel information; a vehicle speed detection means for detecting a vehicle speed of the host vehicle as the travel information, 2. The lane marking recognition device for a vehicle according to claim 1, wherein the first detection accuracy acquisition means calculates the first evaluation value based on at least one of the detection accuracy of the yaw rate detection means and the detection accuracy of the vehicle speed detection means, which change depending on the driving environment.

3. 2. The lane marking recognition device for a vehicle according to claim 1, wherein the second evaluation value acquisition means calculates the second evaluation value based on at least one of the number of the positioning satellites and the signal strength from the positioning satellites when the absolute position information is measured.

4. A processor is provided. The processor, Based on sensing information of the driving environment in front of the vehicle, lane marking information in front of the vehicle is recognized at set intervals, extracting a temporary complement point from the lane marking information at each set period, and calculating a relative coordinate of the temporary complement point with respect to the vehicle; updating the relative coordinates of the temporary complement points up to the present based on the running information of the host vehicle for each set period, and calculating a first lane marking line complement point for complementing the lane marking line information up to a rear of the host vehicle; A first evaluation value relating to detection accuracy of the driving information is obtained for each set period; calculating absolute position information of the temporary complement point from the relative coordinates of the temporary complement point based on absolute position information of the host vehicle measured at each set period using signals from a plurality of positioning satellites, and calculating a second lane marking line complement point for complementing the lane marking line information up to the rear of the host vehicle based on the absolute position information of the temporary complement point up to the present and the current absolute position information of the host vehicle; obtaining a second evaluation value related to detection accuracy of the absolute position information of the vehicle for each set period; A vehicle lane marking recognition device, characterized in that the corresponding first marking line completion point and the second marking line completion point are integrated using a reliability calculated based on the first evaluation value and the second evaluation value.

Images