Outdoor environment recognition device

Abstract

To appropriately identify a road surface.SOLUTION: A road surface identification unit of a vehicle exterior environment recognition device identifies a road surface area corresponding to a road surface in an image, plots the representative distance for each horizontal line in the road surface area at the vertical position to generate a land road surface model (S300), identifies a left lane boundary and plots the relative distance for each horizontal line on the lane boundary at the vertical position to generate a left white road surface model (S302), identifies a right lane boundary and plots the relative distance for each horizontal line on the lane boundary at the vertical position to generate a right white road surface model (S304), and derives a road surface model based on the positional relationship between the land road surface model, the left white road surface model, and the right white road surface model (S310, S314, S318, S320).SELECTED DRAWING: Figure 11

Description

Technical Field

[0001] The present invention relates to an out-of-vehicle environment recognition device that identifies a three-dimensional object existing in the traveling direction of a host vehicle.

Background Art

[0002] Patent Document 1 discloses a technique for detecting a three-dimensional object such as a preceding vehicle located in front of a host vehicle and reducing a collision damage with the preceding vehicle, and a technique for performing follow-up control so as to keep a distance between the host vehicle and the preceding vehicle at a safe distance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Examples of three-dimensional objects existing in the traveling direction of a host vehicle include a preceding vehicle traveling in the same direction and an oncoming vehicle traveling in the opposite direction. In the host vehicle, a three-dimensional object is extracted on the condition that it has a height vertically above the road surface. Therefore, in the host vehicle, first, the road surface must be appropriately identified. If the road surface cannot be appropriately identified, the host vehicle may not be able to extract a three-dimensional object that should originally exist, or may determine a sloped road surface itself as a three-dimensional object.

[0005] In view of such problems, an object of the present invention is to provide an out-of-vehicle environment recognition device capable of appropriately identifying a road surface.

Means for Solving the Problems

[0006] In order to solve the above problems, an out-of-vehicle environment recognition device according to the present invention includes a road surface specifying unit that generates a road surface model corresponding to a road surface in an image, and a three-dimensional object specifying unit that groups blocks located vertically above the specified road surface model to specify a three-dimensional object. The road surface specifying unit specifies a left lane boundary line in the image and a right lane boundary line located on one side in the horizontal direction of the image screen with respect to the left lane boundary line in the image, specifies a road surface area corresponding to the road surface in the image based on the left lane boundary line and the right lane boundary line, calculates a representative value of relative distances for each of a plurality of horizontal lines having equal positions in the vertical direction, which is the vertical direction of the image screen in the road surface area, and on a plane including a first axis indicating the relative distance and a second axis indicating the position in the vertical direction, generates a land road surface model, which is a line approximating a point group representing the representative value and the position in the vertical direction of each of the plurality of horizontal lines, generates a left white line road surface model, which is a line approximating a point group representing the relative distance for each position in the vertical direction of the left lane boundary line on the plane, generates a right white line road surface model, which is a line approximating a point group representing the relative distance for each position in the vertical direction of the right lane boundary line on the plane, specifies one end point having the smallest relative distance among a first end point, which is a point having the largest relative distance of the land road surface model, a second end point, which is a point having the largest relative distance of the left white line road surface model, and a third end point, which is a point having the largest relative distance of the right white line road surface model, and based on the positional relationship of a first corresponding point, which is an intersection of the reference line, which is a straight line parallel to the second axis and passing through the one end point, and the land road surface model, a second corresponding point, which is an intersection of the reference line and the left white line road surface model, and a third corresponding point, which is an intersection of the reference line and the right white line road surface model on the plane, selects one or more of the land road surface model, the left white line road surface model, and the right white line road surface model, and derives a road surface model based on one or more of the selected land road surface model, the left white line road surface model, and the right white line road surface model. When the specific road surface portion is such that the first corresponding point is located between the second corresponding point and the third corresponding point on the reference line, the distance between the first corresponding point and the second corresponding point is greater than a predetermined value, and the distance between the first corresponding point and the third corresponding point is greater than the predetermined value, the land road surface model, the left white line road surface model, and the right white line road surface model may be selected. When the specific road surface portion is such that the first corresponding point is not located between the second corresponding point and the third corresponding point on the reference line, the distance between the first corresponding point and the second corresponding point is greater than a predetermined value, and the distance between the first corresponding point and the third corresponding point is greater than the predetermined value, the land road surface model may be selected. When the distance between the first corresponding point and the second corresponding point is less than or equal to a predetermined value and the first corresponding point is located closer to the second corresponding point than the third corresponding point on the reference line, the land road surface model and the left white line road surface model are selected. When the distance between the first corresponding point and the third corresponding point is less than or equal to the predetermined value and the first corresponding point is located closer to the third corresponding point than the second corresponding point on the reference line, the land road surface model and the right white line road surface model may be selected. When the specific road surface part is such that the first corresponding point is located between the second corresponding point and the third corresponding point on the reference line, and the distance between the first corresponding point and the second corresponding point is greater than a predetermined value, and the distance between the first corresponding point and the third corresponding point is greater than the predetermined value, the land road surface model, the left white line road surface model, and the right white line road surface model are selected, and the land road surface model, the left white line road surface model, and the right white line road surface model are mixed to derive the road surface model. When the first corresponding point is not located between the second corresponding point and the third corresponding point on the reference line, and the distance between the first corresponding point and the second corresponding point is greater than the predetermined value, and the distance between the first corresponding point and the third corresponding point is greater than the predetermined value, the land road surface model is selected and the road surface model is derived based on the land road surface model. When the distance between the first corresponding point and the second corresponding point is less than or equal to the predetermined value and the first corresponding point is located closer to the second corresponding point than the third corresponding point on the reference line, the land road surface model and the left white line road surface model are selected, and the land road surface model and the left white line road surface model are mixed to derive the road surface model. When the distance between the first corresponding point and the third corresponding point is less than or equal to the predetermined value and the first corresponding point is located closer to the third corresponding point than the second corresponding point on the reference line, the land road surface model and the right white line road surface model may be selected, and the land road surface model and the right white line road surface model may be mixed to derive the road surface model 。

Advantages of the Invention

[0008] According to the present invention, the road surface can be appropriately specified.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, with reference to the accompanying drawings, preferred embodiments of the present invention will be described in detail. The dimensions, materials, and other specific numerical values shown in such embodiments are merely examples for facilitating the understanding of the invention, and do not limit the present invention unless otherwise specified. In the present specification and drawings, elements having substantially the same function and configuration are denoted by the same reference numerals to omit redundant description, and elements not directly related to the present invention are not shown.

[0011] (Vehicle exterior environment recognition system 100) FIG. 1 is a block diagram showing the connection relationship of the vehicle exterior environment recognition system 100. The vehicle exterior environment recognition system 100 is provided in the host vehicle 1 and includes an imaging device 110, a vehicle exterior environment recognition device 120, and a vehicle control device 130.

[0012] The imaging device 110 is composed of an imaging element such as a CCD (Charge-Coupled Device) or a CMOS (Complementary Metal-Oxide Semiconductor). The imaging devices 110 are arranged at a distance in a substantially horizontal direction such that the optical axes of the two imaging devices 110 are substantially parallel on the traveling direction side of the host vehicle 1. The imaging device 110 images the vehicle exterior environment in front of the host vehicle 1 and generates a luminance image (color image or monochrome image) including at least luminance information. The imaging device 110 continuously generates a luminance image obtained by imaging a three-dimensional object existing in the detection area in front of the host vehicle 1, for example, every frame of 1 / 60 second (60 fps). Here, the three-dimensional objects and specific objects recognized by the vehicle exterior environment recognition device 120 include not only independently existing objects such as bicycles, pedestrians, vehicles, traffic lights, road signs, guardrails, buildings, and side walls on the roadside, but also objects that can be specified as a part thereof, such as the rear and side surfaces of a vehicle and the wheels of a bicycle. Here, the rear surface of the vehicle refers to the surface facing the host vehicle 1 in front of the host vehicle 1, and does not refer to the rear surface of the vehicle itself.

[0013] The external environment recognition device 120 acquires luminance images from each of the two imaging devices 110 and generates a distance image using so-called pattern matching. Based on the luminance image and the distance image, the external environment recognition device 120 groups, as a three-dimensional object, blocks that are located above the road surface, have equal color values, and whose three-dimensional position information is adjacent to each other. Then, the external environment recognition device 120 identifies which specific object the three-dimensional object is. For example, the external environment recognition device 120 identifies that the three-dimensional object is a preceding vehicle. Also, when the external environment recognition device 120 identifies the preceding vehicle in this way, it performs collision damage mitigation control between the host vehicle 1 and the preceding vehicle, and also performs follow-up control to make the host vehicle 1 follow the preceding vehicle. The specific operations of the external environment recognition device 120 will be described in detail later.

[0014] The vehicle control device 130 is composed of an ECU (Electronic Control Unit) or the like, receives the driver's operation inputs through the steering wheel 132, the accelerator pedal 134, and the brake pedal 136, and controls the host vehicle 1 by transmitting them to the steering mechanism 142, the drive mechanism 144, and the braking mechanism 146. Also, the vehicle control device 130 controls the steering mechanism 142, the drive mechanism 144, and the braking mechanism 146 according to the instructions of the external environment recognition device 120.

[0015] (External environment recognition device 120) FIG. 2 is a functional block diagram showing the schematic functions of the external environment recognition device 120. As shown in FIG. 2, the external environment recognition device 120 includes an I / F unit 150, a data holding unit 152, and a central control unit 154.

[0016] The I / F unit 150 is an interface for performing two-way information exchange with the imaging device 110 and the vehicle control device 130. The data holding unit 152 is composed of a RAM, a flash memory, an HDD, etc., and holds various information necessary for the processing of each of the following functional units.

[0017] The central control unit 154 is composed of a semiconductor integrated circuit including a central processing unit (CPU), a ROM storing programs and the like, a RAM serving as a work area, etc., and controls the I / F unit 150, the data holding unit 152, etc. through the system bus 156. Further, in the present embodiment, the central control unit 154 also functions as a position derivation unit 170, a road surface identification unit 172, a three-dimensional object identification unit 174, and a specific object identification unit 176. Hereinafter, the method for recognizing the external environment of the vehicle will be described in detail in consideration of the operations of the respective functional units of the central control unit 154.

[0018] (Method for Recognizing External Environment of Vehicle) FIG. 3 is a flowchart showing the flow of the method for recognizing the external environment of the vehicle. The external environment recognition device 120 executes the method for recognizing the external environment of the vehicle at every predetermined interruption time. In the method for recognizing the external environment of the vehicle, the position derivation unit 170 derives the three-dimensional position in block units in the luminance image acquired from the imaging device 110 (position derivation process S200). The road surface identification unit 172 generates a road surface model corresponding to the road surface in the image (road surface identification process S202). The three-dimensional object identification unit 174 groups the blocks located vertically above the identified road surface model to identify three-dimensional objects (three-dimensional object identification process S204). The specific object identification unit 176 identifies specific objects such as the preceding vehicle from the three-dimensional objects (specific object identification process S206).

[0019] (Position Derivation Process S200) FIG. 4 (FIGS. 4A to 4C) is an explanatory diagram for explaining the luminance image and the distance image. The position derivation unit 170 acquires a plurality (here, 2) of luminance images captured at the same timing with different optical axes by the imaging device 110. Here, it is assumed that the position derivation unit 170 has acquired, as the luminance image 212, the first luminance image 212a captured by the imaging device 110 located relatively on the right side of the host vehicle 1 shown in FIG. 4A and the second luminance image 212b captured by the imaging device 110 located relatively on the left side of the host vehicle 1 shown in FIG. 4B.

[0020] Referring to FIGS. 4A and 4B, it can be understood that due to the difference in the imaging positions of the imaging device 110, the image positions of the three-dimensional objects included in the images are different in the horizontal direction between the first luminance image 212a and the second luminance image 212b. Here, the horizontal direction indicates the horizontal direction of the captured image screen, and the vertical direction indicates the vertical direction of the captured image screen.

[0021] Based on the first luminance image 212a shown in FIG. 4A and the second luminance image 212b shown in FIG. 4B, both of which are acquired by the position derivation unit 170, the position derivation unit 170 generates a distance image 214 as shown in FIG. 4C that can specify the distance of the imaging target.

[0022] Specifically, the position derivation unit 170 uses so-called pattern matching to derive parallax and parallax information including the image position indicating the position within the image of any block. Then, the position derivation unit 170 searches for a block corresponding to a block arbitrarily extracted from one luminance image (here, the first luminance image 212a) from the other luminance image (here, the second luminance image 212b). Here, the block is represented by an array of, for example, 4 horizontal pixels × 4 vertical pixels. Also, pattern matching is a method of searching for a block corresponding to a block arbitrarily extracted from one luminance image from the other luminance image.

[0023] For example, as functions for evaluating the degree of coincidence between blocks in pattern matching, there are methods such as SAD (Sum of Absolute Difference) that takes the difference in luminance, SSD (Sum of Squared intensity Difference) that uses the squared difference, and NCC (Normalized Cross Correlation) that takes the similarity of the variance value obtained by subtracting the average value from the luminance of each pixel.

[0024] The position derivation unit 170 performs such parallax derivation processing in units of blocks for all blocks mapped in a detection area of, for example, 600 pixels × 200 pixels. Here, the position derivation unit 170 processes the blocks as 4 pixels × 4 pixels, but it may also be processed with any number of pixels.

[0025] The position derivation unit 170 converts the parallax information for each block of the distance image 214 into a three-dimensional position in real space including the horizontal distance x, height y, and relative distance z using a so-called stereo method. Here, the stereo method is a technique that uses triangulation to derive the relative distance z of a block with respect to the imaging device 110 from the parallax in the distance image 214 of the block. At this time, the position derivation unit 170 derives the height y of the block from the road surface based on the relative distance z of the block and the detection distance on the distance image 214 between the point on the road surface at the same relative distance z as the block and the block. Then, the position derivation unit 170 re-associates the derived three-dimensional position with the distance image 214. Since various known techniques can be applied to the derivation process of such relative distance z and the identification process of three-dimensional position, the description thereof is omitted here.

[0026] (Road surface identification process S202) The road surface identification unit 172 first identifies a road surface area corresponding to the road surface in the luminance image 212 and the distance image 214.

[0027] Specifically, the road surface identification unit 172 identifies the road surface in front of the host vehicle 1 based on lane boundary lines such as white lines located on the left and right of the lane in which the host vehicle 1 is traveling. Note that the road surface identification unit 172 can also identify the road surface in front of the host vehicle 1 based on not only the lane boundary lines on the left and right of the lane but also three-dimensional objects such as road markings, walls, steps provided on the road shoulder, poles, and barriers.

[0028] FIG. 5 (FIGS. 5A and 5B) is an explanatory diagram for explaining the road surface identification process. Lane boundary lines for smooth vehicle travel are marked on the road surface. For example, in the example of FIG. 5A, on the road 220 in the luminance image 212, there are two lanes 222a and 222b, and accordingly, a total of three lane boundary lines 230a, 230b, and 230c, one in the center in the width direction and one at each end, are included.

[0029] Note that the road surface specific part 172 makes the blocks having the same color values of white or yellow and the difference in three-dimensional positions within a predetermined range in the distance image 214 continuous in the depth direction to specify each of the lane boundary lines 230a, 230b, and 230c. Therefore, the road surface specific part 172 grasps the three-dimensional positions of the front-end points and the depth-direction end points of each of the lane boundary lines 230a, 230b, and 230c.

[0030] Here, as shown in FIG. 5B, the road surface specific part 172 sets a virtual left limit line 232a indicated by a broken line at a position further to the left by a predetermined distance (for example, 10 cm) from the left end of the lane boundary line 230a located to the left of the lane 222a in which the host vehicle 1 is traveling. Further, the road surface specific part 172 sets a virtual right limit line 232b indicated by a broken line at a position further to the right by a predetermined distance (for example, 10 cm) from the right end of the lane boundary line 230b located to the right of the lane 222a in which the host vehicle 1 is traveling. Then, the road surface specific part 172 sets a region to the right of the left limit line 232a and to the left of the right limit line 232b, that is, the hatched region in FIG. 5B as the road surface region 234.

[0031] Subsequently, the road surface specific part 172 extracts all the blocks in the road surface region 234 for which the relative distance z has been obtained by pattern matching, and generates a land road surface model. The land road surface model is a road surface model generated for the road surface region 234, and is distinguished from the left white line road surface model and the right white line road surface model generated for the lane boundary lines 230a and 230b described later. Hereinafter, the generation procedure of the land road surface model will be described.

[0032] FIG. 6 (FIGS. 6A and 6B) is an explanatory diagram for explaining a histogram, and FIGS. 7 (FIGS. 7A and 7B) and 8 (FIGS. 8A and 8B) are explanatory diagrams for explaining a land road surface model. The road surface specifying unit 172 first generates a histogram of the relative distance z for a horizontal line extending in the horizontal direction of the road surface area 234. Specifically, in the road surface area 234, the road surface specifying unit 172 votes on the relative distance z of all the blocks on the horizontal line having the same vertical position as an arbitrary block in the vertical direction, that is, the blocks shown by cross-hatching in FIG. 6A. Thus, as shown in FIG. 6B, a histogram for an arbitrary vertical position is generated.

[0033] Then, the road surface specifying unit 172 sets the relative distance z shown by hatching in FIG. 6B, at which the number of votes in the histogram is the maximum number, as the representative distance at that vertical position. The road surface specifying unit 172 sequentially changes the vertical position and derives the representative distance for all the vertical positions in the road surface area 234.

[0034] The land road surface model represents the gradient in the depth direction of the road surface area 234 by a yz plane with the vertical axis being the vertical position y and the horizontal axis being the relative distance z. The road surface specifying unit 172 plots the representative distance for each vertical position y at that vertical position y. Then, a point group as shown in FIG. 7A is obtained. The road surface specifying unit 172 derives an approximate straight line of the point group, as shown by the solid line in FIG. 7A, by the least squares method or the like, and uses it as the land road surface model. Thus, the road surface specifying unit 172 can derive the gradient of the road surface. Here, for the sake of convenience of explanation, an example in which the road surface specifying unit 172 derives an approximate straight line is given, but the road surface specifying unit 172 may derive a curve of a higher order of approximation.

[0035] However, within the road surface area 234, there may be a case where a representative distance corresponding to noise that has been erroneously pattern-matched is included. If the road surface specifying unit 172 simply derives an approximate straight line by the least squares method or the like, there is a possibility that, as shown in FIG. 7A, the approximate straight line may deviate from the original position and inclination due to the influence of noise.

[0036] Therefore, the road surface specifying unit 172 uses the Hough transform for straight lines, retains only the representative distances that form the same straight line or parallel straight lines that are not the same but within a predetermined relative distance, and excludes the other representative distances, for example, the representative distances surrounded by broken lines in FIG. 7B, as noise. Then, the road surface specifying unit 172 derives an approximate straight line of such a point group by the least squares method or the like, targeting only the remaining representative distances. In this way, as shown in FIG. 7B, the land road surface model 236 appropriately approximated by the original representative distances is derived. Note that since the Hough transform for straight lines is an existing technique for deriving a straight line passing through a plurality of points in common, the description thereof is omitted here.

[0037] In FIG. 7, an example where the gradient of the road surface does not change within the road surface area 234 has been described. However, in the road surface area 234, the gradient of the road surface is not necessarily constant and may change midway. In this case, if the road surface specifying unit 172 simply performs the Hough transform, the representative distances corresponding to the road surface after the gradient changes will be mostly excluded as noise.

[0038] For example, assume that the road surface specifying unit 172 obtains a point group of representative distances as shown in FIG. 8A. The road surface specifying unit 172 retains only the representative distances that form the same straight line or parallel straight lines that are not the same but within a predetermined relative distance, and excludes the other representative distances, for example, the representative distances surrounded by broken lines in FIG. 8A, as noise. Then, the road surface specifying unit 172 derives an approximate straight line of such a point group by the least squares method or the like, targeting only the remaining representative distances.

[0039] However, among the representative distances excluded as noise, the point group 238 has a predetermined number or more and the point group has continuity. Also, above a predetermined vertical position, there are no representative distances in the vicinity of the approximate straight line. That is, the point group 238 is not noise but shows a correct road surface with a different gradient.

[0040] Therefore, when there are a predetermined number or more of representative distances excluded by the Huff transform and the point group has continuity, the road surface specifying unit 172 newly derives an approximate straight line only from the point group 238. Specifically, as shown in FIG. 8B, the road surface specifying unit 172 leaves only the representative distances that form the same straight line or parallel straight lines within a predetermined relative distance (although not the same) among the point groups 238 located above a predetermined vertical position, and excludes the other representative distances, for example, the representative distances surrounded by the broken lines in FIG. 8B, as noise. Then, the road surface specifying unit 172 derives an approximate straight line of such a point group by the least squares method or the like, targeting only the remaining representative distances.

[0041] In this way, a land road surface model 236 in which two approximate straight lines 236a and 236b having different gradients are continuous at a predetermined vertical position is newly generated. Note that the intersection of the two derived approximate straight lines may intersect at a predetermined angle, or may be connected through a transition curve having a predetermined R. Also, here, an example in which the road surface specifying unit 172 generates two approximate straight lines has been described, but when the gradient change of the road surface is large, three or more may be generated.

[0042] By the way, on a curved portion of the road on which the host vehicle 1 travels, there may be a cant provided so that the vehicle can travel smoothly. When there is such a cant on the road, the specific accuracy of the representative distance of the land road surface model 236 may decrease.

[0043] FIG. 9 (FIGS. 9A and 9B) is an explanatory diagram for explaining the land road surface model 236 when a cant is provided. As described with reference to FIG. 6, the road surface specifying unit 172 votes on the relative distances z of all the blocks on the horizontal line having the same vertical position as an arbitrary block in the vertical direction in the road surface area 234.

[0044] Here, when there is no cant or it is equivalent to having no cant, the relative distances z of the blocks on the horizontal line are approximately equal. Therefore, as shown in FIG. 6B, in the histogram, the number of votes for the relative distance z of 1 becomes significantly larger than the number of votes for the other relative distances z, and the representative distance is uniquely specified.

[0045] On the one hand, when there is a cant, the horizontal gradient of the road disperses the relative distance z. For example, when there is a cant with a gradually increasing elevation from left to right, even if the vertical position in the distance image 214 is the same, the relative distance z of the left block becomes larger and the relative distance z of the right block becomes smaller. Also, when the road is evenly inclined, the number of votes is also even with respect to the relative distance z, so as shown in FIG. 9A, the number of votes of 1 does not stand out. Here, when the road surface specifying unit 172 plots the representative distance for each vertical position y at that vertical position y, as shown in FIG. 9B, the representative distance is dispersed within a predetermined range of the relative distance z surrounded by the broken line.

[0046] Also, in the histogram, in order to prevent a noisy relative distance z from becoming the representative distance, the representative distance is specified on the condition that the number of votes exceeds a predetermined threshold. However, as shown in FIG. 9B, when the relative distance z is dispersed, none of the relative distances z exceeds the threshold, and there may be a case where the representative distance itself cannot be specified. Thus, the specification accuracy of the representative distance decreases, and there may be a case where the land road surface model 236 is not effectively generated.

[0047] Here, it is also possible to specify the horizontal gradient in the road surface area 234 and treat the road surface area 234 as a plane, but the processing load will increase significantly. Therefore, in addition to the land road surface model 236, a left white line road surface model and a right white line road surface model based on the left and right lane boundary lines 230a, 230b are used, and one road surface model is derived based on their positional relationship.

[0048] FIG. 10 (FIGS. 10A and 10B) is an explanatory diagram for explaining a left white line road surface model and a right white line road surface model. The road surface specifying unit 172 plots the relative distance z for each vertical position y of the lane boundary line 230a located on the left, at that vertical position y. For example, in the example of FIG. 10A, the representative distance of the road surface area 234 is indicated by a black circle (●), and the relative distance z of the lane boundary line 230a located on the left is indicated by a white circle (○). The road surface specifying unit 172 derives an approximate straight line of such a point group by the least squares method or the like, targeting only the relative distance z of the lane boundary line 230a located on the left. Thus, as shown in FIG. 10A, a left white line road surface model 240, which is a road surface model based on the relative distance z of the lane boundary line 230a located on the left, is generated. As can be understood with reference to FIG. 10A, the left white line road surface model 240 may bear the boundary of the scattered representative distances in the road surface area 234.

[0049] Also, the road surface specifying unit 172 plots the relative distance z for each vertical position y of the lane boundary line 230b located on the right, in the same manner as the left white line road surface model 240, at that vertical position y. For example, in the example of FIG. 10B, the representative distance of the road surface area 234 is indicated by a black circle (●), and the relative distance z of the lane boundary line 230b located on the right is indicated by a white circle (○). The road surface specifying unit 172 derives an approximate straight line of such a point group by the least squares method or the like, targeting only the relative distance z of the lane boundary line 230b located on the right. Thus, as shown in FIG. 10B, a right white line road surface model 242, which is a road surface model based on the relative distance z of the lane boundary line 230b located on the right, is generated. As can be understood with reference to FIG. 10B, the right white line road surface model 242 may bear the boundary on the opposite side of the left white line road surface model 240 of the scattered representative distances in the road surface area 234.

[0050] The widths of the lane boundary lines 230a and 230b are defined, and they are shorter than the road surface area 234. Then, the road surface specifying unit 172 can stably obtain a uniform relative distance z regardless of whether the road is cambered or not. Therefore, although the left white line road surface model 240 and the right white line road surface model 242 are not road surface models of the road surface area 234 itself, they may have a higher specific accuracy than the land road surface model 236.

[0051] When generating the land road surface model 236, the left white line road surface model 240, and the right white line road surface model 242, the road surface specifying unit 172 derives an optimal road surface model based on the positional relationship among these three road surface models.

[0052] FIG. 11 is a flowchart showing the flow of the road surface model generation process of the road surface specifying unit 172, and FIGS. 12 to 15 are explanatory diagrams for explaining the derivation process of the road surface model. In FIGS. 12 to 15, the land road surface model 236 is indicated by a dashed line, the left white line road surface model 240 is indicated by a one-dot chain line, and the right white line road surface model 242 is indicated by a two-dot chain line. The road surface specifying unit 172 executes the road surface model generation process at every predetermined interrupt time.

[0053] First, the road surface specifying unit 172 specifies the road surface area 234, plots the representative distance for each horizontal line in the road surface area 234 at its vertical position to generate the land road surface model 236 (S300). Next, the road surface specifying unit 172 specifies the left lane boundary line 230a, plots the relative distance z for each horizontal line in the lane boundary line 230a at its vertical position to generate the left white line road surface model 240 (S302). Subsequently, the road surface specifying unit 172 specifies the right lane boundary line 230b, plots the relative distance z for each horizontal line in the lane boundary line 230b at its vertical position to generate the right white line road surface model 242 (S304).

[0054] Subsequently, the road surface specifying unit 172 derives a road surface model 244 by mixing any one or more of the land road surface model 236, the left white line road surface model 240, and the right white line road surface model 242 based on the positional relationship with reference to the end point in the depth direction of any one of the land road surface model 236, the left white line road surface model 240, and the right white line road surface model 242.

[0055] Specifically, in step S300, the road surface specifying unit 172 determines whether the land road surface model 236 has been effectively generated (S306). Here, if the land road surface model 236 cannot be effectively generated, such as when the distance information is not sufficiently obtained (NO in S306), as shown in FIG. 12, the road surface specifying unit 172 determines whether the shorter one of the end points in the depth direction of the left white line road surface model 240 and the right white line road surface model 242, here the end point of the right white line road surface model 242, and the corresponding point of the left white line road surface model 240 with the same relative distance z from the end point satisfy the first condition (S308). The first condition is that the relative distance z between the end point of the right white line road surface model 242 and the corresponding point of the left white line road surface model 240 is both a predetermined distance, for example, 30 m or more, and the vertical distance between the end point of the right white line road surface model 242 and the corresponding point of the left white line road surface model 240 is separated by a predetermined number of pixels, for example, 7 pixels or more.

[0056] Here, if the first condition is satisfied (YES in S308), the road surface specifying unit 172 determines that the road is banked, and as shown in FIG. 12, the left white line road surface model 240 and the right white line road surface model 242 are mixed at a ratio of 1:1 to generate a road surface model 244 (S310). Note that "mixing" in this embodiment means averaging each point of the target model among the land road surface model 236, the left white line road surface model 240, and the right white line road surface model 242 for each same relative distance. Also, if the first condition is not satisfied (NO in S308), the road surface specifying unit 172 ends the road surface model generation process.

[0057] Also, if the land road surface model 236 is effectively generated (YES in S306), as shown in FIG. 13, the road surface specifying unit 172 determines whether the shortest end point among the depth direction end points of the land road surface model 236, the left white line road surface model 240, and the right white line road surface model 242, here the end point of the right white line road surface model 242, and the corresponding points of the land road surface model 236 and the left white line road surface model 240 with the same relative distance z from the end point satisfy the second condition (S312). The second condition is that the corresponding point of the land road surface model 236 is located between the position 300 mm in the direction of the left white line road surface model 240 from the end point of the right white line road surface model 242 and the position 300 mm in the direction of the right white line road surface model 242 from the corresponding point of the left white line road surface model 240.

[0058] Here, if the second condition is satisfied (YES in S312), as shown in FIG. 13, the road surface specifying unit 172 generates the road surface model 244 by mixing the land road surface model 236, the left white line road surface model 240, and the right white line road surface model 242 at a ratio of 1:1:1 (S314).

[0059] Also, if the second condition is not satisfied (NO in S312), as shown in FIG. 14, the road surface specifying unit 172 determines whether the shortest end point among the depth direction end points of the land road surface model 236, the left white line road surface model 240, and the right white line road surface model 242, here the end point of the right white line road surface model 242, and the corresponding points of the land road surface model 236 and the left white line road surface model 240 with the same relative distance z from the end point satisfy the third condition (S316). The third condition is that the corresponding point of the land road surface model 236 is located within the range of ±300 mm in the vertical direction from the end point of the right white line road surface model 242 or within the range of ±300 mm in the vertical direction from the corresponding point of the left white line road surface model 240.

[0060] Here, if the third condition is satisfied (YES in S316), as shown in FIG. 14, the road surface specifying unit 172 generates the road surface model 244 by mixing, at a ratio of 1:1, the one of the left white line road surface model 240 and the right white line road surface model 242 closer to the corresponding point of the land road surface model 236 and the land road surface model 236 (S318).

[0061] Also, if the third condition is not satisfied (YES in S316), the road surface specifying unit 172 gives priority to the road surface area 234, and as shown in FIG. 15, directly adopts only the land road surface model 236 without using the left white line road surface model 240 and the right white line road surface model 242 (S320).

[0062] With such a configuration, even when there is a cant on the road, it is possible to derive a road surface model 244 that appropriately represents the vertical position y and the relative distance z.

[0063] (Three-dimensional object specifying process S204) Returning to the description of FIG. 3, the three-dimensional object specifying unit 174 is located above the road surface model 244 specified by the road surface specifying unit 172, groups blocks having the same color value and within a predetermined range of the difference in three-dimensional positions in the distance image 214 to specify a three-dimensional object. Specifically, the three-dimensional object specifying unit 174 groups blocks in the distance image 214 whose differences in horizontal distance x, height y, and relative distance z are within a predetermined range (for example, 0.1 m), assuming that they correspond to the same specified object. In this way, a virtual block group is generated.

[0064] FIG. 16 (FIGS. 16A to 16C) is an explanatory diagram for explaining the three-dimensional object specifying process S204. Here, assume that the position deriving unit 170 generates a distance image 214 as shown in FIG. 16A, for example. The three-dimensional object specifying unit 174 groups blocks from such a distance image 214. In this way, a plurality of grouped block groups are extracted as shown in FIG. 16B. In FIG. 16B, the three-dimensional object specifying unit 174 sets an outer contour line including all the grouped blocks, for example, a rectangular frame (plane) composed of a horizontal line and a vertical line, or a line extending in the depth direction and a vertical line, as the three-dimensional object 250 (250a, 250b, 250c, 250d, 250e, 250f, 250g).

[0065] In FIG. 16B, when the three-dimensional objects 250a, 250b, 250c, 250d, 250e, 250f, 250g on the distance image 214 are represented on a two-dimensional horizontal plane indicated by the horizontal distance x and the relative distance z, they become the three-dimensional objects 250a, 250b, 250c, 250d, 250e, 250f, 250g in FIG. 16C.

[0066] (Specific object identification process S206) The specific object identification unit 176 identifies which specific object the three-dimensional object identified by the three-dimensional object identification unit 174 is. For example, in front of the host vehicle 1, the specific object identification unit 176 identifies a three-dimensional object traveling in the same direction as the host vehicle 1 as a preceding vehicle.

[0067] In FIG. 16C, the three-dimensional object identification unit 174 identifies the three-dimensional object 250b and the three-dimensional object 250c as different three-dimensional objects. However, in reality, the three-dimensional object 250b constitutes the rear surface of the preceding vehicle, and the three-dimensional object 250c constitutes the side surface of the same preceding vehicle. Therefore, the three-dimensional objects 250b and 250c should originally be recognized as a pair constituting an integral and identical three-dimensional object. Similarly, the three-dimensional objects 250e and 250f should also be recognized as the same three-dimensional object. Therefore, the specific object identification unit 176 pairs the three-dimensional objects 250 that should be the rear surface and the side surface of the same three-dimensional object.

[0068] The specific object identification unit 176 determines, for example, whether a pair of the three-dimensional object 250b corresponding to the rear surface and the three-dimensional object 250c corresponding to the side surface satisfies the conditions as a preceding vehicle. Specifically, the specific object identification unit 176 determines whether the three-dimensional object constituted by the pair of the three-dimensional object 250b and the three-dimensional object 250c has a size, shape, and relative speed similar to a vehicle, and whether it has a light source such as a brake lamp or a high-mounted stop lamp at a predetermined position in the rear. When such conditions are satisfied, the specific object identification unit 176 identifies the three-dimensional object 250 constituted by the pair of the three-dimensional object 250b and the three-dimensional object 250c as a preceding vehicle.

[0069] As described above, in the present embodiment, since the road surface can be appropriately identified, specific objects can be accurately identified.

[0070] In addition, there are also provided a program for causing a computer to function as the out-vehicle environment recognition device 120, and a computer-readable storage medium such as a flexible disk, a magneto-optical disk, a ROM, a CD, a DVD, a BD, etc. that records the program. Here, the program refers to data processing means described in any language and description method.

[0071] As described above, the preferred embodiments of the present invention have been described with reference to the accompanying drawings. Needless to say, the present invention is not limited to such embodiments. It is obvious that those skilled in the art can conceive of various modification examples or correction examples within the scope described in the claims, and it is naturally understood that those also belong to the technical scope of the present invention.

[0072] For example, in the above-described embodiment, the road surface specifying unit 172 derives the road surface model 244 by mixing any one or more of the land road surface model 236, the left white line road surface model 240, and the right white line road surface model 242, for example, at a ratio of 1:1:1, based on the positional relationship among them. However, not limited to such a case, the road surface specifying unit 172 may derive the road surface model 244 by mixing the land road surface model 236, the left white line road surface model 240, and the right white line road surface model 242 at a ratio other than 1:1:1 according to their priorities and the like.

[0073] Note that each step of the out-vehicle environment recognition method in this specification does not necessarily need to be processed in time series in the order described as a flowchart, and may include parallel or subroutine processing.

Industrial Applicability

[0074] The present invention can be used in an out-vehicle environment recognition device that specifies a three-dimensional object existing in the traveling direction of the host vehicle.

Explanation of Reference Numerals

[0075] 110 Imaging device 120 Vehicle exterior environment recognition device 170 Position derivation unit 172 Road surface identification unit 174 Three-dimensional object identification unit 176 Specific object identification unit

Claims

1. A road surface specifying unit that generates a road surface model corresponding to the road surface in the image, A three-dimensional object specifying unit that groups blocks located vertically above the specified road surface model to specify a three-dimensional object, Comprising, The road surface specifying unit, Specifies the left lane boundary line in the image and the right lane boundary line located on one side in the horizontal direction of the image screen relative to the left lane boundary line in the image, Based on the left lane boundary line and the right lane boundary line, specifies a road surface area corresponding to the road surface in the image, Calculates a representative value of the relative distance for each of a plurality of horizontal lines having the same position in the vertical direction, which is the vertical direction of the image screen in the road surface area, On a plane including a first axis indicating the relative distance and a second axis indicating the position in the vertical direction, generates a land road surface model that is a line approximating a point group representing the representative value and the position in the vertical direction of each of the plurality of horizontal lines, On the plane, generates a left white line road surface model that is a line approximating a point group representing the relative distance for each position in the vertical direction of the left lane boundary line, On the plane, generates a right white line road surface model that is a line approximating a point group representing the relative distance for each position in the vertical direction of the right lane boundary line, Among a first end point that is a point with the largest relative distance of the land road surface model, a second end point that is a point with the largest relative distance of the left white line road surface model, and a third end point that is a point with the largest relative distance of the right white line road surface model, specifies one end point with the smallest relative distance, Based on the positional relationship between a first corresponding point that is an intersection of the reference line, which is a straight line parallel to the second axis and passing through the one end point, and the land road surface model, a second corresponding point that is an intersection of the reference line and the left white line road surface model, and a third corresponding point that is an intersection of the reference line and the right white line road surface model on the plane, selects one or more of the land road surface model, the left white line road surface model, and the right white line road surface model, An out-of-vehicle environment recognition device that derives the road surface model based on one or more of the selected land road surface model, the left white line road surface model, and the right white line road surface model.

2. The road surface specifying unit selects the land road surface model, the left white line road surface model, and the right white line road surface model when the first corresponding point is located between the second corresponding point and the third corresponding point on the reference line, the distance between the first corresponding point and the second corresponding point is greater than a predetermined value, and the distance between the first corresponding point and the third corresponding point is greater than the predetermined value. The vehicle external environment recognition device according to claim 1.

3. The road surface specifying unit selects the land road surface model when the first corresponding point is not located between the second corresponding point and the third corresponding point on the reference line, the distance between the first corresponding point and the second corresponding point is greater than a predetermined value, and the distance between the first corresponding point and the third corresponding point is greater than the predetermined value. The vehicle external environment recognition device according to claim 1.

4. The road surface specifying unit selects the land road surface model and the left white line road surface model when the distance between the first corresponding point and the second corresponding point is less than or equal to a predetermined value and the first corresponding point is located closer to the second corresponding point than the third corresponding point on the reference line, selects the land road surface model and the right white line road surface model when the distance between the first corresponding point and the third corresponding point is less than or equal to a predetermined value and the first corresponding point is located closer to the third corresponding point than the second corresponding point on the reference line. The vehicle external environment recognition device according to claim 1.

5. The road surface specifying unit selects the land road surface model, the left white line road surface model, and the right white line road surface model when the first corresponding point is located between the second corresponding point and the third corresponding point on the reference line, the distance between the first corresponding point and the second corresponding point is greater than a predetermined value, and the distance between the first corresponding point and the third corresponding point is greater than the predetermined value, and derives the road surface model by mixing the land road surface model, the left white line road surface model, and the right white line road surface model, selects the land road surface model when the first corresponding point is not located between the second corresponding point and the third corresponding point on the reference line, the distance between the first corresponding point and the second corresponding point is greater than a predetermined value, and the distance between the first corresponding point and the third corresponding point is greater than the predetermined value, and derives the road surface model based on the land road surface model. When the distance between the first corresponding point and the second corresponding point is equal to or less than the predetermined value, and the first corresponding point is located closer to the second corresponding point than the third corresponding point on the reference line, the land road surface model and the left white line road surface model are selected, and the land road surface model and the left white line road surface model are mixed to derive the road surface model. The vehicle external environment recognition device according to claim 1, wherein when the distance between the first corresponding point and the third corresponding point is equal to or less than the predetermined value, and the first corresponding point is located closer to the third corresponding point than the second corresponding point on the reference line, the land road surface model and the right white line road surface model are selected, and the land road surface model and the right white line road surface model are mixed to derive the road surface model.

Images