External environment recognition device

Abstract

The present invention provides an external vehicle environment recognition device, which appropriately determines a road surface. The external vehicle environment recognition device uses a computer as a first road surface determination unit and a second road surface determination unit, wherein the first road surface determination unit determines a first road surface area corresponding to a road surface in an image, and plots the representative distance of each horizontal line in the first road surface area at its vertical position to generate a first road surface model (216), and the second road surface determination unit determines a second road surface area, which is equivalent to a road surface farther than the first road surface area and has a longer horizontal length than the first road surface area when converted into a three-dimensional space, and plots the representative distance of each horizontal line in the second road surface area at its vertical position to generate a second road surface model (228).

Description

Technical Field

[0001] The present invention relates to a vehicle exterior environment recognition device for identifying a specific object existing in the traveling direction of a vehicle. Background Art

[0002] In the past, the following technologies are known (for example, Patent Document 1): detecting a solid object such as a vehicle in front of the vehicle to avoid a collision with the preceding vehicle (collision avoidance control) or controlling the vehicle to maintain a safe distance from the preceding vehicle (cruise control).

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent No. 3349060 Summary of the invention

[0006] Technical issues

[0007] The three-dimensional objects existing in the traveling direction of the host vehicle include the preceding vehicle traveling in the same direction, pedestrians crossing the traveling road in the lateral direction of the host vehicle, etc. One of the extraction conditions of such three-dimensional objects is that they have a height vertically upward from the road surface.

[0008] However, the extraction of the three-dimensional object is premised on being able to properly determine the road surface. If the road surface cannot be properly determined, the road with a slope itself will be judged as a three-dimensional object, or the three-dimensional object that should exist cannot be extracted. In particular, for a road in the distance, the image (parallax) representing the road surface has low reliability, making it difficult to determine the road surface.

[0009] The present invention has been made in view of such a problem, and an object of the present invention is to provide a vehicle exterior environment recognition device capable of appropriately identifying a road surface.

[0010] Technical Solution

[0011] In order to solve the above-mentioned problems, the vehicle exterior environment recognition device of the present invention uses a computer as a first road surface determination unit and a second road surface determination unit. The first road surface determination unit determines a first road surface area in the image that corresponds to the road surface, and draws the representative distance of each horizontal line in the first road surface area at its vertical position to generate a first road surface model. The second road surface determination unit determines a second road surface area, which corresponds to a road surface farther away than the first road surface area and has a longer horizontal length than the first road surface area when converted into three-dimensional space, and draws the representative distance of each horizontal line in the second road surface area at its vertical position to generate a second road surface model.

[0012] The second road surface identification unit extracts a relative distance of a block composed of a plurality of pixels toward the upper vertical direction, and uses the relative distance as a position where a change in height in a three-dimensional space of the second road surface region to the extracted block is equal to or greater than a predetermined threshold.

[0013] The computer may also function as a horizontal gradient determination unit that determines the gradient in the horizontal direction based on the relative distance between each horizontal position on the horizontal line in the first road surface area and the second road surface area.

[0014] Technical Effects

[0015] According to the present invention, the road surface can be appropriately determined. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a block diagram showing the connection relationship of the vehicle exterior environment recognition system.

[0017] Figure 2 This is an explanatory diagram for explaining a brightness image and a distance image.

[0018] Figure 3 This is a functional block diagram showing the main functions of the vehicle exterior environment recognition device.

[0019] Figure 4 This is a flowchart showing the flow of the vehicle exterior environment recognition process.

[0020] Figure 5 It is an explanatory diagram for explaining the process of specifying the first road surface area.

[0021] Figure 6 This is an explanatory diagram for explaining a histogram.

[0022] Figure 7 It is an explanatory diagram for explaining the first road surface model.

[0023] Figure 8 It is an explanatory diagram for explaining the process of specifying the second road surface area.

[0024] Fig. 9 This is an explanatory diagram for explaining a histogram.

[0025] Fig.10 It is an explanatory diagram for explaining the second road surface model.

[0026] Fig.11 This is an illustration for explaining the horizontal slope.

[0027] Fig.12 This is an explanatory diagram for explaining a process of specifying a three-dimensional object by a three-dimensional object specifying unit.

[0028] Explanation of symbols

[0029] 1; own vehicle, 126; brightness image, 128; distance image, 160; first road surface determination unit, 162; second road surface determination unit, 164; horizontal slope determination unit, 166; three-dimensional object determination unit, 200; road, 214; first road surface area, 216; first road surface model, 226; second road surface area, 228; second road surface model, 230; horizontal slope DETAILED DESCRIPTION

[0030] Hereinafter, the preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings. The dimensions, materials, and other specific values ​​shown in the embodiment are only examples for easy understanding of the invention, and do not limit the present invention unless otherwise specified. It should be noted that in this specification and the accompanying drawings, elements having substantially the same function and structure are marked with the same symbols to omit repeated descriptions, and the illustrations of elements that are not directly related to the present invention are omitted.

[0031] (External environment recognition system 100)

[0032] Figure 1 1 is a block diagram showing the connection relationship of the vehicle exterior environment recognition system 100. The vehicle exterior environment recognition system 100 includes a photographing device 110, a vehicle exterior environment recognition device 120, and a vehicle control device (ECU: Engine Control Unit) 130.

[0033] The photographic device 110 is configured to include a photographing element such as a CCD (Charge-Coupled Device) or a CMOS (Complementary Metal-Oxide Semiconductor), and is capable of photographing the exterior environment in front of the vehicle 1, and generating a brightness image (a color image and / or a monochrome image) containing at least brightness information. In addition, the photographic device 110 is arranged at intervals in a substantially horizontal direction on the side of the traveling direction of the vehicle 1 in such a manner that the optical axes of the two photographic devices 110 are substantially parallel. The photographic device 110 continuously generates a brightness image of a three-dimensional object existing in the detection area in front of the vehicle 1 at, for example, 1 / 60 second per frame (60fps). Here, the three-dimensional object recognized by the photographic device 110 includes not only independently existing objects such as bicycles, pedestrians, vehicles, traffic lights, road signs, guardrails, buildings, etc., but also objects that can be identified as parts of them, such as bicycle wheels.

[0034] In addition, the vehicle exterior environment recognition device 120 obtains brightness images from the two imaging devices 110, respectively, and uses so-called pattern matching to derive disparity information including disparity and image positions indicating the positions of arbitrary blocks within the images. The so-called pattern matching is to search for blocks corresponding to blocks (for example, an array of 4 pixels horizontally × 4 pixels vertically) arbitrarily extracted from another brightness image in one brightness image. Here, horizontal refers to the horizontal direction of the image of the captured image, and vertical refers to the vertical direction of the image of the captured image. As this pattern matching, it is considered to compare brightness (Y) in arbitrary block units between a pair of images. For example, there are methods such as SAD (Sum of Absolute Difference) that obtains the difference in brightness, SSD (Sum of Squared intensity Difference) that uses the square of the difference, and NCC (Normalized Cross Correlation) that obtains the similarity of dispersion values ​​obtained by subtracting the average value from the brightness of each pixel. The vehicle exterior environment recognition device 120 performs such a block-by-block parallax derivation process on all blocks reflected in the detection area (for example, 600 pixels×200 pixels). Here, the block is set to 4 pixels×4 pixels, but the number of pixels in the block can be set arbitrarily.

[0035] In the vehicle exterior environment recognition device 120, although the disparity can be derived for each block as a detection resolution unit, it is not possible to identify which three-dimensional object the block is a part of. Therefore, the disparity information is derived independently not in three-dimensional object units but in detection resolution units (e.g., block units) in the detection area. Here, the image associated with the disparity information derived in this way is called a distance image, which is different from the above-mentioned brightness image.

[0036] Figure 2 126 and the distance image 128. For example, two imaging devices 110 generate a brightness image 126 and a distance image 128 for the detection area 124. Figure 2 Here, for easy understanding, only one of the two brightness images 126 is schematically shown. The vehicle exterior environment recognition device 120 obtains the parallax of each block from these brightness images 126 to form Figure 2 The distance image 128 is shown in (b). Each block in the distance image 128 is associated with the parallax of the block. Here, for convenience of explanation, the blocks from which the parallax is derived are indicated by black dots.

[0037] In addition, the vehicle exterior environment recognition device 120 first determines the road surface and groups the blocks with the same color value and close three-dimensional position information as three-dimensional objects, thereby determining whether the three-dimensional object in the detection area in front of the host vehicle 1 corresponds to a certain object (for example, a preceding vehicle or a bicycle) by using the brightness value (color value) based on the brightness image 126 and the position information of the three-dimensional space in the real space. The three-dimensional position information is calculated based on the distance image 128 and includes the relative distance to the host vehicle 1. In addition, if the vehicle exterior environment recognition device 120 determines the three-dimensional object in this way, the host vehicle 1 is controlled so as to avoid collision with the three-dimensional object (collision avoidance control) or to keep the distance between the vehicle and the preceding vehicle at a safe distance (cruise control). It should be noted that the above relative distance is obtained by converting the parallax information of each block in the distance image 128 into three-dimensional position information using the so-called stereo method. Here, the stereo method is a method of deriving the relative distance of the three-dimensional object with respect to the imaging device 110 from the parallax of the three-dimensional object by using the triangulation method.

[0038] The vehicle control device 130 receives the driver's operation input through the steering wheel 132, the accelerator pedal 134, and the brake pedal 136, and controls the host vehicle 1 by transmitting it to the steering mechanism 142, the drive mechanism 144, and the brake mechanism 146. In addition, the vehicle control device 130 controls the steering mechanism 142, the drive mechanism 144, and the brake mechanism 146 according to the instruction of the vehicle external environment recognition device 120.

[0039] As described above, in the vehicle exterior environment recognition system 100, blocks with equal color values ​​and close three-dimensional position information are grouped and set as three-dimensional objects, based on the condition that they have a height vertically above the road surface. Among them, in order to properly extract the three-dimensional object in this way, first, the road surface must be properly determined. For example, if the road surface cannot be properly determined, the sloping road itself will be judged as a three-dimensional object protruding vertically (gravity) above the road surface, or a three-dimensional object that should exist on the downhill slope cannot be judged as being located vertically below the road surface and extracted. In particular, for a road in the distance, the reliability of its image (parallax) indicating the road surface becomes low, and it is difficult to determine the road surface. Therefore, in the present embodiment, the purpose is to properly determine the road surface.

[0040] Here, in this embodiment, the process of determining the road surface in the detection area in front of the vehicle 1 as a feature is described in detail, and the description of the structure not related to the feature of this embodiment is omitted.

[0041] (External environment recognition device 120)

[0042] Figure 3 1 is a functional block diagram showing the main functions of the vehicle exterior environment recognition device 120. Figure 3 As shown, the vehicle exterior environment recognition device 120 is configured to include an I / F unit 150 , a data holding unit 152 , and a central control unit 154 .

[0043] The I / F unit 150 is an interface for bidirectionally exchanging information with the imaging device 110 and the vehicle control device 130. The data storage unit 152 is composed of a RAM, a flash memory, a HDD, etc., and stores various information required for processing of each functional unit described below.

[0044] The central control unit 154 is composed of a semiconductor integrated circuit including a central processing unit (CPU), a ROM storing programs, a RAM as a work area, etc., and controls the I / F unit 150, the data holding unit 152, etc. through the system bus 156. In addition, in the present embodiment, the central control unit 154 also functions as a first road surface determination unit 160, a second road surface determination unit 162, a horizontal slope determination unit 164, and a three-dimensional object determination unit 166. The following also describes in detail the vehicle exterior environment recognition process of determining the road surface, which is a feature of the present embodiment, based on the operation of each functional unit of the central control unit 154.

[0045] (External environment recognition and processing)

[0046] Figure 4 1 is a flowchart showing the flow of the vehicle exterior environment recognition process. In the vehicle exterior environment recognition process, the first road surface determination unit 160 is used to perform a first road surface determination process (S200) of determining a first road surface area in an image that corresponds to a road surface and generating a first road surface model, the second road surface determination unit 162 is used to perform a second road surface determination process (S202) of determining a second road surface area that corresponds to a road surface farther than the first road surface area and generating a second road surface model, the horizontal slope determination unit 164 is used to perform a horizontal slope determination process (S204) of determining a horizontal slope based on a relative distance between each horizontal line in the first road surface area and the second road surface area, and the three-dimensional object determination unit 166 is used to perform a three-dimensional object determination process (S206) of determining a three-dimensional object in the image based on at least the first road surface model and the second road surface model.

[0047] (First Road Surface Determination Process S200)

[0048] First, the first road surface identification unit 160 identifies a first road surface area corresponding to the road surface on the short-distance side in the brightness image 126 and / or the distance image 128 .

[0049] Here, as a method for determining the first road surface area, considering the expected travel path that the vehicle 1 intends to travel in the future based on the current steering angle, turning angular velocity (yaw rate) and speed, for example, the area within ±2.5m in the horizontal direction from the curve representing the expected travel path extending to the front of the vehicle 1 is set as the first road surface area.

[0050] However, the curve representing the expected path of travel is easily changed by the driver's operation, and there is a case where the error between the curve and the actual path of the vehicle 1 is large. Therefore, the first road surface area cannot be properly determined all the way to the far end. Therefore, in the present embodiment, the first road surface area is determined based on the lanes (white lines, etc.) on the left and right of the lane where the vehicle 1 is traveling on the road surface. However, it is not limited to the case where the first road surface area is determined only based on the lanes of the road surface. In addition, the first road surface area can also be determined based on the expected path of travel, and the accuracy of determining the first road surface area can be improved by interpolating the two.

[0051] Figure 5 FIG. 1 is an explanatory diagram for explaining the determination process of the first road surface area. A driving lane for a vehicle to travel smoothly is marked on the road surface. Figure 5 In the example of (a), on the road 200 on the brightness image 126, there are two lanes 202a and 202b, and accordingly, there is one lane 210b in the center in the width direction and one lane 210a and one lane 210c at each end, for a total of three lanes.

[0052] Here, if Figure 5 As shown in (b), the first road surface determination unit 160 sets an imaginary left boundary line 212a indicated by a dotted line at a position that is a predetermined distance (e.g., 10 cm) to the left of the left end of the leftmost lane 210a among the lanes 210a and 210b forming the lane 202a on which the host vehicle 1 is traveling. In addition, the first road surface determination unit 160 similarly sets an imaginary right boundary line 212b indicated by a dotted line at a position that is a predetermined distance (e.g., 10 cm) to the right of the rightmost lane 210b among the lanes 210a and 210b forming the lane 202a on which the host vehicle 1 is traveling. Furthermore, the first road surface determination unit 160 sets an area that is to the right of the left boundary line 212a and to the left of the right boundary line 212b, that is, a shaded area between the left boundary line 212a and the right boundary line 212b in the horizontal direction as the first road surface area 214.

[0053] Next, the first road surface identification unit 160 extracts all blocks having relative distances (relative distances obtained by pattern matching) from the first road surface region 214 in the distance image 128 and generates a first road surface model. The following describes how the first road surface model is generated.

[0054] Figure 6 is an illustration for explaining a histogram. Figure 7 is an explanatory diagram for explaining the first road surface model. The first road surface determination unit 160 first generates a histogram of relative distances for horizontal lines in the horizontal direction of the first road surface area 214. Specifically, in the first road surface area 214, for blocks whose vertical positions (vertical positions on the image) are equal to any blocks in the vertical direction, i.e. Figure 6 The relative distance of all blocks at the horizontal line shown by the mesh in (a) is used for voting. Figure 6 As shown in (b), a histogram associated with an arbitrary vertical position is generated.

[0055] Next, the first road surface identification unit 160 selects the road surface with the largest number of votes in the histogram. Figure 6 The relative distance indicated by hatching in (b) is set as the representative distance of the vertical position. In the first road surface region 214, the vertical position is changed and the above process is repeated. Thus, the representative distance is obtained for each of the plurality of vertical positions of the first road surface region 214.

[0056] In the first road surface model, the vertical axis represents the vertical position, and the horizontal axis represents the relative distance of the distance image 128. The first road surface determination unit 160 plots the representative distance of each vertical position (horizontal line) at its vertical position. Figure 7 The first road surface determination unit 160 uses the least square method, etc., such as Figure 7 As shown by the solid line in (a), an approximate straight line of the point group is derived and set as the first road surface model 216. In this way, the way the road surface changes can be understood, for example, the slope of the road can be understood. It should be noted that here, for the sake of convenience, an example of deriving an approximate straight line is cited for explanation, but multiple subsequent approximate curves can also be derived.

[0057] If the approximate straight line is derived simply by using the least square method, for example, in the first road surface area 214, there may be a representative distance (noise representative distance) as noise that is different from the actual relative distance due to erroneous pattern matching. If the approximate straight line is derived while including the noise representative distance that should be excluded, it will be affected by the noise representative distance. Figure 7 As shown in (a), the approximate straight line may be displaced from its original position and / or its inclination may be deviated, thereby causing the sloped road itself to be determined as a three-dimensional object, or failing to extract the three-dimensional object that should have existed.

[0058] Therefore, in this embodiment, Hough transform related to straight lines is used to retain only the representative distances of parallel straight lines that form the same straight line or are not the same but form a close distance (within a predetermined relative distance), and other representative distances such as Figure 7 The representative distances enclosed by the dotted lines in (a) are excluded as noise representative distances. Then, the first road surface determination unit 160 takes only the remaining representative distances as objects and uses the least square method or the like to derive an approximate straight line of such a point group. Figure 7 As shown in (b), the first road surface model 216 appropriately approximated by using the original representative distance is derived. It should be noted that since the Hough transform related to the straight line is a conventional technology for deriving a straight line passing through a plurality of points at the same time, its description is omitted here.

[0059] In addition, here, the change of the road surface is represented by an approximate straight line, but the road in the detection area is not limited to changing in the same way. For example, there are cases where the slope of the road changes significantly. In this case, if the Hough transform is performed as described above, the representative distance corresponding to the road surface after the slope change is almost excluded as a noise representative distance.

[0060] Therefore, in this embodiment, when the representative distances excluded by the Hough transform are more than a predetermined number and their point group is continuous, a new approximate straight line is derived only from the point group. In this case, it is of course possible to further exclude the noise representative distances with respect to the approximate straight line. Figure 7 As shown in (c) of FIG. 1 , a first road surface model 216 is derived in which two approximate straight lines are continuous.

[0061] It should be noted that the intersection of the two derived approximate straight lines may intersect at a predetermined angle and be connected by a gentle curve having a predetermined curvature radius R. In addition, here, an example of generating two approximate straight lines is given for explanation, but when there are many changes in the road, three or more approximate straight lines may be generated.

[0062] If we refer to the generated example Figure 7 For the first road surface model 216 of (b), the vertical position in the first road surface area 214 moves downward, the relative distance becomes closer (the parallax becomes larger), and the vertical position moves upward, the relative distance becomes farther (the parallax becomes smaller).

[0063] Next, the first road surface determination unit 160 derives the horizontal center line extending in the vertical direction of the first road surface region 214. For example, a block having a vertical position equal to any block in the vertical direction in the first road surface region 214, i.e. Figure 6The center position of the left and right ends of the horizontal block shown by the mesh pattern in (a) is set as the central block. And the first road surface determination unit 160 changes the vertical position and repeats this process. Thus, the central block is obtained with respect to multiple vertical positions of the first road surface area 214. By connecting such central blocks, the center line 218 in the vertical direction is derived.

[0064] In addition, as described above, the blocks in the first road surface area 214 include blocks corresponding to the noise representative distance. As a result, the center line 218 may be affected by the blocks to be excluded and may be offset from the original position and / or have a deviation in inclination. Therefore, the central block and the center line 218 are derived by removing the excluded blocks through the Hough transform.

[0065] (Second Road Surface Determination Process S202)

[0066] The first road surface identification process S200 derives the first road surface model 216 and the center line 218 of the road surface near the front. However, for a road in the distance, since the reliability of the image (parallax) representing the road surface becomes low, it is difficult to properly identify the road surface by only extracting the relative distance between the left boundary line 212a and the right boundary line 212b in the horizontal direction. Figure 7 As shown by the dotted line in (b), the first road surface model 216 can be determined only up to a predetermined vertical position.

[0067] Therefore, the second road surface determination unit 162 uses a different logic from that of the first road surface determination unit 160 to determine the second road surface region that is farther than the first road surface region 214 in the brightness image 126 and / or the distance image 128 .

[0068] Figure 8 2 is an explanatory diagram for explaining the second road surface area determination process. The first road surface determination process S200 is used to derive the first road surface model 216 and the center line 218 of the road surface closer to the front side, but both are within the range of the first road surface area 214 closer to the front side. Figure 8 The way in which the horizontal line 220 shown by the solid line in (a) changes toward the road surface in the distance is unclear.

[0069] Here, it is considered that the change mode of the road surface from the horizontal line 220 to the front is used and the change mode continues to the distance, for example, extending Figure 7 The approximate straight line of the first road surface model 216 of (b) is considered to determine the three-dimensional object based on the road surface. However, in this case, the sloping road itself in the distance may be determined as a three-dimensional object, or the three-dimensional object in the distance that should exist may not be extracted.

[0070] Here, the second road surface determination unit 162 first expands the start line of the second road surface area on the horizontal line 220. For example, the second road surface determination unit 162 sets the start line of the second road surface area on the horizontal line 220, with the intersection 222 between the horizontal line 220 and the center line 218 derived by the first road surface determination unit 160 as the center, and sets the range whose horizontal length when converted into three-dimensional space is longer than the first road surface area 214 by a predetermined distance (e.g., 5 m) as the new start line of the second road surface area.

[0071] Among them, since it is not necessary to set the second road surface area until the block that has no clear relationship with the road, when a three-dimensional object is detected on the horizontal line 220, the inner side of the three-dimensional object is set as the starting line of the second road surface area. That is, the second road surface determination unit 162 moves the horizontal line 220 to the left with the intersection 222 as the center, and if no three-dimensional objects such as walls are detected until it moves 5m to the left, the position of 5m is set as the left end, and if a three-dimensional object is detected, its previous position is set as the left end. Similarly, the second road surface determination unit 162 moves the horizontal line 220 to the right with the intersection 222 as the center, and if no three-dimensional objects such as walls are detected until it moves 5m to the right, the position of 5m is set as the right end, and if a three-dimensional object is detected, the previous position is set as the right end. Thus, for example, Figure 8 As shown in (b) of FIG. 1 , the start line 224 of the second road surface region is set. Therefore, the horizontal length of the second road surface region when converted into the three-dimensional space is longer than that of the first road surface region 214 .

[0072] Next, the second road surface determination unit 162 extracts the relative distance of the blocks from the starting line 224 of the second road surface area toward the upper vertical direction. Specifically, the blocks whose horizontal positions on the starting line 224 are equal to any blocks in the horizontal direction are extracted toward the upper vertical direction (far away). Figure 8 The relative distance of the block extending in the vertical direction shown by the mesh in (b). Furthermore, if the change in the height of the extracted block in the three-dimensional space, that is, the differential value of the line segment when the height is continuous, is greater than a predetermined threshold, the road ends at this point, the extraction of the block is stopped, and the block is used as the boundary of the second road surface area.

[0073] This process is repeated by changing the horizontal position (the horizontal position on the image). Figure 8 As shown in (c) of FIG. 2 , a block group is formed from the starting line of the second road surface area toward the vertical direction. Here, the range indicated by the hatching is referred to as the second road surface area 226 .

[0074] Next, the second road surface identification unit 162 generates a second road surface model for only the block group included in the second road surface region 226 extracted in this way.

[0075] Fig. 9 is an illustration for explaining a histogram. Fig.10 is an explanatory diagram for explaining the second road surface model. The second road surface determination unit 162 first generates a histogram of relative distances along the horizontal direction of the second road surface region 226. Specifically, in the second road surface region 226, for blocks whose vertical positions are equal to any block in the vertical direction, i.e. Fig. 9 The relative distances of all blocks on the horizontal line shown by the mesh in (a) are used to vote. Fig. 9 As shown in (b), a histogram associated with an arbitrary vertical position is generated.

[0076] Then, the second road surface identification unit 162 selects the road with the largest number of votes in the histogram. Fig. 9 The relative distance indicated by hatching in (b) is set as the representative distance of the vertical position. In the second road surface region 226, the vertical position is changed and the above process is repeated. Thus, the representative distance is obtained for each of the plurality of vertical positions of the second road surface region 226.

[0077] The second road surface model is similar to the first road surface model 216, in which the vertical axis represents the vertical position and the horizontal axis represents the relative distance from the distance image 128. The second road surface determination unit 162 plots the representative distance of each vertical position (horizontal line) at its vertical position. Figure 7 Thus, the first road surface model 216 shown in (b) is overlapped. Fig.10 The second road surface determination unit 162 uses the least square method, etc., such as Fig.10 As shown by the solid line, an approximate straight line of the point group is derived to be used as the second road surface model 228. Thus, similarly to the first road surface model 216, the change of the road surface in the second road surface region 226 can be understood, for example, the slope of the road can be understood.

[0078] At this time, similarly to the first road surface determination process S200, the Hough transform related to the straight line is used to retain only the representative distances of parallel straight lines that form the same straight line or are not the same but form a close distance (within a predetermined relative distance), and the other representative distances, such as Fig.10 The representative distances enclosed by the middle dashed line are excluded as noise representative distances. The second road surface identification unit 162 uses only the remaining representative distances as objects and derives an approximate straight line of the point group using the least square method or the like.

[0079] In addition, when the number of representative distances excluded by the Hough transform is greater than a predetermined number and the point group is continuous, a new approximate straight line can be derived using only the point group. In this case, of course, noise representative distances can be further excluded from the approximate straight line.

[0080] Here, on the basis of forming the second road surface model 228 in the distance, the second road surface area 226 is widened in the width direction, and all blocks within the range of the road considered to be near the three-dimensional object are set as objects, so that even if it is far away, it can be as Fig.10 As shown, the changing pattern of the road surface is acquired with high precision.

[0081] In addition, in the first road surface determination processing S200, the histogram in the horizontal direction is directly derived, but in the second road surface determination processing S202, since the block is extracted in the vertical direction until the height change is greater than the predetermined threshold, the appropriate second road surface area 226 is derived, so the determination accuracy can be further improved.

[0082] In addition, although the first road surface model 216 and the second road surface model 228 are obtained independently here, since the first road surface area 214 and the second road surface area 226 should be continuous, the second road surface model 228 in the second road surface area 226 can also be derived in a manner continuous with the first road surface model 216 based on this continuity.

[0083] (Horizontal Slope Determination Process S204)

[0084] Through the above processing, the slope in the depth direction of the first road surface model 216 and the second road surface model 228 can be determined with high accuracy. However, although the slope in the depth direction of the road is obtained, the slope in the horizontal direction of the road cannot be grasped (assuming that it is uniform). Therefore, when there is a slope in the horizontal direction, a high road surface may be recognized as a three-dimensional object, or a low three-dimensional object may not be detected.

[0085] Here, the horizontal gradient determination unit 164 determines the gradient in the horizontal direction based on the relative distance between each horizontal position (horizontal line) in the first road surface region 214 and the second road surface region 226 .

[0086] Fig.11 is an illustration for explaining the horizontal slope. In the first road surface area 214 and the second road surface area 226, extract the blocks whose vertical positions are equal to any blocks in the vertical direction, that is, Fig.11 All relative distances at the vertical position (horizontal line) shown by the mesh in (a).

[0087] And, the extracted relative distance is plotted at its horizontal position. Fig.11 The horizontal slope determination unit 164 uses the least square method, etc., such as Fig.11As shown by the solid line in (b), an approximate straight line of the point group is derived and set to a horizontal slope of 230. Thus, the way in which the road surface changes can be understood, for example, the slope of the road can be understood. It should be noted that here, for the sake of convenience, an example of deriving an approximate straight line is given for explanation, but multiple subsequent approximate curves can also be derived.

[0088] Furthermore, the horizontal gradient determination unit 164 can generate the gradients of the plane in the horizontal direction and the depth direction (road surface distance map) by obtaining the horizontal gradient 230 at each vertical position in the first road surface model 216 and the second road surface model 228 .

[0089] (3D Object Determination Process S206)

[0090] The three-dimensional object identification unit 166 identifies a three-dimensional object having a height vertically above the road surface based on the first road surface model 216 and the second road surface model 228 or based on the road surface distance map to which the horizontal gradient 230 is added.

[0091] Fig.12 3D is an explanatory diagram for explaining the process of specifying a three-dimensional object by the three-dimensional object specifying unit 166. Fig.12 In the figure, any one of the first road model 216 of the first road area 214 and the second road model 228 of the second road area 226 at any time is represented by a straight line, and blocks with relative distances in the entire image are represented by points at their relative distances and vertical positions.

[0092] The three-dimensional object identification unit 166 determines whether the block corresponds to a three-dimensional object by comparing the relative distance of each block with the first road surface model 216 and the second road surface model 228. Specifically, the three-dimensional object identification unit 166 compares the relative distance of each block with the first road surface model 216 and the second road surface model 228. Fig.12 The block located inside the road surface range 240 in which the upper and lower limits are indicated by the dotted lines is determined to correspond to the road surface.

[0093] On the other hand, since the block located outside (particularly on the upper side) of the road surface range 240 protrudes in the height direction from the road surface, the three-dimensional object identification unit 166 regards it as a candidate for a three-dimensional object. Furthermore, if the relative distances of the plurality of blocks that are candidates for three-dimensional objects having a height vertically above the road surface are equal, the three-dimensional object identification unit 166 uses Fig.12 The points are grouped in such a way as to be surrounded by the ellipse 242 in the figure and determined as a three-dimensional object. As for the determination of whether such a point group as a three-dimensional object seems to be a three-dimensional object (shape, size, etc.) and whether the three-dimensional object is a certain object, the detailed description is omitted here because various existing technologies can be used.

[0094] As described above, by appropriately identifying the road surface, a three-dimensional object having a height vertically above the road surface can also be appropriately identified.

[0095] In addition, a program for causing a computer to function as the vehicle exterior environment recognition device 120 and / or a storage medium such as a floppy disk, a magneto-optical disk, a ROM, a CD, a DVD, a BD, etc. storing the program and being readable by a computer are also provided. Here, the program refers to a data processing means described in any language and / or description method.

[0096] The preferred embodiment of the present invention is described above with reference to the accompanying drawings, but the present invention is certainly not limited to this embodiment. For those skilled in the art, it is known that various variations or modifications can be thought of within the scope of the patent claims, and it is understood that these also belong to the technical scope of the present invention.

[0097] It should be noted that the steps of the vehicle exterior environment recognition process in this specification do not need to be processed in chronological order according to the order described in the flowchart, and may also include processing based on parallel or subroutines.

[0098] Industrial Applicability

[0099] The present invention can be used in a vehicle exterior environment recognition device that identifies an object that exists in the traveling direction of the vehicle.

Claims

1. A vehicle exterior environment recognition device, characterized in that: The computer functions as a first road surface determining unit and a second road surface determining unit. The first road surface identification unit identifies a first road surface region corresponding to a road surface in the image, and generates a first road surface model by mapping a representative distance of each horizontal line in the first road surface region in correspondence with a vertical position of the horizontal line. The second road surface determination unit determines a second road surface area, the second road surface area corresponds to a road surface farther than the first road surface area, and is expanded in such a way that the horizontal length of the road surface farther than the first road surface area when converted into a three-dimensional space is longer than that of the first road surface area, and generates a second road surface model by drawing a representative distance of each horizontal line in the second road surface area in correspondence with the vertical position of the horizontal line. The second road surface identification unit extracts a relative distance of a block composed of a plurality of pixels toward the upper side in the vertical direction, and uses the relative distance as a value indicating a change in height of the second road surface region in a three-dimensional space until the extracted block is equal to or greater than a predetermined threshold.

2. The vehicle exterior environment recognition device according to claim 1, characterized in that: The computer functions as a horizontal slope determination unit. The horizontal gradient determination unit determines a gradient in a horizontal direction based on a relative distance between each horizontal position on a horizontal line in the first road surface area and the second road surface area.

Images