Outdoor environment recognition device

Abstract

To prevent false recognition of a preceding vehicle to be tracked.SOLUTION: A surrounding environment recognition apparatus includes one or more processors and one or more memories connected to the processors. The processor functions as: a position derivation unit which derives three-dimensional positions by block in a captured image in cooperation with a program included in the memory; a three-dimensional object specifying unit which specifies a three-dimensional object by grouping the blocks of which differences of three-dimensional positions fall within a predetermined range; a specific object specifying unit which specifies preceding vehicles 230a, 230b, and sidewalls 232a, 232b from the three-dimensional object; and a vehicle tracking unit which predicts future positions of the preceding vehicles and tracks the preceding vehicles. The vehicle tracking unit determines whether the preceding vehicles to be tracked are hidden by the sidewalls on the basis of a border line 240 between a blind area and a visible area.SELECTED DRAWING: Figure 10

Description

Technical Field

[0001] The present invention relates to an out-of-vehicle environment recognition device that identifies a three-dimensional object existing in the traveling direction of a host vehicle.

Background Art

[0002] Patent Document 1 discloses a technique for detecting a preceding vehicle located in front of a host vehicle and reducing a collision damage with the preceding vehicle, and a technique for performing follow-up control so as to keep a distance between the host vehicle and the preceding vehicle at a safe distance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In order to reduce a collision damage between a host vehicle and a preceding vehicle and to realize follow-up control with respect to the preceding vehicle, the host vehicle first identifies a three-dimensional object existing in the traveling direction and determines whether or not the three-dimensional object is a specific object such as a preceding vehicle. Then, the host vehicle realizes various controls with respect to the preceding vehicle by tracking the identified preceding vehicle.

[0005] However, when the host vehicle travels on a curve, the preceding vehicle to be tracked may be blocked by a side wall located on the side of the road or another preceding vehicle located in front of the preceding vehicle to be tracked. Then, the host vehicle may not be able to track the preceding vehicle and may misidentify another specific object as the preceding vehicle to be tracked.

[0006] In view of such problems, an object of the present invention is to provide an out-of-vehicle environment recognition device capable of suppressing misidentification of a preceding vehicle to be tracked.

Means for Solving the Problems

[0007] To solve the above problems, the vehicle external environment recognition device of the present invention includes one or more processors and one or more memories connected to the processors. The processor functions as a position derivation unit that derives the three-dimensional position in block units in the captured image in cooperation with the program included in the memory, a three-dimensional object identification unit that groups blocks whose three-dimensional position differences are within a predetermined range to identify a three-dimensional object, a specific object identification unit that identifies a preceding vehicle and a side wall from the three-dimensional object, and a vehicle tracking unit that predicts the future position of the preceding vehicle and tracks the preceding vehicle. The vehicle tracking unit Whether the visible boundary line, which is a tangent line to the approximate curve of the side wall passing through the imaging device of the host vehicle, intersects with the velocity vector of the preceding vehicle to be tracked, or whether the visible boundary line intersects with the tangent line to the approximate curve of the side wall at the depth position of the preceding vehicle determines whether or not the preceding vehicle to be tracked is blocked by a side wall based on

[0008] If the vehicle tracking unit determines that the preceding vehicle is blocked by a side wall, it may stop tracking the preceding vehicle. When the condition is satisfied, the vehicle tracking unit may increment by a predetermined value the point used for determining that the preceding vehicle to be tracked is blocked by the side wall, and when the point becomes equal to or greater than a predetermined threshold value, it may be determined that the preceding vehicle to be tracked is blocked by the side wall.

[0009] In order to solve the above problems, an out-of-vehicle environment recognition device of the present invention includes one or more processors and one or more memories connected to the processors. The processors cooperate with a program included in the memories to function as a position derivation unit that derives the three-dimensional position in block units in the captured image, a three-dimensional object identification unit that groups blocks whose difference in three-dimensional position is within a predetermined range to identify a three-dimensional object, a specific object identification unit that identifies a preceding vehicle and a side wall from the three-dimensional object, and a vehicle tracking unit that predicts the future position of the preceding vehicle and tracks the preceding vehicle. The vehicle tracking unit determines whether the preceding vehicle to be tracked is blocked by the side wall based on the boundary line between the blind spot and the visible area, and when it is determined that the preceding vehicle is blocked by the side wall, the tracking of the preceding vehicle is stopped. To solve the above problems, another vehicle external environment recognition device of the present invention includes one or more processors and one or more memories connected to the processors. The processor functions as a position derivation unit that derives the three-dimensional position in block units in the captured image in cooperation with the program included in the memory, a three-dimensional object identification unit that groups blocks whose three-dimensional position differences are within a predetermined range to identify a three-dimensional object, a specific object identification unit that identifies a preceding vehicle from the three-dimensional object, and a vehicle tracking unit that predicts the future position of the preceding vehicle and tracks the preceding vehicle. The vehicle tracking unit determines whether or not the preceding vehicle to be tracked is blocked by another preceding vehicle based on the positional relationship of a plurality of preceding vehicles When it is determined that the preceding vehicle is blocked by another preceding vehicle, the tracking of the preceding vehicle is stopped and makes a determination.

Advantages of the Invention

[0011] According to the present invention, it is possible to suppress misrecognition of the preceding vehicle to be tracked.

Brief Description of the Drawings

[0012]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. Dimensions, materials, and other specific numerical values shown in such embodiments are merely examples for facilitating the understanding of the invention, and do not limit the present invention unless otherwise specified. In the present specification and drawings, elements having substantially the same functions and configurations are denoted by the same reference numerals to omit redundant descriptions, and elements not directly related to the present invention are not shown.

[0014] (Vehicle exterior environment recognition system 100) FIG. 1 is a block diagram showing the connection relationship of the vehicle exterior environment recognition system 100. The vehicle exterior environment recognition system 100 is provided in the host vehicle 1 and includes an imaging device 110, a vehicle exterior environment recognition device 120, and a vehicle control device 130.

[0015] The imaging device 110 is composed of an imaging element such as a CCD (Charge-Coupled Device) or a CMOS (Complementary Metal-Oxide Semiconductor). The imaging devices 110 are arranged at a distance from each other in a substantially horizontal direction such that the optical axes of the two imaging devices 110 are substantially parallel on the traveling direction side of the host vehicle 1. The imaging device 110 images the vehicle exterior environment in front of the host vehicle 1 and generates a luminance image (color image or monochrome image) including at least luminance information. The imaging device 110 continuously generates a luminance image obtained by imaging a three-dimensional object existing in the detection area in front of the host vehicle 1, for example, every 1 / 60 second frame (60 fps). Here, the three-dimensional objects and specific objects recognized by the vehicle exterior environment recognition device 120 include not only independently existing objects such as bicycles, pedestrians, vehicles, traffic lights, road signs, guardrails, buildings, and sidewalls on the roadside, but also objects that can be specified as a part thereof, such as the rear and side surfaces of a vehicle and the wheels of a bicycle. Here, the rear surface of the vehicle refers to the surface facing the host vehicle 1 in front of the host vehicle 1, and does not refer to the rear surface of the vehicle itself.

[0016] The external environment recognition device 120 acquires luminance images from each of the two imaging devices 110 and generates a distance image using so-called pattern matching. Based on the luminance image and the distance image, the external environment recognition device 120 groups, as a three-dimensional object, blocks that are located above the road surface, have equal color values, and have three-dimensional position information adjacent to each other. Then, the external environment recognition device 120 identifies which specific object the three-dimensional object is. For example, the external environment recognition device 120 identifies that the three-dimensional object is a preceding vehicle. Further, when the external environment recognition device 120 identifies the preceding vehicle in this way, it performs collision damage mitigation control between the host vehicle 1 and the preceding vehicle, and also performs following control to make the host vehicle 1 follow the preceding vehicle. The specific operation of the external environment recognition device 120 will be described in detail later.

[0017] The vehicle control device 130 is composed of an ECU (Electronic Control Unit) or the like, receives the operation input of the driver through the steering wheel 132, the accelerator pedal 134, and the brake pedal 136, and transmits it to the steering mechanism 142, the drive mechanism 144, and the brake mechanism 146 to control the host vehicle 1. Further, the vehicle control device 130 controls the steering mechanism 142, the drive mechanism 144, and the brake mechanism 146 according to the instructions of the external environment recognition device 120.

[0018] (External environment recognition device 120) FIG. 2 is a functional block diagram showing the schematic functions of the external environment recognition device 120. As shown in FIG. 2, the external environment recognition device 120 includes an I / F unit 150, a data holding unit 152, and a central control unit 154.

[0019] The I / F unit 150 is an interface for performing two-way information exchange with the imaging device 110 and the vehicle control device 130. The data holding unit 152 is composed of a RAM, a flash memory, an HDD, etc., and holds various information necessary for the processing of each of the following functional units.

[0020] The central control unit 154 is composed of a semiconductor integrated circuit including a processor 154a, a ROM 154b storing programs and the like, a RAM 154c serving as a work area, etc., and controls the I / F unit 150, the data holding unit 152, etc. through the system bus 156. The ROM 154b and the RAM 154c may be collectively referred to as a memory. Further, in the present embodiment, in the central control unit 154, the processor 154a, the ROM 154b, and the RAM 154c cooperate to also function as a position derivation unit 170, a three-dimensional object identification unit 172, a specific object identification unit 174, and a vehicle tracking unit 176. Hereinafter, the method for recognizing the external environment of the vehicle will be described in detail in consideration of the operations of the respective functional units of the central control unit 154.

[0021] (Method for Recognizing External Environment of Vehicle) FIG. 3 is a flowchart showing the flow of the method for recognizing the external environment of the vehicle. The external environment recognition device 120 executes the method for recognizing the external environment of the vehicle at every predetermined interrupt time. In the method for recognizing the external environment of the vehicle, the position derivation unit 170 derives the three-dimensional position in block units in the luminance image acquired from the imaging device 110 (position derivation process S200). The three-dimensional object identification unit 172 groups the blocks to identify the three-dimensional object (three-dimensional object identification process S202). The specific object identification unit 174 identifies the preceding vehicle and the side wall from the three-dimensional object (specific object identification process S204). The vehicle tracking unit 176 predicts the future position of the preceding vehicle and tracks the preceding vehicle (vehicle tracking process S206).

[0022] (Position Derivation Process S200) FIG. 4 (FIGS. 4A to 4C) is an explanatory diagram for explaining the luminance image and the distance image. The position derivation unit 170 acquires a plurality (here, 2) of luminance images captured at the same timing with different optical axes by the imaging device 110. Here, assume that the position derivation unit 170 acquires, as the luminance image 212, a first luminance image 212a captured by the imaging device 110 located relatively on the right side of the host vehicle 1 shown in FIG. 4A and a second luminance image 212b captured by the imaging device 110 located relatively on the left side of the host vehicle 1 shown in FIG. 4B.

[0023] Referring to FIGS. 4A and 4B, it can be understood that due to the difference in the imaging positions of the imaging device 110, the image positions of the three-dimensional objects included in the images are different in the horizontal direction between the first luminance image 212a and the second luminance image 212b. Here, the horizontal direction indicates the horizontal direction of the captured image screen, and the vertical direction indicates the vertical direction of the captured image screen.

[0024] Based on the first luminance image 212a shown in FIG. 4A and the second luminance image 212b shown in FIG. 4B, which are acquired by the position derivation unit 170, the position derivation unit 170 generates a distance image 214 as shown in FIG. 4C that can specify the distance of the imaging target.

[0025] Specifically, the position derivation unit 170 uses so-called pattern matching to derive parallax and parallax information including the image position indicating the position within the image of any block. Then, the position derivation unit 170 searches for a block corresponding to the block arbitrarily extracted from one luminance image (here, the first luminance image 212a) from the other luminance image (here, the second luminance image 212b). Here, the block is represented by an array of, for example, 4 pixels horizontally × 4 pixels vertically. Also, pattern matching is a method of searching for a block corresponding to a block arbitrarily extracted from one luminance image from the other luminance image.

[0026] For example, as functions for evaluating the degree of coincidence between blocks in pattern matching, there are methods such as SAD (Sum of Absolute Difference) that takes the difference in luminance, SSD (Sum of Squared intensity Difference) that uses the squared difference, and NCC (Normalized Cross Correlation) that takes the similarity of the variance value obtained by subtracting the average value from the luminance of each pixel.

[0027] The position derivation unit 170 performs such parallax derivation processing in units of blocks for all the blocks mapped in a detection region of, for example, 600 pixels × 200 pixels. Here, although the position derivation unit 170 processes the block as 4 pixels × 4 pixels, it may be processed with any number of pixels.

[0028] The position derivation unit 170 converts the parallax information for each block of the distance image 214 into a three-dimensional position in real space including the horizontal distance x, the height y, and the relative distance z using a so-called stereo method. Here, the stereo method is a technique that uses triangulation to derive the relative distance z of a block with respect to the imaging device 110 from the parallax in the distance image 214 of the block. At this time, the position derivation unit 170 derives the height y of the block from the road surface based on the relative distance z of the block and the detection distance on the distance image 214 between the point on the road surface at the same relative distance z as the block and the block. Then, the position derivation unit 170 associates the derived three-dimensional position with the distance image 214 again. Since various known techniques can be applied to the derivation process of such relative distance z and the specification process of the three-dimensional position, the description thereof is omitted here.

[0029] (Three-dimensional object specification process S202) The three-dimensional object specification unit 172 groups blocks that are located above the road surface, have equal color values, and have a difference in three-dimensional position in the distance image 214 within a predetermined range to specify a three-dimensional object. Specifically, the three-dimensional object specification unit 172 groups blocks in the distance image 214 whose differences in horizontal distance x, height y, and relative distance z are within a predetermined range (for example, 0.1 m) assuming that they correspond to the same specified object. In this way, a virtual block group is generated.

[0030] FIG. 5 (FIGS. 5A to 5C) is an explanatory diagram for explaining the operation of the three-dimensional object specification unit 172. Here, assume that the position derivation unit 170 generates a distance image 214 as shown in FIG. 5A, for example. The three-dimensional object specification unit 172 groups blocks from such a distance image 214. In this way, a plurality of grouped block groups are extracted as shown in FIG. 5B. The three-dimensional object specification unit 172 sets an outer contour line that includes all of the grouped blocks in FIG. 5B, for example, a rectangular frame (plane) composed of a horizontal line and a vertical line, or a line extending in the depth direction and a vertical line, as the three-dimensional object 220 (220a, 220b, 220c, 220d, 220e, 220f, 220g).

[0031] In FIG. 5B, when the three-dimensional objects 220a, 220b, 220c, 220d, 220e, 220f, 220g on the distance image 214 are represented on a two-dimensional horizontal plane indicated by the horizontal distance x and the relative distance z, they become the three-dimensional objects 220a, 220b, 220c, 220d, 220e, 220f, 220g in FIG. 5C.

[0032] (Specific object identification process S204) The specific object identification unit 174 identifies which specific object the three-dimensional object identified by the three-dimensional object identification unit 172 is. For example, in front of the host vehicle 1, the specific object identification unit 174 identifies a three-dimensional object traveling in the same direction as the host vehicle 1 as a preceding vehicle.

[0033] For example, in FIG. 5C, the three-dimensional object identification unit 172 identifies the three-dimensional object 220b and the three-dimensional object 220c as different three-dimensional objects. However, in reality, the three-dimensional object 220b constitutes the rear surface of the preceding vehicle, and the three-dimensional object 220c constitutes the side surface of the preceding vehicle. Therefore, the three-dimensional objects 220b and 220c should originally be recognized as a pair that constitutes an integral and identical three-dimensional object. Similarly, the three-dimensional objects 220e and 220f should also be recognized as the same three-dimensional object. Therefore, the specific object identification unit 174 pairs the three-dimensional objects 220 that should be the rear surface and the side surface of the same three-dimensional object.

[0034] FIG. 6 is an explanatory diagram for explaining the operation of the specific object identification unit 174. FIG. 6 is a projection of the three-dimensional object 220 onto a horizontal plane and is represented only by the horizontal distance x and the relative distance z.

[0035] The specific object identification unit 174 projects the three-dimensional object 220 onto a horizontal plane, distinguishes between the rear surface and the side surface based on the angle with respect to the depth direction, and if the relationship between the rear surface and the side surface satisfies a predetermined condition, pairs the rear surface and the side surface to identify a pair.

[0036] Specifically, the specific object specifying unit 174 first derives the angle of the approximate straight line of the three-dimensional object 220 with respect to the axis of the relative distance corresponding to the depth direction. If the absolute value of the derived angle is 45 degrees or more and less than 135 degrees, the specific object specifying unit 174 regards the three-dimensional object 220 as the back surface. On the other hand, if the absolute value of the derived angle is 0 degrees or more and less than 45 degrees or 135 degrees or more and 180 degrees or less, the specific object specifying unit 174 regards the three-dimensional object 220 as the side surface.

[0037] For example, assuming that as a result of the three-dimensional object specifying unit 172 grouping, the three-dimensional objects 220h and 220i are derived as shown in FIG. 6A. In such an example, as shown in FIG. 6B, since the absolute value of the angle of the approximate straight line of the three-dimensional object 220h is 45 degrees or more and less than 135 degrees, the specific object specifying unit 174 determines that the three-dimensional object 220h is the back surface. On the other hand, since the absolute value of the angle of the approximate straight line of the three-dimensional object 220i is 0 degrees or more and less than 45 degrees, the specific object specifying unit 174 determines that the three-dimensional object 220i is the side surface. In this way, the specific object specifying unit 174 can generally distinguish between the back surface and the side surface.

[0038] At this time, if the distance between the back surface and the side surface is within an appropriate distance (for example, 2 m) as a part of the vehicle, the speeds of the back surface and the side surface are both stable, the lengths of the back surface and the side surface are both equal to or greater than a predetermined value (for example, 1 m), and the central position thereof is within the detection area, the specific object specifying unit 174 may perform pairing. By adding such preconditions for pairing, the specific object specifying unit 174 can appropriately limit the pairing target and reduce the processing load.

[0039] The specific object specific part 174 determines whether a pair of three-dimensional objects 220h corresponding to the back surface and 220i corresponding to the side surface satisfies the conditions as a preceding vehicle. Specifically, the specific object specific part 174 determines whether the three-dimensional object constituted by the pair of the three-dimensional objects 220h and 220i has a size, shape, and relative speed similar to those of a vehicle, and whether it has a light source such as a brake lamp or a high-mounted stop lamp at a predetermined position in the rear. When such conditions are satisfied, the specific object specific part 174 specifies the three-dimensional object 220 constituted by the pair of the three-dimensional objects 220h and 220i as the preceding vehicle.

[0040] Note that the specific object specific part 174 may specify the three-dimensional object 220 as the preceding vehicle on the condition that the difference in position between the three-dimensional object specified by the three-dimensional object specific part 172 and the three-dimensional object 220 predicted by the vehicle tracking part 176 is within a predetermined range based on the prediction result predicted in the previous frame in the vehicle tracking part 176 described later.

[0041] In addition, the specific object specific part 174 specifies a three-dimensional object standing on the roadside in front of the host vehicle 1 as a side wall.

[0042] For example, in FIG. 5C, the three-dimensional object specific part 172 sets the three-dimensional object 220a standing on the left side of the road as the left side wall. Also, the three-dimensional object specific part 172 sets the three-dimensional object 220g standing on the right side of the road as the right side wall. Generally, it is known that the integrally formed side walls have continuity. Therefore, as shown in FIG. 5C, when the three-dimensional objects 220a and 220g have a linear locus in the horizontal plane, the three-dimensional object specific part 172 can determine the three-dimensional object 220a as a linear side wall. Also, when the three-dimensional object has a regular curved locus such as an equal radius of curvature in the horizontal plane, the three-dimensional object specific part 172 can determine the three-dimensional object as a curved side wall.

[0043] At this time, the specific object identification unit 174 may identify the three-dimensional objects 220a and 220g as side walls on the condition that the relative speeds of the three-dimensional objects 220a and 220g are equal to the speed of the host vehicle 1, the horizontal distance and the vertical distance from the road are stable within a predetermined range, and there is continuity in the depth direction.

[0044] When the specific object identification unit 174 identifies the three-dimensional object 220 as a side wall, for example, an approximation curve by the least squares method is derived for the blocks constituting the three-dimensional object 220. The specific object identification unit 174 associates the derived approximation curve with the three-dimensional object 220 as a locus on the horizontal plane of the side surface.

[0045] (Vehicle tracking process S206) The vehicle tracking unit 176 predicts the future position of the preceding vehicle and tracks the preceding vehicle. Here, tracking means determining the identity of the preceding vehicles specified for each frame and deriving the temporal movement transition of the preceding vehicles regarded as the same, thereby managing the preceding vehicles in time series.

[0046] FIG. 7 is a flowchart showing the flow of the vehicle tracking process, and FIGS. 8 (FIGS. 8A and 8B) are explanatory diagrams for explaining the operation of the vehicle tracking unit 176. FIG. 8 is represented only by the horizontal distance x and the relative distance z. For example, it is assumed that the three-dimensional object identification unit 172 has identified the three-dimensional objects 220h and 220i as shown in FIG. 8A. Further, it is assumed that the specific object identification unit 174 has paired the three-dimensional object 220h as the rear surface and the three-dimensional object 220i as the side surface and identified them as the preceding vehicle. Here, the intersection point between the approximation curve of the blocks constituting the three-dimensional object 220h and the approximation curve of each block of the three-dimensional object 220i is defined as the actually measured intersection point. Further, among the blocks constituting the three-dimensional object 220h, the point farthest from the actually measured intersection point is defined as the actually measured rear end point, and among the blocks constituting the three-dimensional object 220i, the point farthest from the actually measured intersection point is defined as the actually measured side end point.

[0047] As shown in FIG. 8B, the vehicle tracking unit 176 uses such measured intersection points, measured rear end points, and measured side end points, and the measured intersection points, measured rear end points, and measured side end points for multiple times derived in the past, and uses, for example, a Kalman filter or the like to derive the next predicted intersection point, predicted rear end point, and predicted side end point (S300). At this time, the vehicle tracking unit 176 predicts the next predicted intersection point, predicted rear end point, and predicted side end point in consideration of the relative speeds of the rear and side surfaces, the angular velocity around the Y axis, and the ego motion. Then, the vehicle tracking unit 176 stores the prediction results (predicted intersection point, predicted rear end point, predicted side end point) of the current frame in the data holding unit 152 for the next frame. Here, although the vehicle tracking unit 176 predicts the movement transition of the preceding vehicle using the end points and intersection points of the preceding vehicle, the present invention is not limited to such a case, and various conventional movement transition prediction techniques can be used.

[0048] Next, the vehicle tracking unit 176 reads out the prediction results predicted in the previous frame from the data holding unit 152. The vehicle tracking unit 176 compares the measured results (measured intersection point, measured rear end point, measured side end point) of the current frame with the prediction results in the previous frame to determine the identity of the three-dimensional object (S302). Specifically, the vehicle tracking unit 176 identifies the measured result of the current frame that has the shortest distance from the prediction result in the previous frame and the difference in the size of the three-dimensional object is within a predetermined range. Then, if the difference in the three-dimensional position between the identified measured result of the current frame and the prediction result in the previous frame is within a predetermined range, the vehicle tracking unit 176 determines that the preceding vehicles identified for each frame are the same preceding vehicle. In this way, the vehicle tracking unit 176 can derive the movement transition of the preceding vehicle to be tracked for each frame.

[0049] When the vehicle tracking unit 176 derives the movement transition of the preceding vehicle to be tracked, the out-vehicle environment recognition device 120 can perform collision damage mitigation control between the host vehicle 1 and the preceding vehicle and perform following control to make the host vehicle 1 follow the preceding vehicle using the movement transition of the preceding vehicle.

[0050] Incidentally, when the host vehicle 1 travels along a curve, the preceding vehicle to be tracked may be blocked by a side wall located beside the road. In such a case, the host vehicle 1 may be unable to track the preceding vehicle and may misidentify another specific object as the preceding vehicle to be tracked.

[0051] FIG. 9 (FIGS. 9A and 9B) is an explanatory diagram showing an example of the travel of the host vehicle 1. FIG. 9A shows a luminance image 212 when the host vehicle 1 is traveling along a curve, and FIG. 9B shows the state of the detection area projected onto a horizontal plane. As shown in FIG. 9A, the host vehicle 1 is traveling along a leftward curve, and two preceding vehicles 230a and 230b are located ahead.

[0052] Here, side walls 232a and 232b are located beside the curved road. Depending on the positional relationship between the host vehicle 1 and the preceding vehicle 230a, there may be a case where the specific object identification unit 174 cannot identify the preceding vehicle 230a as a specific object. For example, as shown in FIG. 9B, even if the preceding vehicle 230a is traveling on the lane, as seen from the host vehicle 1, as shown in FIG. 9A, the preceding vehicle 230a is partially blocked by the left side wall 232a.

[0053] When the preceding vehicle 230a is blocked by the side wall 232a in this way and the three-dimensional object identification unit 172 cannot identify the entire three-dimensional object, the identification accuracy of the preceding vehicle 230a by the specific object identification unit 174 decreases, and the tracking of the preceding vehicle 230a by the vehicle tracking unit 176 becomes unstable. Also, as described above, the vehicle tracking unit 176 tracks the preceding vehicle using the measurement result of the current frame that has the shortest distance from the prediction result of the current frame in the previous frame. Therefore, when the preceding vehicle 230a is completely blocked by the side wall 232a, the vehicle tracking unit 176 may track, as the preceding vehicle 230a, the preceding vehicle 230b that is located near the preceding vehicle 230a and has a size approximate to that of the preceding vehicle 230a.

[0054] Therefore, the vehicle tracking unit 176 determines whether or not the preceding vehicle 230a to be tracked will be blocked by the side wall 232a in the future (S304).

[0055] Figures 10 and 11 are explanatory diagrams showing the operation of the vehicle tracking unit 176. In Figure 10, the blind spots of the imaging device 110 are indicated by hatching. Therefore, the area other than the hatching is the visible area that can be imaged by the imaging device 110. The visible area is determined by the blind spots of the imaging device 110, the trajectories of the side walls 232a and 232b, and the blind spots due to the curve of the side wall 232a.

[0056] Here, a visible boundary line 240 indicating the boundary between the blind spot due to the curve of the side wall 232a and the visible area is defined. The visible boundary line 240 shown by the thick line is a tangent line to the approximate curve of the side wall 232a when the trajectory of the side wall 232a in the horizontal plane passing through the imaging device 110 of the host vehicle 1 can be approximated by a curve, and refers to a straight line extended in the depth direction from the contact point with the side wall 232a.

[0057] Here, as shown in Figure 10, when the velocity vector 242 (hereinafter simply referred to as the velocity vector 242) starting from the center point of the preceding vehicle 230a to be tracked by the vehicle tracking unit 176 intersects the visible boundary line 240, the vehicle tracking unit 176 determines that the preceding vehicle 230a is shielded by the side wall 232a. With such a configuration, the vehicle tracking unit 176 can appropriately determine that the preceding vehicle 230a is shielded by the side wall 232a.

[0058] Also, when the tangent line 244 in the horizontal plane to the approximate curve of the side wall 232a at the depth position of the preceding vehicle 230a, shown by the dashed line in Figure 11, intersects the visible boundary line 240, the vehicle tracking unit 176 may determine that the preceding vehicle 230a is shielded by the side wall 232a. With such a configuration, the vehicle tracking unit 176 can appropriately determine that the preceding vehicle 230a is shielded by the side wall 232a.

[0059] However, when the velocity vector 242 or the tangent line 244 intersects with the visible boundary line 240, if the vehicle tracking unit 176 immediately determines that the leading vehicle 230a is shielded by the side wall 232a, the following problems may occur. For example, due to noise, the shielding determination may be made in a situation where it should not be made originally. Also, the following control may become unstable due to chattering. Therefore, when the vehicle tracking unit 176 satisfies the shielding determination conditions such as the velocity vector 242 or the tangent line 244 intersecting with the visible boundary line 240, it adds points accordingly. When the points reach a predetermined threshold or more, it determines that the leading vehicle 230a is shielded by the side wall 232a.

[0060] For example, when the velocity vector 242 or the tangent line 244 intersects with the visible boundary line 240, the vehicle tracking unit 176 increments the shielding point by 1. Also, when there is another specific object on the side opposite to the side wall 232a with respect to the leading vehicle 230a, the vehicle tracking unit 176 increments the shielding point by 1. Further, when the leading vehicle 230a cannot be identified as a specific object, that is, when the leading vehicle 230a does not have a relative distance z, the vehicle tracking unit 176 increments the loss point by 1. Note that when the side wall 232a is not located in the vicinity of the leading vehicle 230a, the vehicle tracking unit 176 resets the shielding point and the loss point to 0.

[0061] Then, when the shielding points of the vehicle tracking unit 176 reach a predetermined value (for example, 3 points) or more, it determines that the preceding vehicle 230a is shielded by the side wall 232a. Also, the vehicle tracking unit 176 may determine that the preceding vehicle 230a is shielded by the side wall 232a when the loss points reach a predetermined value (for example, 2 points) or more. Further, it may determine that the preceding vehicle 230a is shielded by the side wall 232a when the shielding points reach a predetermined value (for example, 2 points) or more and the loss points reach a predetermined value (for example, 2 points) or more. Note that when there is another specific object on the side opposite to the side wall 232a with respect to the preceding vehicle 230a, the vehicle tracking unit 176 may lower the threshold value in order to make it easier to determine that the preceding vehicle 230a is shielded by the side wall 232a. With such a configuration, the vehicle tracking unit 176 can appropriately and stably determine that the preceding vehicle 230a is shielded by the side wall 232a.

[0062] Returning to FIG. 7, when the vehicle tracking unit 176 determines that the preceding vehicle 230a is shielded by the side wall 232a as described above (YES in S304), it stops tracking the preceding vehicle 230a (S306). Specifically, the vehicle tracking unit 176 excludes the preceding vehicle 230a determined to be shielded by the side wall 232a from the tracking target. With such a configuration, since the preceding vehicle 230a is excluded from the tracking target at an early stage, it is possible to prevent the vehicle tracking unit 176 from being unable to track the preceding vehicle 230a and from misidentifying another specific object as the preceding vehicle being tracked. When it is determined that the preceding vehicle 230a is not shielded by the side wall 232a (NO in S304), the process proceeds to the preceding vehicle shielding determination S308 described later.

[0063] Also, when the host vehicle 1 is traveling on a curve, the preceding vehicle being tracked may be shielded by another preceding vehicle traveling parallel. Then, similar to the case where the preceding vehicle 230a being tracked is shielded by the side wall 232a, the host vehicle 1 may be unable to track the preceding vehicle and may misidentify another specific object as the preceding vehicle being tracked.

[0064] FIG. 12 (FIGS. 12A and 12B) is an explanatory diagram showing a driving example of the host vehicle 1. FIG. 12A shows a luminance image 212 when the host vehicle 1 is traveling on a curve, and FIG. 12B is a projection of the state of the detection area onto a horizontal plane. As shown in FIG. 12A, the host vehicle 1 is traveling on a left curve, and two preceding vehicles 230c and 230d are located ahead. Also, side walls 232a and 232b are located beside the curved road.

[0065] Here, depending on the positional relationship between the host vehicle 1 and the preceding vehicles 230c and 230d, there may be a case where the specific object specifying unit 174 cannot specify the preceding vehicle 230d as a specific object. For example, as shown in FIG. 12B, even if the preceding vehicle 230d is traveling on the lane, as viewed from the host vehicle 1, as shown in FIG. 12A, the preceding vehicle 230d is partially shielded by the preceding vehicle 230c located on the left side of the preceding vehicle 230d. If the preceding vehicle 230d is completely shielded by the preceding vehicle 230c, the vehicle tracking unit 176 may misidentify the side wall 232b located in the vicinity of the preceding vehicle 230d as the preceding vehicle 230d.

[0066] Therefore, the vehicle tracking unit 176 determines whether or not the target preceding vehicle 230d is shielded by the preceding vehicle 230c based on the positional relationship between the plurality of preceding vehicles 230c and 230d (S308).

[0067] FIG. 13 (FIGS. 13A and 13B) is an explanatory diagram showing the operation of the vehicle tracking unit 176. The vehicle tracking unit 176 determines whether or not the outer contour line 250c representing the preceding vehicle 230c and the outer contour line 250d representing the preceding vehicle 230d are in contact with each other in the luminance image 212 in the current frame shown in FIG. 13A, in other words, whether or not there is a gap therebetween. Then, if the outer contour line 250c and the outer contour line 250d are in contact with each other, the vehicle tracking unit 176 determines that the preceding vehicle 230d is shielded by the preceding vehicle 230c. With such a configuration, the vehicle tracking unit 176 can appropriately determine that the preceding vehicle 230d is shielded by the preceding vehicle 230c.

[0068] Further, in the calculation space as shown in FIG. 13B, the vehicle tracking unit 176 determines whether or not an area 260c that extends radially from the host vehicle 1 and in which the preceding vehicle 230c occupies the distance image 214 and an area 260d that extends radially from the host vehicle 1 and in which the preceding vehicle 230d occupies the distance image 214 overlap in the xz plane. At this time, the vehicle tracking unit 176 identifies the position of the preceding vehicle 230d based on the prediction result in the previous frame predicted by the vehicle tracking unit 176. If the areas 260c and 260d that extend radially from the host vehicle 1 overlap in the xz plane, the vehicle tracking unit 176 determines that the preceding vehicle 230d is blocked by the preceding vehicle 230c. With such a configuration, the vehicle tracking unit 176 can appropriately determine that the preceding vehicle 230d is blocked by the preceding vehicle 230c. Further, in the example of FIG. 13B, as shown in FIG. 13A, not only the current frame but also the amount of movement of the preceding vehicle 230d from the previous frame is a determination target, so the vehicle tracking unit 176 can perform the occlusion determination with higher accuracy.

[0069] Further, when the vehicle tracking unit 176 satisfies the occlusion determination condition, such as the outer contour line 250c and the outer contour line 250d being in contact with each other, it adds points accordingly. When the points reach a predetermined threshold or more, the vehicle tracking unit 176 determines that the preceding vehicle 230d is blocked by the preceding vehicle 230c.

[0070] For example, if the outer contour line 250c and the outer contour line 250d are in contact, the vehicle tracking unit 176 increments the occlusion point by 1. Further, if the region 260c and the region 260d overlap in the xz plane as shown in FIG. 13B, the vehicle tracking unit 176 increments the occlusion point by 2. Also, when there is another specific object on the side opposite to the preceding vehicle 230c with respect to the preceding vehicle 230d as a reference, the vehicle tracking unit 176 increments the occlusion point by 1. Further, when the preceding vehicle 230d cannot be identified as a specific object, that is, when the preceding vehicle 230d does not have the relative distance z, the vehicle tracking unit 176 increments the loss point by 1. Note that when another preceding vehicle 230c running parallel is not located in the vicinity of the preceding vehicle 230d, the vehicle tracking unit 176 resets the occlusion point and the loss point to 0.

[0071] Then, if the occlusion point becomes equal to or greater than a predetermined value (for example, 3 points), the vehicle tracking unit 176 determines that the preceding vehicle 230d is occluded by the preceding vehicle 230c. Further, if the loss point becomes equal to or greater than a predetermined value (for example, 2 points), the vehicle tracking unit 176 may determine that the preceding vehicle 230d is occluded by the preceding vehicle 230c. Furthermore, if the occlusion point is equal to or greater than a predetermined value (for example, 2 points) and the loss point is equal to or greater than a predetermined value (for example, 2 points), the vehicle tracking unit 176 may determine that the preceding vehicle 230d is occluded by the preceding vehicle 230c. Note that when there is another specific object on the side opposite to the preceding vehicle 230c with respect to the preceding vehicle 230d as a reference, the vehicle tracking unit 176 may lower the threshold value for determining that the preceding vehicle 230d is occluded by the preceding vehicle 230c. With such a configuration, the vehicle tracking unit 176 can appropriately and stably determine that the preceding vehicle 230d is occluded by the preceding vehicle 230c.

[0072] Returning to FIG. 7, when the vehicle tracking unit 176 determines that the preceding vehicle 230d is blocked by the preceding vehicle 230c as described above (YES in S308), it stops tracking the preceding vehicle 230d (S306). Specifically, the vehicle tracking unit 176 excludes the preceding vehicle 230d determined to be blocked by the preceding vehicle 230c from the tracking target. With such a configuration, since the preceding vehicle 230d is excluded from the tracking target at an early stage, it is possible to prevent the vehicle tracking unit 176 from being unable to track the preceding vehicle 230d and from misidentifying another specific object as the preceding vehicle being tracked. When it is determined that the preceding vehicle 230d is not blocked by the preceding vehicle 230c (NO in S308), the vehicle tracking process S206 is terminated without stopping the tracking of the preceding vehicle 230d.

[0073] Note that in FIG. 7, an example has been described in which the vehicle tracking unit 176 executes both the process of determining whether the preceding vehicle 230a to be tracked will be blocked by the side wall 232a in the future (S304) and the process of determining whether the preceding vehicle 230d to be tracked is blocked by the preceding vehicle 230c (S308). However, the present invention is not limited to such a case, and it is also possible to execute only one of them.

[0074] In addition, a program that causes a computer to function as the vehicle external environment recognition device 120, and a computer-readable storage medium such as a flexible disk, a magneto-optical disk, a ROM, a CD, a DVD, a BD, etc. on which the program is recorded are also provided. Here, the program refers to data processing means described in an arbitrary language and description method.

[0075] As described above, the preferred embodiments of the present invention have been described with reference to the accompanying drawings. Needless to say, the present invention is not limited to such embodiments. It is obvious that those skilled in the art can conceive of various modification examples or correction examples within the scope described in the claims, and it is naturally understood that they also belong to the technical scope of the present invention.

[0076] For example, in the above-described embodiment, an example in which the side wall 232a and other preceding vehicles 230c are located on the left side of the preceding vehicles 230a and 230d to be tracked has been described. However, the present invention is not limited to such a case, and it goes without saying that the same processing can be applied when a side wall or other preceding vehicles are located on the right side of the preceding vehicles 230a and 230d.

[0077] In addition, each step of the vehicle external environment recognition method in this specification does not necessarily need to be processed in time series in the order described as a flowchart, and may include parallel or subroutine processing.

Industrial Applicability

[0078] The present invention can be used in an in-vehicle external environment recognition device that identifies a three-dimensional object existing in the traveling direction of a host vehicle.

Explanation of Signs

[0079] 110 Imaging device 120 In-vehicle external environment recognition device 170 Position derivation unit 172 Three-dimensional object identification unit 174 Specific object identification unit 176 Vehicle tracking unit

Claims

1. One or more processors, One or more memories connected to the processor, And having, The processor, in cooperation with a program included in the memory, A position derivation unit that derives a three-dimensional position in block units in the captured image, A three-dimensional object identification unit that groups blocks whose differences in the three-dimensional positions are within a predetermined range to identify a three-dimensional object, A specific object identification unit that identifies a preceding vehicle and a side wall from the three-dimensional object, A vehicle tracking unit that predicts a future position of the preceding vehicle and tracks the preceding vehicle, Functioning as, The vehicle tracking unit is an out-of-vehicle environment recognition device that determines whether or not the preceding vehicle to be tracked is blocked by the side wall based on whether or not a visible boundary line, which is a tangent to an approximate curve of the side wall passing through an imaging device of the host vehicle, intersects with a velocity vector of the preceding vehicle to be tracked, or based on whether or not the visible boundary line intersects with a tangent to the approximate curve of the side wall at a depth position of the preceding vehicle.

2. The out-of-vehicle environment recognition device according to claim 1, wherein when the vehicle tracking unit determines that the preceding vehicle is blocked by the side wall, the vehicle tracking unit stops tracking the preceding vehicle.

3. When the vehicle tracking unit satisfies the condition, the vehicle tracking unit increments by a predetermined value a point used for determining that the preceding vehicle to be tracked is blocked by the side wall, and when the point becomes equal to or greater than a predetermined threshold value, the vehicle tracking unit determines that the preceding vehicle to be tracked is blocked by the side wall. The out-of-vehicle environment recognition device according to claim 1 or claim 2.

4. One or more processors, One or more memories connected to the processor, And having, The processor, in cooperation with a program included in the memory, A position derivation unit that derives a three-dimensional position in block units in the captured image, A three-dimensional object identification unit that groups blocks whose differences in the three-dimensional positions are within a predetermined range to identify a three-dimensional object, A specific object identification unit that identifies a preceding vehicle and a side wall from the three-dimensional object, A vehicle tracking unit that predicts a future position of the preceding vehicle and tracks the preceding vehicle, Functioning as, The vehicle tracking unit, Based on a boundary line between a blind spot and a visible area, determines whether or not the preceding vehicle to be tracked is blocked by the side wall, An out-of-vehicle environment recognition device that stops tracking the preceding vehicle when it is determined that the preceding vehicle is blocked by the side wall.

5. one or more processors, one or more memories connected to the processor, and having, the processor cooperates with a program included in the memory to, a position derivation unit that derives a three-dimensional position in block units in the captured image, a solid object identification unit that groups blocks whose difference in the three-dimensional position is within a predetermined range to identify a solid object, a specific object identification unit that identifies a preceding vehicle from the solid object, a vehicle tracking unit that predicts the future position of the preceding vehicle and tracks the preceding vehicle, function as, the vehicle tracking unit, based on the positional relationship of a plurality of preceding vehicles, determines whether the preceding vehicle to be tracked is blocked by another preceding vehicle, an out-of-vehicle environment recognition device that stops tracking the preceding vehicle when it is determined that the preceding vehicle is blocked by the other preceding vehicle.

Images