Moving body recognition device

Abstract

To provide a moving body recognition device which can suitably recognize a dimension of a moving body in a captured image in a situation in which the moving body is moving.SOLUTION: A moving body recognition device 1 includes: a camera 21 which captures an image of a front of a vehicle V; a distance measurement sensor 22 which detects relative movement information on a moving body included in a capturing range of the camera 21 relative to the vehicle V; a first movement amount acquisition unit which acquires a first movement amount as a movement amount of the moving body with the distance measurement sensor 22 as a reference on the basis of the relative movement information and an exposure time of the camera 21; a second movement amount acquisition unit which acquires a second movement amount as a movement amount of the moving body with a world coordinate as a reference on the basis of the first movement amount; and a dimension correction unit which corrects a dimension on the captured image of the moving body on the basis of the captured image of the camera 21 and the second movement amount.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a moving object recognition device. [Background technology]

[0002] Conventionally, a moving object detection device is known that generates a blurred image by processing one captured image based on captured images with different exposure times and movement information of the imaging device, and detects a moving object from the difference between the blurred image and the other captured image (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2016-170522 A Summary of the Invention [Problem to be solved by the invention]

[0004] The blurred image in the above-mentioned conventional technology is generated based on a blur filter capable of simulating blur caused by the movement of an imaging device, so the blurred image includes only pseudo blur caused by the movement of the imaging device, and does not include pseudo blur caused by the movement of a subject. However, in a situation where a moving object is moving, blur occurs on the captured image due to the movement of the moving object itself, so there is room for improvement in order to properly recognize the dimensions of the moving object on the captured image. [Means for solving the problem]

[0005] A moving object recognition device according to one embodiment of the present invention includes a camera that captures an image of the area in front of a vehicle, a moving object sensor that detects relative movement information of a moving object included in the camera's imaging range with respect to the vehicle, a first movement amount acquisition unit that acquires a first movement amount, which is the movement amount of the moving object based on the moving object sensor, based on the relative movement information and an exposure time of the camera, a second movement amount acquisition unit that acquires a second movement amount, which is the movement amount of the moving object based on world coordinates, based on the first movement amount, and a dimension correction unit that corrects the dimensions of the moving object on the captured image based on the camera's captured image and the second movement amount.

[0006] According to a moving object recognition device according to an aspect of the present invention, a first movement amount, which is a movement amount of a moving object based on a moving object sensor, is acquired based on relative movement information and an exposure time of a camera. Since the relative movement information is acquired by a moving object sensor, not a camera, the first movement amount does not include the influence of blur. Based on such a first movement amount, the dimensions of the moving object on the captured image are corrected using a second movement amount, which is a movement amount of the moving object based on world coordinates. Therefore, by correcting the dimensions of the moving object on the captured image using the first movement amount that is not affected by blur on the captured image, the influence of blur can be reduced from the dimensions of the moving object on the captured image even in a situation where the moving object is moving. Therefore, according to the moving object recognition device, it is possible to appropriately recognize the dimensions of the moving object on the captured image in a situation where the moving object is moving.

[0007] In one embodiment, the dimension correction unit may correct the depth dimension as the dimension of the moving object on the captured image based on the component of the relative movement information along the traveling direction of the vehicle. In this case, it becomes possible to appropriately recognize the dimension on the captured image of an oncoming vehicle traveling in the opposite lane of the lane in which the vehicle is traveling, as an example of the moving object.

[0008] In one embodiment, the dimension correction unit may correct the width dimension as the dimension of the moving object on the captured image based on the component of the relative movement information along the direction intersecting the traveling direction of the vehicle. In this case, it becomes possible to appropriately recognize the dimension on the captured image of a crossing vehicle traveling on a road that crosses the lane on which the vehicle is traveling, as an example of the moving object.

[0009] In one embodiment, the moving object recognition device further includes a turning detection unit that detects a turning speed of the vehicle, and the dimension correction unit may correct the dimensions of the moving object on the captured image based on the second movement amount, the turning speed, and the exposure time of the camera. In this case, in addition to the influence of the second movement amount of the moving object, the influence of blur according to the turning speed of the vehicle can be reduced from the dimensions of the moving object on the captured image. Effect of the Invention

[0010] According to the moving object recognition device of the present invention, it is possible to appropriately recognize the dimensions of a moving object on a captured image while the moving object is moving. [Brief description of the drawings]

[0011] [Figure 1] 1 is a block diagram showing an example of a moving object recognition device according to an embodiment. [Diagram 2] 1 is an example of a diagram showing the surroundings of a vehicle traveling straight ahead as viewed from above. [Diagram 3] 3 is an example of a schematic captured image of the situation in FIG. 2. [Figure 4] 13 is another example of a view showing the surroundings of the vehicle traveling straight from above. [Diagram 5] 5 is an example of a schematic captured image of the situation in FIG. 4. [Figure 6] 1 is an example of a diagram showing the surroundings of a turning vehicle as viewed from above. [Figure 7] 7 is an example of a schematic captured image of the situation in FIG. 6. [Figure 8] 2 is a flowchart showing an example of a process of the moving object recognition device of FIG. 1. [Figure 9]13 is an example of an impact level based on a vehicle's turning speed and the direction of movement of another vehicle. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, exemplary embodiments will be described with reference to the drawings. In each drawing, the same or corresponding elements are denoted by the same reference numerals, and duplicated explanations will be omitted.

[0013] 1, the moving object recognition device 1 includes a moving object recognition ECU [Electronic Control Unit] 10, a camera 21, a distance measurement sensor (moving object sensor) 22, and an internal sensor 23. The moving object recognition device 1 is mounted on, for example, a vehicle V. The moving object recognition device 1 detects objects (e.g., moving objects) around the vehicle V based on the captured image of the camera 21 and detection data of the distance measurement sensor 22, and recognizes the objects on the captured image of the camera 21.

[0014] The following description will focus on a configuration for appropriately recognizing the dimensions of other vehicles (moving bodies) on the captured image of the camera 21, even when the other vehicles (moving bodies) are moving around the vehicle V as objects. In this embodiment, the moving body recognition device 1 detects a moving body in front of the vehicle V. The front of the vehicle V may be, for example, any position forward of the center of the vehicle V in the vehicle longitudinal direction in plan view.

[0015] The moving object recognition ECU 10 is an electronic control unit having a central processing unit (CPU), a read only memory (ROM), a random access memory (RAM), etc. In the moving object recognition ECU 10, for example, a program recorded in the ROM is loaded into the RAM, and the program loaded into the RAM is executed by the CPU, thereby realizing various functions. The moving object recognition ECU 10 may be composed of multiple electronic units.

[0016] The camera 21 is an imaging device that captures an image of the area in front of the vehicle V. The camera 21 is provided, for example, on the rear side of the windshield of the vehicle V. The camera 21 transmits an image of the area in front of the vehicle V and imaging information including an exposure time of the camera 21 to the moving object recognition ECU 10. The imaging information may also include information that affects the degree of blur in the captured image, such as the brightness of the surroundings of the vehicle V. The camera 21 may be a monocular camera or a stereo camera. A plurality of cameras 21 may be provided.

[0017] The distance measurement sensor 22 detects relative movement information of other vehicles (moving objects) included in the imaging range of the camera 21 with respect to the vehicle V. The relative movement information is information regarding the relative movement of the other vehicle with respect to the vehicle V (more specifically, the distance measurement sensor 22) as a reference. The relative movement information includes, for example, at least the relative distance of the other vehicle with respect to the vehicle V and the relative speed of the other vehicle with respect to the vehicle V. The relative movement information may also include the movement direction of the other vehicle with respect to the vehicle V. The distance measurement sensor 22 transmits the detected relative movement information to the moving object recognition ECU 10.

[0018] The distance measurement sensor 22 may include a radar sensor. A radar sensor is a detection device that detects objects around the vehicle V using radio waves (e.g., millimeter waves) or light. Radar sensors include, for example, millimeter wave radar or LiDAR (Light Detection and Ranging). A radar sensor detects an object by transmitting radio waves or light to the periphery (here, at least the front) of the vehicle V and receiving the radio waves or light reflected by the object. The distance measurement sensor 22 may be a stereo camera as long as it can detect at least the relative distance and the relative speed.

[0019] The internal sensor 23 is an in-vehicle sensor that detects the traveling state of the vehicle V. The internal sensor 23 includes at least a yaw rate sensor (turning detection unit). The yaw rate sensor is a detector that detects the yaw rate (rotational angular velocity, turning speed) around the vertical axis of the center of gravity of the vehicle V. For example, a gyro sensor can be used as the yaw rate sensor. The yaw rate sensor transmits the detected yaw rate information of the vehicle V to the moving object recognition ECU 10.

[0020] Next, a description will be given of an example of the functional configuration of the moving object recognition ECU 10. The moving object recognition ECU 10 has a captured image processing unit 11, a running state recognition unit 12, a blur correction amount calculation unit (a first movement amount acquisition unit, a second movement amount acquisition unit) 13, a dimension correction unit 14, and a moving object recognition unit 15.

[0021] The captured image processing unit 11 identifies other vehicles in the captured image, for example, by deep learning or pattern matching, based on the captured image of the camera 21. The captured image processing unit 11 acquires the apparent dimensions of other vehicles in the captured image by a known method. The apparent dimensions are dimensions of other vehicles in the captured image without correcting blur, which will be described later, and correspond to length dimensions including blur extending in the captured image according to the exposure time, for example.

[0022] Fig. 2 is an example of a view of the surroundings of a vehicle traveling straight from above. In this embodiment, as an example, a situation will be described in which another vehicle X1 is present as an oncoming vehicle of the vehicle V and is approaching the vehicle V as shown in Fig. 2.

[0023] FIG. 3 is an example of a schematic captured image of the situation in FIG. 2. In the captured image of the camera 21, the other vehicle X1 traveling on the oncoming lane RW1 is captured approaching the near side with respect to the position of the camera 21. In the captured image of the camera 21 in FIG. 3, blur BL1 occurs in the other vehicle X1 due to the movement of the other vehicle X1 with respect to the position of the camera 21 during the exposure time of the camera 21. The blur BL1 means an afterimage or blur that occurs naturally (not artificially) due to the movement of the other vehicle X1. Such blur tends to appear more prominently in the captured image in a situation where the surroundings of the vehicle V are dark, such as at night or inside a tunnel, compared to a situation where the surroundings of the vehicle V are bright, such as daytime or outdoors, because the exposure time of the camera 21 is longer.

[0024] In the case of Fig. 3, the vehicle X1 traveling in the oncoming lane RW1 moves in the depth direction on the captured image, so the blur BL1 extends in the depth direction on the captured image. The blur BL1 extends in the depth direction with a length DB1 according to the distance traveled by the vehicle X1 during the exposure time of the camera 21.

[0025] In the example of Fig. 3, the other vehicle X1 that moved during the exposure time of the camera 21 is identified by the captured image processing unit 11 as a dashed rectangular parallelepiped CZ1 including a blur BL1 that occurs on the captured image in appearance. In the case of Fig. 3, the captured image processing unit 11 acquires a length DZ1 as the apparent depth dimension of the other vehicle X1 on the captured image. The captured image processing unit 11 may acquire an angle of the other vehicle with respect to the vehicle V on the captured image (the moving direction of the other vehicle). The captured image processing unit 11 may identify a moving object other than the other vehicle on the captured image.

[0026] The running state recognition unit 12 recognizes the running state of the vehicle V based on the detection result of the internal sensor 23. The running state recognition unit 12 recognizes at least the yaw rate (turning speed) of the vehicle V as the running state based on yaw rate information from a yaw rate sensor. The running state recognition unit 12 may recognize the vehicle speed of the vehicle V as the running state based on vehicle speed information from a vehicle speed sensor.

[0027] The blur correction amount calculation unit 13 calculates a blur correction amount for correcting the dimensions of other vehicles on the captured image.

[0028] More specifically, the blur correction amount calculation unit 13 acquires a first movement amount, which is the movement amount of another vehicle based on the distance measurement sensor 22, based on the relative movement information detected by the distance measurement sensor 22 and the exposure time of the camera. Since the first movement amount is based on the relative movement information detected by the distance measurement sensor 22, it is not affected by blur, unlike the apparent dimensions in the captured image.

[0029] The blur correction amount calculation unit 13 may acquire the first movement amount by using a component of relative movement information corresponding to the angle of the other vehicle with respect to the vehicle V recognized on the captured image of the camera 21, or the movement direction of the other vehicle with respect to the vehicle V detected by the distance measurement sensor 22. For example, when the other vehicle is an oncoming vehicle, the blur correction amount calculation unit 13 may acquire the first movement amount along the traveling direction of the vehicle V by using a component of the relative speed along the traveling direction of the vehicle V. For example, when the other vehicle is a crossing vehicle, the blur correction amount calculation unit 13 may acquire the first movement amount along the direction intersecting the traveling direction of the vehicle V by using a component of the relative speed along the direction intersecting the traveling direction of the vehicle V. Note that the crossing vehicle is another vehicle traveling along a direction intersecting the traveling direction of the vehicle V, and does not necessarily have to cross in front of the vehicle V.

[0030] The blur correction amount calculation unit 13 acquires a second movement amount, which is a movement amount of the other vehicle based on the world coordinates, based on the first movement amount. Since the second movement amount is based on the world coordinates, it is not influenced by the movement of the vehicle V (for example, the influence of the vehicle speed and turning speed of the vehicle V).

[0031] The blur correction amount calculation unit 13 acquires a third movement amount, which is the movement amount of the other vehicle on the captured image of the camera 21, based on the second movement amount. The third movement amount is based on the second movement amount based on the world coordinates, and is therefore the movement amount of the other vehicle on the captured image of the camera 21 excluding the influence of the movement of the vehicle V. The blur correction amount calculation unit 13 can use a known coordinate conversion method to acquire the first movement amount, the second movement amount, and the third movement amount.

[0032] In the example of FIG. 3, the position of the other vehicle X1 at the beginning of the exposure time (here, the position of the front end of the other vehicle X1) is indicated by a two-dot chain line LZ1, and the position of the other vehicle X1 at the end of the exposure time is indicated by a dashed line LX1. A distance D1 on the captured image from the two-dot chain line LZ1 to the dashed line LX1 is a third movement amount of the other vehicle X1 on the captured image, which corresponds to the second movement amount in the world coordinate system. Here, the blur BL1 occurs due to the movement of the other vehicle X1 from the beginning to the end of the exposure time, so its length DB1 is equivalent to the distance D1 on the captured image from the two-dot chain line LZ1 to the dashed line LX1. Therefore, the blur correction amount calculation unit 13 uses the third movement amount as a blur correction amount corresponding to the length DB1 of the blur BL1 on the captured image.

[0033] The dimension correcting unit 14 corrects the dimension of the other vehicle on the captured image based on the captured image of the camera 21 and the second movement amount. In the case of FIG. 3, the dimension correcting unit 14 corrects the depth dimension as the dimension on the captured image of the other vehicle X1 based on the component of the relative speed (relative movement information) along the traveling direction of the vehicle V. The dimension correcting unit 14 removes the influence of the blur BL1 on the captured image by subtracting the third movement amount, which is a blur correction amount corresponding to the length DB1 of the blur BL1 on the captured image, from the length DZ1, which is the apparent depth dimension on the captured image of the other vehicle X1. The dimension correcting unit 14 corrects the depth dimension on the captured image of the other vehicle X1 to be the length DX1 obtained by subtracting the length DB1 from the length DZ1.

[0034] The moving object recognition unit 15 recognizes, on the captured image, the other vehicle X1 whose depth dimension on the captured image has been corrected by the dimension correction unit 14. In the example of Fig. 3, the moving object recognition unit 15 recognizes the other vehicle X1 that moved during the exposure time of the camera 21 as a rectangular parallelepiped CX1 indicated by a thick solid line with the blur BL1 generated on the captured image removed.

[0035] As another example, the dimension correction unit 14 may correct the width dimension as the dimension on the captured image of the other vehicle based on the component of the relative speed (relative movement information) along the direction intersecting the traveling direction of the vehicle V. Fig. 4 is another example of a view of the surroundings of a vehicle traveling straight from above. As shown in Fig. 4, a situation will be described in which another vehicle X2 exists as a crossing vehicle of the vehicle V, and the other vehicle X2 is traveling from the left to the right of the vehicle V.

[0036] Fig. 5 is an example of a schematic captured image of the situation in Fig. 4. In the captured image of the camera 21, another vehicle X2 traveling on the oncoming lane RW2 is captured traveling in the left-right direction (the width direction of the captured image) based on the position of the camera 21. The left-right direction here is the width direction of the captured image, which corresponds to the vehicle width direction of the vehicle V. In the captured image of the camera 21 in Fig. 5, blur BL2 occurs in the other vehicle X2 due to the movement of the other vehicle X2 based on the position of the camera 21 during the exposure time of the camera 21.

[0037] 5, the vehicle X2 traveling in the oncoming lane RW2 moves along the width direction on the captured image, so the blur BL2 extends along the width direction on the captured image. The blur BL2 extends along the width direction on the captured image with a length DB2 according to the distance traveled by the vehicle X2 during the exposure time of the camera 21.

[0038] In the example of Fig. 5, the other vehicle X2 that moved during the exposure time of the camera 21 is identified by the captured image processor 11 as a dashed rectangular parallelepiped CZ2 including a blur BL2 that occurs on the captured image. In the case of Fig. 5, the captured image processor 11 acquires a length DZ2 as the apparent width dimension of the other vehicle X2 on the captured image.

[0039] In the example of FIG. 5, the position of the other vehicle X2 at the beginning of the exposure time (here, the position of the front end of the other vehicle X2) is indicated by a two-dot chain line LZ2, and the position of the other vehicle X2 at the end of the exposure time is indicated by a dashed line LX2. A distance D2 on the captured image from the two-dot chain line LZ2 to the dashed line LX2 is a third movement amount of the other vehicle X2 on the captured image, which corresponds to the second movement amount in the world coordinate system. Here, the blur BL2 occurs due to the movement of the other vehicle X2 from the beginning to the end of the exposure time, so its length DB2 is equivalent to the distance D2 on the captured image from the two-dot chain line LZ2 to the dashed line LX2. Therefore, the blur correction amount calculation unit 13 uses the third movement amount as a blur correction amount corresponding to the length DB2 of the blur BL2 on the captured image.

[0040] 5, the dimension corrector 14 removes the influence of the blur BL2 on the captured image by subtracting a third movement amount, which is a blur correction amount corresponding to the length DB2 of the blur BL2 on the captured image, from the length DZ2, which is the apparent width dimension of the other vehicle X2 on the captured image. The dimension corrector 14 corrects the width dimension of the other vehicle X2 on the captured image to be the length DX2 obtained by subtracting the length DB2 from the length DZ2.

[0041] The moving object recognition unit 15 recognizes, on the captured image, the other vehicle X2 whose width dimension on the captured image has been corrected by the dimension correction unit 14. In the example of Fig. 5, the moving object recognition unit 15 recognizes the other vehicle X2 that moved during the exposure time of the camera 21 as a rectangular parallelepiped CX2 indicated by a thick solid line with the blur BL2 generated on the captured image removed.

[0042] As another example, the dimension corrector 14 may correct the dimensions of the other vehicle on the captured image based on the second movement amount, the turning speed, and the exposure time of the camera. The blur correction amount calculator 13 may calculate a fourth movement amount for correcting turning blur, which is the blur of the other vehicle on the captured image generated according to the turning speed of the vehicle V, based on the yaw rate information detected by the internal sensor 23, the exposure time of the camera, and the relative distance.

[0043] Fig. 6 is an example of a view of the surroundings of a turning vehicle as seen from above. As shown in Fig. 6, a situation will be described in which another vehicle X3 is present as an oncoming vehicle of the vehicle V, the other vehicle X3 is approaching the vehicle V, and the vehicle V is turning in a right direction at a turning speed Y.

[0044] Fig. 7 is an example of a schematic captured image of the situation in Fig. 6. In the captured image of the camera 21, another vehicle X3 traveling on the oncoming lane RW3 is captured approaching the near side with respect to the position of the camera 21. In the captured image of the camera 21 in Fig. 7, blur BL3 and blur BL4 are generated in the other vehicle X3 due to the fact that the other vehicle X3 has moved with respect to the position of the camera 21 during the exposure time of the camera 21 and that the vehicle V is turning in the right direction at a turning speed Y. Blur BL4 means an afterimage or blur that occurs naturally (not artificially) due to the turning of the vehicle V.

[0045] In the case of FIG. 7, the other vehicle X3 traveling on the oncoming lane RW3 moves along the depth direction on the captured image, so the blur BL3 extends along the depth direction on the captured image. The blur BL3 extends along the depth direction with a length DB3 according to the distance traveled by the other vehicle X3 during the exposure time of the camera 21. On the other hand, the vehicle V turns in the right turn direction at a turning speed Y, so the blur BL4 extends along the right side of the other vehicle X3 in the width direction on the captured image. The blur BL4 extends along the width direction to the right of the other vehicle X3 with a length DB4 according to the distance traveled by the vehicle V in the right turn direction at the turning speed Y during the exposure time of the camera 21. The blur correction amount calculation unit 13 may calculate, as the fourth movement amount, the perimeter of a virtual arc when the vehicle V rotates in the right turn direction at the turning speed Y for the exposure time period, with the relative distance from the vehicle V to the other vehicle X3 as the radius.

[0046] In the example of Fig. 7, the other vehicle X3 that moved during the exposure time of the camera 21 is identified by the captured image processor 11 as a dashed rectangular parallelepiped CZ34 including blurs BL3 and BL4 that appear on the captured image. In the case of Fig. 7, the captured image processor 11 acquires a length DZ3 as the apparent depth dimension of the other vehicle X3 on the captured image, and acquires a length DZ4 as the apparent width dimension of the other vehicle X3 on the captured image.

[0047] 2 and 3, the dimension corrector 14 removes the influence of blur BL3 on the captured image by subtracting a third movement amount, which is a blur correction amount corresponding to the length DB3 of blur BL3 on the captured image, from the length DZ3, which is the apparent depth dimension of the other vehicle X3 on the captured image. The dimension corrector 14 corrects the depth dimension of the other vehicle X3 on the captured image to be length DX3, which is the length DZ3 minus length DB3.

[0048] In the width direction, the dimension correcting unit 14 removes the influence of the blur BL4 on the captured image by subtracting a fourth movement amount, which is a turning blur correction amount corresponding to the length DB4 of the blur BL4 on the captured image, from the length DZ4, which is the apparent width dimension of the other vehicle X3 on the captured image. The dimension correcting unit 14 corrects the width dimension of the other vehicle X3 on the captured image to be the length DX4 obtained by subtracting the length DB4 from the length DZ4.

[0049] The moving object recognition unit 15 recognizes, on the captured image, the other vehicle X3 whose depth dimension and width dimension on the captured image have been corrected by the dimension correction unit 14. In the example of Fig. 7, the moving object recognition unit 15 recognizes the other vehicle X3 as a rectangular parallelepiped CX34 indicated by a thick solid line, from which blur BL3 generated on the captured image due to the movement of the other vehicle X3 during the exposure time of the camera 21 and blur BL4 generated on the captured image due to imaging from the vehicle V turning in the right turn direction at a turning speed Y have been removed.

[0050] Next, a flow of processing performed by the moving object recognition ECU 10 of the moving object recognition device 1 will be described with reference to the flowchart of Fig. 8. The processing shown in Fig. 8 may be restarted from the start after a predetermined time has elapsed since the processing ended.

[0051] 8, the moving object recognition ECU 10 of the moving object recognition device 1, as S01, identifies other vehicles in the captured image by, for example, deep learning or pattern matching, based on the captured image of the camera 21, using the captured image processing unit 11. The captured image processing unit 11 acquires apparent dimensions of the other vehicles in the captured image.

[0052] In S02, the moving object recognition ECU 10 recognizes the traveling state of the vehicle V by the traveling state recognition unit 12 based on the detection result of the internal sensor 23. The traveling state recognition unit 12 recognizes, for example, the yaw rate (turning speed) of the vehicle V as the traveling state based on yaw rate information of the yaw rate sensor.

[0053] In S03 and S04, the moving object recognition ECU 10 acquires a first movement amount, which is a movement amount of another vehicle based on the distance measurement sensor 22, using the blur correction amount calculation unit 13 based on the relative movement information detected by the distance measurement sensor 22 and the exposure time of the camera. In S03, the moving object recognition ECU 10 acquires the first movement amount along the traveling direction of the vehicle V using the component of the relative speed along the traveling direction of the vehicle V using the blur correction amount calculation unit 13. In S04, the moving object recognition ECU 10 acquires the first movement amount along the direction intersecting the traveling direction of the vehicle V using the component of the relative speed along the direction intersecting the traveling direction of the vehicle V using the blur correction amount calculation unit 13.

[0054] In S05, the moving object recognition ECU 10 causes the blur correction amount calculation unit 13 to acquire a second movement amount, which is a movement amount of the other vehicle with respect to the world coordinate system, based on the first movement amount.

[0055] In S06, the moving object recognition ECU 10 causes the blur correction amount calculation unit 13 to acquire a third movement amount, which is the movement amount of the other vehicle on the image captured by the camera 21, based on the second movement amount.

[0056] In S07, the moving object recognition ECU 10 calculates a turning blur correction amount (fourth movement amount) for correcting turning blur based on the yaw rate information detected by the internal sensor 23 and the exposure time and relative distance of the camera using the blur correction amount calculation unit 13.

[0057] In S08, the moving object recognition ECU 10 causes the dimension correction unit 14 to correct the dimension of the other vehicle on the captured image based on the image captured by the camera 21 and the second movement amount. The dimension correction unit 14 may correct the dimension of the other vehicle on the captured image using a third movement amount acquired based on the second movement amount. The dimension correction unit 14 may correct the width dimension of the other vehicle on the captured image using a fourth movement amount acquired based on the turning speed (yaw rate).

[0058] In S09, the moving object recognition ECU 10 causes the moving object recognition unit 15 to recognize, on the captured image, another vehicle whose dimensions (depth dimension and / or width dimension) on the captured image have been corrected by the dimension correction unit 14. Thereafter, the moving object recognition ECU 10 ends the process of FIG.

[0059] According to the moving object recognition device 1 described above, a first movement amount, which is the movement amount of the other vehicle based on the distance measurement sensor 22, is acquired based on the relative movement information and the exposure time of the camera 21. Since the relative movement information is acquired by the distance measurement sensor 22, not the camera 21, the first movement amount does not include the influence of blur. Based on such a first movement amount, the dimensions of the other vehicle on the captured image are corrected using a second movement amount, which is the movement amount of the other vehicle based on the world coordinates. Therefore, by correcting the dimensions of the other vehicle on the captured image using the first movement amount that is not affected by blur on the captured image, even in a situation where the other vehicle is moving, the influence of blur can be reduced from the dimensions of the other vehicle on the captured image. Therefore, according to the moving object recognition device 1, it is possible to appropriately recognize the dimensions of the other vehicle on the captured image in a situation where the other vehicle is moving. By appropriately recognizing the dimensions of the other vehicle on the captured image, for example, when the recognized other vehicle is displayed typically on a display or the like provided on the vehicle V, the dimensions of the other vehicle on the display become more appropriate with the influence of blur reduced.

[0060] In the moving object recognition device 1, the dimension correction unit 14 corrects the depth dimension as the dimension on the captured image of the other vehicle based on the component of the relative speed along the traveling direction of the vehicle V. This makes it possible to appropriately recognize the dimension on the captured image of an oncoming vehicle traveling in the opposite lane to the lane on which the vehicle V is traveling, as an example of the other vehicle.

[0061] In the moving object recognition device 1, the dimension correction unit 14 corrects the width dimension as the dimension on the captured image of the other vehicle based on the component of the relative speed along the direction intersecting the traveling direction of the vehicle V. This makes it possible to appropriately recognize the dimension on the captured image of a crossing vehicle traveling on a road that crosses the lane on which the vehicle V is traveling, as an example of the other vehicle.

[0062] The moving object recognition device 1 further includes an internal sensor 23 that detects the turning speed of the vehicle V. The dimension correction unit 14 corrects the dimensions of the other vehicle on the captured image based on the second movement amount, the turning speed, and the exposure time of the camera 21. This makes it possible to reduce the influence of blurring according to the turning speed of the vehicle V, in addition to the influence of the second movement amount of the other vehicle, from the dimensions of the other vehicle on the captured image.

[0063] Although the embodiment of the present disclosure has been described above, the present invention is not limited to the above embodiment. The present invention can be embodied in various forms including the above embodiment and various modifications and improvements based on the knowledge of those skilled in the art.

[0064] In the above embodiment, as shown in Fig. 6 and Fig. 7, when the other vehicle is an oncoming vehicle, the width dimension of the other vehicle on the captured image is corrected based on the turning speed of the vehicle V, but this is not limited to this. When the other vehicle is a crossing vehicle, the width dimension of the other vehicle on the captured image may be corrected based on the turning speed of the vehicle V. For example, in Fig. 4 and Fig. 5, the turning blur can be corrected by a fourth movement amount calculated based on the turning speed of the vehicle V.

[0065] In this case, the internal sensor 23 may detect the turning direction of the vehicle V, and the dimension correction unit 14 may correct the dimensions of the moving body on the captured image based on the second movement amount, the turning speed, and the exposure time of the camera 21 when the moving direction of the moving body is opposite to the turning direction. When the moving direction of the moving body is opposite to the turning direction, the blur caused by the movement of the moving body and the blur caused by the turning of the vehicle V are added together, so that the overall effect of the blur is large. In this way, by correcting the dimensions when the moving direction of the moving body is opposite to the turning direction, which is a case in which the effect of the blur caused by the turning of the vehicle V is large, the effect of the blur can be effectively reduced from the dimensions of the moving body on the captured image.

[0066] Furthermore, when the turning speed is equal to or greater than a predetermined turning speed threshold, the dimension correction unit 14 may correct the dimensions of the moving object on the captured image based on the second movement amount, the turning speed, and the exposure time of the camera 21. FIG. 9 shows an example of the influence level based on the turning speed of the vehicle and the moving direction of another vehicle. FIG. 9 shows the influence level according to the magnitude relationship between the turning speed (yaw rate) and a predetermined turning speed threshold γth. The influence level is an index that expresses the degree of influence in a plurality of levels, in which the longer the extension length of the blur on the captured image, the greater the influence. The influence level is, for example, four levels, from level 1 to level 4. Level 1 is the minimum level, and level 4 is the maximum level. The extension length of the blur on the captured image is longer at level 2 than at level 1, longer at level 3 than at level 2, and longer at level 4 than at level 3.

[0067] 9, when the moving direction of the other vehicle is fixed, the influence level when the turning speed is equal to or greater than the turning speed threshold value γth is equal to or greater than the influence level when the turning speed is less than the turning speed threshold value γth. In this way, the influence of blur when the turning speed is equal to or greater than the turning speed threshold value, which is the case where the influence of blur due to turning of the vehicle V becomes large, can be reduced from the dimensions of the moving body on the captured image.

[0068] In the example of Fig. 9, when the moving direction of the other vehicle is fixed, the influence level when the moving speed V of the moving body is equal to or greater than a predetermined moving speed threshold Vth is equal to or greater than the influence level when the moving speed V of the moving body is less than the moving speed threshold Vth. In the example of Fig. 9, the influence level when the moving direction of the moving body is opposite to the turning direction is greater than the influence level when the moving direction of the moving body is the same as the turning direction. Depending on the magnitude relationship of such influence levels, the dimension modification unit 14 may modify the dimensions of the moving body on the captured image.

[0069] In the above embodiment, the depth dimension of the moving object on the captured image is corrected based on the component of the relative movement information along the traveling direction of the vehicle, but it does not necessarily have to be based on the component of the relative movement information along the traveling direction of the vehicle. In the above embodiment, the width dimension of the moving object on the captured image is corrected based on the component of the relative movement information along the direction intersecting the traveling direction of the vehicle, but it does not necessarily have to be based on the component of the relative movement information along the direction intersecting the traveling direction of the vehicle.

[0070] In the above embodiment, the first movement amount along each direction was obtained based on the component of relative movement information along the vehicle's traveling direction and / or the component of relative movement information along a direction intersecting the vehicle's traveling direction, but the first movement amount may be obtained regardless of each component, and then decomposed into components along each direction when obtaining the second movement amount or the third movement amount.

[0071] In the above embodiment, the dimensions of the moving object on the captured image are corrected based on the turning speed of the vehicle V. However, it does not necessarily have to be based on the turning speed of the vehicle V.

[0072] In the above embodiment, the third movement amount is obtained based on the second movement amount, but obtaining the third movement amount is not essential as long as the dimensions of the other vehicle in the captured image are corrected based on the captured image and the second movement amount. [Explanation of symbols]

[0073] 1...moving object recognition device, 13...blur correction amount calculation unit (first movement amount acquisition unit, second movement amount acquisition unit), 14...dimension correction unit, 21...camera, 22...distance measurement sensor (moving object sensor), V...vehicle.

Claims

1. A camera that captures an image of the front of the vehicle; a moving body sensor that detects relative movement information of a moving body included in an imaging range of the camera with respect to the vehicle; a first movement amount acquisition unit that acquires a first movement amount, which is a movement amount of the moving body based on the moving body sensor, based on the relative movement information and an exposure time of the camera; a second movement amount acquisition unit that acquires a second movement amount, which is a movement amount of the moving object based on a world coordinate system, based on the first movement amount; a dimension correction unit that corrects a dimension of the moving object on the captured image based on the captured image of the camera and the second movement amount; A turning detection unit that detects a turning speed of the vehicle, A moving object recognition device, wherein the dimension correction unit corrects the dimension of the moving object on the captured image based on the second movement amount, the rotation speed, and an exposure time of the camera.

2. The moving object recognition device according to claim 1 , wherein the dimension correction unit corrects a depth dimension as the dimension of the moving object on the captured image based on a component of the relative movement information along a traveling direction of the vehicle.

3. The moving object recognition device according to claim 1 or 2, wherein the dimension correction unit corrects a width dimension as the dimension of the moving object on the captured image based on a component of the relative movement information along a direction intersecting a traveling direction of the vehicle.

Images