Image processing device, image processing method, and program

Abstract

An image processing device includes: a detection unit that detects another vehicle present in an image captured by an image sensor mounted on a host vehicle; and a correction unit that corrects a vanishing point of the image, the correction unit correcting the vanishing point on the basis of a result of comparison between the detected first width of the other vehicle and a second width of the other vehicle estimated using the vanishing point.

Description

TECHNICAL FIELD

[0001] The present application relates to an image processing apparatus, an image processing method, and a program. BACKGROUND

[0002] A technique of extracting a vanishing point from an image captured by a camera mounted on a vehicle is known. For example, a technique of calculating a motion vector between a plurality of images captured by a camera and calculating a vanishing point coordinate within an image using the calculated motion vector is disclosed in Patent Literature 1.

[0003] PRIOR ART LITERATURE

[0004] PATENT LITERATURE

[0005] Patent Literature 1: Japanese Patent Application Publication No. 2012-123751 SUMMARY

[0006] PROBLEMS TO BE SOLVED BY THE INVENTION

[0007] The technique described in Patent Literature 1 calculates a motion vector for each pixel based on a plurality of images captured by a camera using a block matching method, a gradient method, or the like. However, such a conventional technique sometimes requires a large calculation cost and a large battery consumption in order to improve the accuracy of a vanishing point.

[0008] The present application has been achieved in view of such a situation, and one object thereof is to provide an image processing apparatus, an image processing method, and a program capable of calculating a vanishing point with high accuracy while suppressing a calculation cost and a battery consumption to be low.

[0009] MEANS FOR SOLVING THE PROBLEMS

[0010] The image processing apparatus, the image processing method, and the program according to the present application have the following structure.

[0011] (1) One aspect of the present application relates to an image processing apparatus including a detection unit that detects another vehicle present in an image captured by an image sensor mounted on a host vehicle, and a correction unit that corrects a vanishing point of the image based on a comparison result between a first width of the another vehicle detected and a second width of the another vehicle estimated using the vanishing point.

[0012] (2) In the aspect described in (1) above, in a case where the first width of the another vehicle traveling in either of left and right directions with respect to the host vehicle is smaller than the second width, the correction unit corrects the vanishing point in the direction.

[0013] (3): In the aspect of (1) described above, in a case where the first width of the other vehicle traveling in either of left and right directions with respect to the host vehicle is larger than the second width, the correction section corrects the vanishing point in a direction opposite to the direction.

[0014] (4): In the aspect of (2) or (3) described above, in a case where a distance between the host vehicle and the same other vehicle detected in a time series is close, the correction section performs the correction.

[0015] (5): In the aspect of (2) or (3) described above, in a case where a distance between the host vehicle and the other vehicle is below a threshold value, the correction section performs the correction.

[0016] (6): Another aspect of the present application relates to an image processing method, in which the image processing method causes a computer to perform processing of: detecting an other vehicle existing in an image captured by an image sensor mounted on a host vehicle; and correcting a vanishing point of the image, the correction being based on a comparison result between a first width of the detected other vehicle and a second width of the other vehicle estimated using the vanishing point.

[0017] (7): Another aspect of the present application relates to a program, in which the program causes a computer to perform processing of: detecting an other vehicle existing in an image captured by an image sensor mounted on a host vehicle; and correcting a vanishing point of the image, the correction being based on a comparison result between a first width of the detected other vehicle and a second width of the other vehicle estimated using the vanishing point.

[0018] Effects of Invention

[0019] According to (1) to (7), the vanishing point can be calculated with high accuracy while suppressing the calculation cost and the battery consumption low. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 is a diagram illustrating an example of an environment in which a terminal device 100 mounted on a host vehicle M is used.

[0021] Figure 2 is a diagram illustrating an example of a structure of the terminal device 100.

[0022] Figure 3 is a diagram illustrating an example of a scene in which a distance estimation section 120 estimates a distance between the host vehicle M and an other vehicle M1.

[0023] Figure 4is a diagram for explaining a method by which the distance estimation section 120 estimates the distance in the longitudinal direction between the host vehicle M and the other vehicle M1.

[0024] Figure 5 is a diagram for explaining a method by which the distance estimation section 120 estimates the distance in the lateral direction of the other vehicle M1.

[0025] Figure 6 is a diagram showing an example of the bird's-eye view and the camera image displayed on the display section 20.

[0026] Figure 7 is a diagram showing another example of the bird's-eye view and the camera image displayed on the display section 20.

[0027] Figure 8 is a diagram for explaining a determination method by which the vanishing point correction section 140 determines the longitudinal position of the vanishing point V.

[0028] Figure 9 is a diagram showing an example of a scenario in which the vanishing point correction section 140 corrects the lateral position of the vanishing point V.

[0029] Figure 10 is a diagram for explaining a correction method by which the vanishing point correction section 140 corrects the lateral position of the vanishing point V.

[0030] Figure 11 is another diagram for explaining a correction method by which the vanishing point correction section 140 corrects the lateral position of the vanishing point V.

[0031] Figure 12 is a flowchart showing an example of the flow of processing performed by the terminal device 100. DETAILED DESCRIPTION

[0032] Embodiments of the image processing device, the image processing method, and the program of the present application will be described below with reference to the drawings. In the present embodiment, the image processing device is, for example, a terminal device 100 such as a smartphone having a camera (image sensor) and a display (display section). However, the present application is not limited to such a structure, and the image processing device can be at least a computer device having a calculation function obtained by removing the camera and the display from the structure of the terminal device 100 described later. In this case, the functions of the present application are realized by the cooperation of the camera, the display, and the image processing device.

[0033] [Structure]

[0034] Figure 1is a diagram showing an example of an environment in which the terminal device 100 mounted on the host vehicle M is used. The host vehicle M is, for example, a two-wheeled, three-wheeled, four-wheeled, or the like vehicle, and its drive source is an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination thereof. The electric motor operates using power generated by a generator coupled to the internal combustion engine, or power discharged from a secondary battery or a fuel cell.

[0035] As shown in Figure 1 , the terminal device 100 is disposed on the host vehicle M in a manner capable of photographing a region in front of the host vehicle M in a traveling direction. The terminal device 100 is held by a not-shown vehicle-mounted support mounted on a cowl of the host vehicle M, for example, and is used to photograph a region in front of the host vehicle M. As will be described later, a user of the terminal device 100 positions the terminal device 100 at a prescribed height in accordance with a guide line displayed on the display portion 20. Figure 1 The result of the positioning of the terminal device 100 indicates a case in which the terminal device 100 photographs a vicinity of an upper end of another vehicle M1 of the host vehicle M in a substantially horizontal direction with respect to a road surface.

[0036] Figure 2 is a diagram showing an example of a structure of the terminal device 100. As Figure 2As illustrated, the terminal device 100 includes, for example, the camera 10, the display section 20, the object detection section 110, the distance estimation section 120, the bird's-eye view generation section 130, and the vanishing point correction section 140. The object detection section 110, the distance estimation section 120, the bird's-eye view generation section 130, and the vanishing point correction section 140 are realized, for example, by a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of these components can be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), a GPU (Graphics Processing Unit), or the like, and can be realized by a combination of software and hardware. The program can be stored in advance in a storage device (a storage device including a non-transitory storage medium) such as an HDD (Hard Disk Drive), a flash memory, or the like, or can be stored in a removable storage medium (a non-transitory storage medium) such as a DVD, a CD-ROM, or the like, and installed by mounting the storage medium in a drive device. In the following description, the functions of the object detection section 110, the distance estimation section 120, the bird's-eye view generation section 130, and the vanishing point correction section 140 are collectively referred to as a "bird's-eye view application". The bird's-eye view application is mounted on the terminal device 100, and is started, for example, when the user of the terminal device 100 starts driving the host vehicle M. The camera 10 is, for example, a digital camera that uses a solid-state imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The display section 20 is, for example, a display device such as a touch panel or a liquid crystal display.

[0037] [Estimation of Distance]

[0038] The object detection section 110 detects an object mapped in an image captured by the camera 10. More specifically, for example, the object detection section 110 detects an object using a learned model learned in such a manner that information on the presence, position, category, or the like of an object is output when an image captured by the camera 10 is input. For example, the object detection section 110 can detect the presence and position of a vehicle and the presence and position of a road division line using the learned model.

[0039] The distance estimation section 120 estimates a distance between a detected vehicle and the camera 10 when the object detected by the object detection section 110 is a vehicle (in this case, the vehicle refers to a vehicle of two wheels, three wheels, four wheels, or the like).Figure 3 is an example of a scene in which the distance estimation section 120 estimates the distance between the host vehicle M and the other vehicle Ml. In Figure 3 , the reference sign h A indicates the height from the road position corresponding to the lower end of the display section 20 to the vanishing point of the image, and the reference sign h B indicates the height from the road position corresponding to the lower end of the detected other vehicle Ml to the vanishing point of the image. The method of determining the vanishing point will be described later.

[0040] Figure 4 is a diagram for explaining the method by which the distance estimation section 120 estimates the distance in the longitudinal direction between the host vehicle M and the other vehicle Ml. In Figure 4 , the reference sign IS indicates the image sensor included in the camera 10, the reference sign D indicates the display included in the camera 10 (the end portion of the camera 10), the reference sign O indicates the central portion of the image sensor, the reference sign A indicates the position on the display D corresponding to the road position F mapped to the bottom of the display D, the reference sign B indicates the position on the display D corresponding to the road position G of the rear end portion of the other vehicle Ml, the reference sign C indicates the intersection of the photographing direction of the image sensor and the display D, the reference sign H indicates the height of the camera 10 from the road surface, and the reference sign D A indicates the distance from the position of the image sensor IS to the road position F mapped to the bottom of the display D, and the reference sign D B indicates the distance from the position of the image sensor IS to the road position G of the rear end portion of the other vehicle Ml.

[0041] In Figure 4 , the triangle OAC is similar to the triangle OEF, and the triangle OBC is similar to the triangle OEG. That is, L: h A = D A : H, and L: h B = D B : H, so D A = L x H / h A , and D B = L x H / h B . Therefore, the distance estimation section 120 can calculate the distance to the road position G of the rear end portion of the other vehicle Ml using the calculation formula of D B = D A x h A / h B . Here, the height h A , h B to the vanishing point can be calculated in advance on the basis of the image photographed by the camera 10, and is independent of the distance D AThis can be calculated in advance according to the set position of the terminal device 100. Note that the above calculation can be performed using only the height information of the vanishing point, without all coordinate information of the vanishing point.

[0042] The distance estimation section 120 further estimates the lateral distance of the detected vehicle in the case where the object detected by the object detection section 110 is a vehicle. Figure 5 is a diagram for explaining a method by which the distance estimation section 120 estimates the lateral distance of the other vehicle Ml. In Figure 5 , reference sign V shows the vanishing point of the image, reference sign Wb shows the pixel number of the lateral direction of the other vehicle Ml with the vanishing point V as a reference, and reference sign Wa shows the pixel number when the pixel number Wb is moved to the lowermost end of the display D.

[0043] In Figure 5 , the triangle VT'T has a similar relationship with the triangle VS'S. That is, Wa: h A == Wb: h B is established, so it is transformed to Wa = Wb x h A / h B . Here, when it is assumed that the total pixel number Wsc of the lower end of the display D and the road width Wrd on which the host vehicle M is traveling are known, the distance estimation section 120 can calculate the actual lateral distance W corresponding to the pixel number Wa using the calculation formula of W = Wrd x Wa / Wsc. As described above, the distance estimation section 120 calculates the longitudinal distance and the lateral distance between the host vehicle M and the other vehicle Ml.

[0044] [Generation of Aerial View]

[0045] The aerial view generation section 130 generates an aerial view showing the surrounding situation of the host vehicle M based on the longitudinal distance and the lateral distance between the host vehicle M and the other vehicle Ml estimated by the distance estimation section 120. The aerial view generation section 130 causes the generated aerial view to be displayed on the display section 20 together with the camera image captured by the camera 10.

[0046] Figure 6 is a diagram showing an example of the aerial view and the camera image displayed on the display section 20. Figure 6 The left portion of Figure 6 shows the aerial view, Figure 6 The right portion of shows the camera image. InIn this case, reference sign GL shows a guide line for setting the position of the terminal device 100 at the time of starting the bird's-eye view application (or at the time of starting the host vehicle M) by the user. The user sets the terminal device 100 in the vehicle in such a manner that the intersection of the displayed guide line coincides with the vanishing point V (i.e., a point at which the road can no longer be seen). The intersection set at this time is read into the bird's-eye view application as an initial value of the vanishing point V. Figure 6 An example in which only the host vehicle M is displayed in the bird's-eye view immediately after starting the bird's-eye view application is shown. Instead of this, the bird's-eye view application can automatically set the vanishing point V, which is the intersection of the guide line, at the central position of the display portion 20.

[0047] Figure 7 is a view showing another example of the bird's-eye view and the camera image displayed in the display portion 20. Figure 7 An example in which the distance estimation portion 120 estimates the distance in the longitudinal direction and the distance in the lateral direction with the host vehicle M as a reference with respect to four other vehicles and one motorcycle detected by the object detection portion 110, and the bird's-eye view generation portion 130 generates a bird's-eye view based on the estimated distances is shown. As shown in the left portion of Figure 7 As shown in the left portion of

[0048] [Decision of Vanishing Point]

[0049] As described above, the vanishing point V of the image captured by the camera 10 is utilized when estimating the distance between the host vehicle M and other vehicles. The vanishing point V is automatically calibrated to the central position of the display portion 20 immediately after starting the bird's-eye view application, or is manually decided by the user using a guide line, but when the host vehicle M starts running, the vanishing point V of the image can change, for example, depending on the condition of the road. Therefore, in order to accurately estimate the distance between the host vehicle M and other vehicles and generate a bird's-eye view, it is required to accurately decide and correct the vanishing point V. Hereinafter, a method of deciding and correcting the vanishing point V performed by the vanishing point correction portion 140 will be described.

[0050] [Decision of Longitudinal Position of Vanishing Point]

[0051] The vanishing point correction portion 140 sets a bounding box with respect to the other vehicle Ml detected by the object detection portion 110 from the camera image, and decides the longitudinal position of the vanishing point V as a position of a length (a length of a prescribed ratio) prescribed downward from the upper end of the set bounding box. This is because, in the present embodiment, the terminal device 100 including the camera 10 is disposed at the upper portion of the front window or the rear window of the host vehicle M as shown in Figure 1 Therefore, it is assumed that the height of the vanishing point corresponds to the height at which the terminal device 100 is disposed.

[0052] Figure 8is a view for explaining a method of determining the longitudinal position of the vanishing point V. As shown in Figure 8 The vanishing point correction section 140 sets a bounding box BB that surrounds the other vehicle Ml detected, and determines the longitudinal position VL of the vanishing point V as a position of an amount of length 1 defined downward from the upper end of the set bounding box BB. In a case where a plurality of other vehicles Ml are detected from the camera image, the vanishing point correction section 140 selects the other vehicle Ml closest to the host vehicle M among them as a target vehicle for setting the longitudinal position VL of the vanishing point V. Note that the vanishing point correction section 140 can change the defined length 1 in a case where the host vehicle M on which the terminal device 100 is installed is a two-wheeled vehicle and in a case where it is a four-wheeled vehicle. In this way, the longitudinal position of the vanishing point V is determined.

[0053] [Decision of Lateral Position of Vanishing Point]

[0054] On the other hand, regarding the lateral position of the vanishing point, the vanishing point correction section 140 corrects the lateral position of the vanishing point V based on a comparison result between the width (measured value) of the bounding box BB of the detected other vehicle Ml and the width (estimated value) of the other vehicle Ml estimated using the last decided vanishing point VP. Figure 9 is a view showing an example of a scene in which the vanishing point correction section 140 corrects the lateral position of the vanishing point V. Figure 9 As an example, a case where the other vehicle Ml is detected in a lane on the left side of the lane on which the host vehicle M is traveling is indicated. In Figure 9 , x indicates the width of the bounding box BB, and y indicates the height of the bounding box BB.

[0055] As shown in Figure 9 , the vanishing point correction section 140, in a case where the other vehicle Ml traveling in either of the left and right directions with respect to the host vehicle M is detected, performs the lateral position correction of the vanishing point V explained below. The "case where the other vehicle Ml traveling in either of the left and right directions with respect to the host vehicle M is detected" can be, for example, a case where the other vehicle Ml is detected as traveling on a lane different from the lane on which the host vehicle M is traveling, or a case where it is detected that the lateral distance from the host vehicle M estimated based on the camera image is a predetermined distance or more.

[0056] Further, the vanishing point correction section 140 performs the lateral position correction of the vanishing point V explained below in a case where the distance between the host vehicle M and the same other vehicle Ml detected in time series is approaching (in other words, in a case where the length of the relative coordinates of the other vehicle Ml with the host vehicle M as a reference is becoming small). This is based on the fact that the lateral position correction of the vanishing point V explained below is effective in a case where the host vehicle M and the other vehicle Ml are not too far apart. Therefore, instead of this, the vanishing point correction section 140 can perform the correction explained below in a case where the distance between the host vehicle M and the other vehicle Ml is below a threshold value. Further, for example, the vanishing point correction section 140 can perform the correction explained below in a case where the side surface of the other vehicle Ml is recognized to be above a prescribed area.

[0057] Figure 10 is a view for explaining a correction method of the lateral position of the vanishing point V corrected by the vanishing point correction section 140. Figure 10 represents a bird's-eye view obtained by performing bird's-eye view conversion on the image shown in Figure 9 by the vanishing point correction section 140. That is, the vanishing point correction section 140 estimates the width of the other vehicle Ml from the bird's-eye view obtained using the last decided vanishing point VP, and compares it with the width of the other vehicle Ml actually measured from the bounding box BB, thereby verifying the last decided vanishing point VP, and correcting the lateral position of the vanishing point VP according to the comparison result.

[0058] In Figure 10 , the angle a represents the angle of the diagonal line P1-P2 of the other vehicle Ml formed as a rectangle on the bird's-eye view with respect to the lane, and the angle β represents the angle with respect to the other vehicle Ml when the other vehicle Ml is observed from the host vehicle M. Further, the length a represents the vehicle width of the other vehicle Ml, and the length b represents the vehicle length of the other vehicle Ml, and each is set to the average vehicle width and the average vehicle length of vehicles.

[0059] At this time, the vanishing point correction section 140 can calculate the angles a and β as a = arctan (a / b), β = arctan (x / y), respectively. The vanishing point correction section 140 can estimate the width w of the other vehicle Ml as w = L sin (a + β) = V (a2+ b2) sin (a + β) using the above values. That is, the estimated width w represents the width of the other vehicle Ml estimated on the premise of the last decided vanishing point VP.

[0060] Next, the vanishing point correction section 140 compares the estimated width w of the other vehicle Ml with the width x of the bounding box BB actually measured from the camera image, and corrects the lateral position of the vanishing point VP according to the comparison result. Figure 11is another diagram for explaining a correction method of the lateral position of the vanishing point V corrected by the vanishing point correction section 140. Figure 11 Pattern (a) of the comparison result shown indicates a case where the estimated width w of the other vehicle Ml is larger than the measured width x (i.e., w = x + Δ), Figure 11 Pattern (b) of the comparison result shown indicates a case where the estimated width w of the other vehicle Ml is smaller than the measured width x (i.e., x = w + Δ).

[0061] As shown in pattern (a), in the case where the estimated width w of the other vehicle Ml is larger than the measured width x, this means that the other vehicle Ml is actually going to the left direction more than estimated. That is, the vanishing point VP used for generating the bird's-eye view should be located on the left side more, and thus the vanishing point correction section 140 corrects the vanishing point VP to the left direction from the vanishing point VP decided last time. On the other hand, as shown in pattern (b), in the case where the estimated width w of the other vehicle Ml is smaller than the measured width x, this means that the other vehicle Ml is actually going to the right direction more than estimated. That is, the vanishing point VP used for generating the bird's-eye view should be located on the right side more, and thus the vanishing point correction section 140 corrects the vanishing point VP to the right direction from the vanishing point VP decided last time.

[0062] The vanishing point correction section 140 repeatedly performs the above-described decision of the longitudinal position of the vanishing point VP and the correction of the lateral position at a prescribed control cycle (e.g., every several seconds). Such decision and correction of the vanishing point according to the present embodiment do not cause complicated calculation accompanying the use of a learned model or the like, nor communication with an external server or the like. Thus, the vanishing point can be found with high accuracy while suppressing the calculation cost and the battery consumption low. In particular, in the present embodiment, the terminal device 100 is a terminal held by a user such as a smartphone, and thus the convenience as a user can be improved by suppressing the calculation cost and the battery consumption low.

[0063] Note that, in the above description, the case where the other vehicle Ml is detected has been described. However, the present embodiment is not limited to this. For example, the present embodiment can be applied to a case where a pedestrian or the like is detected. Figure 9 to Figure 11In the explanation of the above-described example, the case where the other vehicle Ml travels in front of the host vehicle M on the left side thereof is explained, but the vanishing point correction section 140 can also correct the lateral position of the vanishing point VP in the same manner in the case where the other vehicle Ml travels in front of the host vehicle M on the right side thereof. More specifically, the vanishing point correction section 140 corrects the vanishing point VP to the right direction from the vanishing point VP decided last time in the case where the estimated width w of the other vehicle Ml traveling in front of the host vehicle M on the right side thereof is larger than the measured width x, which means that the other vehicle Ml actually travels to the right direction more than estimated. In the case where the estimated width w of the other vehicle Ml is smaller than the measured width x, which means that the other vehicle Ml actually travels to the left direction more than estimated. Thus, the vanishing point correction section 140 corrects the vanishing point VP to the left direction from the vanishing point VP decided last time.

[0064] Next, the flow of the process performed by the terminal device 100 will be explained with reference to the flowchart shown in FIG. 6. Figure 12 Figure 12 is a flowchart showing an example of the flow of the process performed by the terminal device 100. Figure 12 The process of the flowchart shown is performed, for example, in a prescribed control cycle in the travel of the host vehicle M. Figure 12 The process of the flowchart shown is premised on the start of the terminal device 100 and the initial value of the vanishing point VP being calibrated.

[0065] First, the terminal device 100 acquires the camera image captured by the camera 10 and detects the other vehicle (step S100). Next, the terminal device 100 sets a bounding box on the detected other vehicle and sets the longitudinal position of the vanishing point to a position of a length prescribed downward from the upper end of the set bounding box (step S102). Next, the terminal device 100 generates an aerial view based on the camera image and the vanishing point (step S104).

[0066] Next, the terminal device 100 calculates the relative coordinates of the other vehicle with the host vehicle as a reference in time series on the aerial view (step S106). Next, the terminal device 100 determines whether the distance between the host vehicle M and the other vehicle is approaching based on the relative coordinates calculated in time series (step S108). In the case where it is determined that the distance between the host vehicle M and the other vehicle is not approaching, the terminal device 100 ends the process of the flowchart. In this case, the vanishing point obtained by correcting only the longitudinal position is obtained.

[0067] ​On the other hand, in a case where it is determined that the distance between the host vehicle and the other vehicle is approaching, the terminal device 100 compares the estimated width w of the other vehicle calculated on the bird's-eye view and the measured width w of the bounding box set for the other vehicle (step S110). Next, the terminal device 100 corrects the lateral position of the vanishing point on the basis of the comparison result between the estimated width w and the measured width w (step S112). Thereby, the processing of the present flowchart ends.

[0068] Note that, in step S108 of the above flowchart, the terminal device 100 determines whether the distance between the host vehicle M and the other vehicle is approaching on the basis of the relative coordinates calculated in time series. However, the present application is not limited to such a configuration, and the terminal device 100 can determine whether the distance between the host vehicle M and the other vehicle M1 is below a threshold value in step S108.

[0069] According to the present embodiment as explained above, the other vehicle existing in an image captured by a camera mounted on the host vehicle is detected, and the lateral position of the vanishing point is corrected on the basis of the comparison result between the first width, which is a measured value of the bounding box of the detected other vehicle, and the second width, which is an estimated value of the width of the other vehicle estimated using the last vanishing point. Thereby, the vanishing point can be calculated with high accuracy while suppressing the calculation cost and the battery consumption to be low.

[0070] The above-described embodiment can be expressed as follows.

[0071] An image processing device configured to include:

[0072] a storage medium that stores computer-readable instructions; and

[0073] a processor connected to the storage medium,

[0074] the processor executing the computer-readable instructions to:

[0075] detect an other vehicle existing in an image captured by an image sensor mounted on a host vehicle; and

[0076] correct a vanishing point of the image,

[0077] The vanishing point is corrected based on a comparison between a first width of the other vehicle detected and a second width of the other vehicle estimated using the vanishing point.

[0078] The above describes the specific embodiments of the present application using the embodiments, but the present application is by no means limited to such embodiments, and various modifications and substitutions can be made within the scope of the gist of the present application.

[0079] Reference Signs Description

[0080] 10 camera

[0081] 20 display section

[0082] 110 object detection section

[0083] 120 distance estimation section

[0084] 130 bird's-eye view generation section

[0085] 140 vanishing point correction section

Claims

1. An image processing apparatus, wherein, The image processing device includes: The inspection department detects other vehicles present in images captured by the image sensors mounted on this vehicle; and The correction unit corrects the vanishing point of the image. The correction unit corrects the vanishing point based on a comparison between the detected first width of the other vehicle and the second width of the other vehicle estimated using the vanishing point.

2. The image processing apparatus according to claim 1, wherein, If the first width of another vehicle traveling in either the left or right direction relative to this vehicle is smaller than the second width, the correction unit corrects the vanishing point in that direction.

3. The image processing apparatus according to claim 1, wherein, If the first width of another vehicle traveling in either the left or right direction relative to this vehicle is greater than the second width, the correction unit corrects the vanishing point in the direction opposite to the vanishing point.

4. The image processing apparatus according to claim 2 or 3, wherein, The correction unit performs the correction when the distance between the vehicle and another vehicle detected in the time sequence is approaching.

5. The image processing apparatus according to claim 2 or 3, wherein, When the distance between the vehicle and the other vehicles is below a threshold, the correction unit performs the correction.

6. An image processing method, wherein, The image processing method causes the computer to perform the following processing: Detect other vehicles present in images captured by the image sensors mounted on this vehicle; as well as Correct the vanishing point of the image. The correction is based on a comparison between the detected first width of the other vehicle and the second width of the other vehicle estimated using the vanishing point, and the vanishing point is corrected accordingly.

7. A program in which, The program causes the computer to perform the following processing: Detect other vehicles present in images captured by the image sensors mounted on this vehicle; as well as Correct the vanishing point of the image. The correction is based on a comparison between the detected first width of the other vehicle and the second width of the other vehicle estimated using the vanishing point, and the vanishing point is corrected accordingly.

Images