Distance Estimation Device, Distance Estimation Method, and Computer Program for Distance Estimation

Abstract

To provide a distance estimation apparatus capable of estimating a distance to a target object in the surroundings of a vehicle without relying on stereo cameras for ranging.SOLUTION: A distance estimation apparatus identifies a first partial region from a first surrounding image generated at a first point of time by a first camera mounted on a vehicle. The distance estimation apparatus generates an interpolated second surrounding image corresponding to the first point of time by using a preceding second surrounding image and a succeeding second surrounding image. The distance estimation apparatus estimates an interpolated position of a second camera corresponding to the first point of time. The distance estimation apparatus detects one or more feature points for a target object represented in both the first partial region and the interpolated second surrounding image, and estimates the distance between the vehicle and the target object, using at least a relationship between the position of the first camera at the first point of time and the interpolated position of the second camera, and a relationship between the position of one feature point among the one or more feature points in the first partial region and the position of the feature point corresponding to the one feature point in the interpolated second surrounding image.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present disclosure relates to a distance estimation device, a distance estimation method, and a computer program for distance estimation that estimate the distance from a vehicle to a surrounding object.

Background Art

[0002] In order to appropriately assist the driving of a vehicle, it is important to appropriately grasp the distance from the vehicle to a surrounding object. Patent Document 1 describes a vehicle peripheral monitoring device that estimates the distance to an object within a predetermined distance from the vehicle and tracks the position of the object.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The vehicle peripheral monitoring device described in Patent Document 1 estimates the distance from the vehicle to an object based on the parallax of the image portions of the same object imaged by two cameras (stereo cameras) arranged side by side in the vehicle so that their optical axes are parallel. That is, according to the vehicle peripheral monitoring device of Patent Document 1, it is necessary to mount a dedicated stereo camera for estimating the distance from the vehicle to an object, which causes an increase in cost. There is a need for a distance estimation device that does not require dedicated hardware such as a stereo camera.

[0005] An object of the present disclosure is to provide a distance estimation device that can estimate the distance to an object around a vehicle without using a stereo camera for distance measurement.

Means for Solving the Problems

[0006] The distance estimation device according to the present disclosure includes: a specifying unit that specifies a first partial region corresponding to a region represented by a preceding second peripheral image generated at a second time before the first time by a second camera having a second imaging range narrower than the first imaging range of a first camera mounted on the vehicle, or a region represented by a subsequent second peripheral image generated at a third time after the first time by the second camera, from a first peripheral image representing the situation around the vehicle generated at the first time by the first camera mounted on the vehicle; an interpolation unit that generates an interpolated second peripheral image corresponding to the first time using the preceding second peripheral image and the subsequent second peripheral image, and estimates an interpolated position of the second camera corresponding to the first time using the position of the second camera at the second time and the position of the second camera at the third time, or the position of the first camera at the first time; a detection unit that detects one or more feature points for a target object commonly represented from each of the first partial region and the interpolated second peripheral image; and an estimation unit that estimates the distance between the vehicle and the target object using at least the relationship between the position of the first camera at the first time and the interpolated position of the second camera, and the relationship between the position of one of the one or more feature points in the first partial region and the position of the feature point corresponding to the one feature point in the interpolated second peripheral image.

[0007] The distance estimation device according to the present disclosure preferably further includes a conversion unit that converts the first partial region and the interpolated second peripheral image so that they have the same resolution by changing the resolution of at least one of the first partial region and the interpolated second peripheral image.

[0008] The distance estimation method according to the present disclosure includes identifying a first partial region corresponding to a region represented in a preceding second peripheral image generated at a second time before the first time by a second camera having a second imaging range narrower than the first imaging range of a first camera mounted on the vehicle, or a region represented in a subsequent second peripheral image generated at a third time after the first time by the second camera, from a first peripheral image representing the situation around the vehicle generated at the first time by the first camera having the first imaging range mounted on the vehicle, generating an interpolated second peripheral image corresponding to the first time using the preceding second peripheral image and the subsequent second peripheral image, estimating an interpolated position of the second camera corresponding to the first time using the position of the second camera at the second time and the position of the second camera at the third time, or the position of the first camera at the first time, detecting one or more feature points for an object represented in common from each of the first partial region and the interpolated second peripheral image, and estimating the distance between the vehicle and the object using at least the relationship between the position of the first camera at the first time and the interpolated position of the second camera, and the relationship between the position of one of the one or more feature points in the first partial region and the position of the feature point corresponding to the one feature point in the interpolated second peripheral image.

[0009] The computer program for distance estimation according to the present disclosure identifies a first partial region corresponding to a region represented in a preceding second surrounding image generated at a second time earlier than the first time by a second camera having a second imaging range narrower than the first imaging range of a first camera mounted on a vehicle, or a region represented in a subsequent second surrounding image generated at a third time later than the first time by the second camera, from a first surrounding image representing the situation around the vehicle generated at the first time by the first camera mounted on the vehicle, generates an interpolated second surrounding image corresponding to the first time using the preceding second surrounding image and the subsequent second surrounding image, estimates an interpolated position of the second camera corresponding to the first time using the position of the second camera at the second time and the position of the second camera at the third time, or the position of the first camera at the first time, detects one or more feature points for a target object commonly represented from each of the first partial region and the interpolated second surrounding image, and estimates the distance between the vehicle and the target object using at least the relationship between the position of the first camera at the first time and the interpolated position of the second camera, and the relationship between the position of one of the one or more feature points in the first partial region and the position of the feature point corresponding to the one feature point in the interpolated second surrounding image, and causes a computer to execute the same.

[0010] According to the distance estimation device of the present disclosure, the distance to a target object around the vehicle can be estimated without using a stereo camera for distance measurement.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Modes for Carrying Out the Invention

[0012] Hereinafter, with reference to the drawings, a distance estimation device that can estimate the distance to a target object around a vehicle without using a stereo camera for distance measurement will be described in detail. The distance estimation device identifies a first partial region from a first peripheral image representing the situation around the vehicle generated by a first camera mounted on the vehicle at a first time. The first partial region corresponds to a region represented in a preceding second peripheral image generated by a second camera mounted on the vehicle at a second time before the first time, or a region represented in a subsequent second peripheral image generated by the second camera at a third time after the first time. The first camera has a first shooting range, and the second camera has a second shooting range smaller than the first shooting range. Next, the distance estimation device generates an interpolated second peripheral image corresponding to the first time using the preceding second peripheral image and the subsequent second peripheral image. Also, the distance estimation device generates an interpolated position of the second camera corresponding to the first time using the position of the second camera at the second time and the position of the second camera at the third time, or the position of the first camera at the first time. Next, the distance estimation device detects one or more feature points for a target object commonly represented from each of the first partial region and the interpolated second peripheral image. Then, the distance estimation device estimates the distance between the vehicle and the target object using at least the relationship between the position of the first camera at the first time and the interpolated position of the second camera, and the relationship between the position of one of the one or more feature points in the first partial region and the position of one of the feature points in the interpolated second peripheral image.

[0013] FIG. 1 is a schematic configuration diagram of a vehicle in which the distance estimation device is implemented.

[0014] The vehicle 1 includes a wide-angle camera 2, a telephoto camera 3, a GNSS (Global Navigation Satellite System) receiver 4, and a distance estimation device 5. The wide-angle camera 2, the telephoto camera 3, and the GNSS receiver 4 and the distance estimation device 5 are communicably connected via an in-vehicle network conforming to a standard such as a controller area network.

[0015] The wide-angle camera 2 is an example of a first camera for generating image data including a first peripheral image representing the situation around the vehicle. The wide-angle camera 2 includes a two-dimensional detector composed of an array of photoelectric conversion elements sensitive to visible light, such as a CCD or a C-MOS, and an imaging optical system that forms an image of an area to be imaged on the two-dimensional detector. The wide-angle camera 2 can capture an image within a first imaging range, for example, a range of ±30 degrees in the horizontal direction from the optical axis of the imaging optical system. The wide-angle camera 2 is disposed, for example, at the upper front inside the vehicle cabin, facing forward. The wide-angle camera 2 has a clock synchronized with the time supplied by, for example, the GNSS receiver 4. The wide-angle camera 2 captures the situation around the vehicle 1 through the windshield at a predetermined imaging period (for example, 1 / 30 second to 1 / 10 second) and outputs image data corresponding to the situation around. The image data output by the wide-angle camera 2 can be used to obtain detailed information about an object located at a position with a relatively small distance from the vehicle 1.

[0016] The telephoto camera 3 is an example of a second camera for generating image data including a second peripheral image representing the situation around the vehicle. The telephoto camera 3 includes a two-dimensional detector composed of an array of photoelectric conversion elements sensitive to visible light, such as a CCD or a C-MOS, and an imaging optical system that forms an image of an area to be imaged on the two-dimensional detector. The telephoto camera 3 can capture an image within a second imaging range that is narrower than the first imaging range, for example, a range of ±15 degrees in the horizontal direction from the optical axis of the imaging optical system. The telephoto camera 3 is disposed, for example, at the upper front inside the vehicle cabin, facing forward and arranged side by side with the wide-angle camera 2 left and right. The telephoto camera 3 has a clock synchronized with the time supplied by, for example, the GNSS receiver 4. The telephoto camera 3 captures the situation around the vehicle 1 through the windshield at a predetermined imaging period (for example, 1 / 30 second to 1 / 10 second) and outputs image data corresponding to the situation around. The image data output by the telephoto camera 3 can be used to obtain information about an object located at a position with a relatively large distance from the vehicle 1.

[0017] The wide-angle camera 2 and the telephoto camera 3 may be arranged side by side left and right so as to face the rear or side of the vehicle 1 instead of or in addition to the front of the vehicle 1, so that the same object can be photographed.

[0018] The GNSS receiver 4 is an example of a positioning sensor. It receives GNSS signals from GNSS satellites at a predetermined period by a GNSS antenna (not shown), and determines the self-position of the vehicle 1 based on the received GNSS signals. The GNSS receiver 4 outputs a positioning signal representing the positioning result of the self-position of the vehicle 1 based on the GNSS signals to the distance estimation device 5 via the in-vehicle network at a predetermined period.

[0019] The distance estimation device 5 is an ECU (Electronic Control Unit) having a communication interface, a memory, and a processor. The distance estimation device 5 receives images from the wide-angle camera 2 and the telephoto camera 3 via the communication interface. The distance estimation device estimates the distance between the vehicle 1 and an object existing around the vehicle 1 using the received images.

[0020] FIG. 2 is a hardware schematic diagram of the distance estimation device 5. The distance estimation device 5 includes a communication interface 51, a memory 52, and a processor 53.

[0021] The communication interface 51 is an example of a communication unit and has a communication interface circuit for connecting the distance estimation device 5 to the in-vehicle network. The communication interface 51 supplies the received data to the processor 53. Also, the communication interface 51 outputs the data supplied from the processor 53 to the outside.

[0022] Memory 52 is an example of a storage unit and includes a volatile semiconductor memory and a non-volatile semiconductor memory. Memory 52 stores various data used for processing by processor 53, such as the attachment positions, attachment directions of each of the wide-angle camera 2 and the telephoto camera 3, and internal parameters such as the focal length of the imaging optical system and the pixel size of the peripheral image, and a parameter group (number of layers, layer configuration, kernel, weight coefficient, etc.) for defining a neural network used as an identifier for detecting feature points of an object from an image. Further, memory 52 stores various application programs, such as a distance estimation program for executing distance estimation processing.

[0023] Processor 53 is an example of a control unit and includes one or more processors and their peripheral circuits. Processor 53 may further include other arithmetic circuits such as a logical arithmetic unit, a numerical arithmetic unit, or a graphic processing unit.

[0024] FIG. 3 is a functional block diagram of processor 53 included in distance estimation device 5.

[0025] Processor 53 of distance estimation device 5 includes, as functional blocks, a specifying unit 531, an interpolation unit 532, a conversion unit 533, a detection unit 534, and an estimation unit 535. Each of these units included in processor 53 is a functional module implemented by a program executed on processor 53. A computer program for realizing the functions of each unit of processor 53 may be provided in a form recorded on a computer-readable portable recording medium such as a semiconductor memory, a magnetic recording medium, or an optical recording medium. Alternatively, each of these units included in processor 53 may be implemented in distance estimation device 5 as an independent integrated circuit, microprocessor, or firmware.

[0026] The specific unit 531 identifies a first partial region from a first peripheral image representing the situation around the vehicle 1 generated by the wide-angle camera 2 at the first time. The first partial region corresponds to a region represented in a preceding second peripheral image generated by the telephoto camera 3 at a second time before the first time, or a region represented in a subsequent second peripheral image generated by the telephoto camera 3 at a third time after the first time.

[0027] FIG. 4 is a schematic diagram showing an example of the processing of the peripheral image.

[0028] The wide-angle camera 2 generates a first peripheral image SP1 at the first time t1. The telephoto camera 3 generates a preceding second peripheral image SP2 p at a second time t2 before the first time t1, and generates a subsequent second peripheral image SP2 s at a third time t3 after the first time t1.

[0029] The specific unit 531 refers to the mounting positions, mounting directions, and internal parameters of the wide-angle camera 2 and the telephoto camera 3 stored in the memory 52. Further, the specific unit 531 calculates the amount of movement of the vehicle 1 between the second time t2 and the first time t1, or the amount of movement of the vehicle 1 between the first time t1 and the third time t3.

[0030] The specific unit 531 calculates the amount of movement of the vehicle 1 between the second time t2 and the first time t1, or the amount of movement of the vehicle 1 between the first time t1 and the third time t3, from the positions of the vehicle 1 at the first time t1, the second time t2, and the third time t3 respectively. The specific unit 531 identifies the positions of the vehicle 1 at the second time t2 and the third time t3 respectively, based on the positioning signals corresponding to the second time t2 and the third time t3 acquired from the GNSS receiver 4 (GNSS positioning). Note that the specific unit 531 acquires information about the features around the position corresponding to the positioning signal acquired from the GNSS receiver 4, from a storage device (not shown) mounted on the vehicle 1 and storing high-precision map information, and the first peripheral image SP1, the preceding second peripheral image SP2 p , or the subsequent second peripheral image SP2 sBy collating with the ground objects shown, the position of the vehicle 1 (the position of the wide-angle camera 2 or the telephoto camera 3) may be specified (localization process).

[0031] Further, regardless of the position of the vehicle 1 at each of the first time t1, the second time t2, and the third time t3, the specifying unit 531 may calculate the movement amount of the vehicle 1 between the second time t2 and the first time t1, or the movement amount of the vehicle 1 between the first time t1 and the third time t3. For example, the specifying unit 531 may calculate the movement amount of the vehicle 1 in the movement using data (odometry information) representing the operation of the vehicle 1 obtained from an acceleration sensor, an angular velocity sensor, etc. mounted on the vehicle 1 at a plurality of times while the vehicle 1 is moving.

[0032] Based on the parameters of each camera and the movement amount of the vehicle 1, the specifying unit 531 specifies the first partial region A1 from the first peripheral image SP1 by specifying the region occupied by the range shown in the preceding second peripheral image SP2 acquired at the second time t2 by the telephoto camera 3 in the field of view of the wide-angle camera 2 at the first time t1. Further, based on the parameters of each camera and the movement amount of the vehicle 1, the specifying unit 531 specifies the first partial region A1 from the first peripheral image SP1 by specifying the region occupied by the range shown in the subsequent second peripheral image SP2 acquired at the third time t3 by the telephoto camera 3 in the field of view of the wide-angle camera 2 at the first time t1. p Returning to FIG. 3, the interpolation unit 532 generates the interpolated second peripheral image SP2 corresponding to the first time t1 using the preceding second peripheral image SP2 s and the subsequent second peripheral image SP2

[0033] Returning to FIG. 3, the interpolation unit 532 generates the interpolated second peripheral image SP2 corresponding to the first time t1 using the preceding second peripheral image SP2 p and the subsequent second peripheral image SP2 s and generates the interpolated position of the telephoto camera 3 corresponding to the first time t1 using the position of the telephoto camera 3 at the second time t2 and the position of the telephoto camera 3 at the third time t3. i The interpolation unit 532 generates the interpolated second peripheral image SP2 corresponding to the first time t1 using, for example, the block matching method, using the preceding second peripheral image SP2

[0034] The interpolation unit 532 generates the interpolated second peripheral image SP2 corresponding to the first time t1 using, for example, the block matching method, using the preceding second peripheral image SP2 p and the subsequent second peripheral image SP2s For each block of a predetermined size (e.g., 16 pixels × 16 pixels) obtained by dividing s , a motion vector is obtained. Then, the interpolation unit 532 places the preceding second peripheral image SP2 p or the subsequent second peripheral image SP2 s at a position obtained by proportionally dividing the motion vector according to the time interval between the second time t2 and the first time t1 and the time interval between the first time t1 and the third time t3, thereby generating the interpolated second peripheral image SP2 i .

[0035] The interpolation unit 532 acquires the position of the vehicle 1 at each of the second time t2 and the third time t3 from the specifying unit 531. The interpolation unit 532 estimates the position of the vehicle 1 corresponding to the first time t1 by proportionally dividing the positions of the vehicle 1 at the second time t2 and the third time t3 according to the time interval between the second time t2 and the first time t1 and the time interval between the first time t1 and the third time t3. Then, the interpolation unit 532 estimates the interpolated position of the telephoto camera 3 corresponding to the first time t1 using the relationship between the predetermined position of the vehicle 1 represented by the acquired position of the vehicle 1 and the mounting position of the telephoto camera 3 stored in the memory 52. The predetermined position of the vehicle 1 represented by the acquired position of the vehicle 1 is, for example, the position of the GNSS antenna when specified by GNSS positioning. Also, the predetermined position of the vehicle 1 represented by the acquired position of the vehicle 1 is, for example, the position of the wide-angle camera 2 when specified by localization processing. In any case, the predetermined position of the vehicle 1 represented by the acquired position of the vehicle 1 may be corrected to represent a specific position (e.g., the position of the driver's seat) in the vehicle 1. Also, when the position of the telephoto camera 3 is acquired as the position of the vehicle 1 by localization processing, it is not necessary to estimate the interpolated position of the telephoto camera 3 using the relationship between the acquired predetermined position of the vehicle 1 and the mounting position of the telephoto camera 3. Note that when the specifying unit 531 specifies the movement amount of the vehicle 1 regardless of the position of the vehicle 1, the interpolation unit 532 may specify the position of the vehicle 1 by executing a process using the positioning signal acquired from the GNSS receiver 4 described as the operation of the specifying unit 531, or a process using the ground features included in the high-precision map information.

[0036] The interpolation unit 532 may estimate, as the interpolation position of the telephoto camera 3 corresponding to the first time t1, the relative position of the telephoto camera 3 based on the position of the wide-angle camera 2 at the first time t1, based on the respective mounting positions of the wide-angle camera 2 and the telephoto camera 3 stored in the memory 52. In this case, the interpolation unit 532 acquires the position of the wide-angle camera 2 as the position of the vehicle 1 by the localization process.

[0037] The conversion unit 533 changes at least one of the resolution of the first partial region A1 and the interpolated second peripheral image SP2 according to a predetermined upsampling method or downsampling method, so that each of the first partial region A1 and the interpolated second peripheral image SP2 i has the same resolution. i

[0038] If each of the first partial region A1 and the interpolated second peripheral image SP2 i already has the same resolution without conversion, it is not necessary to execute the conversion by the conversion unit 533. Therefore, in the vehicle 1 in which the first partial region A1 and the interpolated second peripheral image SP2 i (that is, the preceding second peripheral image SP2 p and the subsequent second peripheral image SP2 s ) have the same resolution, the distance estimation device 5 may not have the conversion unit 533.

[0039] The detection unit 534 detects one or more feature points for the target object commonly represented from each of the first partial region A1 and the interpolated second peripheral image SP2. i

[0040] The detection unit 534 inputs the first partial region A1 and the interpolated second peripheral image SP2 i to a discriminator pre-trained to detect feature points from an image, for example, to detect one or more feature points (for example, the contact points of the tires) for a target object (for example, another vehicle) existing around the vehicle 1.

[0041] The identifier can be, for example, a convolutional neural network (CNN) having a plurality of convolutional layers connected in series from an input side to an output side. By inputting a plurality of images representing a predetermined object to be detected and having feature points tagged as teacher data into the CNN and performing pre-training, the CNN operates as an identifier for detecting the feature points of the object.

[0042] Further, the detection unit 534 may extract feature points from each of the first partial region A1 and the interpolated second peripheral image SP2 according to a feature point extraction method such as a Harris corner detector or a Scale-Invariant Feature Transform (SIFT). i from each of them.

[0043] In the example shown in FIG. 4, the detection unit 534 detects feature points including the feature point FP1 from the first partial region A1, and the feature points including the feature point FP2 corresponding to the feature point FP1 from the interpolated second peripheral image SP2. i from the interpolated second peripheral image SP2. i including the feature point FP2 corresponding to the feature point FP1.

[0044] The estimation unit 535 estimates the distance between the vehicle 1 and the target object by using at least the relationship between the position of the wide-angle camera 2 and the interpolated position of the telephoto camera 3 at the first time, and the relationship between the position of one feature point among one or more feature points in the first partial region A1 and the position of one feature point in the interpolated second peripheral image SP2. i using at least the relationship between the position of the wide-angle camera 2 and the interpolated position of the telephoto camera 3 at the first time, and the relationship between the position of one feature point among one or more feature points in the first partial region A1 and the position of one feature point in the interpolated second peripheral image SP2.

[0045] For example, the estimation unit 535 calculates the distance between the wide-angle camera 2 and the telephoto camera 3 from the relationship between the position of the wide-angle camera 2 and the interpolated position of the telephoto camera 3 at the first time. Further, the estimation unit 535 calculates the parallax by multiplying the pixel size by the interval (number of pixels) on the image between the feature point FP1 and the feature point FP2 when the first partial region A1 and the interpolated second peripheral image SP2 are overlaid. Then, the estimation unit 535 divides the product of the distance between the wide-angle camera 2 and the telephoto camera 3 and the focal length of the optical system of the telephoto camera 3 by the parallax to obtain the distance between the telephoto camera 3 and the feature point FP1 (or the feature point FP2). i when the first partial region A1 and the interpolated second peripheral image SP2 are overlaid. i including the feature point FP2). i)Calculate the distance to [object], and estimate it as the distance between Vehicle 1 and the target object.

[0046] FIG. 5 is a flowchart of the distance estimation process. The distance estimation device 5 repeatedly executes the process at a predetermined time interval (for example, every 1 / 10 second) while the vehicle 1 is traveling.

[0047] First, the specifying unit 531 of the distance estimation device 5 specifies a first partial region A1 from a first peripheral image SP1 generated by the wide-angle camera 2 at the first time t1 (step S1).

[0048] Next, the interpolation unit 532 of the distance estimation device 5 uses the preceding second peripheral image SP2 generated by the telephoto camera 3 at the second time t2 p , and the subsequent second peripheral image SP2 generated by the telephoto camera 3 at the third time t3 s to generate an interpolated second peripheral image SP2 corresponding to the first time t1. i Also, the interpolation unit 532 generates an interpolated position of the telephoto camera 3 corresponding to the first time t1 using the position of the telephoto camera 3 at the second time t2 and the position of the telephoto camera 3 at the third time t3 (step S2).

[0049] Next, the conversion unit 533 of the distance estimation device 5 changes at least one of the resolutions of the first partial region A1 and the interpolated second peripheral image SP2 i so that each of the first partial region A1 and the interpolated second peripheral image SP2 i has the same resolution (step S3).

[0050] Next, the detection unit 534 of the distance estimation device 5 detects one or more feature points for the target object commonly represented from each of the first partial region A1 and the interpolated second peripheral image SP2 i (step S4).

[0051] The estimation unit 535 of the distance estimation device 5 determines the relationship between the position of the wide-angle camera 2 and the interpolated position of the telephoto camera 3 at the first time t1, and the position of one of the one or more feature points in the first partial region A1 and the interpolated second peripheral image SP2i Estimate the distance between the vehicle 1 and the target object by using at least the relationship with the position of one feature point in (step S5), and end the distance estimation process.

[0052] By executing the distance estimation process in this way, the distance estimation device 5 can estimate the distance to the target object around the vehicle with appropriate accuracy without using a stereo camera for distance measurement.

[0053] The distance estimation device 5 transmits the estimated distance between the vehicle 1 and the target object to a travel control device that transmits a control signal to a travel mechanism such as the engine, brakes, and steering of the vehicle 1. The travel control device can control the travel of the vehicle 1 so that the distance between the vehicle 1 and the objects around the vehicle 1 is equal to or greater than a predetermined value by using the estimated distance between the vehicle 1 and the target object.

[0054] According to a modification, instead of or in addition to detecting feature points from the interpolated second peripheral image SP2 i the detection unit 534 performs detection of feature points from the preceding second peripheral image SP2 p and the subsequent second peripheral image SP2 s respectively. Then, the estimation unit 535 estimates the position of the feature point represented in the image (corresponding to the interpolated second peripheral image SP2 p when it is assumed that the image was generated at the first time t1 by the telephoto camera 3) from the positions of the feature points detected from the preceding second peripheral image SP2 s and the subsequent second peripheral image SP2 i respectively. The estimation unit 535 estimates the distance between the vehicle 1 and the target object by using the estimated position of the feature point. In this case, the interpolation unit 532 does not have to generate the interpolated second peripheral image SP2 i either.

[0055] Those skilled in the art should understand that various changes, substitutions, and modifications can be added to this without departing from the spirit and scope of the present invention.

Explanation of Reference Numerals

[0056] 1 Vehicle 5 Distance estimation device 531 Specific part 532 Interpolation part 533 Conversion part 534 Detection part 535 Estimation part

Claims

1. From a first peripheral image representing the situation around the vehicle generated at a first time by a first camera having a first shooting range mounted on the vehicle, an area represented by a preceding second peripheral image generated at a second time before the first time by a second camera having a second shooting range narrower than the first shooting range mounted on the vehicle, or a subsequent second peripheral image generated at a third time after the first time by the second camera, a specifying unit that specifies a first partial area corresponding to the area; An interpolation unit that generates an interpolated second peripheral image corresponding to the first time using the preceding second peripheral image and the subsequent second peripheral image, and estimates an interpolated position of the second camera corresponding to the first time using the position of the first camera at the first time; A detection unit that detects one or more feature points for an object commonly represented from each of the first partial area and the interpolated second peripheral image; An estimation unit that estimates the distance between the vehicle and the object using at least the relationship between the position of the first camera at the first time and the interpolated position of the second camera, and the relationship between the position of one of the one or more feature points in the first partial area and the position of the feature point corresponding to the one feature point in the interpolated second peripheral image; A distance estimation device comprising the above.

2. The distance estimation device according to claim 1, further comprising a conversion unit that converts the first partial area and the interpolated second peripheral image so that they have the same resolution by changing the resolution of at least one of the first partial area and the interpolated second peripheral image.

3. From a first peripheral image representing the situation around the vehicle generated at a first time by a first camera having a first shooting range mounted on the vehicle, an area represented by a preceding second peripheral image generated at a second time before the first time by a second camera having a second shooting range narrower than the first shooting range mounted on the vehicle, or a subsequent second peripheral image generated at a third time after the first time by the second camera, a first partial area corresponding to the area is specified, An interpolated second peripheral image corresponding to the first time is generated using the preceding second peripheral image and the subsequent second peripheral image, and an interpolated position of the second camera corresponding to the first time is estimated using the position of the first camera at the first time, Detect one or more feature points for a target object commonly represented from each of the first partial area and the interpolated second peripheral image. Estimate the distance between the vehicle and the target object using at least the relationship between the position of the first camera at the first time and the interpolated position of the second camera, and the relationship between the position of one of the one or more feature points in the first partial area and the position of the one feature point in the interpolated second peripheral image. A distance estimation method including this.

4. From a first peripheral image representing the situation around the vehicle generated at a first time by a first camera having a first shooting range mounted on the vehicle, an area represented by a preceding second peripheral image generated at a second time before the first time by a second camera having a second shooting range narrower than the first shooting range mounted on the vehicle, or a subsequent second peripheral image generated at a third time after the first time by the second camera Identify a first partial area corresponding to the area represented by, Generate an interpolated second peripheral image corresponding to the first time using the preceding second peripheral image and the subsequent second peripheral image, and estimate the interpolated position of the second camera corresponding to the first time using the position of the first camera at the first time. Detect one or more feature points for a target object commonly represented from each of the first partial area and the interpolated second peripheral image. Estimate the distance between the vehicle and the target object using at least the relationship between the position of the first camera at the first time and the interpolated position of the second camera, and the relationship between the position of one of the one or more feature points in the first partial area and the position of the one feature point in the interpolated second peripheral image. A computer program for distance estimation for causing a computer to execute this.

Images