Vehicle-mounted imaging device

Abstract

The present invention addresses the problem of providing an onboard imaging device for correcting deviation in a measured distance due to lens distortion characteristics using a corrective value for each plurally divided image height region. This onboard imaging device 100 has an imaging unit 301 for capturing an image using a camera that includes a lens and an imaging element, a region information holding unit 305 for holding the positions of image height regions obtained by sectioning the light-receiving surface of the imaging element at a plurality of image height positions, a corrective value holding unit 307 for holding a corrective value determined for each of the image height regions, a region identification unit 304 for identifying which image height region a feature portion belongs to, and a distance calculation unit 303 for calculating a target distance on the basis of information on the imaging element.

Description

[Technical field]

[0001] The present invention relates to an imaging device that is mounted on a vehicle and is capable of measuring the distance to an object. [Background technology]

[0002] In recent years, the development of self-driving vehicles in which a system takes over part or all of the vehicle's operation is underway. Self-driving vehicles require external recognition sensors that can accurately capture objects outside the vehicle. Among external recognition sensors, imaging devices using cameras are being developed to capture the accurate distance to objects ahead.

[0003] In order to accommodate driving assistance and autonomous driving at intersections, it is important to use a wide-angle lens to provide a wide camera field of view and accurately obtain the distance to objects in order to avoid collisions, etc. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 10-341458 Summary of the Invention [Problem to be solved by the invention]

[0005] However, when using a wide-angle lens, there is a problem that the amount of positional deviation of the image of an object reflected on the periphery of the image increases, and the measurement accuracy of the distance to the object reflected on the periphery of the image also decreases. [Means for solving the problem]

[0006] an extraction unit that extracts a characteristic portion of a measurement object contained in the image; a distance calculation unit that calculates a target distance from the imaging unit to the characteristic portion based on the image; a region information storage unit that stores information on positions of a plurality of image height regions obtained by dividing a light receiving surface of the imaging element into a plurality of different image height positions from a position corresponding to an optical axis of the lens based on distortion characteristics of the lens; a correction value storage unit that is determined for each of the plurality of image height regions and stores information on a correction value used to calculate the target distance; a region identification unit that identifies which of the plurality of image height regions the characteristic portion belongs to; and a correction value acquisition unit that acquires information on the correction value from the correction value storage unit based on information on the image height region to which the characteristic portion belongs, wherein the distance calculation unit corrects the target distance using the correction value acquired by the correction value acquisition unit. Effect of the Invention

[0007] According to the present invention, it is possible to provide an in-vehicle imaging device that can quickly measure the distance to an object with less processing load than conventional devices. Further features related to the present invention will become apparent from the description of this specification and the accompanying drawings. In addition, problems, configurations, and effects other than those described above will become apparent from the following description of the embodiments. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram of an in-vehicle imaging device according to a first embodiment. [Diagram 2] FIG. 4 is a diagram showing an example of a captured image including a characteristic portion of a measurement target. [Diagram 3] 5A and 5B are diagrams illustrating the relationship between lens characteristics and an image height shift. [Figure 4] A diagram showing a captured image and an image height region superimposed on each other. [Diagram 5] 4 is a flowchart showing an example of processing performed by the vehicle-mounted imaging device according to the first embodiment. [Figure 6] FIG. 11 is a block diagram of a vehicle-mounted imaging device according to a second embodiment. [Figure 7] 10 is a flowchart showing an example of processing performed by the vehicle-mounted imaging device according to the second embodiment. [Figure 8] FIG. 11 is a block diagram of an in-vehicle imaging device according to a third embodiment. [Figure 9] FIG. 9(a) is a diagram in which the captured image and image height region at time t1 are superimposed, and FIG. 9(b) is a diagram in which the captured image and image height region at time t2 are superimposed. [Figure 10] FIG. 11 is a flowchart showing an example of processing performed by the vehicle-mounted imaging device according to the third embodiment. [Figure 11] FIG. 11(a) is a diagram showing two images captured at different times and their image height regions superimposed when point O is positioned at the center of the captured image, and FIG. 11(b) is a diagram showing two images captured at different times and their image height regions superimposed when point O is positioned below the center of the captured image. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, a number of embodiments according to the present invention will be described with reference to the drawings. Note that in the present embodiments, a case where vehicles keep to the left will be described as an example.

[0010] [Embodiment 1] FIG. 1 is a block diagram of the vehicle-mounted imaging device according to the first embodiment. The features of this embodiment can be applied to a single or multiple camera configuration. In Fig. 1, 301 is an imaging unit, 302 is an extraction unit, 303 is a distance calculation unit, 304 is a region identification unit, 305 is a region information storage unit, 306 is a correction value acquisition unit, 307 is a correction value storage unit, and 100 is an in-vehicle imaging device.

[0011] <Imaging unit> The imaging unit 301 is mounted on a vehicle and captures an image using a camera including a lens and an imaging element. The image includes objects to be measured, such as people, road signs, utility poles, traffic lights, and curbs. In this embodiment, the imaging unit 301 is configured as a stereo camera system having a pair of cameras. The lenses of the pair of cameras are arranged at approximately the same height in the vertical direction with a predetermined distance between their optical axes in the horizontal direction. The pair of cameras are set to have the same optical specifications, including the distortion characteristics of each lens, and the same specifications, such as the size of the light receiving surface of each imaging element and the number of effective pixels.

[0012] FIG. 2 is a diagram showing an example of a captured image including a characteristic portion of a measurement target. 2 was captured by an image capture unit 301 of a vehicle traveling in a vehicle's own lane 201, which is sandwiched between a left road side 202 and a center line 203 of a single lane in each direction. In the captured image, 201 is the vehicle's own lane, 202 is the left road side, 203 is a curb, 204 is the center line, 205 is a speed sign to be measured, and 206 is the numeric part of the speed sign, which is a characteristic part.

[0013] <Extraction part> The extraction unit 302 extracts a characteristic portion of the measurement target contained in the image. The extraction unit 302 performs image processing based on the image data of the captured image, recognizes the measurement target using a known method such as pattern recognition, and extracts the characteristic portion.

[0014] <Distance calculation section> The distance calculation unit 303 calculates the target distance, which is the distance from the imaging unit 301 to the characteristic part. Here, the target refers to an object to be measured, such as an object or a person, and the target distance refers to the distance from the imaging unit 301 to the target. The distance calculation unit 303 calculates the target distance using an image captured by the imaging unit 301. The target distance is calculated by a known method, such as using parallax, and a detailed description thereof will be omitted. In the case of a stereo camera, if the distance between the optical axes of the lenses is d, the focal length of the lenses is f, and the parallax is q', the target distance p' can be calculated from the relationship between the lengths of similar triangles. p = (f × d) / q (1) It can be calculated using the following formula.

[0015] The distance calculation unit 303 calculates the value of the parallax q' from the positions of the characteristic parts on each image captured by the pair of cameras. Next, the distance calculation unit 303 reads the focal length f and the distance d stored in a storage unit (not shown), and calculates the target distance p' by combining the parallax q' with the formula (1).

[0016] Here, p' and q' in the above formula (1) are the target distance and the parallax when the influence of the image position shift caused by the distortion characteristics of the lens, which will be described later, is large. As will be described later, the distance calculation unit 303 corrects the target distance p' using the correction value acquired from the correction value storage unit 307, and calculates the target distance p when the influence of the image position shift caused by the distortion characteristics of the lens is small.

[0017] Next, image position shift caused by the lens will be described. FIG. 3 is a diagram illustrating the relationship between the lens characteristics and the image height shift. In the captured image 110 shown in Fig. 3, the center of the field of view of the captured image 110 coincides with the optical axis of the lens, and is an optical axis corresponding point. It is assumed that the person 103 captured in the center of the field of view of the captured image 110 shown in Fig. 3 and the person 105 captured at the side edge of the captured image 110 are the same person and are the same distance away from the imaging unit 301. In the case of a lens with ideal characteristics without distortion, the person 103 and the person 105 are captured at positions with the same size and height in the captured image 110.

[0018] However, lenses generally have distortion characteristics that are symmetrical about the optical axis, and distortion is greater in the periphery than in the center of the field of view, so the size and position of person 105 differ from the actual size and position (shift) compared to person 103. In particular, in the case of a wide-angle lens, the field of view is wide, so the change in size of targets at the edges of the captured image and the amount of positional shift also become large (large image height shift).

[0019] The distortion characteristics of a lens not only change depending on the lens specifications, but also change due to individual lens variations, temperature, aging, and other factors even when the specifications are the same. For example, in the case of a lens such as an inexpensive resin lens, whose temperature change and aging behavior differs between the center and periphery of the field of view, the change in size of the target and the amount of positional deviation differ significantly between the center and periphery of the field of view. Therefore, in the example shown in Figure 3, although people 103 and 105 should be imaged at their positions and sizes, there is a risk that they will be imaged at their positions and sizes as people 104 and 106 due to temperature changes and aging.

[0020] Such a phenomenon of deviation in size and position of the target occurs in both the vertical and horizontal directions of the captured image, and appears as an image height shift phenomenon symmetrical about the optical axis center, which is the center of the field of view of the captured image. Therefore, the difference in the deviation between the center and periphery of the lens' field of view may result in a different deviation in the distance measurement, which may affect vehicle control based on the measured distance.

[0021] In the vehicle-mounted imaging device of this embodiment, focusing on the distortion characteristics that are symmetrical around the optical axis of the lens, the image height area is divided into multiple zone areas as shown by multiple concentric circles in Figure 3, and the amount of deviation is grasped and corrected on a zone area basis.

[0022] FIG. 4 is a diagram in which a captured image and an image height region are superimposed. Here, reference numeral 15 denotes an effective pixel area of ​​a captured image, and O denotes an optical axis corresponding point of the lens. In Fig. 4, the optical axis corresponding point O of the lens is set at a position where the diagonal lines of the effective pixel area 15 intersect. The image height in Fig. 4 is the distance from the optical axis corresponding point O of the lens, and the distance from the optical axis corresponding point O of the lens to each end of the diagonal line of the effective pixel area 15 is set to be 100% of the image height.

[0023] In this embodiment, the image sensor of the imaging unit is divided into a plurality of image height regions at a plurality of different image height positions from the position corresponding to the optical axis of the lens based on the distortion characteristics of the lens. The boundaries of each image height region are set to form concentric circles centered on the position corresponding to the optical axis of the lens.

[0024] <Area information holding section> The region information storage unit 305 stores information about the positions of a plurality of image height regions obtained by dividing the light receiving surface of the image sensor into a plurality of different image height positions from a position corresponding to the optical axis of the lens based on the distortion characteristics of the lens.

[0025] <Area identification part> The region specifying unit 304 specifies which of a plurality of image height regions the characteristic portion belongs to. The region specifying unit 304 refers to position information corresponding to each image height region stored in the region information storage unit 305, and specifies which image height region the characteristic portion of the measurement target falls within.

[0026] <Correction value storage section> The correction value storage unit 307 stores information on a correction value for calculating a target distance, which is determined for each of a plurality of image height regions. The correction value storage unit 307 stores a correction value (distance correction coefficient) corresponding to each image height region.

[0027] <Correction value acquisition section> The correction value acquisition unit 306 acquires a correction value (distance correction coefficient) corresponding to the image height region specified by the region specification unit 304 from the correction value storage unit 307 .

[0028] <Distance calculation section> The distance calculation unit 303 calculates the target distance p when the effect of image position shift due to lens distortion characteristics is small by correcting the target distance when the effect of image position shift due to lens distortion characteristics is large using the correction value acquired by the correction value acquisition unit 306. The target distance p' obtained by equation (1) is calculated using the position on the captured image. The distance calculation unit 303 calculates the target distance p by integrating the correction value (distance correction coefficient) acquired by the correction value acquisition unit 306 based on the target distance p'.

[0029] Although the first embodiment has two cameras, the present invention can also be applied to a single camera. In the case of a single camera, for example, a method of determining the target distance p' from the shape of the image blur using a color filter can be used. In other words, the target distance p' can be determined based on the information on the image sensor on which the characteristic portion is imaged, and the target distance p can be calculated in the same manner.

[0030] <Operation of the vehicle-mounted imaging device> FIG. 5 is a flow diagram showing an example of processing of the car-mounted imaging device 100 according to the first embodiment. First, the imaging unit 301 captures an image including a measurement target (S101). The extraction unit 302 performs image processing on the captured image, recognizes the measurement target using a known method such as pattern recognition, and extracts characteristic portions (S102). The region specifying unit 304 specifies to which of the plurality of image height regions the characteristic portion belongs by referring to the information stored in the region information storage unit 305 (S103). The correction value acquisition unit 306 acquires a correction value (distance correction coefficient) corresponding to the section identified by the area identification unit 304 from the correction value storage unit 307 (S104). The distance calculation unit 303 calculates the target distance p by multiplying the target distance p' calculated from the parallax q' for the characteristic portion by a correction value (distance correction coefficient) (S105).

[0031] The processes performed by the imaging unit 301, extraction unit 302, area specification unit 304, distance calculation unit 303, etc. are performed by a microcomputer or dedicated LSI in the camera, a CPU on a PC, etc. (not shown).

[0032] As described above, in the first embodiment, the target distance is calculated using distance correction coefficients set for each of multiple image height regions. This eliminates the need to store a large number of tables, thereby reducing memory capacity, and also reduces the processing load because complex function calculations are not required, thereby enabling faster calculation of the target distance.

[0033] Furthermore, since the cause of the image position shift is corrected only at the stage of calculating the target distance, the processing load is reduced and the target distance can be calculated more quickly than in a method in which the target distance is calculated after converting the image position.

[0034] [Embodiment 2] Next, a second embodiment will be described. In an in-vehicle imaging device, the refractive index of the lens changes with time, and the lens expands and contracts due to its temperature, which may affect the positional deviation of the image. If the image positional deviation is affected, the measurement accuracy of the target distance decreases, so the correction value needs to be updated to an appropriate value. Therefore, in the second embodiment, when the difference between the target distance and the reference distance is equal to or greater than a predetermined value, a calibration process is performed to update the correction value to an appropriate value. Here, the reference distance is acquired, for example, from the traveling speed information of the vehicle information acquisition unit 311. In other words, the distance of the vehicle traveling between two different times t1 and t2 is used as the reference.

[0035] 6 is a block diagram of an in-vehicle imaging device 100A according to the second embodiment. This block diagram is divided into three parts. The first block is a block for calculating a target distance by the in-vehicle imaging device. The second block is a block for calculating a difference between two target distances calculated from the same measurement target belonging to the same image height region at different times. The third block is a block for comparing a predetermined vehicle travel distance with the difference between the two target distances, and determining whether or not to update the correction value.

[0036] The first block has the same configuration as in the first embodiment, so the same reference numerals are used in the drawings and detailed description thereof will be omitted.

[0037] <Second Block> In FIG. 6, 311 is a vehicle information acquisition unit, 313 is a first distance holding unit, 314 is an elapsed time gate, 315 is a second distance holding unit, and 316 is a difference calculation unit.

[0038] <First distance maintaining unit> The first distance storage unit 313 stores a first target distance calculated by the distance calculation unit 303 for a characteristic portion based on an image captured at a first time. For example, the first distance storage unit 313 stores a first target distance calculated for a characteristic portion W extracted from an image captured at time t1.

[0039] <Second distance maintaining unit> The second distance storage unit 315 stores a second target distance calculated by the distance calculation unit 303 for the same characteristic portion based on an image captured at a second time later than the first time. For example, the second distance storage unit 315 stores a second target distance calculated for a characteristic portion W extracted from an image captured at a time t2 later than the time t1.

[0040] <Vehicle information acquisition section> The vehicle information acquisition unit 311 acquires information related to the travel distance of the vehicle. The vehicle information acquisition unit 311 can acquire the vehicle speed from a vehicle speed sensor that generates a pulse signal proportional to the rotation speed of the axle. In addition, the vehicle information acquisition unit 311 may acquire information related to the distance other than the vehicle speed from a V-CAN, a locator, or the like.

[0041] <Elapsed time gate> The elapsed time gate 314 opens the gate, and causes the second distance storage unit 315 to store the second target distance calculated by the distance calculation unit 303 for the same characteristic portion based on the image captured at the second time t2.

[0042] <Difference calculation part> The difference calculation unit 316 calculates the distance difference between the first target distance held in the first distance holding unit 313 and the second target distance held in the second distance holding unit 315. The difference calculation unit 316 calculates the distance difference Dc between the first target distance calculated for the characteristic portion W at time t1 held in the first distance holding unit 313 and the second target distance calculated for the characteristic portion W at time t2 held in the second distance holding unit 315.

[0043] <Third Block> Next, the third block will be described. In Fig. 6, 322 is a comparison section, 323 is a correction value calculation section, 324 is a determination section, and 325 is an update gate.

[0044] <Comparison section> The comparison unit 322 compares the distance traveled by the vehicle between the first time and the second time, acquired by the vehicle information acquisition unit 311, with the difference in distance calculated by the difference calculation unit 316. The travel distance (vehicle travel distance) Dm acquired by the vehicle information acquisition unit 311 of the second block and the difference in distance Dc between the target distance calculated by the difference calculation unit 316 of the second block are sent to the comparison unit 322. The comparison unit 322 calculates |(Dm-Dc) / Dm|. Here, |(Dm-Dc) / Dm| is assumed to be K.

[0045] <Judgment part> The determination unit 324 determines whether to update the correction value based on the moving distance Dm, the difference in distance Dc, and a predetermined threshold. The determination unit 324 updates the correction value (distance correction coefficient) stored in the correction value storage unit 307 of the first block when K is equal to or greater than the threshold of 3%, for example, and sets the update gate 325 so as not to update the correction value when K is less than 3%.

[0046] <Correction value calculation section> The correction value calculation unit 323 calculates a new correction value based on the movement distance Dm, the difference in distance Dc, and the correction value. The correction value calculation unit 323 calculates the correction value so that K becomes zero.

[0047] When the determination unit 324 determines that the correction value is to be updated, the correction value information corresponding to the region to which the characteristic portion belonged and held by the correction value holding unit 307 is updated to the new correction value information calculated by the correction value calculation unit 323. When the determination unit 324 determines that the correction value is not to be updated, the correction value information corresponding to the region to which the characteristic portion belonged and held by the correction value holding unit 307 is not updated. When K is, for example, 3% or more, the determination unit 324 opens the gate of the update gate 325 and sends the new correction value calculated by the correction value calculation unit 323 to the correction value holding unit 307. When K is less than 3%, the update gate 325 does not open, and the correction value held by the correction value holding unit 307 is maintained (the correction value is not updated). In the above method, the updates may accumulate and a correction value significantly different from the initial value may be given. Therefore, when the correction value has changed by 10% or more from the initial value, a warning may be displayed on a monitor of the vehicle to notify the user of the vehicle.

[0048] <Operation of the vehicle-mounted imaging device> FIG. 7 is a flow diagram showing an example of processing of the car-mounted imaging device 100A according to the second embodiment.

[0049] The extraction unit 302 performs image processing based on the image data, recognizes the measurement target using a known method such as pattern recognition, and extracts characteristic portions (S201).

[0050] The distance calculation unit 303 causes the first distance storage unit 313 to store the first target distance calculated by the distance calculation unit 303 from the captured image at time t1 (S202).

[0051] The vehicle information acquisition unit 311 acquires the time t2 that has been stored in advance (S203).

[0052] The vehicle information acquisition unit 311 judges whether the current time has reached t2, and if not, repeats the judgment (S204).

[0053] When time t2 is reached, the vehicle information acquisition unit 311 acquires the vehicle's traveling distance from time t1 to time t2, opens the elapsed time gate 314, and causes the second target distance calculated by the distance calculation unit 303 from the captured image at time t2 to be stored in the second distance storage unit 315 (S205).

[0054] The difference calculation unit 316 calculates the difference Dc between the first target distance and the second target distance (S206).

[0055] The determination unit 324 determines whether |(Dm-Dc) / Dm| (=K) calculated by the comparison unit 322 is equal to or greater than a threshold value (S207).

[0056] If K is equal to or greater than the threshold value, the determination unit 324 updates the correction value (S208).

[0057] The processes performed by the vehicle information acquisition unit 311, the comparison unit 322, the difference calculation unit 316, the determination unit 324, and the like are performed by a microcomputer or dedicated LSI in the camera (not shown), a CPU on a PC, or the like.

[0058] The above calibration process, i.e., the determination of whether or not to update the correction value by the determination unit 324, may be set to be performed immediately after the vehicle starts traveling, each time the target distance is calculated, or at least one timing at any arbitrary time interval.

[0059] In addition, after the correction value has been updated, when calculating the target distance to the measurement target in the updated image height area, the above-mentioned calibration process may be performed immediately beforehand to confirm that the updated correction value is correct.

[0060] Also, two thresholds may be provided, and if the higher threshold is exceeded, the above calibration operation may be immediately performed again with another target to confirm authenticity.

[0061] Furthermore, if the higher of the two thresholds is exceeded consecutively, the abnormal value may be stored in a log, an alert signal may be generated, and a lamp or the like may be used to notify the user.

[0062] It is preferable that the calibration process by the vehicle-mounted image capture device 100A according to the second embodiment is performed on a substantially straight road that continues for a predetermined length or more.

[0063] [Embodiment 3] In the above-described second embodiment, the image height regions to which the feature portions belong at time t1 and time t2 may not be the same. As a result, the measurement accuracy may not be high. In the third embodiment, a form will be described in which the distance acquisition unit 312 causes a specific feature portion to remain within the same specific image height region at both time t1 and time t2 so that the difference between the two target distances is less susceptible to the influence of image position shifts. In the third embodiment, the same components as those in the second embodiment are denoted by the same reference numerals, and detailed descriptions thereof will be omitted.

[0064] 8 is a block diagram of an in-vehicle image capture device 100B according to embodiment 3. The difference from the block diagram of embodiment 2 is that a distance acquisition unit 312 is added, and it is related to a region identification unit 304, a region information storage unit 305, an elapsed time gate 314, and a comparison unit 322.

[0065] <Distance acquisition section> The distance acquisition unit 312 acquires the distance and time between two points where a characteristic part of a target moving on an image remains within the same specific image height region. The distance acquisition unit 312 calculates the two-point distance and the two-point time for each image height region by referring to a calculation table using vehicle speed information and the image height region where the target exists. The distance acquisition unit 312 in this embodiment corresponds to the two-point time calculation unit in the claims.

[0066] The distance acquisition unit 312 makes the specific feature part belong to the same specific image height region at both time t1 and time t2. Specifically, the distance acquisition unit 312 stores in advance the longest travel distance of the vehicle where the specific feature part belongs to the same specific image height region as a table in which the longest travel distance is associated with the position of the feature part and the position of the image height region in the image. The term "longest" means that the longest travel distance is determined so that the specific feature part belongs to the same specific image height region even when various measurement error factors, individual variations in the device, etc. are taken into account. Here, the image height region where the feature part belongs at the first time t1 specified by the region specification unit 304 is zx, and the position where the vehicle is traveling at the first time t1 is Lx. The distance acquisition unit 312 acquires Dx (target value Dm), which is the longest travel distance of the vehicle from the position Lx where the feature part belongs to the image height region zx, based on the table and the position of the feature part and the position of the image height region in the image.

[0067] The distance acquisition unit 312 sequentially obtains the travel distance of the vehicle from Lx based on the vehicle speed information acquired from the vehicle information acquisition unit 311 .

[0068] Distance acquisition unit 312 determines the time when the vehicle's travel distance from Lx reaches Dx (target value Dm) as the second time, and causes second distance storage unit 315 to store the second target distance calculated by distance calculation unit 303 for the same characteristic part based on the image captured at the second time. Distance acquisition unit 312 determines the time when the travel distance reaches target value Dm as the second time t2, opens elapsed time gate 314, and causes second distance storage unit 315 to store the second target distance calculated by distance calculation unit 303 for the same characteristic part based on the image captured at the second time t2.

[0069] Fig. 9 is a diagram showing that the characteristic portion W belongs to the same image height region z2 at time t1 and time t2. Fig. 9(a) is a diagram in which the captured image and image height region at time t1 are superimposed. Fig. 9(b) is a diagram in which the captured image and image height region at time t2 are superimposed.

[0070] <Operation of the vehicle-mounted imaging device> 10 is a flow diagram showing an example of the process of the car-mounted imaging device 100B according to the embodiment 3. The differences from the embodiment 2 are two steps, S303 and S304, and a description of the overlapping steps will be omitted.

[0071] The distance acquisition unit 312 holds in advance the longest travel distance of the vehicle, where a specific feature portion belongs to a specific image height region, as a table in which the longest travel distance is associated with the position of the feature portion and the position of the image height region in the image. Here, the image height region to which the feature portion belongs at the first time specified by the region specification unit 304 is zx, and the position at which the vehicle is traveling at the first time is Lx. The distance acquisition unit 312 acquires Dx (target value Dm), which is the longest travel distance of the vehicle from Lx, where the feature portion belongs to zx, based on the table and the position of the feature portion and the position of the image height region in the image (S303).

[0072] The distance acquisition unit 312 sequentially obtains the travel distance of the vehicle from Lx based on the information acquired from the vehicle information acquisition unit 311. The distance acquisition unit 312 determines whether the travel distance has reached the target value Dm (S304).

[0073] When the distance acquisition unit 312 determines that the traveling distance has reached the target value Dm, it opens the elapsed time gate 314, determining that the point at which the traveling distance has reached the target value Dm is the second time, and causes the second distance storage unit 315 to store the second target distance calculated by the distance calculation unit 303 for the same characteristic part based on the image captured at the second time (S205).

[0074] The processing performed by the distance acquisition unit 312 and the like is performed by a microcomputer or dedicated LSI in the camera (not shown), a CPU on a PC, or the like.

[0075] The above calibration process, i.e., the determination of whether or not to update the correction value by the determination unit 324, may be set to be performed immediately after the vehicle starts traveling, each time the target distance is calculated, or at least one timing at any arbitrary time interval.

[0076] In addition, after the correction value has been updated, when calculating the target distance to the measurement target in the updated image height area, the above-mentioned calibration process may be performed immediately beforehand to confirm that the updated correction value is correct.

[0077] Also, two thresholds may be provided, and if the higher threshold is exceeded, the above calibration operation may be immediately performed again with another target to confirm authenticity.

[0078] Furthermore, if the higher of the two thresholds is exceeded consecutively, the abnormal value may be stored in a log, an alert signal may be generated, and a lamp or the like may be used to notify the user.

[0079] It is preferable that the calibration process by the vehicle-mounted image capture device 100B according to the third embodiment is performed on a substantially straight road that continues for a predetermined length or more.

[0080] In this way, since each target distance is calculated from images taken at two times when both feature portions are in the same image height region, the measurement accuracy of the target distance can be improved.

[0081] In addition, since the longest travel distance in which the characteristic portion belongs to a specific image height region can be obtained, the measurement accuracy of the target distance can be improved.

[0082] [Embodiment 4] In the third embodiment, the accuracy of calibration is improved by making the interval between two driving positions where the same feature part exists in a specific image height region as long as possible when the vehicle is traveling. However, calibration is not possible unless the same feature part exists in a specific image height region. Furthermore, if the interval between two driving positions where the same feature part exists in a specific image height region cannot be made long, the accuracy of calibration decreases. Therefore, in the fourth embodiment, the range for cutting out the effective pixel region from the captured image is changed, and the optical axis corresponding point O of the lens is moved from the range arranged at the center of the captured image to the range arranged below the center of the captured image.

[0083] FIG. 11(a) is a diagram in which the effective pixel areas and image height areas of two images captured at different times are superimposed on each other when point O is placed at the center of the captured images. FIG. 11(b) is a diagram showing an overlap of the effective pixel areas and image height areas of two images captured at different times when the range for cutting out the effective pixel area from the captured image is changed and the optical axis corresponding point O of the lens is positioned below the center of the captured image.

[0084] In this way, by changing the range in which the effective pixel area is cut out from the captured image and relatively moving the position of the optical axis corresponding point O of the lens, it is possible to increase the vehicle travel distance over which the characteristic part of the object to be measured exists in the image height area z2, thereby improving the measurement accuracy.

[0085] In addition, when calibrating a correction value (distance correction coefficient) for an image height region where the image height is large, highly accurate calibration can be performed even when there are no suitable features or when the vehicle travels a short distance in the image height region.

[0086] Although the embodiments of the present invention have been described above in detail, the present invention is not limited to the above-described embodiments, and various design changes can be made without departing from the spirit of the present invention described in the claims. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those having all of the configurations described. In addition, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. Furthermore, it is possible to add, delete, or replace a part of the configuration of each embodiment with another configuration. [Explanation of symbols]

[0087] 15: effective pixel area, 100, 100A, 100B: vehicle-mounted imaging device, 301: imaging unit, 302: extraction unit, 303: distance calculation unit, 304: area identification unit, 305: area information storage unit, 306: correction value acquisition unit, 307: correction value storage unit, 311: vehicle information acquisition unit, 312: distance acquisition unit (point-to-point time calculation unit), 322: comparison unit, 324: judgment unit.

Claims

1. an imaging unit mounted on the vehicle and configured to capture images using one or more cameras including a lens and an imaging element; an extraction unit that extracts a characteristic portion of the measurement object included in the image; a distance calculation unit that calculates a target distance from the imaging unit to the characteristic portion based on the image; a region information storage unit that stores information on the positions of a plurality of image height regions obtained by dividing a light receiving surface of the image sensor into a plurality of different image height positions from a position corresponding to an optical axis of the lens based on a distortion characteristic of the lens; a correction value storage unit that stores information about a correction value for calculating the target distance, the correction value being determined for each of the plurality of image height regions; a region specifying unit that specifies to which of the plurality of image height regions the characteristic portion belongs; a correction value acquisition unit that acquires information about the correction value from the correction value storage unit based on information about the image height region to which the characteristic portion belongs; having The distance calculation unit corrects the target distance using the correction value acquired by the correction value acquisition unit, a first distance storage unit that stores a first target distance calculated by the distance calculation unit for the characteristic portion based on the image captured at a first time; a second distance storage unit that stores a second target distance calculated by the distance calculation unit for the same feature portion based on the image captured at a second time that is later than the first time; and a difference calculation unit that calculates a difference in distance between the first target distance held in the first distance holding unit and the second target distance held in the second distance holding unit; A vehicle information acquisition unit that acquires information regarding a mileage of the vehicle; a comparison unit that compares a travel distance of the vehicle from the first time to the second time acquired by the vehicle information acquisition unit with a difference in distance calculated by the difference calculation unit; a determination unit that determines whether or not to update the correction value based on the movement distance, the difference between the distances, and a predetermined threshold value; having a correction value calculation unit that calculates a new correction value based on a difference between the moving distance and the distance; when determining that the correction value should be updated, the determination unit updates information of the correction value corresponding to the image height region to which the characteristic portion belonged, which is stored in the correction value storage unit, to information of the new correction value calculated by the correction value calculation unit; When the determination unit determines not to update the correction value, the determination unit does not update the information of the correction value corresponding to the image height region to which the characteristic portion belonged, which is stored in the correction value storage unit, a two-point time calculation unit that calculates a time between two points during which the characteristic portion remains within the same specific image height region by using vehicle speed information of the vehicle and information about the image height region to which the characteristic portion belongs, The vehicle-mounted imaging device is characterized in that the second distance storage unit stores the second target distance calculated by the distance calculation unit, with the second time being the time that the time between the two points has elapsed from the first time.

2. The two-point time calculation unit storing in advance, as a table, the longest travel distance of the vehicle for which a specific feature portion belongs to a specific image height region, in association with a position of the feature portion and a position of the image height region in the image; Let zx be the image height region to which the characteristic portion belongs at the first time, as identified by the region identification unit, and let Lx be the position of the vehicle at which the vehicle is traveling at the first time, obtaining a longest travel distance Dx of the vehicle from Lx where the characteristic portion belongs to zx, based on the table, the position of the characteristic portion in the image, and the position of the image height region; Sequentially calculating a travel distance of the vehicle from Lx based on information acquired from the vehicle information acquisition unit; A time point when the travel distance of the vehicle from the Lx reaches the Dx is defined as the second time, and the second target distance calculated by the distance calculation unit for the same characteristic portion based on the image captured at the second time is stored in the second distance storage unit. The vehicle-mounted imaging device according to claim 1 .

Images