Image processing device

Abstract

This image processing device comprises a computation device, into which is input an image in which the area in front of a vehicle is captured by an imaging device. The computation device: calculates, for each of a plurality of regions included in the image, the parallax of a subject image included in the region; and determines whether a tunnel exit is present in a particular region among the plurality of regions, on the basis of the position of a region, among the plurality of regions, for which the parallax of the subject image could not be calculated, and on the basis of the parallax calculated for another region in the vicinity of the region for which the parallax of the subject image could not be calculated.

Description

[Technical Field]

[0001] The present invention relates to an image processing device. [Background technology]

[0002] Conventionally, there is known a technique for predicting abrupt changes in brightness so as not to lose sight of a preceding vehicle even in an environment where brightness changes suddenly, such as at the exit of a tunnel. Patent Document 1 describes a technique for setting a monitoring area around an object detected as a preceding vehicle on an image, and predicting that the preceding vehicle will enter a bright area (or a dark area) using the number of pixels in the monitoring area that have a brightness equal to or greater than a threshold (or that have a brightness less than the threshold). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2008-301342 Summary of the Invention [Problem to be solved by the invention]

[0004] The technology described in Patent Document 1 has a problem in that it must first detect the preceding vehicle, since it detects a bright area such as the exit of a tunnel based on the position of the preceding vehicle. [Means for solving the problem]

[0005] An image processing device according to one aspect of the present invention includes a calculation device to which an image captured by an imaging device in front of a vehicle is input, and the calculation device calculates the parallax of the subject image contained in each of a plurality of regions contained in the image, and determines whether a tunnel exit is present in a particular region among the plurality of regions based on the position of a region among the plurality of regions for which the parallax of the subject image could not be calculated and the parallax calculated in other regions located near the region for which the parallax of the subject image could not be calculated. [Effects of the Invention]

[0006] According to the present invention, it is possible to provide an image processing device that can detect the entrance or exit of a tunnel without waiting for the detection of a preceding vehicle. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a block diagram showing a schematic configuration of a vehicle equipped with an image processing device according to the first embodiment. [Figure 2] FIG. 2 is a block diagram schematically showing the hardware configuration of the image processing apparatus according to the first embodiment. [Figure 3] FIG. 3 is a functional block diagram of the image processing apparatus according to the first embodiment. [Figure 4] FIG. 4 is a diagram schematically illustrating a left image, a right image, and a parallax image. [Figure 5] FIG. 5 is a flowchart of the calculation process performed by the processing device. [Figure 6] FIG. 6 is a functional block diagram similar to FIG. 3, showing an image processing apparatus according to the second embodiment. [Figure 7] FIG. 7 is a diagram schematically illustrating a left image, a right image, and a parallax image. [Figure 8] FIG. 8 is a diagram similar to FIG. 5, and is a flowchart showing an example of the arithmetic processing executed by the processing device according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] (First embodiment) An image processing device according to an embodiment of the present invention will be described with reference to FIGS.

[0009] 1 is a block diagram showing a schematic configuration of a vehicle equipped with an image processing device according to the first embodiment. The vehicle 1 is provided with a left camera 2L, a right camera 2R, and an image processing device 3. In the following description, the left camera 2L and the right camera 2R are collectively referred to as cameras 2.

[0010] The camera 2 is mounted near the rearview mirror inside the vehicle 1. The camera 2 captures images of the area in front of the vehicle 1. The left camera 2L and the right camera 2R are installed so that their optical axes are parallel and at the same height. The image captured by the left camera 2L is referred to as the left image. The left camera 2L periodically captures the left image and inputs it to the image processing device 3. The image captured by the right camera 2R is referred to as the right image. The right camera 2R periodically captures the right image and inputs it to the image processing device 3. In this embodiment, the left camera 2L and the right camera 2R simultaneously capture the left image and the right image. In other words, the left image and the right image are images of the area in front of the vehicle 1 captured at the same time. Note that the optical axes of the left camera 2L and the right camera 2R do not need to be parallel. In this case, it is desirable to collimate the left image and the right image in the camera 2 or the image processing device 3.

[0011] FIG. 2 is a block diagram schematically illustrating the hardware configuration of an image processing device 3 according to the first embodiment. The image processing device 3 is configured as a computer including a processing device 31 such as a central processing unit (CPU), a micro processing unit (MPU), or a digital signal processor (DSP), a non-volatile memory 32 such as a read-only memory (ROM), a flash memory, or a hard disk drive, a volatile memory 33 known as a random access memory (RAM), an input interface 34, an output interface 35, and other peripheral circuits (not shown). The processing device 31 may also be an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or the like. The non-volatile memory 32 and the volatile memory 33 function as storage devices for storing information (data).

[0012] The nonvolatile memory 32 stores programs capable of executing various calculations. In other words, the nonvolatile memory 32 is a storage device (storage medium) from which the programs for realizing the functions of this embodiment can be read. The processing device 31 is an arithmetic device that loads the programs stored in the nonvolatile memory 32 into the volatile memory 33 and executes the programs, and performs predetermined arithmetic processing on data taken in from the input interface 34, the nonvolatile memory 32 which is a storage device, and the volatile memory 33 in accordance with the programs. The functions realized by the programs will be described later.

[0013] The input interface 34 converts the left and right images captured by the camera 2 so that they can be calculated by the processing device 31. The output interface 35 generates an exposure control signal according to the calculation results of the processing device 31 and outputs it to the camera 2.

[0014] The image processing device 3 may be configured with a single computer or multiple computers. When the image processing device 3 is configured with multiple computers, the functions of the image processing device 3 are realized by multiple processing devices 31.

[0015] 3 is a functional block diagram of the image processing device 3. The image processing device 3 functions as a parallax calculation unit 301, an exit determination unit 302, a distance calculation unit 303, a preceding vehicle detection unit 304, an inter-vehicle distance calculation unit 305, a relative vehicle speed calculation unit 306, and an exposure control unit 307 by the processing device 31 executing a program stored in the non-volatile memory 32.

[0016] The disparity calculation unit 301 generates a disparity image using left and right images captured simultaneously by the left camera 2L and right camera 2R. The exit determination unit 302 uses the disparity image calculated by the disparity calculation unit 301 to determine whether there is a tunnel entrance or exit on the path of vehicle 1. The distance calculation unit 303 calculates the distance from vehicle 1 to the tunnel entrance or exit. The preceding vehicle detection unit 304 detects other vehicles located in front of vehicle 1, i.e., preceding vehicles. The inter-vehicle distance calculation unit 305 calculates the distance from vehicle 1 to the preceding vehicle, i.e., the inter-vehicle distance. The relative vehicle speed calculation unit 306 calculates the relative vehicle speed between vehicle 1 and the preceding vehicle. For example, when the vehicle speed of vehicle 1 and the preceding vehicle are equal, the relative vehicle speed calculated by the relative vehicle speed calculation unit 306 is 0 kilometers per hour. Furthermore, when the speed of vehicle 1 is 30 kilometers per hour and the speed of the preceding vehicle is 50 kilometers per hour, the relative vehicle speed calculated by relative vehicle speed calculation unit 306 is 20 kilometers per hour. Exposure control unit 307 controls the exposure of camera 2. In other words, exposure control unit 307 adjusts the amount of exposure of camera 2 so that the exposure of camera 2 is appropriate.

[0017] (Method of generating parallax images by the parallax calculation unit 301) FIG. 4 is a diagram schematically illustrating a left image, a right image, and a parallax image. The left image 4L and the right image 4R illustrated in FIG. 4 are images captured inside a tunnel. A tunnel exit 6 and a preceding vehicle 7 are captured in the left image 4L and the right image 4R. Here, it is assumed that the tunnel exit 6 is much brighter than the inside of the tunnel, and therefore the tunnel exit 6 is overexposed in the left image 4L and the right image 4R. A method for generating a parallax image 5 from the left image 4L and the right image 4R using the left image 4L, the right image 4R, and the parallax image 5 illustrated in FIG. 4 will be described below.

[0018] The parallax calculation unit 301 divides the left image 4L into a plurality of pixel blocks in a grid pattern. Each pixel block is, for example, 16 pixels vertically and 16 pixels horizontally. In other words, each pixel block contains a total of 256 pixels. Note that the size of the pixel blocks may be different from this. Furthermore, the shape of the pixel blocks may be rectangular rather than square. The parallax calculation unit 301 divides the right image 4R into a plurality of pixel blocks in a grid pattern, similar to the left image 4L.

[0019] In the following explanation, pixel blocks in the left image 4L will be denoted as BL(x,y). Here, x is an index value representing the horizontal position of the pixel block. x is 1 for the leftmost pixel block and increases by one as you move to the right. Similarly, y is an index value representing the vertical position of the pixel block. y is 1 for the topmost pixel block and increases by one as you move down. As with the left image 4L, pixel blocks in the right image 4R will be denoted as BR(x,y). x and y have the same meanings as for pixel block BL(x,y) in the left image 4L.

[0020] The parallax calculation unit 301 calculates the parallax of the subject image contained in a row of pixel blocks BL(x,y) by comparing multiple pixel blocks BL(x,y) arranged horizontally in a row in the left image 4L with multiple pixel blocks BR(x,y) arranged in a row at the same position as those multiple pixel blocks BL(x,y) in the right image 4R.

[0021] First, focus on pixel block BL(1,1) at the top left corner of left image 4L. For pixel blocks BR(1,1), BR(2,1), BR(3,1), ... in right image 4R, i.e., for pixel blocks aligned horizontally and at the same vertical position (y=1) as the pixel block BL(1,1), the degree of similarity between the pixel block BL(1,1) and the pixel block is calculated. Here, the degree of similarity is a numerical value that quantifies the degree to which the luminance values ​​of multiple pixels in one pixel block resemble those of multiple pixels in the other pixel block. The closer the luminance values, the greater the degree of similarity. Specific methods for calculating the degree of similarity will be described later.

[0022] The disparity calculation unit 301 uses the calculated degree of matching to identify the pixel block BR(i,1) that most closely matches the pixel block BL(1,1) of interest. In this case, the disparity at position x=1, y=1 is |i-1|. In other words, the difference in horizontal position between the pixel block BR(i,1) that most closely matches the pixel block BL(1,1) of interest is the disparity of the subject image at the position of the pixel block BL(1,1) of interest.

[0023] The parallax calculation unit 301 repeats this process for all pixel blocks BL(1,1), BL(2,1), BL(3,1), . . . at position y=1 in the left image 4L, and calculates all parallaxes at position y=1.

[0024] The parallax calculation unit 301 repeatedly performs the above process while gradually changing the position in the vertical direction of the left image 4L, thereby calculating all parallaxes for the object images at positions y=2, y=3, .... That is, for the positions of all pixel blocks BL(x, y) included in the left image 4L, the parallax corresponding to the object images at those positions is calculated. The parallax image is data in which the parallaxes calculated in this way are arranged two-dimensionally according to the positions at which the parallaxes are calculated.

[0025] The parallax image 5 illustrated in FIG. 4 has five regions, A, B, C, D, and E, in order of decreasing parallax. That is, region A has the smallest parallax, and region E has the largest parallax. In addition to regions A to E, there is also region X, for which parallax could not be calculated. Note that the parallax image 5 illustrated in FIG. 4 is shown schematically for ease of explanation. In reality, parallax is calculated in more detail than five stages. Therefore, there are many more regions with different parallaxes than those illustrated in FIG. 4, with more detailed stages.

[0026] When the exposure of the camera 2 is controlled for the dark environment inside the tunnel, blown-out highlights occur in the captured image in the area where the tunnel exit 6 is located. In other words, the brightness values ​​of the pixels corresponding to the tunnel exit 6 become saturated. For example, if the brightness value is expressed as an integer value between 0 and 255, the brightness value of the pixel corresponding to the tunnel exit 6 becomes the maximum value of 255. When the brightness values ​​of all the pixels included in one pixel block become saturated in this way, it becomes impossible to calculate a disparity value from that pixel block. In other words, it becomes impossible to calculate a disparity value for the pixel block corresponding to the tunnel exit 6.

[0027] When the brightness values ​​of all pixels in a pixel block in the left image 4L are maximum, the parallax calculation unit 301 determines that parallax cannot be calculated in the corresponding region X of the parallax image 5. When the parallax calculation unit 301 determines that parallax cannot be calculated at a certain position, it uses a special value indicating that parallax calculation was not possible (e.g., a negative value) as the parallax value for that position. In the following description, this "special value indicating that parallax calculation was not possible" is referred to as an invalid parallax value. Furthermore, the region X where parallax calculation was not possible and an invalid parallax value was set is referred to as an invalid parallax region. In contrast, the regions A to E that are not invalid parallax regions are referred to as valid parallax regions.

[0028] For example, when capturing an image of a very dark area, such as a section inside a tunnel where no illumination light reaches, black crush may occur instead of white blowout. For example, when brightness values ​​are expressed as integers between 0 and 255, the brightness value of a pixel corresponding to such a very dark area is the minimum value, 0. Even if all pixels in a pixel block have the minimum brightness value, the disparity value cannot be calculated from that pixel block. In a pixel block in the left image 4L, even if all pixels in that pixel block have the minimum brightness value, the disparity calculation unit 301 determines that disparity cannot be calculated for the corresponding area in the disparity image 5.

[0029] As can be seen from the parallax image 5 illustrated in FIG. 4, a relatively large disparity value is obtained in an area E close to the vehicle (host vehicle) 1, while a relatively small disparity value is obtained in an area A far from the vehicle 1. In this way, the disparity gradually decreases from the vicinity of the vehicle 1 to the distance away. Therefore, the disparity value of the road surface portion in the parallax image 5 gradually decreases from the bottom to the top in the vertical direction. On the other hand, if there is a three-dimensional object such as a vehicle, the same disparity value is obtained along the vertical direction within the range where the three-dimensional object exists in the parallax image 5. Therefore, by finding an area in the parallax image 5 where the disparity value is the same along the vertical direction, it can be determined that a three-dimensional object exists at the position of that area. In other words, by finding an area in the parallax image 5 where the disparity value is the same along the vertical direction, a three-dimensional object can be detected.

[0030] Using the following equation (1), the distance from the vehicle 1 to an object present in any area can be calculated from the parallax value of that area. d=(L+f) / p …(1) Here, d is the distance to the target object, L is the baseline length of the left camera 2L and the right camera 2R, f is the focal length of the left camera 2L and the right camera 2R, and p is the parallax value.

[0031] A method for calculating the degree of match will now be described. In this embodiment, the degree of match is the sum of the absolute values ​​of the luminance differences (SAD) between each pixel in a pixel block. For example, if one pixel block is 16 pixels vertically and 16 pixels horizontally, one pixel block contains 256 pixels. When one pixel block in the left image 4L is juxtaposed with one pixel block in the right image 4R, there are 256 pairs of left and right pixels. For each of these 256 pairs, the absolute value of the difference in luminance value is calculated to obtain the absolute values ​​of the 256 differences in luminance value. The sum of these 256 absolute values ​​is the degree of match in this embodiment.

[0032] The method for calculating the degree of coincidence is not limited to the method using the SAD described above, and other methods may also be used. Also, depending on the value used as the degree of coincidence, it may be determined that the disparity cannot be calculated for reasons other than those caused by the above-mentioned blown-out highlights and blocked-up shadows. Even in such cases, the region may be treated as an invalid disparity region, as in the above-mentioned method.

[0033] (Explanation of tunnel exit determination method by exit determination unit 302) A method for determining a tunnel exit by the exit determination unit 302 will be described using the left image 4L, the right image 4R, and the parallax image 5 illustrated in FIG.

[0034] The exit determination unit 302 first checks whether an invalid parallax area exists near the optical axis position of the camera 2 (the vanishing point, in this case, the center of the parallax image 5). For example, it focuses on an area corresponding to an 11×11 pixel block centered on the center point of the parallax image 5 and checks whether an invalid parallax area exists within it. Note that the search range for invalid parallax areas, "11×11 pixel blocks centered on the center point of the parallax image 5," is just an example, and the presence or absence of an invalid parallax area may be checked within a different range. For example, by setting this search range to a horizontally long rectangle, it becomes possible to more accurately detect a tunnel exit when one is located at the end of a curve. Conversely, by shortening the search range horizontally, it is possible to reduce the amount of calculations.

[0035] If an invalid parallax area is found, the exit determination unit 302 calculates the total area of ​​the invalid parallax area and all invalid parallax areas that exist consecutively with that invalid parallax area. Here, "exist consecutively" means that the invalid parallax areas are adjacent to each other. If the calculated total area is equal to or greater than a predetermined threshold, the exit determination unit 302 checks the proportion of saturated pixels among the pixels included in the consecutive invalid parallax areas. Here, saturated pixels are pixels whose brightness values ​​are equal to or greater than a predetermined threshold. For example, if the number of pixels included in the consecutive invalid parallax areas is 1,000, and 400 of those pixels have brightness values ​​equal to or greater than the predetermined threshold, the calculated proportion is 400 / 1,000 = 0.4.

[0036] The exit determination unit 302 checks whether the calculated ratio is equal to or greater than a predetermined threshold (e.g., 0.8). If the calculated ratio is equal to or greater than the predetermined threshold, the exit determination unit 302 further calculates the center of gravity when these consecutive invalid parallax regions are joined together. Then, the exit determination unit 302 searches for a valid parallax region, i.e., a region for which parallax can be calculated, moving leftward from the center of gravity. The exit determination unit 302 cycles through valid parallax regions that are aligned leftward from the first found valid parallax region and checks whether the parallax values ​​corresponding to these regions gradually increase. That is, it checks whether the parallax values ​​gradually increase from the center of gravity to the leftward of the image. For example, in the parallax image 5 illustrated in FIG. 4, regions A, B, C, D, and E exist in this order from region X to the leftward of the image. That is, the parallax values ​​gradually increase from region X to the leftward of the image. If the parallax values ​​gradually increase, the exit determination unit 302 performs the same process for the rightward direction. If the disparity value gradually increases not only to the left side of the image but also to the right side of the image, the exit determination unit 302 determines that those invalid parallax areas are tunnel exits, i.e., that the tunnel exit exists at the position of those invalid parallax areas.

[0037] On both sides of the tunnel exit, there should be outer walls of the tunnel along the road leading from the tunnel exit to vehicle 1. Because the parallax value increases as the distance of an object from vehicle 1 increases, the parallax values ​​calculated in the regions corresponding to the left and right tunnel outer walls should gradually increase from the tunnel exit toward vehicle 1. Based on this concept, exit determination unit 302 executes the above-described process and determines the presence of a tunnel exit. That is, based on the position of the region where the parallax of the subject image could not be calculated and the parallax calculated in other regions located near the region where the parallax of the subject image could not be calculated, exit determination unit 302 determines whether a tunnel exit exists in the region where the parallax of the subject image could not be calculated.

[0038] (Explanation of how the distance calculation unit 303 calculates the distance from the vehicle 1 to the tunnel exit) The distance calculation unit 303 calculates the distance from the vehicle 1 to the tunnel exit by substituting the disparity value in the vicinity of the invalid parallax area into the above-mentioned formula (1). For example, the distance calculation unit 303 calculates the distance to the tunnel exit by substituting the disparity value calculated at a position adjacent to either the left or right of the invalid parallax area corresponding to the tunnel exit into the above-mentioned formula (1). Alternatively, the distance to the tunnel exit may be calculated by substituting the average value of two disparity values ​​calculated at positions adjacent to the left and right of the invalid parallax area into the above-mentioned formula (1).

[0039] (Explanation of a method for detecting a preceding vehicle by the preceding vehicle detection unit 304) The preceding vehicle detection unit 304 checks whether the same (or sufficiently close) disparity values ​​exist continuously along the vertical direction for a predetermined length or more near the center of the disparity image. If there is nothing but a road surface at that location, the disparity value should gradually change depending on the distance to the road surface. On the other hand, if the same (or sufficiently close) disparity values ​​exist continuously along the vertical direction for a predetermined length or more near the center of the disparity image, it is highly likely that some kind of object exists there. If the same (or sufficiently close) disparity values ​​exist continuously along the vertical direction for a predetermined length or more near the center of the disparity image, the preceding vehicle detection unit 304 determines that a preceding vehicle exists there.

[0040] (Explanation of arithmetic processing by the processing device 31) 5 is a flowchart of the calculation process by the processing device 31. In step S100, a left image captured by the left camera 2L and a right image simultaneously captured by the right camera 2R are input to the processing device 31 via the input interface 34. In step S110, the parallax calculation unit 301 generates a parallax image using the left image and right image input in step S100.

[0041] In step S120, the preceding vehicle detection unit 304 determines whether or not a preceding vehicle exists ahead of the vehicle 1. The preceding vehicle detection unit 304 checks whether the same (or sufficiently close) disparity values ​​exist consecutively along the vertical direction for a predetermined length or more near the center of the disparity image. If the same (or sufficiently close) disparity values ​​exist consecutively along the vertical direction for a predetermined length or more near the center of the disparity image, the preceding vehicle detection unit 304 determines that a preceding vehicle exists there, and the process proceeds to step S130. On the other hand, if the same (or sufficiently close) disparity values ​​do not exist consecutively along the vertical direction for a predetermined length or more near the center of the disparity image, the preceding vehicle detection unit 304 does not detect a preceding vehicle, and the process proceeds to step S150.

[0042] In step S130, inter-vehicle distance calculation unit 305 calculates the distance between vehicle (host vehicle) 1 and the preceding vehicle (another vehicle) detected in step S120, i.e., the inter-vehicle distance from the preceding vehicle. Inter-vehicle distance calculation unit 305 calculates the distance to the preceding vehicle by substituting the parallax value at the position where the preceding vehicle is detected into the above-mentioned equation (1). In step S140, relative vehicle speed calculation unit 306 calculates the relative vehicle speed between vehicle 1 and the preceding vehicle detected in step S120. Relative vehicle speed calculation unit 306 calculates the relative vehicle speed between vehicle 1 and the preceding vehicle by dividing the difference between the inter-vehicle distance calculated in the previous step S130 and the inter-vehicle distance calculated in the current step S130 by the difference between the time when step S130 was executed last time and the time when step S130 was executed this time. Thereafter, the process proceeds to step S150.

[0043] In step S150, the exit determination unit 302 identifies an invalid parallax region located near the vanishing point (the center of the optical axis of the camera) from the parallax image calculated in step S110. In step S160, the exit determination unit 302 determines whether the area of ​​the identified invalid parallax region is equal to or greater than a predetermined threshold. If the area of ​​the identified invalid parallax region is less than the predetermined threshold, the exit determination unit 302 determines that the identified invalid parallax region is not a tunnel exit, and the process shown in Fig. 6 ends. If the area of ​​the identified invalid parallax region is equal to or greater than the predetermined threshold, the process proceeds to step S170.

[0044] In step S170, the exit determination unit 302 determines whether the disparity value gradually increases leftward or rightward from the center of gravity of the invalid parallax region. If the disparity value does not gradually increase leftward or rightward from the center of gravity of the invalid parallax region, the exit determination unit 302 determines that the identified invalid parallax region is not a tunnel exit, and the process shown in Fig. 6 ends. If the disparity value gradually increases leftward or rightward from the center of gravity of the invalid parallax region, the process proceeds to step S180. In step S180, the exit determination unit 302 calculates the proportion of pixels whose brightness values ​​are equal to or greater than a predetermined threshold among all pixels included in the invalid parallax region, and determines whether this proportion is equal to or greater than the predetermined threshold. If the calculated proportion is less than the predetermined threshold, the exit determination unit 302 determines that the identified invalid parallax region is not a tunnel exit, and the process shown in Fig. 6 ends. If the calculated proportion is equal to or greater than the predetermined threshold, the exit determination unit 302 determines that the invalid parallax region is a tunnel exit, and the process proceeds to step S190.

[0045] In step S190, the distance calculation unit 303 calculates the distance from the vehicle 1 to the tunnel exit. The distance to the tunnel exit is calculated by substituting the parallax value in the vicinity of the invalid parallax area into the above-mentioned formula (1). For example, the distance calculation unit 303 calculates the distance to the tunnel exit by substituting the parallax value calculated at a position adjacent to either the left or right of the invalid parallax area into the above-mentioned formula (1).

[0046] In step S200, exposure control unit 307 determines whether a leading vehicle was detected in step S120. If a leading vehicle was not detected, processing proceeds to step S230. On the other hand, if a leading vehicle was detected, processing proceeds to step S210. In step S210, exposure control unit 307 calculates the distance from the leading vehicle to the tunnel exit. Exposure control unit 307 calculates the distance from the leading vehicle to the tunnel exit by subtracting the inter-vehicle distance (the distance from vehicle 1 to the leading vehicle) calculated in step S130 from the distance from vehicle 1 to the tunnel exit calculated in step S190.

[0047] In step S220, the exposure control unit 307 adjusts the exposure of camera 2 according to the distance from the leading vehicle to the tunnel exit calculated in step S210 so that the exposure of camera 2 matches the lighting environment outside the tunnel when the leading vehicle reaches (passes) the tunnel exit. For example, the exposure control unit 307 adjusts the exposure so that the exposure is appropriate for the luminance values ​​of pixels in the invalid parallax area. Alternatively, the exposure control unit 307 may record the exposure before vehicle 1 enters the tunnel, and adjust the exposure of camera 2 so that it matches the recorded exposure when the leading vehicle passes the tunnel exit. Then, the processing shown in FIG. 6 ends.

[0048] A distance-exposure amount table corresponding to the distance from the preceding vehicle to the tunnel exit is stored in advance in non-volatile memory 32. In step S220, exposure control unit 307 converts the distance-exposure amount table into a time-exposure amount table using the speed of the preceding vehicle calculated from the relative speed of the preceding vehicle and the speed of vehicle 1, and determines a schedule for exposure amount control. Note that instead of using such a table, the schedule for exposure amount control may be determined using a mathematical formula that uses the distance from the preceding vehicle to the tunnel exit, the speed of the preceding vehicle, and the elapsed time.

[0049] In step S230, the exposure control unit 307 adjusts the exposure of camera 2 in accordance with the distance from vehicle 1 to the tunnel exit calculated in step S190 so that the exposure of camera 2 matches the lighting environment outside the tunnel just before vehicle 1 reaches (passes) the tunnel exit. For example, the exposure control unit 307 adjusts the exposure so that the exposure is appropriate for the luminance values ​​of pixels in the invalid parallax area. Alternatively, the exposure control unit 307 may record the exposure before vehicle 1 enters the tunnel, and adjust the exposure of camera 2 so that it matches the recorded exposure just before vehicle 1 reaches (passes) the tunnel exit. Then, the processing shown in FIG. 6 ends.

[0050] A distance-exposure amount table corresponding to the distance from vehicle 1 to the tunnel exit is stored in advance in non-volatile memory 32. In step S230, exposure control unit 307 converts the distance-exposure amount table into a time-exposure amount table using the speed of vehicle 1, and determines a schedule for exposure amount control. Note that instead of using such a table, the schedule for exposure amount control may be determined using a mathematical formula that uses the distance from vehicle 1 to the tunnel exit, the speed of vehicle 1, and the elapsed time.

[0051] Note that exposure control unit 307 may use a well-known technique to adjust the exposure amount. For example, the shutter speed of camera 2, i.e., the exposure time, may be adjusted, the aperture diameter of the diaphragm of camera 2 may be adjusted, or the sensitivity (amplification factor) of the image sensor of camera 2 may be adjusted.

[0052] According to the above-described first embodiment, the following effects are achieved.

[0053] (1) The processing device (arithmetic device) 31 calculates the parallax of the subject image included in each of a plurality of regions included in the image captured by the camera 2, and determines whether a tunnel exit exists in a specific region among the plurality of regions based on the position of the region among the plurality of regions for which the parallax of the subject image could not be calculated and the parallax calculated for other regions located near the region for which the parallax of the subject image could not be calculated. This makes it possible to detect the entrance or exit of a tunnel without waiting for the detection of a preceding vehicle. Furthermore, it is possible to detect the entrance or exit of a tunnel regardless of the presence of a preceding vehicle.

[0054] (2) When there is an area near the optical axis of the camera 2 (imaging device) where the parallax of the subject image could not be calculated, the processing device 31 determines that a tunnel exit is present in that area if the parallax calculated for multiple areas lined up to the left of that area and multiple areas lined up to the right of that area gradually increases with increasing distance from the area.In this way, the processing device 31 not only checks for the presence or absence of invalid parallax areas but also uses a parallax pattern that is characteristic of a tunnel exit, making it possible to detect the tunnel exit with high accuracy.

[0055] (3) The processing device 31 controls the exposure of the camera 2 (imaging device) while the vehicle 1 is traveling to the tunnel exit based on the distance from the vehicle 1 to the tunnel exit and the speed of the vehicle 1. Therefore, appropriate exposure can be maintained at all times from the time the tunnel exit is detected until the vehicle 1 reaches the tunnel exit.

[0056] (4) The processing device 31 detects a preceding vehicle traveling ahead of the vehicle 1 from the image, and controls the exposure of the camera 2 (image capture device) while the preceding vehicle travels to the tunnel exit based on the positional relationship between the preceding vehicle and the tunnel exit. Therefore, appropriate exposure can be maintained at all times from the time the tunnel exit is detected until the preceding vehicle reaches the tunnel exit, and therefore the preceding vehicle can be continuously detected without losing sight of it.

[0057] (Second embodiment) An image processing device according to a second embodiment of the present invention will be described with reference to Figures 6 to 8. Note that components that are the same as or equivalent to those described in the first embodiment are given the same reference symbols, and differences will be mainly described.

[0058] 6 is a functional block diagram of an image processing device 300 according to the second embodiment. The image processing device 300 includes an exit determination unit 1302 instead of the exit determination unit 302. The exit determination unit 1302 determines whether or not there is a tunnel exit on the path of the vehicle 1, using the disparity image calculated by the disparity calculation unit 301.

[0059] FIG. 7 is a diagram illustrating a left image 40L, a right image 40R, and a disparity image 50. The left image 40L and the right image 40R illustrated in FIG. 7 are images captured inside a tunnel. The tunnel exit 6 and the preceding vehicle 7 are captured in the left image 40L and the right image 40R. Assume that the exposure of the camera 2 is suppressed to a level that prevents overexposure at the tunnel exit 6. In this case, the disparity calculation unit 301 can correctly calculate the disparity at the tunnel exit 6. However, because the exposure of the camera 2 is suppressed, blackout occurs in dark areas inside the tunnel, such as areas where external light from the tunnel exit or illumination light inside the tunnel is difficult to reach. In this case, the disparity calculation unit 301 cannot calculate the disparity in such dark areas. In other words, in the disparity image 50 generated by the disparity calculation unit 301, such dark areas become invalid disparity areas.

[0060] The parallax image 50 illustrated in FIG. 7 has five regions, F, G, H, I, and J, in order of decreasing parallax. That is, region F is the region with the smallest parallax, and region J is the region with the largest parallax. In addition to regions F to J, there is also region X, where the parallax could not be calculated. As can be seen from the parallax image 50 illustrated in FIG. 7, a relatively large parallax value is obtained in region J, which is close to the vehicle 1, and a relatively small parallax value is obtained in region F (corresponding to the tunnel exit 6), which is far from the vehicle 1. On the other hand, region X, which corresponds to an area far from the tunnel exit 6 and where external light from outside the tunnel does not easily reach, is an invalid parallax region.

[0061] The exit determination unit 1302 first checks whether invalid parallax areas exist in the left and right edge portions of the parallax image 50. For example, it focuses on pixel blocks with a width of 10 pixels from the left and right edges of the parallax image 50 and checks whether invalid parallax areas exist within these. If invalid parallax areas exist, the exit determination unit 1302 cycles through valid parallax areas aligned from these invalid parallax areas toward the optical axis and checks whether the parallax values ​​corresponding to these areas gradually decrease. For example, in the parallax image 50 illustrated in FIG. 7, areas J, I, H, G, and F exist in this order from area X at the left and right edges toward the center of the image. That is, the parallax value gradually decreases from area X toward the center of the image. If the parallax value gradually decreases, the exit determination unit 1302 determines that the area with the smallest parallax value near the center of the image is the tunnel exit, i.e., that the tunnel exit is located at the position of the area with the smallest parallax value.

[0062] Fig. 8 is a flowchart similar to Fig. 5, showing an example of arithmetic processing executed by the processing device according to the second embodiment. In the flowchart of Fig. 8, processing of steps S250 to S260 is executed instead of the processing of steps S150 to S180 in the flowchart of Fig. 5.

[0063] In step S250, the exit determination unit 1302 focuses on pixel blocks each having a width of 10 pixels from the left and right ends of the parallax image calculated in step S110, and determines whether or not an invalid parallax area exists within the pixel blocks. If an invalid parallax area does not exist on at least one of the left and right sides, the exit determination unit 1302 determines that the tunnel exit cannot be detected, and the process shown in Fig. 8 ends. If an invalid parallax area exists on both the left and right sides, the process proceeds to step S260.

[0064] In step S260, the exit determination unit 1302 cycles through the valid parallax areas from these invalid parallax areas toward the vicinity of the optical axis, and determines whether the parallax values ​​corresponding to these areas are gradually decreasing. If the parallax values ​​are not gradually decreasing in at least one of the left and right areas (for example, if there is another invalid parallax area before reaching the vicinity of the optical axis), the exit determination unit 1302 determines that the tunnel exit cannot be detected, and the process shown in Fig. 8 ends. If the parallax values ​​are gradually decreasing in both the left and right areas, the process proceeds to step S190.

[0065] According to the above-described second embodiment, the following advantageous effects are achieved.

[0066] (1) When there is an area near the optical axis of the camera 2 (imaging device) where the parallax of the subject image can be calculated, and there are areas to the left and right of that area where the parallax of the subject image cannot be calculated, the processing device 31 determines that the tunnel exit is located in that area. This allows for detection of the tunnel exit that is resistant to blackout rather than whiteout.

[0067] The following modified examples are also within the scope of the present invention, and it is possible to combine the configuration shown in the modified example with the configuration described in the above embodiment, to combine the configurations described in the different embodiments above, or to combine the configurations described in the different modified examples below.

[0068] <Variation 1> Instead of the area of ​​the invalid parallax region near the optical axis, the shape (outline) of the invalid parallax region may be examined. For example, the exit determination unit 302 may examine whether the shape of the invalid parallax region is a semicircular shape, which is assumed to be a tunnel exit, and determine that a tunnel exit exists only if the shape is semicircular or close to semicircular. This can improve the accuracy of detecting the tunnel exit.

[0069] <Variation 2> In the left image, rather than determining that disparity cannot be calculated if the brightness values ​​of all pixels in a pixel block are at the maximum value, it may be determined that disparity cannot be calculated in the corresponding area of ​​the disparity image if the number of pixels with the maximum brightness values ​​in the pixel block is equal to or greater than a certain number (or a certain percentage).

[0070] <Variation 3> The first and second embodiments may be combined to address both overexposure and underexposure, thereby providing a more accurate function for detecting the tunnel exit.

[0071] Although the embodiments of the present invention have been described above, the above embodiments merely illustrate some of the application examples of the present invention, and it is not intended that the technical scope of the present invention be limited to the specific configurations of the above embodiments. [Explanation of symbols]

[0072] 1...vehicle, 2...camera, 2L...left camera, 2R...right camera, 3...image processing device, 31...processing device (arithmetic device), 32...non-volatile memory, 33...volatile memory, 34...input interface, 35...output interface, 301...parallax calculation unit, 302, 1302...exit determination unit, 303...distance calculation unit, 304...preceding vehicle detection unit, 305...inter-vehicle distance calculation unit, 306...relative vehicle speed calculation unit, 307...exposure control unit

Claims

1. a computing device to which an image of the area ahead of the vehicle captured by an imaging device is input; The computing device For each of a plurality of regions included in the image, a parallax of a subject image included in the region is calculated; determining whether or not a tunnel exit exists in a specific area among the plurality of areas based on a position of an area among the plurality of areas for which the parallax of the subject image could not be calculated and a parallax calculated in another area located near the area for which the parallax of the subject image could not be calculated; Image processing device.

2. 2. The image processing device according to claim 1, The image processing device determines that an invalid parallax area, which is an area where the parallax of the subject image cannot be calculated, is the specific area when, when an invalid parallax area exists near the optical axis of the imaging device, the parallax calculated in multiple areas lined up to the left, starting from an area where the parallax was calculated to the left of the invalid parallax area, gradually increases with increasing distance from the invalid parallax area, and when the parallax calculated in multiple areas lined up to the right, starting from an area where the parallax was calculated to the right of the invalid parallax area, gradually increases with increasing distance from the invalid parallax area.

3. 2. The image processing device according to claim 1, The image processing device determines that the area with the smallest parallax near the optical axis is the specific area when a first invalid parallax area, which is an area where the parallax of the subject image could not be calculated, exists near the left edge of the image and a second invalid parallax area, which is an area where the parallax of the subject image could not be calculated, exists near the right edge of the image, and when the parallax calculated in a plurality of areas lined up in a row toward the optical axis, starting from an area where the parallax could be calculated and located closer to the optical axis of the imaging device than the first invalid parallax area, gradually decreases with increasing distance from the first invalid parallax area, and when the parallax calculated in a plurality of areas lined up in a row toward the optical axis, starting from an area where the parallax could be calculated and located closer to the optical axis than the second invalid parallax area, gradually decreases with increasing distance from the second invalid parallax area.

4. 2. The image processing device according to claim 1, The computing device is an image processing device that controls the exposure of the imaging device while the vehicle is traveling to the tunnel exit, based on the distance from the vehicle to the tunnel exit and the speed of the vehicle.

5. 2. The image processing device according to claim 1, The computing device Detecting a preceding vehicle traveling ahead of the vehicle from the image; an image processing device that controls the exposure of the imaging device while the leading vehicle travels to the tunnel exit based on the positional relationship between the leading vehicle and the tunnel exit;

Images