Image processing apparatus

Abstract

An image processing device is provided with a calculation device to which an image captured by an imaging device in front of a vehicle is input, the calculation device calculating, for each of a plurality of regions included in the image, a parallax of a subject image included in the region, and calculating, for each of the plurality of regions included in the image, a parallax of the subject image included in the region. And determining whether or not there is a tunnel exit in a specific region among the plurality of regions on the basis of the position of a region in which the parallax of the subject image cannot be calculated among the plurality of regions and the parallax calculated in another region located in the vicinity of the region in which the parallax of the subject image cannot be calculated.

Description

Technical Field

[0001] The present invention relates to an image processing device. Background Art

[0002] Conventionally, there is a known technology for predicting a sudden change in brightness difference so that the preceding vehicle will not be lost even in an environment such as a tunnel exit where the brightness difference changes sharply. Patent document 1 describes a technology for predicting that the preceding vehicle enters a bright area (or a dark area) by setting a monitoring area around an object detected as a preceding vehicle on an image and using the number of pixels having brightness above a threshold (or having brightness below the threshold) in the monitoring area. Prior art literature Patent Literature

[0003] Patent Document 1: Japanese Patent Application Publication No. 2008-301342 Summary of the invention Problem that the invention aims to solve

[0004] The technology described in Patent Document 1 detects a bright area such as a tunnel exit based on the position of a preceding vehicle, and therefore has a problem that the preceding vehicle must be detected first. Technical means of solving problems

[0005] An image processing device according to one embodiment of the present invention includes a computing device, to which an image captured by a camera device in front of a vehicle is input, the computing device calculates the parallax of a subject image contained in each of a plurality of regions contained in the image, and determines whether there is a tunnel exit in a specific region among the plurality of regions based on the position of a region among the plurality of regions where the parallax of the subject image cannot be calculated and the parallax calculated in other regions near the region where the parallax of the subject image cannot be calculated. Effects of the Invention

[0006] According to the present invention, it is possible to provide an image processing device capable of detecting the entrance and exit of a tunnel without waiting for detection of a preceding vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 It is a block diagram schematically showing the configuration of a vehicle equipped with the image processing device according to the first embodiment. Figure 2 It is a block diagram schematically showing the hardware configuration of the image processing device according to the first embodiment. Figure 3 This is a functional block diagram of the image processing device according to the first embodiment. Figure 4A diagram schematically showing a left image, a right image, and a parallax image. Figure 5 This is a flowchart of the calculation process of the processing device. Figure 6 is with Figure 3 The same figure is a functional block diagram of the image processing device according to the second embodiment. Figure 7 A diagram schematically showing a left image, a right image, and a parallax image. Figure 8 is with Figure 5 The same figure is a flowchart showing an example of arithmetic processing executed by the processing device of the second embodiment. DETAILED DESCRIPTION

[0008] (First Embodiment) Reference Figure 1 to Figure 6 An image processing device according to an embodiment of the present invention will be described.

[0009] Figure 1 1 is a block diagram schematically showing the configuration of a vehicle equipped with the image processing device according to the first embodiment. A left camera 2L, a right camera 2R, and an image processing device 3 are provided on a vehicle 1. In the following description, the left camera 2L and the right camera 2R are collectively referred to as cameras 2.

[0010] The camera 2 is installed near the interior rearview mirror in the cabin of the vehicle 1. The camera 2 captures the front of the vehicle 1. The left camera 2L and the right camera 2R are arranged so that their optical axes are parallel and at the same height. The image captured by the left camera 2L is referred to as the left image. The left camera 2L periodically captures the left image and inputs it to the image processing device 3. The image captured by the right camera 2R is referred to as the right image. The right camera 2R periodically captures the right image and inputs it to the image processing device 3. In the present embodiment, the left camera 2L and the right camera 2R capture the left image and the right image at the same time. In other words, the left image and the right image are images of the front of the vehicle 1 captured at the same time. In addition, the optical axes of the left camera 2L and the right camera 2R may not be parallel. In this case, it is preferred to parallelize the left image and the right image in the camera 2 or the image processing device 3.

[0011] Figure 2This is a block diagram schematically showing the hardware configuration of the image processing device 3 of the first embodiment. The image processing device 3 is composed of a computer, which includes a processing device 31 such as a CPU (Central Processing Unit), an MPU (Micro Processing Unit), and a DSP (Digital Signal Processor), a non-volatile memory 32 such as a ROM (Read-Only Memory), a flash memory, and a hard disk drive, a volatile memory 33 called a so-called RAM (Random Access Memory), an input interface 34, an output interface 35, and other peripheral circuits (not shown). In addition, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), etc. can also be used as the processing device 31. The non-volatile memory 32 and the volatile memory 33 are used as storage devices for storing information (data).

[0012] The nonvolatile memory 32 stores programs that can perform various operations. That is, the nonvolatile memory 32 is a storage device (storage medium) that can read the program that realizes the functions of this embodiment. The processing device 31 is an operation device that expands the program stored in the nonvolatile memory 32 to the volatile memory 33 and performs operations. According to the program, the data taken from the input interface 34, the nonvolatile memory 32 as a storage device, and the volatile memory 33 are subjected to predetermined operation processing. The functions realized by the program will be described later.

[0013] The input interface 34 converts the left image and the right image captured by the camera 2 into images that can be calculated by the processing device 31. In addition, the output interface 35 generates an exposure control signal corresponding to the calculation result of the processing device 31 and outputs it to the camera 2.

[0014] The image processing device 3 may be composed of a single computer or a plurality of computers. When the image processing device 3 is composed of a plurality of computers, the functions of the image processing device 3 are realized by a plurality of processing devices 31 .

[0015] Figure 33 is a functional block diagram of the image processing device 3. The image processing device 3 functions as a parallax calculation unit 301, an exit determination unit 302, a distance calculation unit 303, a preceding vehicle detection unit 304, a vehicle-to-vehicle distance calculation unit 305, a relative vehicle speed calculation unit 306, and an exposure control unit 307 by executing a program stored in a nonvolatile memory 32 through a processing unit 31.

[0016] The parallax calculation unit 301 generates a parallax image using the left image and the right image simultaneously captured by the left camera 2L and the right camera 2R. The exit determination unit 302 determines whether there is an entrance or exit of a tunnel on the path of the vehicle 1 using the parallax image calculated by the parallax calculation unit 301. The distance calculation unit 303 calculates the distance from the vehicle 1 to the entrance or exit of the tunnel. The preceding vehicle detection unit 304 detects other vehicles located in front of the vehicle 1, that is, the preceding vehicle. The inter-vehicle distance calculation unit 305 calculates the distance from the vehicle 1 to the preceding vehicle, that is, the inter-vehicle distance. The relative speed calculation unit 306 calculates the relative speed between the vehicle 1 and the preceding vehicle. For example, when the speed of the vehicle 1 is the same as the speed of the preceding vehicle, the relative speed calculated by the relative speed calculation unit 306 is 0 kilometers per hour. In addition, when the speed of the vehicle 1 is 30 kilometers per hour and the speed of the preceding vehicle is 50 kilometers per hour, the relative speed calculated by the relative speed calculation unit 306 is 20 kilometers per hour. The exposure control unit 307 performs exposure control of the camera 2. In other words, the exposure control unit 307 adjusts the exposure amount of the camera 2 so that the exposure of the camera 2 becomes appropriate exposure.

[0017] (Method of Generating Parallax Images by Parallax Calculation Unit 301) Figure 4 A diagram schematically showing a left image, a right image, and a parallax image. Figure 4 The illustrated left image 4L and right image 4R are images captured in a tunnel. The tunnel exit 6 and the preceding vehicle 7 are captured in the left image 4L and the right image 4R. Since the tunnel exit 6 is much brighter than the inside of the tunnel, it is assumed that the tunnel exit 6 is overexposed in the left image 4L and the right image 4R. Figure 4 The method of generating the parallax image 5 from the left image 4L and the right image 4R is described with reference to the left image 4L, the right image 4R, and the parallax image 5 shown in FIG.

[0018] The parallax calculation unit 301 divides the left image 4L into a plurality of pixel blocks in a grid shape. A pixel block is, for example, 16 pixels in the vertical direction and 16 pixels in the horizontal direction. That is, a pixel block includes a total of 256 pixels. In addition, the size of the pixel block may be different. In addition, the shape of the pixel block may be a rectangle instead of a square. Similar to the left image 4L, the parallax calculation unit 301 divides the right image 4R into a plurality of pixel blocks in a grid shape.

[0019] In the following description, the pixel blocks in the left image 4L are marked as BL (x, y). Here, x is an index value indicating the position of the horizontal pixel block. In the pixel block at the left end, x is 1, and increases by 1 each time as it moves to the right. Similarly, y is an index value indicating the position of the vertical pixel block. In the pixel block at the upper end, y is 1, and increases by 1 each time as it moves downward. Similar to the left image 4L, the pixel blocks in the right image 4R are marked as BR (x, y). The meanings of x and y are the same as those of the pixel blocks BL (x, y) of the left image 4L.

[0020] The parallax calculation unit 301 compares a plurality of pixel blocks BL(x, y) arranged in a row horizontally in the left image 4L with a plurality of pixel blocks BR(x, y) arranged in a row at the same positions as the plurality of pixel blocks BL(x, y) in the right image 4R, thereby calculating the parallax of the subject image included in the row of pixel blocks.

[0021] First, focus on the pixel block BL(1, 1) at the upper left end of the left image 4L. For the pixel blocks BR(1, 1), BR(2, 1), BR(3, 1), ... of the right image 4R, that is, for each of the pixel blocks arranged in a row horizontally and having the same vertical position (y=1) as the pixel block BL(1, 1) of interest, calculate the degree of consistency with the pixel block BL(1, 1) of interest. Here, consistency refers to a numerical value that quantifies the degree of similarity between the brightness values ​​of multiple pixels contained in one pixel block and the brightness values ​​of multiple pixels contained in another pixel block. The closer the brightness values ​​are to each other, the greater the degree of consistency. The specific method for calculating the degree of consistency will be described later.

[0022] The parallax calculation unit 301 uses the calculated consistency to determine the pixel block BR(i, 1) that has the highest consistency with the pixel block BL(1, 1) of interest. At this time, the parallax at the position x=1, y=1 is |i-1|. That is, the difference in the horizontal position of the pixel block BR(i, 1) that is most consistent with the pixel block BL(1, 1) of interest is the parallax of the subject image at the position of the pixel block BL(1, 1) of interest.

[0023] The parallax calculation unit 301 repeats this process for all pixel blocks BL(1,1), BL(2,1), BL(3,1), . . . at position y=1 of the left image 4L, and calculates all parallaxes at position y=1.

[0024] The parallax calculation unit 301 repeatedly performs the above-mentioned processing while changing the position little by little in the longitudinal direction of the left image 4L, thereby calculating all the parallaxes in the subject image at the positions y=2, y=3, ... That is, at the positions of all the pixel blocks BL(x, y) included in the left image 4L, the parallax corresponding to the subject image at the position is calculated. Data in which the thus calculated parallaxes are arranged two-dimensionally according to the positions at which the parallaxes are calculated is a parallax image.

[0025] exist Figure 4 In the parallax image 5 shown in the example, there are five regions A, B, C, D, and E in the order of the parallax from small to large. That is, region A is the region with the smallest parallax, and region E is the region with the largest parallax. In addition, unlike these regions A to E, there is a region X where the parallax cannot be calculated. In addition, Figure 4 The parallax image 5 shown is schematically shown for the sake of simplicity. In reality, the parallax is calculated more finely than five stages. Figure 4 The more finely divided the stages are as shown, the more regions with different parallaxes will exist.

[0026] When the exposure control of the camera 2 is performed in the dark environment in the tunnel, overexposure occurs in the area where the tunnel exit 6 is present in the camera image. That is, the brightness value of the pixel corresponding to the tunnel exit 6 is saturated. For example, when the brightness value is represented by an integer value of 0 to 255, the brightness value of the pixel corresponding to the tunnel exit 6 becomes the maximum value 255. When the brightness values ​​of all pixels included in one pixel block are saturated like this, the disparity value cannot be calculated from the pixel block. That is, in the pixel block corresponding to the tunnel exit 6, the disparity value cannot be calculated.

[0027] In a pixel block in the left image 4L, when the brightness values ​​of all pixels in the pixel block are maximum values, the parallax calculation unit 301 determines that the parallax cannot be calculated in the corresponding area X of the parallax image 5. When the parallax calculation unit 301 determines that the parallax cannot be calculated at a certain position, the parallax calculation unit 301 uses a special value indicating that the parallax cannot be calculated as the parallax value of the position (for example, a negative value, etc.). In the following description, this "special value indicating that the parallax cannot be calculated" is referred to as an invalid parallax value. In addition, the area X where the parallax cannot be calculated and the invalid parallax value is set is referred to as an invalid parallax area. On the contrary, areas A to E that are not invalid parallax areas are referred to as valid parallax areas.

[0028] For example, when photographing a very dark part such as a part inside a tunnel where the lighting light cannot reach, underexposure rather than overexposure may occur. For example, when the brightness value is represented by an integer value of 0 to 255, the brightness value of the pixel corresponding to such a very dark place becomes the minimum value 0. In the case where the brightness value of all pixels included in a pixel block becomes the minimum value like this, the parallax value cannot be calculated from the pixel block. In the pixel block in the left image 4L, even when the brightness value of all pixels in the pixel block is the minimum value, the parallax calculation unit 301 determines that the parallax cannot be calculated in the corresponding area of ​​the parallax image 5.

[0029] from Figure 4 It can be seen from the parallax image 5 illustrated in the figure that a relatively large parallax value is obtained in the area E close to the vehicle (the vehicle itself) 1, and a relatively small parallax value is obtained in the area A far from the vehicle 1. In this way, the parallax gradually decreases from the vicinity of the vehicle 1 to the distance. Therefore, in the parallax image 5, the parallax value of the road surface portion gradually decreases as it moves from the lower part to the upper part in the longitudinal direction. On the other hand, when there is a three-dimensional object such as a vehicle, the same parallax value is obtained in the longitudinal direction within the range where the three-dimensional object exists in the parallax image 5. Therefore, by finding an area with the same parallax value in the longitudinal direction in the parallax image 5, it is possible to determine the position where the three-dimensional object exists in the area. That is, by finding an area with the same parallax value along the longitudinal direction in the parallax image 5, the three-dimensional object can be detected.

[0030] By using the following formula (1), the distance from the vehicle 1 to an object existing in an arbitrary area can be obtained from the parallax value in the area. d=(L+f) / p…(1) Here, d is the distance to the target object, L is the baseline length of the left camera 2L and the right camera 2R, f is the focal length of the left camera 2L and the right camera 2R, and p is the parallax value.

[0031] The calculation method of the consistency is explained. The consistency in this embodiment is the sum of the absolute values ​​of the brightness differences between each pixel in the pixel block (SAD: Sum of the Absolute Differences, the sum of absolute differences). For example, when a pixel block has a size of 16 vertical pixels and 16 horizontal pixels, a pixel block includes 256 pixels. When a pixel block in the left image 4L is opposite to a pixel block in the right image 4R, there are 256 groups of left and right pixels. For each of these 256 groups, the absolute value of the difference in brightness values ​​is calculated to obtain the absolute values ​​of the difference in 256 brightness values. The sum of these 256 absolute values ​​is the consistency in this embodiment.

[0032] In addition, the method for calculating the degree of consistency is not limited to the method using the above-mentioned SAD, and other methods may be used. In addition, depending on what value is used as the degree of consistency, it may be determined that the parallax cannot be calculated for reasons different from the reasons caused by overexposure or underexposure described above, but even in this case, it can be treated as an invalid parallax area in the same way as the above-mentioned method.

[0033] (Description of a method for determining a tunnel exit by the exit determination unit 302) use Figure 4 The method by which the exit determination unit 302 determines the tunnel exit will be described using the left image 4L, the right image 4R, and the parallax image 5 shown in FIG.

[0034] The exit determination unit 302 first checks whether there is an invalid parallax area near the optical axis position of the camera 2 (the vanishing point, which is the center of the parallax image 5 here). For example, focus on the area corresponding to the 11×11 pixel blocks centered on the center point of the parallax image 5, and check whether there is an invalid parallax area in these areas. In addition, the inspection range of the invalid parallax area of ​​"the 11×11 pixel blocks centered on the center point of the parallax image 5" is an example, and it is also possible to check whether there is an invalid parallax area in a range different from it. For example, by setting the inspection range to a horizontally long rectangle, it is possible to better detect the tunnel exit when there is a tunnel exit at the front end of the curve. In addition, on the contrary, by shortening the inspection range in the horizontal direction, the amount of calculation can be reduced.

[0035] In the case where an invalid parallax area exists, the exit determination unit 302 calculates the total area of ​​the invalid parallax area and all invalid parallax areas that exist continuously with the invalid parallax area. Here, "continuous existence" means that the invalid parallax areas are adjacent to each other. In the case where the calculated total area is a size greater than a threshold, the exit determination unit 302 checks the proportion of saturated pixels in the pixels contained in those continuously existing invalid parallax areas. Here, a saturated pixel is a pixel whose brightness value is greater than a specified threshold. For example, the number of pixels contained in a continuous invalid parallax area is 1000, of which 400 pixels have a brightness value greater than a specified threshold, and the proportion calculated here is 400 / 1000=0.4.

[0036] The exit determination unit 302 checks whether the calculated ratio is above a specified threshold value (for example, 0.8). When the calculated ratio is above the specified threshold value, the exit determination unit 302 further calculates the center of gravity position when these consecutive invalid parallax areas are connected to each other. Then, the valid parallax area is searched from the center of gravity position to the left, that is, the area where the parallax can be calculated is searched. The exit determination unit 302 circulates the valid parallax areas arranged in a row from the initially discovered valid parallax area to the left, and checks whether the parallax values ​​corresponding to these areas are gradually increasing. That is, it checks whether the parallax values ​​are gradually increasing from the center of gravity position to the left of the image. For example, in Figure 4 In the parallax image 5 shown, regions A, B, C, D, and E exist in this order from region X to the left of the image. That is, the parallax value gradually increases from region X to the left of the image. In the case where the parallax value gradually increases, the exit determination unit 302 performs the same processing in the right direction. In addition to the case where the parallax value gradually increases in the right direction of the image, the exit determination unit 302 determines that the invalid parallax area is a tunnel exit, that is, it determines that the tunnel exit exists at the position of the invalid parallax area.

[0037] The outer wall of the tunnel should exist on the left and right sides of the tunnel exit along the road from the tunnel exit to the vehicle 1. Since the closer the object is to the vehicle 1, the larger the parallax value, the parallax value calculated in the area corresponding to the left and right tunnel outer walls should gradually increase from the tunnel exit toward the vehicle 1. The exit determination unit 302 performs the above-mentioned processing with such a concept in mind to determine the existence of the tunnel exit. That is, the exit determination unit 302 determines whether there is a tunnel exit in the area where the parallax of the subject image cannot be calculated based on the position of the area where the parallax of the subject image cannot be calculated and the parallax calculated in other areas located near the area where the parallax of the subject image cannot be calculated.

[0038] (Description of a Calculation Method of the Distance Calculation Unit 303 Calculating the Distance from the Vehicle 1 to the Tunnel Exit) The distance calculation unit 303 calculates the distance from the vehicle 1 to the tunnel exit by substituting the parallax value near the invalid parallax area into the above formula (1). For example, the distance calculation unit 303 calculates the distance to the tunnel exit by substituting the parallax value calculated at the left or right adjacent position of the invalid parallax area corresponding to the tunnel exit into the above formula (1). Alternatively, the distance to the tunnel exit may be calculated by substituting the average of two parallax values ​​calculated at the left and right adjacent positions of the invalid parallax area into the above formula (1).

[0039] (Description of a Detection Method of the Leading Vehicle Detection Unit 304) The leading vehicle detection unit 304 checks whether the same (or close enough) parallax value exists continuously in the longitudinal direction for a prescribed length or more near the left and right centers of the parallax image. When there is nothing but the road surface at this position, the parallax value should gradually change according to the distance to the road surface. On the other hand, when the same (or close enough) parallax value exists continuously in the longitudinal direction for a prescribed length or more near the left and right centers of the parallax image, there is a high possibility that some kind of object exists at that location. When the same (or close enough) parallax value exists continuously in the longitudinal direction for a prescribed length or more near the left and right centers of the parallax image, the leading vehicle detection unit 304 considers that there is a leading vehicle at that location.

[0040] (Description of the calculation process of the processing device 31) Figure 5 2 is a flowchart of the calculation process of the processing device 31. In step S100, the left image captured by the left camera 2L and the right image captured by the right camera 2R are input to the processing device 31 via the input interface 34. In step S110, the parallax calculation unit 301 generates a parallax image using the left image and the right image input in step S100.

[0041] In step S120, the leading vehicle detection unit 304 determines whether there is a leading vehicle in front of the vehicle 1. The leading vehicle detection unit 304 checks whether the same (or sufficiently close) parallax value exists continuously in the longitudinal direction for a predetermined length or more near the left and right centers of the parallax image. When the same (or sufficiently close) parallax value exists continuously in the longitudinal direction for a predetermined length or more near the left and right centers of the parallax image, the leading vehicle detection unit 304 considers that there is a leading vehicle there, and the processing proceeds to step S130. On the other hand, when the same (or sufficiently close) parallax value does not exist continuously in the longitudinal direction for a predetermined length or more near the left and right centers of the parallax image, the leading vehicle detection unit 304 cannot detect the leading vehicle, and the processing proceeds to step S150.

[0042] In step S130, the inter-vehicle distance calculation unit 305 calculates the distance between the preceding vehicle (other vehicle) detected in step S120 and vehicle (this vehicle) 1, that is, the inter-vehicle distance with the preceding vehicle. The inter-vehicle distance calculation unit 305 calculates the distance to the preceding vehicle by substituting the parallax value at the position where the preceding vehicle is detected into the above formula (1). In step S140, the relative vehicle speed calculation unit 306 calculates the relative vehicle speed between the preceding vehicle detected in step S120 and vehicle 1. The relative vehicle speed calculation unit 306 calculates the relative vehicle speed between the preceding vehicle and vehicle 1 by dividing the difference between the inter-vehicle distance calculated in the previous step S130 and the inter-vehicle distance calculated in this step S130 by the difference between the time when the previous step S130 is executed and the time when the current step S130 is executed. Then, the processing enters step S150.

[0043] In step S150, the exit determination unit 302 determines an invalid parallax area located near the vanishing point (the center of the optical axis of the camera) based on the parallax image calculated in step S110. In step S160, the exit determination unit 302 determines whether the area of ​​the determined invalid parallax area is greater than a predetermined threshold. If the area of ​​the determined invalid parallax area is less than the predetermined threshold, the exit determination unit 302 determines that the determined invalid parallax area is not a tunnel exit, and ends the tunnel. Figure 6 When the area of ​​the determined invalid parallax region is equal to or larger than the predetermined threshold, the process proceeds to step S170.

[0044] In step S170, the exit determination unit 302 determines whether the parallax value gradually increases from the center of gravity of the invalid parallax area to the left and right directions. If the parallax value does not gradually increase from the center of gravity of the invalid parallax area to the left and right directions, the exit determination unit 302 determines that the determined invalid parallax area is not a tunnel exit, and ends. Figure 6 When the parallax value gradually increases from the center of gravity of the invalid parallax area in the left-right direction, the process proceeds to step S180. In step S180, the exit determination unit 302 calculates the ratio of pixels whose brightness values ​​are greater than a predetermined threshold value among all pixels included in the invalid parallax area, and determines whether the ratio is greater than the predetermined threshold value. If the calculated ratio is less than the predetermined threshold value, the exit determination unit 302 determines that the determined invalid parallax area is not a tunnel exit, and the process ends. Figure 6 When the calculated ratio is equal to or greater than the predetermined threshold, the exit determination unit 302 determines that the invalid parallax area is a tunnel exit, and the process proceeds to step S190.

[0045] In step S190, the distance calculation unit 303 calculates the distance from the vehicle 1 to the tunnel exit. The distance to the tunnel exit is calculated by substituting the parallax value near the invalid parallax area into the above formula (1). For example, the distance calculation unit 303 calculates the distance to the tunnel exit by substituting the parallax value calculated at the left or right adjacent position of the invalid parallax area into the above formula (1).

[0046] In step S200, the exposure control unit 307 determines whether the preceding vehicle is detected in step S120. In the case where the preceding vehicle is not detected, the process proceeds to step S230. On the other hand, in the case where the preceding vehicle is detected, the process proceeds to step S210. In step S210, the exposure control unit 307 calculates the distance from the preceding vehicle to the tunnel exit. The exposure control unit 307 calculates the distance from the preceding vehicle to the tunnel exit by subtracting the inter-vehicle distance (the distance from vehicle 1 to the preceding vehicle) calculated in step S130 from the distance from vehicle 1 to the tunnel exit calculated in step S190.

[0047] In step S220, the exposure control unit 307 adjusts the exposure of the camera 2 according to the distance from the preceding vehicle to the tunnel exit calculated in step S210, so that the exposure of the camera 2 is suitable for the bright environment outside the tunnel when the preceding vehicle reaches the tunnel exit (passes through the tunnel exit). For example, the exposure control unit 307 adjusts the exposure in such a way that the exposure becomes an appropriate exposure relative to the brightness value of the pixels existing in the invalid parallax area. Alternatively, the exposure control unit 307 may pre-record the exposure of the vehicle 1 before it enters the tunnel, and adjust the exposure of the camera 2 so that it becomes the recorded exposure when the preceding vehicle passes through the tunnel exit. Afterwards, Figure 6 The shown processing ends.

[0048] A distance-exposure table corresponding to the distance from the preceding vehicle to the tunnel exit is pre-stored in the non-volatile memory 32. In step S220, the exposure control unit 307 converts the distance-exposure table into a time-exposure table using the speed of the preceding vehicle calculated based on the relative speed of the preceding vehicle and the speed of the vehicle 1, and determines a schedule for exposure control. Alternatively, instead of using such a table, the schedule for exposure control may be determined by a mathematical formula using the distance from the preceding vehicle to the tunnel exit, the speed of the preceding vehicle, and the elapsed time.

[0049] In step S230, the exposure control unit 307 adjusts the exposure of the camera 2 according to the distance from the vehicle 1 to the tunnel exit calculated in step S190, so that the exposure of the camera 2 is suitable for the bright environment outside the tunnel when the vehicle 1 is about to reach the tunnel exit (pass through the tunnel exit). For example, the exposure control unit 307 adjusts the exposure in such a way that the exposure becomes an appropriate exposure relative to the brightness value of the pixels existing in the invalid parallax area. Alternatively, the exposure control unit 307 may pre-record the exposure of the vehicle 1 before entering the tunnel, and adjust the exposure of the camera 2 so that it becomes the recorded exposure when the vehicle 1 is about to reach the tunnel exit (pass through the tunnel exit). Afterwards, Figure 6 The shown processing ends.

[0050] The nonvolatile memory 32 stores in advance a distance-exposure table corresponding to the distance from the vehicle 1 to the tunnel exit. In step S230, the exposure control unit 307 converts the distance-exposure table into a time-exposure table using the speed of the vehicle 1, and determines a schedule for exposure control. Alternatively, instead of using such a table, the schedule for exposure control may be determined using a mathematical formula using the distance from the vehicle 1 to the tunnel exit, the speed of the vehicle 1, and the elapsed time.

[0051] In addition, the exposure control unit 307 can use a known technique to adjust the exposure amount. For example, the shutter speed of the camera 2, that is, the exposure time, can be adjusted, the aperture diameter of the camera 2 can be adjusted, and the sensitivity (magnification) of the imaging element of the camera 2 can be adjusted.

[0052] According to the above-mentioned first embodiment, the following effects are achieved.

[0053] (1) The processing device (computing device) 31 calculates the parallax of the subject image contained in each of the plurality of regions contained in the image captured by the camera 2, and determines whether there is a tunnel exit in a specific region among the plurality of regions based on the position of the region in which the parallax of the subject image cannot be calculated and the parallax calculated in other regions located near the region in which the parallax of the subject image cannot be calculated. In this way, the entrance and exit of the tunnel can be detected without waiting for the detection of the preceding vehicle. In addition, the entrance and exit of the tunnel can be detected regardless of the presence of the preceding vehicle.

[0054] (2) When there is an area in the vicinity of the optical axis of the camera 2 (camera device) where the parallax of the subject image cannot be calculated, the processing device 31 determines that there is a tunnel exit in the area when the parallax calculated for the plurality of areas arranged in a row in the left direction on the left side of the area and the plurality of areas arranged in a row in the right direction on the right side of the area gradually increases as the distance from the area increases. In this way, not only the presence of the invalid parallax area is observed, but also a characteristic parallax pattern is used at the tunnel exit, so that the tunnel exit can be detected with high accuracy.

[0055] (3) The processing device 31 controls the exposure of the camera 2 (imaging device) while the vehicle 1 is traveling to the tunnel exit, based on the distance from the vehicle 1 to the tunnel exit and the speed of the vehicle 1. Therefore, appropriate exposure can be maintained from the time the tunnel exit is detected until the vehicle 1 reaches the tunnel exit.

[0056] (4) The processing device 31 detects the preceding vehicle traveling in front of the vehicle 1 based on the image, and controls the exposure of the camera 2 (camera device) while the preceding vehicle is traveling to the tunnel exit based on the positional relationship between the preceding vehicle and the tunnel exit. Therefore, appropriate exposure can be maintained from the time the tunnel exit is detected until the preceding vehicle arrives at the tunnel exit, and detection can be continued without missing the preceding vehicle.

[0057] (Second Embodiment) Reference Figure 6 to Figure 8 Hereinafter, an image processing device according to a second embodiment of the present invention will be described. The same reference numerals are used for the same or corresponding components as those described in the first embodiment, and mainly the differences will be described.

[0058] Figure 6 1 is a functional block diagram of the image processing device 300 according to the second embodiment. The image processing device 300 includes an exit determination unit 1302 instead of the exit determination unit 302. The exit determination unit 1302 determines whether there is a tunnel exit on the travel path of the vehicle 1 using the parallax image calculated by the parallax calculation unit 301.

[0059] Figure 7 40 is a diagram schematically showing the left image 40L, the right image 40R, and the parallax image 50 . Figure 7 The left image 40L and the right image 40R shown are images taken in a tunnel. The tunnel exit 6 and the preceding vehicle 7 are captured in the left image 40L and the right image 40R. Here, it is assumed that the exposure of the camera 2 is suppressed to a level that does not cause overexposure at the tunnel exit 6. In this case, the parallax calculation unit 301 can correctly calculate the parallax at the tunnel exit 6. On the other hand, since the exposure of the camera 2 is controlled, underexposure occurs in the dark part of the tunnel, such as the part that is difficult to reach by the external light from the tunnel exit and the illumination light in the tunnel. In this case, the parallax calculation unit 301 cannot calculate the parallax in such a dark part. In other words, in the parallax image 50 generated by the parallax calculation unit 301, such a dark part becomes an invalid parallax area.

[0060] exist Figure 7 In the parallax image 50 shown in the example, there are five regions F, G, H, I, and J in the order of the parallax from small to large. That is, region F is the region with the smallest parallax, and region J is the region with the largest parallax. In addition, there is a region X where the parallax cannot be calculated, unlike these regions F to J. Figure 7As can be seen from the parallax image 50 shown, a relatively large parallax value is obtained in the area J close to the vehicle 1, and a relatively small parallax value is obtained in the area F (corresponding to the tunnel exit 6) far from the vehicle 1. On the other hand, the area X corresponding to the part far from the tunnel exit 6 and difficult for external light from outside the tunnel to reach becomes an invalid parallax area.

[0061] The exit determination unit 1302 first checks whether there are invalid parallax areas at the left and right ends of the parallax image 50. For example, it focuses on pixel blocks with a width of 10 pixels from the left and right ends of the parallax image 50, and checks whether there are invalid parallax areas in these pixel blocks. If there are invalid parallax areas, the exit determination unit 1302 circulates valid parallax areas arranged in a row from these invalid parallax areas toward the vicinity of the optical axis, and checks whether the parallax values ​​corresponding to these areas gradually decrease. For example, Figure 7 In the parallax image 50 shown, regions J, I, H, G, and F exist in order from region X located at the left and right ends toward the center of the image. That is, the parallax value gradually decreases from region X toward the center of the image. When the parallax value gradually decreases, the exit determination unit 1302 determines that the region with the smallest parallax value, which exists near the center of the image, is the tunnel exit, that is, the tunnel exit exists at the position of the region with the smallest parallax value.

[0062] Figure 8 is with Figure 5 The same figure is a flowchart showing an example of a calculation process executed by the processing device of the second embodiment. Figure 8 In the flowchart, replace Figure 5 The processing of steps S150 to S180 of the flowchart is performed, and the processing of steps S250 to S260 is performed.

[0063] In step S250, the exit determination unit 1302 focuses on the pixel blocks with a width of 10 pixels from the left and right ends of the parallax image calculated in step S110, and determines whether there is an invalid parallax area in the blocks. If there is no invalid parallax area on at least one side, the exit determination unit 1302 determines that the tunnel exit cannot be detected, and ends. Figure 8 When there are invalid parallax areas in both the left and right sides, the process proceeds to step S260.

[0064] In step S260, the exit determination unit 1302 traverses the valid parallax areas from these invalid parallax areas toward the vicinity of the optical axis to determine whether the parallax values ​​corresponding to these areas are gradually decreasing. If the parallax value in at least one of the left and right sides does not gradually decrease (for example, there are other invalid parallax areas before reaching the vicinity of the optical axis), the exit determination unit 1302 determines that the tunnel exit cannot be detected and ends. Figure 8When the parallax values ​​on both sides gradually decrease, the process proceeds to step S190.

[0065] According to the second embodiment described above, the following effects are achieved.

[0066] (1) When there is an area near the optical axis of the camera 2 (imaging device) where the parallax of the subject image can be calculated, and there is an area on the left and right of the area where the parallax of the subject image cannot be calculated, the processing device 31 determines that there is a tunnel exit in the area. This makes it possible to detect a tunnel exit that is resistant to underexposure rather than overexposure.

[0067] The following modifications are also within the scope of the present invention. It is also possible to combine the configuration shown in the modification with the configuration described in the above embodiment, or to combine the configurations described in the above different embodiments, or to combine the configurations described in the following different modifications.

[0068] <Modification 1> The shape (outer shape) of the invalid parallax area may be checked instead of the area of ​​the invalid parallax area near the optical axis. For example, the exit determination unit 302 may check whether the shape of the invalid parallax area is a semicircular shape assumed to be a tunnel exit, and determine that there is a tunnel exit only when it is a semicircular shape or a shape close to a semicircular shape. In this way, the detection accuracy of the tunnel exit can be improved.

[0069] <Modification 2> In the left image, it is not that the disparity cannot be calculated when the brightness values ​​of all pixels in the pixel block are maximum values, but if the number of pixels with maximum brightness values ​​in the pixel block is above a certain number (or a certain proportion), it can be determined that the disparity cannot be calculated in the corresponding area of ​​the disparity image.

[0070] <Variation 3> The first embodiment and the second embodiment may be combined to deal with both overexposure and underexposure. In this way, a tunnel exit detection function with higher accuracy can be provided.

[0071] Although the embodiments of the present invention have been described above, the above embodiments merely represent a part of application examples of the present invention, and the technical scope of the present invention is not limited to the specific configurations of the above embodiments. Explanation of symbols

[0072] 1…vehicle, 2…camera, 2L…left camera, 2R…right camera, 3…image processing device, 31…processing device (computing device), 32…non-volatile memory, 33…volatile memory, 34…input interface, 35…output interface, 301…parallax calculation unit, 302…exit determination unit, 303…distance calculation unit, 304…front vehicle detection unit, 305…vehicle-to-vehicle distance calculation unit, 306…relative vehicle speed calculation unit, 307…exposure control unit.

Claims

1. An image processing device, characterized in that: have: a computing device to which an image captured by a camera device in front of the vehicle is input; The computing device calculates, for each of a plurality of regions included in the image, a parallax of the subject image included in the region. The computing device determines whether there is a tunnel exit in a specific area among the plurality of areas based on the position of an area among the plurality of areas where the parallax of the subject image cannot be calculated and the parallax calculated in another area located near the area where the parallax of the subject image cannot be calculated.

2. The image processing device according to claim 1, characterized in that When there is an area near the optical axis of the camera device where the parallax of the subject image cannot be calculated, and when the parallax calculated in the multiple other areas arranged in a row in the left direction on the left side of the area or in the multiple other areas arranged in a row in the right direction on the right side of the area gradually increases as they go away from the area, the computing device determines that there is a tunnel exit in the area.

3. The image processing device according to claim 1, characterized in that When there is an area near the optical axis of the imaging device where the parallax of the subject image can be calculated, and there are areas to the left and right of the area where the parallax of the subject image cannot be calculated, the computing device determines that a tunnel exit exists in the area.

4. The image processing device according to claim 1, characterized in that: The computing device controls the exposure of the imaging device while the vehicle is traveling to the tunnel exit, based on the distance from the vehicle to the tunnel exit and the speed of the vehicle.

5. The image processing device according to claim 1, characterized in that: The computing device detects a preceding vehicle traveling in front of the vehicle based on the image, The exposure of the imaging device is controlled based on the positional relationship between the preceding vehicle and the tunnel exit while the preceding vehicle is traveling to the tunnel exit.

Images