Image display device

Abstract

To provide an image display apparatus for displaying an image of surroundings of a vehicle, configured to properly control exposure level of imaging means even when a captured image includes an area having a singular luminance distribution, thereby effectively reducing over-exposure and / or under-exposure in a display image.SOLUTION: An image display apparatus updates a photometric value by increasing a weighting coefficient of a photometric area having a luminance exceeding a standard luminance range, and adjusts exposure level in imaging processing so that the photometric value may be close to a target photometric value. When over-exposure and / or under-exposure in a display image cannot be satisfactorily reduced by adjusting the exposure level and when intensity of an input signal falls within a dynamic range of an imaging unit, the over-exposure and / or under-exposure may be reduced by adjusting a gamma curve. In addition, when it is determined that a vehicle is located in an environment which is consistent with an environment where an image adjustment routine has been executed, an exposure level, or the like, updated by the image adjustment routine may be applied.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an image display device, and more specifically, to an image display device that displays an image of the periphery of a vehicle, and that can effectively reduce blown-out highlights and / or crushed shadows in a displayed image by suitably controlling an exposure level of an imaging means even when the captured image includes an area with a peculiar luminance distribution. [Background technology]

[0002] In the exposure control of a general imaging device, the exposure level of the imaging means is adjusted (exposure control) based on the average luminance value of the captured image. In such exposure control, the exposure level fluctuates due to the influence of background light, so that, for example, when the background is dark, the subject appears bright, and when the background is bright, the subject appears dark. As a result, there is a problem that the accuracy of advanced image processing such as recognition processing in the subsequent stage is difficult to stabilize. In addition, since the exposure level is adjusted so that the average luminance value of the captured image becomes the target value, exposure control is not performed even when an extremely bright area and / or an extremely dark area exists in the captured image as long as the average luminance value is the target value. As a result, there is a risk that the entire extremely bright area will be displayed monotonously bright (whiteout) or the entire extremely dark area will be displayed monotonously dark (blackout), making it impossible to visually recognize objects present in these places.

[0003] In the technical field, a technique is known that avoids the above-mentioned problems by excluding extremely bright and extremely dark areas and calculating an average brightness value (for example, see Patent Document 1). For example, the photometric area of ​​a captured image is divided into a plurality of blocks, a brightness value histogram is generated for each of the blocks, and a photometric value (average brightness value) is calculated by excluding blocks having abnormal values ​​(portions with extremely high or low brightness), and the exposure level of the imaging means is controlled based on the photometric value. According to this technique, even if a portion with a peculiar brightness distribution exists in the captured image, the exposure level of the imaging means can be appropriately set and the captured image can be displayed clearly.

[0004] Meanwhile, in the recent automobile industry, systems that capture images of the surroundings of a vehicle and display them to the driver are widely used. As a specific example of such a system, for example, a system that displays an image of the surroundings of the vehicle as seen from above the vehicle by mounting cameras on the front, rear, left and right sides of the vehicle is known. The cameras mounted on the left and right sides of the vehicle need to simultaneously capture images of the front and rear of the vehicle. However, when there is a large difference in brightness between the front and rear of the vehicle, for example, and there is an extremely bright area and / or an extremely dark area in the captured image, the general exposure control as described above may result in overexposure in the former area and underexposure in the latter area.

[0005] The above-mentioned situation is particularly likely to occur when, for example, strong light shines from behind the vehicle when the vehicle is backing up at night and / or in a dark multi-storey parking lot. In such a case, it is difficult to suitably adjust the exposure level of the imaging means even if exposure control is performed by excluding a portion having a peculiar luminance distribution in the captured image as described in the above-mentioned Patent Document 1. As a result, for example, problems may occur such as a clear captured image of the rear of the vehicle not being obtained or exposure control taking a long time. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Patent No. 4523629 Summary of the Invention [Problem to be solved by the invention]

[0007] As described above, in the technical field, there is a demand for a technology that can effectively reduce blown-out highlights and / or crushed shadows in a displayed image by suitably controlling the exposure level of an imaging means even when the captured image has an area with a peculiar brightness distribution in an image display device that displays an image of the periphery of a vehicle. [Means for solving the problem]

[0008] Therefore, as a result of intensive research, the inventors have discovered that it is possible to meet the above requirements by adjusting the weighting of areas having unique brightness distribution when calculating the photometric value used for exposure control of the imaging means.

[0009] Specifically, the image display device according to the present invention (hereinafter, may be referred to as the "device of the present invention") is an image display device including an imaging unit, a control unit, and a display unit. The imaging unit executes imaging processing to capture images of the surroundings of the vehicle. The control unit includes an exposure adjustment unit that executes exposure adjustment processing. The exposure adjustment processing is processing to adjust the exposure level in the imaging processing so that the photometric value, which is the average brightness value of the image captured by the imaging unit, approaches a predetermined target photometric value. The display unit displays the image captured by the imaging processing at the exposure level adjusted by the exposure adjustment unit.

[0010] In the device of the present invention, the control unit further includes a peculiar area detection unit and a photometric value update unit. The peculiar region detection unit executes a peculiar region detection process. In the peculiar region detection process, a luminance distribution, which is a distribution of the number of pixels relative to luminance, is acquired for each of a plurality of photometric regions obtained by dividing an image captured by the imaging unit, and the peculiar region detection unit identifies the presence or absence of a first region and / or a second region. The first region is a photometric region in which a first pixel count, which is the number of pixels having a luminance higher than an upper limit luminance, which is the upper limit of a standard luminance range, which is a predetermined range in the luminance distribution, is equal to or greater than a first threshold, which is a predetermined threshold. On the other hand, the second region is a photometric region in which a second pixel count, which is the number of pixels having a luminance lower than a lower limit luminance, which is the lower limit of the standard luminance range, is equal to or greater than a second threshold, which is a predetermined threshold.

[0011] The photometric value update unit executes a photometric value update process when the first region and / or the second region is detected by the peculiar region detection unit. The photometric value update process is a process of updating the photometric value by increasing a first coefficient, which is a weighting coefficient for the first region in the calculation of the photometric value, and / or a second coefficient, which is a weighting coefficient for the second region.

[0012] The exposure adjustment unit is further configured to execute an exposure adjustment process based on the photometric value updated by the photometric value update unit, and the control unit is further configured to execute an image quality adjustment routine that sequentially executes a peculiar area detection process, a photometric value update process, an exposure adjustment process, and an image capture process.

[0013] In one aspect of the present invention, the control unit further includes a gamma curve adjustment unit that executes a gamma curve adjustment process that adjusts the gamma curve so as to reduce the first number of pixels and / or the second number of pixels. Furthermore, the control unit is configured to cause the gamma curve adjustment unit to execute the gamma curve adjustment process when the first region and / or the second region is detected by the peculiar region detection unit and the intensity of the input signal is within the dynamic range of the imaging unit even after the image quality adjustment routine is repeatedly executed a predetermined number of times.

[0014] In another aspect of the present invention, the control unit further includes a parameter recording unit configured to execute a parameter recording process that is a process of linking and recording environmental parameters and image quality parameters. The environmental parameters are data related to the environment in which the image quality adjustment routine is executed, and the image quality parameters are data related to the exposure level of the imaging unit updated by execution of the image quality adjustment routine. Furthermore, when an environmental parameter that matches the environmental parameter of the vehicle at the start of the image quality adjustment routine is present among the environmental parameters recorded in the parameter recording unit, the control unit is configured to read out the image quality parameter corresponding to the environmental parameter from the parameter recording unit and start the image quality adjustment routine based on the image quality parameter. Effect of the Invention

[0015] As described above, in the device of the present invention, the weighting coefficient of the photometric area having a luminance deviating from the standard luminance range is increased to update the photometric value, and the exposure level in the image capture process is adjusted so that the photometric value approaches the target photometric value. As a result, according to the device of the present invention, even if the captured image has an area having a peculiar luminance distribution in the image display device that displays an image of the periphery of the vehicle, the exposure level of the image capture means can be suitably controlled to effectively reduce blown out highlights and / or crushed shadows in the displayed image.

[0016] In one aspect of the present invention, when blown-out highlights and / or crushed shadows in a display image cannot be sufficiently reduced even by adjusting the exposure level as described above and the intensity of the input signal is within the dynamic range of the imaging unit, the first number of pixels and / or the second number of pixels are reduced by adjusting the gamma curve. As a result, according to this aspect, even when it is difficult to effectively reduce blown-out highlights and / or crushed shadows in a display image by controlling the exposure level of the imaging means, it is possible to display an output image that exhibits light and dark gradations over a wider input range.

[0017] In another aspect of the present invention, when it is determined that the vehicle is located in an environment that matches an environment in which the image quality adjustment routine was previously executed, an image is captured and displayed based on the image quality parameters updated by the image quality adjustment routine. As a result, according to this aspect, the exposure level of the imaging means can be suitably controlled to more quickly perform processing that effectively reduces blown out highlights and / or crushed shadows in the displayed image.

[0018] Other objects, features and attendant advantages of the present invention will become readily apparent from the following description of the embodiments of the present invention which are given with reference to the accompanying drawings. [Brief description of the drawings]

[0019] [Figure 1] FIG. 1 is a schematic block diagram showing an example of a configuration of an image display device (first device) according to a first embodiment of the present invention. [Diagram 2] 10 is a flowchart showing an example of a process flow including an image quality adjustment routine executed in the first device. [Diagram 3] 11 is a schematic luminance value histogram showing an example of a luminance distribution acquired by the peculiar region detection unit. [Figure 4] 10A and 10B are schematic diagrams illustrating an example of how a unique area in an image captured by an imaging unit is reduced as the image quality adjustment routine described above is executed. [Diagram 5] 10A to 10C are schematic diagrams illustrating an example of a difference between an image displayed before and after whiteout is reduced by executing an image quality adjustment routine; [Figure 6] FIG. 11 is a schematic block diagram showing an example of the configuration of an image display device (second device) according to a second embodiment of the present invention. [Figure 7] 10 is a flowchart showing an example of a process flow including an image quality adjustment routine executed in the second device. [Figure 8] 10 is a schematic graph illustrating the specific content of a gamma curve adjustment process; [Figure 9] FIG. 11 is a schematic block diagram showing an example of the configuration of an image display device (third device) according to a third embodiment of the present invention. [Figure 10] 10 is a flowchart showing an example of a process flow including an image quality adjustment routine executed in the third device illustrated in FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] First Embodiment An image display device according to a first embodiment of the present invention (hereinafter, may be referred to as a "first device") will be described below with reference to the drawings.

[0021] <composition> Fig. 1 is a schematic block diagram showing an example of the configuration of a first device. However, Fig. 1 does not show the specific structure of the first device and / or the positional relationship of each component, but merely illustrates an example of the configuration of the first device functionally for the purpose of facilitating understanding of the present invention. The first device 101 is an image display device including an imaging unit 10, a control unit 20, and a display unit 30.

[0022] The imaging unit 10 executes an imaging process to capture an image of the periphery of the vehicle. The specific configuration of the imaging unit 10 is not particularly limited as long as it is possible to capture an image of the periphery of the target vehicle. For example, the imaging unit 10 is a camera mounted on the vehicle. Typically, the camera is a digital camera.

[0023] The control unit 20 includes an exposure adjustment unit 21 that executes exposure adjustment processing. The exposure adjustment processing is a processing for adjusting an exposure level in an image capture processing so that a photometric value, which is a value of the average brightness of an image captured by the image capture unit, approaches a predetermined target photometric value. The specific configuration of the control unit 20 is not particularly limited as long as it is possible to achieve various processes and functions required for the first device 101, including the exposure adjustment processing. For example, the control unit 20 is a computer or the like equipped with a CPU, a memory, an interface, and the like. In this case, various processes and functions, including the exposure adjustment processing executed by the exposure adjustment unit 21, can be realized by the computer constituting the control unit 20. That is, the exposure adjustment unit 21 can be realized as a function achieved by the control unit 20. Details of the exposure adjustment processing are well known to those skilled in the art, so a description thereof will be omitted here.

[0024] The display unit 30 displays an image captured by the imaging process executed by the imaging unit 10 at the exposure level adjusted by the exposure adjustment unit 21. The specific configuration of the display unit 30 is not particularly limited as long as it is capable of displaying the image captured by the imaging process with suitable image quality. For example, the display unit 30 is a liquid crystal display.

[0025] In the first device 101, the control unit 20 further includes a peculiar area detection unit 22 and a photometric value update unit . The peculiar region detection unit 22 executes a peculiar region detection process. Specifically, the peculiar region detection unit 22 acquires a luminance distribution, which is a distribution of the number of pixels relative to luminance, for each of a plurality of photometric regions obtained by dividing an image captured by the imaging unit 10. The shape and size of the plurality of photometric regions can be appropriately determined according to, for example, the intended use of the captured image. Typically, the plurality of photometric regions are rectangular regions obtained by dividing the captured image vertically and horizontally. The luminance distribution is a distribution of the number of pixels relative to luminance, and is also called a "luminance value histogram."

[0026] Furthermore, the peculiar region detection unit 22 identifies the presence or absence of the first region and / or the second region. The first region is a photometric region in which the first pixel number, which is the number of pixels having a luminance higher than the upper limit luminance, which is the upper limit of the standard luminance range, which is a predetermined range in the luminance distribution, is equal to or greater than the first threshold, which is a predetermined threshold. As described above, the standard luminance range is a predetermined range in the luminance distribution. Typically, the standard luminance range corresponds to a range of luminance in which the gradation of light and dark can be distinguished by the naked eye. In this case, it is difficult or impossible to distinguish the gradation of light and dark for pixels having a luminance higher than the upper limit luminance, which is the upper limit of the standard luminance range. That is, such pixels constitute a region in which so-called "whiteout" occurs. From this viewpoint, it can be said that the first pixel number is the number of pixels constituting the part in which whiteout occurs. In this case, the first region is a photometric region in which the number of pixels constituting the part in which whiteout occurs is equal to or greater than a predetermined threshold (first threshold). The specific value of the first threshold can be appropriately determined according to, for example, the degree of whiteout permitted in the use of the captured image.

[0027] On the other hand, the second region is a photometric region in which the second pixel number, which is the number of pixels having a luminance lower than the lower limit luminance, which is the lower limit of the standard luminance range, is equal to or greater than the second threshold, which is a predetermined threshold. The standard luminance range is a predetermined range in the luminance distribution as described above, and typically corresponds to a range of luminance in which the gradation of light and dark can be distinguished by the naked eye. In this case, it is difficult or impossible to distinguish the gradation of light and dark for pixels having a luminance lower than the lower limit luminance, which is the lower limit of the standard luminance range. That is, such pixels constitute a region in which so-called "black crush" occurs. From this point of view, it can be said that the second pixel number is the number of pixels constituting the region in which black crush occurs. In this case, the second region is a photometric region in which the number of pixels constituting the region in which black crush occurs is equal to or greater than a predetermined threshold (second threshold). The specific value of the second threshold can be appropriately determined according to, for example, the degree of black crush that is allowed in the use of the captured image.

[0028] The photometric value update unit 23 executes a photometric value update process when the first region and / or the second region are detected by the peculiar region detection unit 22. The photometric value update process is a process of updating the photometric value by increasing the first coefficient, which is a weighting coefficient for the first region in the calculation of the photometric value, and / or the second coefficient, which is a weighting coefficient for the second region. In other words, the photometric value update process is a process of updating the weighting of the first region and / or the second region so that the contribution of the photometric region (first region) where whiteout occurs and / or the photometric region (second region) where blackout occurs becomes larger in the calculation of the photometric value, which is the average luminance value of the image captured by the imaging unit. This allows the luminance of the photometric region where whiteout and / or blackout occurs to be more prominently reflected in the photometric value. Specifically, when whiteout occurs, a larger photometric value is calculated when the photometric value update process is executed compared to when the photometric value update process is not executed. On the other hand, when black crush occurs, if the photometric value update process is executed, a smaller photometric value is calculated compared to when the photometric value update process is not executed.

[0029] In addition, the specific method of increasing the first coefficient, which is the weighting coefficient for the first region, and / or the second coefficient, which is the weighting coefficient for the second region, in the photometric value update process is not particularly limited. For example, the first coefficient and the second coefficient can be updated by the following formulas (1) and (2), respectively.

[0030]

number

[0031] In formulas (1) and (2), Ww is a weighting coefficient (first coefficient) for the photometric region (first region) where whiteout occurs, and Wb is a weighting coefficient (second coefficient) for the photometric region (second region) where blackout occurs. Furthermore, Nw is the number of pixels in the photometric region that have a luminance higher than the upper limit (upper limit luminance) of the standard luminance range, and Nb is the number of pixels in the photometric region that have a luminance lower than the lower limit (lower limit luminance) of the standard luminance range. Furthermore, Rw and Rb are positive coefficients.

[0032] As is clear from formula (1) and formula (2), when calculating the first coefficient Ww and the second coefficient Wb using these formulas, it is assumed that pixels having a luminance higher than the upper limit luminance and pixels having a luminance lower than the lower limit luminance coexist in one photometric region. In formula (1), the larger Nw, which is the number of pixels having a luminance higher than the upper limit luminance, the larger the first coefficient Ww becomes, and the larger Nb, which is the number of pixels having a luminance lower than the lower limit luminance, the smaller the first coefficient Ww becomes. On the other hand, in formula (2), the larger Nb, which is the number of pixels having a luminance lower than the lower limit luminance, the larger the second coefficient Wb becomes, and the larger Nw, which is the number of pixels having a luminance higher than the upper limit luminance, the smaller the second coefficient Wb becomes.

[0033] As described above, Rw and Rb are positive coefficients. Rw and Rb may be constants or variables. In the latter case, for example, Rw may be larger as Nw increases, and Rb may be larger as Nb increases.

[0034] Furthermore, the exposure adjustment unit 21 is configured to execute exposure adjustment processing based on the photometric value updated by the photometric value update unit 23. As described above, the exposure adjustment processing is processing for adjusting the exposure level in the imaging processing so that the photometric value, which is the value of the average luminance of the image captured by the imaging unit 10, approaches a predetermined target photometric value. When whiteout occurs, as described above, when the photometric value update processing is executed, a larger photometric value is calculated compared to when the photometric value update processing is not executed. In the exposure adjustment processing, in order to bring the larger photometric value closer to the target photometric value, it is necessary to lower the exposure level compared to the case of the original photometric value. In this way, the photometric value, which is the value of the average luminance of the image captured by the imaging unit 10, is lowered, and the number of pixels (first pixel number) in which whiteout occurs is reduced (i.e., whiteout is reduced).

[0035] On the other hand, when blackout occurs, as described above, when the photometric value update process is performed, a smaller photometric value is calculated compared to when the photometric value update process is not performed. In the exposure adjustment process, in order to bring the smaller photometric value closer to the target photometric value, it is necessary to increase the exposure level compared to the case of the original photometric value. In this way, the photometric value, which is the average luminance value of the image captured by the imaging unit 10, is raised, and the number of pixels where blackout occurs (the second pixel number) is reduced (i.e., blackout is reduced).

[0036] In addition, the control unit 20 is configured to execute an image quality adjustment routine, which is a routine for sequentially executing the above-mentioned peculiar area detection process, photometric value update process, exposure adjustment process, and image capture process. The control unit 20 may be configured to repeatedly execute the image quality adjustment routine until the first area and the second area are no longer detected. However, in this case, depending on the luminance distribution of the captured image, it may be necessary to repeatedly execute the image quality adjustment routine a very large number of times until the first area and the second area are no longer detected. As a result, there is a risk that it may be difficult to achieve good image quality in which blown-out highlights and / or crushed shadows are sufficiently reduced within a time allowed for the application to which the first device 101 is applied. Therefore, the control unit 20 may be configured to end the image quality adjustment routine after repeatedly executing the routine a predetermined number of times.

[0037] 2 is a flowchart showing an example of a process flow including an image quality adjustment routine executed in first device 101. A CPU constituting control unit 20 can realize various functions including the process shown in the flowchart by executing instructions (programs) stored in memory (ROM).

[0038] For example, the first device 101 is started up in step S101 in response to a user operation such as a gear change to reverse range (R range) by the driver of the vehicle in which the first device 101 is mounted and operation of the start switch of the first device 101, and an image of the surroundings of the vehicle is captured by the imaging unit 10 (imaging processing is executed).

[0039] Next, in step S102, the peculiar region detection unit 22 included in the control unit 20 executes a peculiar region detection process. Specifically, the peculiar region detection unit 22 first acquires a luminance distribution, which is a distribution of the number of pixels relative to luminance, for each of a plurality of photometric regions obtained by dividing the image captured by the imaging unit 10. FIG. 3 is a schematic luminance value histogram showing an example of the luminance distribution acquired by the peculiar region detection unit 22, in which the horizontal axis indicates luminance and the vertical axis indicates the number of pixels exhibiting the luminance indicated by the horizontal axis. The region indicated by the diagonal line slanting upward to the right at the right end of this luminance value histogram represents a range of luminance higher than the upper limit (upper limit luminance) of the above-mentioned standard luminance range. On the other hand, the region indicated by the diagonal line slanting downward to the right at the left end of this luminance value histogram represents a range of luminance lower than the lower limit (lower limit luminance) of the standard luminance range. Therefore, the region of this luminance value histogram that is not shaded represents the standard luminance range.

[0040] In the image corresponding to the brightness value histogram illustrated in Fig. 3, there is a photometric region (first region) where the first pixel number, which is the number of pixels having a brightness higher than the upper limit brightness, is equal to or greater than the first threshold, which is a predetermined threshold. That is, there is a photometric region in which whiteout occurs in the image. On the other hand, there is no photometric region (second region) in which the second pixel number, which is the number of pixels having a brightness lower than the lower limit brightness, is equal to or greater than the second threshold, which is a predetermined threshold. That is, there is no photometric region in which blackout occurs in the image.

[0041] Next, in step S103, it is determined whether or not a peculiar region (i.e., the first region and / or the second region) is detected by the peculiar region detection unit 22. If no peculiar region is detected, (the CPU that realizes the function of) the peculiar region detection unit 22 determines "No", and the process proceeds to step S108, where the image captured by the imaging unit 10 is displayed by the display unit 30.

[0042] On the other hand, if a peculiar region exists as in the luminance value histogram illustrated in Fig. 3, the peculiar region detection unit 22 determines "Yes" and the process proceeds to the next step S104. In step S104, the photometric value update unit 23 executes a photometric value update process. Specifically, the photometric value update unit 23 updates the photometric value by increasing the first coefficient, which is a weighting coefficient for the first region, and / or the second coefficient, which is a weighting coefficient for the second region, in calculating the average luminance value (photometric value) of the image captured by the imaging unit.

[0043] Next, in step S105, the exposure adjustment unit 21 executes exposure adjustment processing based on the photometric value updated by the photometric value update unit 23. As described above, the exposure adjustment processing is processing for adjusting the exposure level in the imaging processing executed by the imaging unit 10 so that the average luminance value (photometric value) of the image captured by the imaging unit 10 approaches a predetermined target photometric value.

[0044] Next, in step S106, the imaging unit 10 executes imaging processing to capture an image of the periphery of the vehicle. In this imaging processing, the exposure level adjusted in step S105 based on the photometric value updated in the above-mentioned step S104 is used.

[0045] Steps S102 to S106 surrounded by a thick solid line in Fig. 2 correspond to the peculiar area detection process, the photometric value update process, the exposure adjustment process, and the image capture process as described above, and constitute an image quality adjustment routine that sequentially executes these processes. The flow of the process shown in the flowchart of Fig. 2 is configured to end the image quality adjustment routine after the image quality adjustment routine has been repeatedly executed a predetermined number of times. Specifically, following step S106 in which the image capture process has been executed, in step S107 it is determined whether the image quality adjustment routine has been executed a predetermined number of times.

[0046] If the image quality adjustment routine has already been executed a predetermined number of times, the control unit 20 (or a CPU constituting the control unit 20) judges "Yes" in step S107, the process proceeds to the next step S108, and the image captured by the imaging unit 10 is displayed on the display unit 30 in step S106. On the other hand, if the image quality adjustment routine has not yet been executed a predetermined number of times, the control unit 20 judges "No" in step S107, and the process returns to the above-mentioned step S102.

[0047] Fig. 4 is a schematic diagram illustrating a state in which the peculiar area in the image captured by the imaging unit 10 is reduced as the image quality adjustment routine described above is executed. In the example shown in Fig. 4, the image is divided into 25 photometry areas. Fig. 4(a) is a schematic diagram of the image captured by the imaging unit 10 in the above-mentioned step S101, and a predetermined weighting coefficient is assigned to each photometry area. As a result, whiteout occurs in the photometry area surrounded by the thick dashed line.

[0048] Therefore, the presence of a peculiar region (in this case, the first region, which is the photometric region where whiteout occurs) is detected based on the luminance distribution acquired by the peculiar region detection process executed in step S102 described above. As a result, the next step S103 is judged as "Yes," and photometric value update processing, exposure adjustment processing, and imaging processing are executed in steps S104 to S106 (i.e., the image quality adjustment routine is executed). At this point, the image quality adjustment routine has only been executed once, and the predetermined number of times has not yet been reached. Therefore, the step S107 is judged as "No," and the process returns to step S102, where the peculiar region detection process is executed.

[0049] 4B is a schematic diagram of an image obtained as a result of executing the image quality adjustment routine as described above. Although the number of unique regions (first regions) has been reduced by executing the image quality adjustment routine, blown-out highlights still occur in the photometry region surrounded by the thick dashed line. Therefore, the next step S103 is judged as "Yes" again, and photometry value update processing, exposure adjustment processing, and image capture processing are executed in steps S104 to S106 (the image quality adjustment routine is executed).

[0050] 4(c) is a schematic diagram of an image obtained as a result of executing the image quality adjustment routine again as described above. By executing the image quality adjustment routine again, the peculiar region (first region) is no longer detected. In other words, an image exhibiting good image quality without any photometry region where whiteout occurs is obtained. Therefore, this time, the determination in step S103 is "No," and the process proceeds to step S108, where the image is displayed by the display unit 30.

[0051] Fig. 5 is a schematic diagram illustrating the difference between images displayed before and after whiteout is reduced by executing an image quality adjustment routine. The image shown in Fig. 5 represents an image viewed from above a vehicle parked in a typical convenience store parking lot, displayed by an image display device mounted on the vehicle. (a) is the image before the image quality adjustment routine is executed, and (b) is the image after the image quality adjustment routine is executed.

[0052] In the image before the image quality adjustment routine is executed, as shown in (a), the light from the store's interior lighting shining through the glass window in front of the store causes whiteout, and the situation behind the vehicle cannot be clearly seen. Specifically, for example, the boundary between the white lines indicating each parking space and the road surface is unclear. On the other hand, in the image after the image quality adjustment routine is executed, as shown in (b), the whiteout behind the vehicle is reduced and the boundary between the white lines indicating the parking space and the road surface can be clearly seen. This is because, as described above, the exposure level set based on the increased average luminance value (photometric value) by increasing the weighting coefficient of the photometric area where whiteout occurs is reduced, and as a result, it is possible to generate light and dark gradations even in the area where whiteout occurs.

[0053] <effect> As described above, in the first device, the weighting coefficient of the photometric area having a luminance deviating from the standard luminance range is increased to update the average luminance value (photometric value), and the exposure level in the imaging process is adjusted so that the photometric value approaches the target photometric value. As a result, according to the first device, even if the captured image has an area having a peculiar luminance distribution in the image display device that displays an image around the vehicle, the exposure level of the imaging means can be suitably controlled to effectively reduce blown out highlights and / or crushed shadows in the displayed image.

[0054] Second Embodiment An image display device according to a second embodiment of the present invention (hereinafter, may be referred to as a "second device") will be described below with reference to the drawings.

[0055] As described above, the control unit included in the image display device (the device of the present invention) according to the present invention, including the first device, may be configured to repeatedly execute the image quality adjustment routine until the first and second regions are no longer detected. However, in this case, depending on the luminance distribution of the captured image, it may be necessary to repeatedly execute the image quality adjustment routine a very large number of times until the first and second regions are no longer detected. As a result, it may be difficult to achieve good image quality with sufficient reduction in blown-out highlights and / or crushed shadows within an allowable time for the intended use. Therefore, the control unit may be configured to end the image quality adjustment routine after repeatedly executing the routine a predetermined number of times.

[0056] However, with only the above configuration, there is a possibility that blown-out highlights and / or crushed shadows are not sufficiently reduced even after the image quality adjustment routine is repeatedly executed a predetermined number of times. Therefore, in the second device, the control unit is configured to reduce the first number of pixels and / or the second number of pixels by adjusting the gamma curve if blown-out highlights and / or crushed shadows in the displayed image are not sufficiently reduced even after the exposure level is adjusted by executing the image quality adjustment routine. However, in order to reduce the first number of pixels and / or the second number of pixels by adjusting the gamma curve, the intensity of the input signal needs to be within the dynamic range of the imaging unit.

[0057] <composition> FIG. 6 is a schematic block diagram showing an example of the configuration of the second device. However, like FIG. 1, FIG. 6 does not show the specific structure of the second device and / or the positional relationship of each component, but merely illustrates an example of the configuration of the second device functionally for the purpose of facilitating understanding of the present invention. The second device 102 illustrated in FIG. 6 is an image display device having a configuration similar to that of the first device 101 illustrated in FIG. 1, except that the control unit 20 further includes a gamma curve adjustment unit 24. Therefore, in the following description, the gamma curve adjustment process executed by the gamma curve adjustment unit 24 will be mainly described.

[0058] That is, the second device 102 is an image display device according to the present invention including the above-mentioned first device 101, and is an image display device in which the control unit 20 further includes a gamma curve adjustment unit 24 that executes gamma curve adjustment processing. The gamma curve adjustment processing is processing for adjusting the gamma curve so as to reduce the first pixel number and / or the second pixel number. The gamma curve adjustment processing will be described in detail later.

[0059] Furthermore, in the second device 102, if the first region and / or the second region is detected by the peculiar region detection unit 22 and the intensity of the input signal is within the dynamic range of the imaging unit even after the image quality adjustment routine has been repeatedly executed a predetermined number of times, the control unit 20 is configured to cause the gamma curve adjustment unit 24 to execute gamma curve adjustment processing. The gamma curve adjustment processing executed by the gamma curve adjustment unit 24 can also be realized by a computer constituting the control unit 20. In other words, the gamma curve adjustment unit 24 can be realized as a function achieved by the control unit 20.

[0060] Fig. 7 is a flowchart showing an example of a process flow including an image quality adjustment routine executed in second device 102. The flowchart shown in Fig. 7 is similar to the flowchart shown in Fig. 2 referred to in the description of first device 101, except that steps S111 to S114 are added between steps S107 and S108. Therefore, the following description will mainly focus on the processes executed after step S107.

[0061] As described for the first device 101, when the image quality adjustment routine is executed by executing steps S102 to S106, it is determined in step S107 whether the image quality adjustment routine has been executed a predetermined number of times. If the image quality adjustment routine has been repeatedly executed a predetermined number of times, the control unit 20 (the CPU constituting the control unit 20) determines "Yes" in step S107. Next, in the first device 101, the process proceeds to step S108 as described above, and an image is displayed by the display unit 30. However, in the second device 102, the process proceeds to steps S111 and S112, and the unique area detection process is executed by the unique area detection unit 22. These steps are similar to the above-mentioned steps S102 and S103, so their description will be omitted here.

[0062] If no unique region exists at this point, the result of step S112 is determined as "No" and the process proceeds to step S108, where the image captured by the imaging unit 10 is displayed by the display unit 30. On the other hand, if a unique region exists, the result of step S112 is determined as "Yes" and the process proceeds to the next step S113. In step S113, it is determined whether or not the intensity of the input signal of the image captured by the imaging unit 10 falls within the dynamic range of the imaging unit 10 (or the light receiving unit such as an image sensor provided therein).

[0063] If the intensity of the input signal is not within the dynamic range of the imaging unit 10, it is not possible to reduce blown out highlights and / or crushed shadows by the gamma curve adjustment process, so a "No" determination is made in step S113, the process proceeds to step S108, and the image is displayed by the display unit 30. On the other hand, if the intensity of the input signal is within the dynamic range of the imaging unit 10, a "Yes" determination is made in step S113, the process proceeds to step S114, and the gamma curve adjustment process is executed.

[0064] 8 is a schematic graph illustrating the change in the gamma curve before and after the gamma curve adjustment process. The horizontal axis of the graph indicates the input, and the vertical axis indicates the output. The dashed curve indicates the gamma curve before the gamma curve adjustment process is performed, and the solid curve indicates the gamma curve after the gamma curve adjustment process is performed.

[0065] It can be seen that the output is saturated and highlights are generated near the upper limit of the input of the gamma curve (dashed line) before the gamma curve adjustment process is executed (area surrounded by a thick dotted line). Therefore, by the gamma curve adjustment process executed in step S114, the gamma curve is shifted to the right as shown by the hollow arrow, and the light and dark gradations are achieved even in the area where highlights were generated. In this way, the gamma curve adjustment process is a process in which so-called "gamma correction" is performed so that the output exhibits light and dark gradations over a wider range of inputs (preferably the entire range of inputs) (within the dynamic range of the imaging means).

[0066] When the gamma curve adjustment process is executed in step S114 as described above, the process proceeds to the next step S108, and an image with good image quality, free of any photometric regions with blown-out highlights, is displayed on the display unit 30.

[0067] <effect> As described above, in the second device, when blown-out highlights and / or crushed shadows in the display image cannot be sufficiently reduced even if the exposure level is adjusted by executing the image quality adjustment routine and the intensity of the input signal is within the dynamic range of the imaging unit, the first pixel number and / or the second pixel number are reduced by executing a gamma curve adjustment process. As a result, according to the second device, even when it is difficult to effectively reduce blown-out highlights and / or crushed shadows in the display image by controlling the exposure level by executing the image quality adjustment routine, it is possible to display an output image that exhibits light and dark gradations over a wider input range.

[0068] Third Embodiment An image display device according to a third embodiment of the present invention (hereinafter, may be referred to as a "third device") will be described below with reference to the drawings.

[0069] As described above, in the first device, the weighting coefficient of the photometric area having a luminance deviating from the standard luminance range is increased to update the average luminance value (photometric value), and the exposure level in the imaging process is adjusted so that the photometric value approaches the target photometric value. As a result, according to the first device, even if the captured image has an area having a peculiar luminance distribution in the image display device that displays an image around the vehicle, the exposure level of the imaging means can be suitably controlled to effectively reduce blown out highlights and / or crushed shadows in the displayed image.

[0070] In the second device, when blown-out highlights and / or crushed shadows in the display image cannot be sufficiently reduced even if the exposure level is adjusted by executing the image quality adjustment routine and the intensity of the input signal is within the dynamic range of the imaging unit, the first pixel number and / or the second pixel number are reduced by executing a gamma curve adjustment process. As a result, according to the second device, even when it is difficult to effectively reduce blown-out highlights and / or crushed shadows in the display image by controlling the exposure level by executing the image quality adjustment routine, it is possible to display an output image that exhibits light and dark gradations over a wider input range.

[0071] Incidentally, for example, if a vehicle is in the same environment as one in which an image quality adjustment routine has been executed in the past, if it is possible to utilize the exposure level, etc. updated by the "previously executed image quality adjustment routine," the process of reducing white blowout and / or black crush in the displayed image can be performed more quickly.

[0072] <composition> FIG. 9 is a schematic block diagram showing an example of the configuration of the third device. However, like FIG. 1 and FIG. 6, FIG. 9 does not show the specific structure of the third device and / or the positional relationship of each component, but merely illustrates an example of the configuration of the third device functionally for the purpose of facilitating understanding of the present invention. The third device 103 illustrated in FIG. 9 is an image display device having a configuration similar to that of the second device 102 illustrated in FIG. 6, except that the control unit 20 further includes a parameter recording unit 25. Therefore, in the following description, the process related to the parameter recording unit 25 will be mainly described.

[0073] That is, the third device 103 is an image display device according to the present invention including the above-mentioned first device 101 and second device 102, and is an image display device in which the control unit 20 further includes a parameter recording unit 25 configured to execute a parameter recording process that is a process of linking and recording environmental parameters and image quality parameters. The parameter recording unit 25 can also be realized by a computer constituting the control unit 20. That is, the parameter recording unit 25 can be realized as a function achieved by the control unit 20. In addition, the environmental parameters and image quality parameters that are linked to each other and recorded by the parameter recording unit 25 can be stored in a data storage means such as a memory (RAM), a hard disk drive (HDD), or a solid state drive (SSD) in the form of, for example, a data table or a data map.

[0074] The environmental parameters are data related to the environment in which the image quality adjustment routine is executed. Specifically, the environmental parameters are data that can specify the altitude (elevation angle) and direction of the sun at the position and date and time when the image quality adjustment routine is executed. More specifically, the environmental parameters are data including, for example, the position and orientation of the vehicle on which the third device is mounted, as well as the date and time. Such data can be acquired, for example, by using a global positioning system (GPS) or the like.

[0075] Furthermore, the environmental parameters may include data that affect the intensity of sunlight. A specific example of such data may be weather data provided by the Japan Meteorological Agency or the like. In this case, the weighting coefficients applied may be changed according to the weather, such as fine, cloudy, rainy, etc. Specifically, the weighting coefficient (first coefficient) of the photometry region (first region) where whiteout occurs may be changed to "fine > cloudy > rainy", and the weighting coefficient (second coefficient) of the photometry region (second region) where blackout occurs may be changed to "fine < cloudy < rainy".

[0076] The image quality parameters are data related to the exposure level of the imaging unit 10 updated by the execution of the image quality adjustment routine. The image quality parameters may be the exposure level of the imaging unit 10 itself set as a result of the execution of the image quality adjustment routine. Alternatively, the image quality parameters may be data capable of specifying the exposure level of the imaging unit 10 set as a result of the execution of the image quality adjustment routine, such as a photometric value which is a value of the average luminance of an image captured by the imaging unit 10, a position and a first coefficient of a first region in the image, and a position and a second coefficient of a second region. In addition, when a means for detecting the illuminance around the vehicle, such as an illuminance sensor, is available, the image quality parameters may include illuminance.

[0077] In the third device 103 illustrated in Fig. 9, the control unit 20 includes a gamma curve adjustment unit 24. That is, the third device 103 has a configuration corresponding to the second device 102 described above. In this case, the image quality parameters may include a gamma curve adjusted by executing a gamma curve adjustment process. Naturally, this does not apply when the control unit 20 does not include the gamma curve adjustment unit 24 as in the first device 101 described above.

[0078] Furthermore, in the third device 103, if an environmental parameter that matches the environmental parameter of the vehicle at the start of the image quality adjustment routine is present among the environmental parameters recorded in the parameter recording unit, the control unit 20 is configured to read out an image quality parameter corresponding to that environmental parameter from the parameter recording unit 25 and start the image quality adjustment routine based on that image quality parameter.

[0079] Fig. 10 is a flowchart showing an example of a process flow including an image quality adjustment routine executed in the third device 103 illustrated in Fig. 9. The flowchart shown in Fig. 10 is similar to the flowchart shown in Fig. 7 referred to in the description of the second device 102, except that steps S121 to S123 are added between steps S101 and S102, and step S124 is added between steps S114 and S108. Therefore, the following description will mainly focus on the processes executed in these added steps.

[0080] In step S101, the first device 101 is started and the imaging unit 10 captures an image of the periphery of the vehicle (imaging process is executed). In the next step S121, the control unit 20 acquires environmental data using the GPS or the like as described above. Next, in step S122, it is determined whether or not an environmental parameter that matches the acquired environmental parameter is present among the environmental parameters recorded in the parameter recording unit 25.

[0081] If there is no environmental parameter that matches the acquired environmental parameter among the environmental parameters recorded in parameter recording unit 25, "No" is determined in step S122, and the process proceeds to step S102, where the singular area detection process is executed. The flow of the process thereafter is the same as that of the flowchart exemplified in Fig. 7. However, in the third device 103, before the image is displayed by the display unit 30 in step S108, the environmental parameter and the image quality parameter are recorded in association with each other by the parameter recording process executed by parameter recording unit 25 in step S124.

[0082] On the other hand, if there is an environmental parameter that matches the acquired environmental parameter among the environmental parameters recorded in parameter recording unit 25, then "Yes" is determined in step S122, and processing proceeds to step S123. In step S123, image quality parameters corresponding to the environmental parameters acquired in step S121 are read from parameter recording unit 25, and the image quality parameters are reflected (applied) in the image quality adjustment routine executed in the next step S102 and thereafter.

[0083] The image quality parameters read out from the parameter recording unit 25 in the above manner are image quality parameters updated by an image quality adjustment routine executed in the past. Therefore, typically, the peculiar area detection process executed in step S102 should not detect peculiar areas (i.e., the photometry area (first area) where whiteout occurs and the photometry area (second area) where blackout occurs). In this case, the next step S103 is judged as "No", the process proceeds to step S124, the parameter recording unit 25 executes the parameter recording process, and the image is displayed by the display unit 30 in the next step S108. If the image quality parameters have not been updated as described above, only the environmental parameters may be updated in step S124.

[0084] Incidentally, even if environmental parameters that match the acquired environmental parameters exist among the environmental parameters recorded in parameter recording unit 25, the luminance distribution in the image captured by imaging unit 10 may change due to, for example, a change in the buildings around the vehicle. In such a case, a peculiar area may be detected in the next step S102 despite the application of the image quality parameters read out from parameter recording unit 25 in step S123 (i.e., image quality parameters updated by an image quality adjustment routine executed in the past).

[0085] If a unique region is detected in step S102 as described above, then the answer is "Yes" in step S103, and processing proceeds to the next step S104, where a photometric value update process is executed. The flow of processing thereafter is the same as that shown in the flowchart of Fig. 7, and in step S124, the environmental parameters and the updated image quality parameters are recorded in association with each other by a parameter recording process executed by parameter recording unit 25.

[0086] As described above, in the third device 103 illustrated in Fig. 9, the control unit 20 includes the gamma curve adjustment unit 24. Therefore, in the flowchart illustrated in Fig. 10, steps S111 to S114 are executed between steps S107 and S124. Therefore, the image quality parameters may include a gamma curve adjusted by the gamma curve adjustment process executed in step S114. However, it goes without saying that, if the control unit 20 does not include the gamma curve adjustment unit 24 like the first device 101 described above, steps S111 to S114 are not executed.

[0087] <effect> As described above, in the third device, when it is determined that the vehicle is located in an environment that matches the environment in which the image quality adjustment routine was previously executed, an image is captured and displayed based on the image quality parameters by the image quality adjustment routine. As a result, the third device can more quickly perform processing to effectively reduce blown-out highlights and / or crushed shadows in the displayed image by suitably controlling the exposure level of the imaging means.

[0088] For the purpose of explaining the present invention, several embodiments and modifications having specific configurations have been described above, sometimes with reference to the attached drawings. However, the scope of the present invention should not be construed as being limited to these exemplary embodiments and modifications, and it goes without saying that appropriate modifications can be made within the scope of the claims and the matters described in the specification. [Explanation of symbols]

[0089] 10 imaging section, 20 control section, 21 exposure adjustment section, 22 peculiar area detection section, 23 photometric value update section, 24 gamma curve adjustment section, 25 parameter recording section, 30 display section, 101, 102, 103 image display device.

Claims

1. An imaging unit that performs imaging processing to capture an image of the periphery of the vehicle; a control unit including an exposure adjustment unit that executes an exposure adjustment process that adjusts an exposure level in the image capture process so that a photometric value, which is an average luminance value of an image captured by the image capture unit, approaches a predetermined target photometric value; a display unit that displays an image captured by the imaging process at the exposure level adjusted by the exposure adjustment unit; An image display device comprising: The control unit is a peculiar region detection unit that executes a peculiar region detection process that acquires a luminance distribution, which is a distribution of the number of pixels relative to luminance, for each of a plurality of photometric regions obtained by dividing an image captured by the imaging unit, and identifies the presence or absence of a first region, which is a photometric region in which a first number of pixels, which is the number of pixels having a luminance higher than an upper limit luminance that is the upper limit of a standard luminance range that is a predetermined range in the luminance distribution, is equal to or greater than a first threshold that is a predetermined threshold, and / or a second region, which is a photometric region in which a second number of pixels, which is the number of pixels having a luminance lower than a lower limit luminance that is the lower limit of the standard luminance range, is equal to or greater than a second threshold that is a predetermined threshold; a photometric value update unit that executes a photometric value update process that updates the photometric value by increasing a first coefficient that is a weighting coefficient for the first region in calculating the photometric value when the first region is detected by the peculiar region detection unit, and by increasing a second coefficient that is a weighting coefficient for the second region in calculating the photometric value when the second region is detected by the peculiar region detection unit; Further comprising: the exposure adjustment unit is configured to execute the exposure adjustment process based on the photometric value updated by the photometric value update unit, the control unit is configured to execute an image quality adjustment routine which is a routine for sequentially executing the peculiar area detection process, the photometric value update process, the exposure adjustment process, and the imaging process.

1. An image display device comprising:

Images