Imaging device

Abstract

To provide an imaging device capable of accurately suppressing streaking by a simple structure.SOLUTION: An imaging device includes: an imaging unit that is arranged at an effective area for sensing light and at least one left or right end of said effective area and outputs video signals with a horizontal light-shielded area; a horizontal light shielding area smoothing unit that detects a black level sinking amount in the horizontal light shielding area from the video signals; a compensation value calculation unit that calculates a compensation value by detecting compensation parameters from the video signals; and a correction value calculation unit that calculates a correction value for correcting the video signals for suppressing a streaking phenomenon by using the black level sinking amount from the horizontal light shielding area smoothing unit and the compensation value from the compensation value calculation unit. The compensation parameter uses a value of the horizontal width of the signal with light intensity above a certain level.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to an imaging device, and more particularly to an imaging device that reduces the influence of streaking that occurs in an image sensor or the like. [Background technology]

[0002] In the past, when strong light was incident on image sensors such as CCD (Charge Coupled Device) image sensors or CMOS (Complementary Metal Oxide Semiconductor) image sensors, horizontal lines not present in the image (streaking) could appear. This is due to the fact that in recent years, as image sensors have become larger and higher pixel count, the power supply lines connecting each pixel from the power supply unit have become thinner and longer. This causes a voltage drop due to the internal resistance of the power supply line, resulting in differences in the voltage supplied to each pixel depending on their position from the power supply unit. These factors are said to be the causes of streaking.

[0003] To address this phenomenon, various methods have been proposed, generally using signals from the light-shielded area (optical black area) of the image sensor to correct signals from the effective area.

[0004] For example, Patent Document 1 proposes a method of generating a streaking correction signal by subtracting the black level in the vertical shading section (vertical OB section) from the signal level of each line in the horizontal shading section (horizontal OB section) of the image sensor, and correcting streaking by subtracting the streaking correction signal from the signal in the effective pixel area.

[0005] Furthermore, Patent Document 2 proposes a method of dividing an effective pixel area into a plurality of blocks and generating a streaking correction signal for each block. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Laid-Open No. 2008-236271 [Patent Document 2] Japanese Patent Publication No. 2020-17910 Summary of the Invention [Problem to be solved by the invention]

[0007] However, the method in Patent Document 1 simply subtracts the average value of the horizontal shading area from the effective pixel area. Therefore, it cannot address the increase in streaking caused by changes in the internal resistance of the power line, which occurs as the position where the high-brightness signal is incident moves away from the horizontal shading area. Furthermore, because the amount of streaking changes depending on the horizontal width of the incident light, the streaking correction signal may be excessive or insufficient, resulting in correction errors remaining in the video signal.

[0008] Furthermore, in the method of Patent Document 2, in image sensors with a high pixel count and a large size, managing streaking correction signals for each of a large number of divided blocks results in a large-scale and complex circuit configuration of the imaging device.

[0009] In view of the above-mentioned problems, an object of the present invention is to provide an imaging device that can accurately suppress streaking with a simple configuration. [Means for solving the problem]

[0010] In order to achieve the above object, one representative imaging device of the present invention includes an imaging element having an effective area for sensing light and a horizontal light-shielded area arranged at least at one of the left and right ends of the effective area and for outputting a video signal, a horizontal light-shielded area smoothing unit that detects from the video signal an amount of black level depression in the horizontal light-shielded area, a compensation value calculation unit that detects a compensation parameter from the video signal and calculates a compensation value, and a correction value calculation unit that calculates a correction value for correcting the video signal to suppress streaking using the amount of black level depression from the horizontal light-shielded area smoothing unit and the compensation value from the compensation value calculation unit, wherein the compensation parameter is a value that is used to correct a signal of light intensity equal to or greater than a predetermined value. the distance between the horizontal shading area and the positionThe value of [Effects of the Invention]

[0011] According to the present invention, in an imaging device, streaking can be suppressed with high precision using a simple configuration. Problems, configurations, and effects other than those described above will become apparent from the following embodiments. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a block diagram of a computer system for implementing aspects according to an embodiment of the present disclosure. [Figure 2] FIG. 2 is a diagram showing the configuration of an embodiment of the imaging device of the present invention. [Figure 3] FIG. 3 shows a first example of a light receiving section of an image sensor that can be used in the imaging device of the present invention, and is a diagram showing a light receiving section of an image sensor that has a horizontal light-shielding region on the left edge. [Figure 4] FIG. 4 shows a second example of a light receiving section of an imaging element that can be used in the imaging device of the present invention, and is a diagram showing a light receiving section of an imaging element that has horizontal light-shielding regions on both ends. [Figure 5] FIG. 5 is a diagram showing an example of the configuration of the streaking correction unit in the imaging device of the present invention. [Figure 6] FIG. 6 is a diagram illustrating a first example of when streaking occurs in the image sensor shown in FIG. 3, where (A) is a diagram showing the light receiving surface of the image sensor, and (B) is a diagram showing the relationship between the coordinate y and the pixel value I at each position of the coordinate x. [Figure 7] FIG. 7 is a diagram illustrating the correction by the imaging device of the present invention for the first example of FIG. 6, where (C-1) shows the state of the streaking signal before correction, and (C-2) shows the state after correction. [Figure 8] FIG. 8 is a diagram illustrating a second example of when streaking occurs in the image sensor shown in FIG. 3, where (A) is a diagram showing the light receiving surface of the image sensor, and (B) is a diagram showing the relationship between the coordinate y and the pixel value I at each position of the coordinate x. [Figure 9]FIG. 9 is a diagram for explaining correction by the imaging device of the present invention for the second example of FIG. 8, where (C-1) shows the state of the streaking signal before correction, (C-2) shows the state after correction without compensation, and (C-3) shows the state after correction with compensation. [Figure 10] FIG. 10 is a diagram illustrating a third example of streaking occurring in the image sensor shown in FIG. 3, where (A) is a diagram showing the light receiving surface of the image sensor, and (B) is a diagram showing the relationship between the coordinate y and the pixel value I at each position of the coordinate x. [Figure 11] FIG. 11 is a diagram for explaining the correction by the imaging device of the present invention for the third example of FIG. 10, where (C) shows the state of the streaking signal before correction, (C-2) shows the state after correction without compensation, and (C-3) shows the state after correction with compensation. DETAILED DESCRIPTION OF THE INVENTION

[0013] An embodiment of the present invention will be described with reference to the drawings.

[0014] <Computer system for implementing aspects according to the embodiment> 1 is a block diagram of a computer system 300 for implementing aspects according to an embodiment of the present disclosure. The mechanisms and devices of various embodiments disclosed herein may be applied to any suitable computing system. The main components of the computer system 300 include one or more processors 302, memory 304, a terminal interface 312, a storage interface 314, an I / O (input / output) device interface 316, and a network interface 318. These components may be interconnected via a memory bus 306, an I / O bus 308, a bus interface unit 309, and an I / O bus interface unit 310.

[0015] Computer system 300 may include one or more processing units 302A and 302B, collectively referred to as processors 302. Each processor 302 executes instructions stored in memory 304 and may include an on-board cache. In some embodiments, computer system 300 may include multiple processors, while in other embodiments, computer system 300 may be a single processing unit system. The processing unit may be a central processing unit (CPU), a field-programmable gate array (FPGA), a graphics processing unit (GPU), a digital signal processor (DSP), or the like.

[0016] In some embodiments, memory 304 may include random-access semiconductor memory, storage devices, or storage media (either volatile or nonvolatile) for storing data and programs. In some embodiments, memory 304 represents the entire virtual memory of computer system 300 and may include virtual memory of other computer systems connected to computer system 300 via a network. While memory 304 may be conceptually considered a single entity, in other embodiments, memory 304 may be a more complex organization, such as a hierarchy of caches and other memory devices. For example, memory may exist as multiple levels of caches, and these caches may be divided by function. As a result, one cache may hold instructions, while other caches hold non-instruction data used by the processor. Memory may also be distributed and associated with various different processing units, such as in a so-called Non-Uniform Memory Access (NUMA) computer architecture.

[0017] Memory 304 may store all or a portion of the programs, modules, and data structures that implement the functions described herein. For example, memory 304 may store latent factor identification application 350. In some embodiments, latent factor identification application 350 may include instructions or descriptions that execute the functions described below on processor 302, or may include instructions or descriptions that are interpreted by other instructions or descriptions. In some embodiments, latent factor identification application 350 may be implemented in hardware via semiconductor devices, chips, logic gates, circuits, circuit cards, and / or other physical hardware devices instead of or in addition to a processor-based system. In some embodiments, latent factor identification application 350 may include data other than instructions or descriptions. In some embodiments, a camera, sensor, or other data input device (not shown) may be provided to communicate directly with bus interface unit 309, processor 302, or other hardware in computer system 300. Such a configuration may reduce the need for processor 302 to access memory 304 and the latent factor identification application.

[0018] Computer system 300 may include a bus interface unit 309 that facilitates communication between processor 302, memory 304, display system 324, and I / O bus interface unit 310. I / O bus interface unit 310 may couple to I / O bus 308 for transferring data to and from various I / O units. I / O bus interface unit 310 may communicate with multiple I / O interface units 312, 314, 316, and 318, also known as I / O processors (IOPs) or I / O adapters (IOAs), via I / O bus 308. Display system 324 may include a display controller, a display memory, or both. The display controller may provide video, audio, or both data to display device 326. Computer system 300 may also include one or more sensors or other devices configured to collect data and provide the data to processor 302. For example, computer system 300 may include environmental sensors that collect humidity data, temperature data, pressure data, etc., and motion sensors that collect acceleration data, movement data, etc. Other types of sensors may also be used. Display memory may be dedicated memory for buffering video data. Display system 324 may be connected to a display device 326, such as a standalone display screen, a television, a tablet, or a mobile device. In some embodiments, display device 326 may include a speaker for rendering audio. Alternatively, the speaker for rendering audio may be connected to an I / O interface unit. In other embodiments, the functionality provided by display system 324 may be implemented by an integrated circuit that includes processor 302. Similarly, the functionality provided by bus interface unit 309 may be implemented by an integrated circuit that includes processor 302.

[0019] The I / O interface unit provides functionality for communicating with various storage or I / O devices. For example, the terminal interface unit 312 may be attached to user I / O devices 320, such as user output devices such as a video display, a television with speakers, and user input devices such as a keyboard, a mouse, a keypad, a touchpad, a trackball, buttons, a light pen, or other pointing device. A user may use a user interface to enter input data or instructions into the user I / O devices 320 and the computer system 300 and receive output data from the computer system 300 by operating the user input devices. The user interface may be displayed on a display, played through speakers, or printed via a printer via the user I / O devices 320, for example.

[0020] Storage interface 314 may be attached to one or more disk drives or direct access storage devices 322 (typically magnetic disk drive storage devices, but may also be an array of disk drives or other storage devices configured to appear as a single disk drive). In some embodiments, storage device 322 may be implemented as any secondary storage device. Contents of memory 304 may be stored in storage device 322 and retrieved from storage device 322 as needed. Network interface 318 may provide a communications path that allows computer system 300 and other devices to communicate with each other. This communications path may be, for example, network 330.

[0021] 1 includes a bus structure providing direct communication paths between processor 302, memory 304, bus interface 309, display system 324, and I / O bus interface unit 310; however, in other embodiments, computer system 300 may include point-to-point links, multiple hierarchical buses, parallel or redundant communication paths in a hierarchical, star, or web configuration. Furthermore, while I / O bus interface unit 310 and I / O bus 308 are shown as a single unit, computer system 300 may actually include multiple I / O bus interface units 310 or multiple I / O buses 308. Additionally, while multiple I / O interface units are shown isolating I / O bus 308 from various communication paths leading to various I / O devices, in other embodiments, some or all of the I / O devices may be directly connected to a single system I / O bus.

[0022] <Configuration of the imaging device> 2 is a block diagram showing an embodiment of an imaging device of the present invention, which includes an imaging element 1, a streaking correction unit 2, an image adjustment unit 3, and an output generation unit 4.

[0023] The imaging element 1 can be an image sensor such as a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor. Here, the image sensor receives light from an object through an optical system such as a lens, forms an image on a light receiving surface, and converts the image photoelectrically into an electrical signal. For example, a two-dimensional image sensor arranged in a plane can be used.

[0024] The streaking correction unit 2 performs a process to correct streaking. Streaking is a horizontal line that appears in the horizontal direction (x direction) of a captured image, and is a type of noise. The streaking correction unit 2 compensates for the correction value as described below, thereby enabling more appropriate streaking correction.

[0025] The image adjustment unit 3 performs image processing on the streaking-corrected image signal. The image processing here can be general image processing, such as gamma correction, knee correction, and detail signal addition.

[0026] The output generation unit 4 outputs the video-processed signal.

[0027] The flow of these processes will be explained with reference to Figure 2. A video signal from the image sensor 1 is output in units of horizontal lines from the top left of the image and enters the streaking correction unit 2. The video signal corrected for streaking in the streaking correction unit 2 undergoes video processing in the video adjustment unit 3, and is then output from the imaging device via the output generation unit 4.

[0028] <Example of light receiving unit> Fig. 3 shows a first example of a light receiving section of an imaging element that can be used in the imaging device of the present invention, and is a diagram showing the light receiving section of an imaging element that has a horizontal light-shielding area on the left edge. Fig. 4 shows a second example of a light receiving section of an imaging element that can be used in the imaging device of the present invention, and is a diagram showing the light receiving section of an imaging element that has horizontal light-shielding areas on both ends. In Figs. 3 and 4, the horizontal direction is the horizontal direction (x direction) and the vertical direction is the vertical direction (y direction).

[0029] The light receiving section of the image sensor 1 shown in Figures 3 and 4 has an effective area 80 that senses light, and a light-shielded area (OB (Optical Black) area) where no light is incident and pixel values do not change. The effective area 80 has a plurality of pixels arranged in a matrix, and receives light from an object imaged by an optical system. The light-shielded area is made up of a plurality of rows of light-shielded pixels arranged around the effective area 80. The light-shielded areas are referred to as horizontal light-shielded areas 81 (horizontal OB area) at the left end or left and right ends of the effective area 80, and vertical light-shielded areas 82 (vertical OB area) at the top of the effective area 80. The horizontal light-shielded area 81 is formed to extend in the vertical direction. The vertical light-shielded area 82 is formed to extend in the horizontal direction.

[0030] The horizontal light-shielding area 81 is present only on the left side in Fig. 3, and is present on both the left and right sides in Fig. 4. The vertical light-shielding area 82 is present at the top in both Figs.

[0031] In the following embodiments, the explanation will be based on the case where the horizontal light-shielding area 81 and the effective area 80 (only on the left side) of the image sensor 1 are arranged as shown in Fig. 3. However, even when the horizontal light-shielding areas 81 and the effective areas 80 (on both sides) of the image sensor 1 are arranged as shown in Fig. 4, the streaking correction effect can be obtained similarly to that of Fig. 3. In this case, the black level can be obtained from the left and right horizontal light-shielding areas 81 to obtain the same streaking correction effect.

[0032] <Configuration of streaking correction unit> 5 is a diagram showing an example of the configuration of the streaking correction section in the imaging device of the present invention, and will explain the detailed configuration of the streaking correction section 2 shown in FIG.

[0033] The streaking correction unit 2 includes a memory unit 10, a horizontal light-shielded area smoothing unit 11, a compensation value calculation unit 12, a correction value calculation unit 13, and a subtractor .

[0034] The memory unit 10 temporarily stores the video signal output from the image sensor 1 .

[0035] The horizontal shading area smoothing unit 11 calculates the average value of the signal level of each horizontal line in the horizontal shading area 81 shown in FIGS. 3 and 4 for the video signal output from the image sensor 1. The calculated average values are output sequentially to the correction value calculation unit 13 as the black level of each horizontal line. At this time, the horizontal shading area smoothing unit 11 detects the amount of black level depression and outputs this to the correction value calculation unit 13. Note that the horizontal shading area 81 generally has fewer pixels than the effective area 80 of the image sensor. For this reason, when calculating the average value of the black level of each line, simply adding up each pixel to calculate the average value may not be enough to remove the effects of random noise. In this case, a configuration can be applied in which the average value calculated by pixel addition is filtered using a time-domain recursive filter to remove the effects of random noise.

[0036] The compensation value calculation unit 12 calculates a compensation value to be used in generating a streaking correction value. To calculate the compensation value, first, a compensation parameter is detected based on the video signal output from the image sensor 1, and a compensation value is calculated based on the detected compensation parameter. The compensation parameters are parameters that affect streaking, such as light intensity, position, and width. Light intensity is detected based on the signal level of a high-luminance signal above a predetermined level. Position is detected by detecting the incident position of each horizontal line of a high-luminance signal above a predetermined level. Width is detected by detecting the horizontal width of a high-luminance signal above a predetermined level. If the compensation parameter is light intensity, a light intensity compensation value (light intensity compensation gain) is calculated based on the detected light intensity. If the compensation parameter is position, a position compensation value (position compensation gain) is calculated based on the detected position. If the compensation parameter is width, a width compensation value (width compensation gain) is calculated based on the detected width.

[0037] The correction value calculation unit 13 generates a streaking correction value using the black level calculated by the horizontal light-shielded area smoothing unit 11 and the compensation value calculated by the compensation value calculation unit 12. The compensation value here includes the light intensity compensation value, position compensation value, and width compensation value described above, and compensates for the correction value based on the amount of sinking of the black level.

[0038] The subtractor 14 subtracts the black level from the video signal temporarily stored in the memory unit 10 using the streaking correction value calculated by the correction value calculation unit 13. This makes it possible to obtain a video signal free from streaking. The video signal temporarily stored in the memory unit 10 is read out from the memory unit in accordance with the timing at which the correction value calculation unit 13 generates the correction value.

[0039] <First example of streaking> 6A and 6B are diagrams illustrating a first example of streaking occurring in the image sensor shown in Fig. 3, with (A) showing the light receiving surface of the image sensor and (B) showing the relationship between the coordinate y and pixel value I at each x coordinate position. In Fig. 6A, the lateral direction (left-right direction) is the horizontal direction (x direction), and the longitudinal direction is the vertical direction (y direction).

[0040] In the example shown in Figure 6, as shown in Figure 6(A), a case occurs in which light 53 stronger than a predetermined level is input to the effective area 50 of the image sensor 1. Light 53 stronger than a predetermined level is, for example, light with a high brightness that far exceeds the rated level of the image processing system of the image pickup device. In this case, black level dips 54 occur on the left and right of the light 53. If no correction is made, streaking will occur due to this black level dip 54.

[0041] Figure 6(B) shows the distribution of pixel values I in the y direction (vertical direction) at x-coordinate points x1, x2, x3, and x4 in Figure 6(A). The vertical direction of the drawing indicates the y-coordinate direction (vertical direction), and the horizontal direction of the drawing indicates pixel values, with higher pixel values indicating higher values further to the right. The point with x-coordinate x1 is the position of the horizontal shading area 51 located to the left of the effective area 50. The point with x-coordinate x2 is the position of the effective area 50 to the left of the light 53. The point with x-coordinate x3 is the position of the light 53. The point with x-coordinate x4 is the position of the effective area 50 to the right of the light 53.

[0042] As shown in FIG. 6(B), at the position x3 where the y coordinate is light 53, the pixel value becomes higher than the black level by D2 due to the incident light of light 53. Also, at the position x2 where the y coordinate corresponds to light 53, depression 54 occurs as described above, and the pixel value becomes lower than the black level of other areas by D1. The same is true for x4. Furthermore, the same is true for horizontal light-shielded area 51 at x1, where the pixel value becomes lower than the black level of other areas by D1 at the position where the y coordinate corresponds to light 53. That is, in the example shown in FIG. 6(B), the value D1 of depression due to the black level detected in horizontal light-shielded area 51 and the value of depression D1 of the black level detected in effective area 50 are approximately the same.

[0043] Fig. 7 is a diagram for explaining correction by the imaging device of the present invention for the first example of Fig. 6, where (C-1) shows the state of the streaking signal before correction and (C-2) shows the state after correction. In Fig. 7, the horizontal direction of the drawing indicates the x-coordinate direction (horizontal direction), and the vertical direction of the drawing indicates the pixel value.

[0044] In the state before correction shown in FIG. 7(C-1), the black level depression is constant at D1. Therefore, the depression in horizontal shading region 51 shown in FIG. 6 also becomes D1. Therefore, correction value calculation unit 13 shown in FIG. 5 performs correction based on the black level value output from horizontal shading region smoothing unit 11, taking into account only this depression D1. In other words, correction is performed without taking into account the compensation value of compensation value calculation unit 12. Therefore, correction is performed by reducing the black level by D1 in the left and right regions of light 53.

[0045] By performing such a correction, the area where the black level sinks as shown in FIG. 7(C-2) disappears, and the streaking phenomenon can be eliminated.

[0046] <Second example of streaking> 8A and 8B are diagrams illustrating a second example of streaking occurring in the image sensor shown in Fig. 3, where (A) is a diagram showing the light receiving surface of the image sensor, and (B) is a diagram showing the relationship between the coordinate y and pixel value I at each position of the coordinate x. In Fig. 8A, the lateral direction (left-right direction) is the horizontal direction (x direction), and the longitudinal direction is the vertical direction (y direction).

[0047] In the example shown in FIG. 8, as shown in FIG. 8(A), a case occurs in which light 63 strong enough to be a predetermined level or more is input to the effective area 60 of the image sensor 1. The light 63 strong enough to be a predetermined level or more is, for example, light with a high brightness that significantly exceeds the rated level of the image processing system of the image sensor. Meanwhile, as shown in FIG. 8(A), an example is shown in which the distance A in the x direction from the light 63 to the horizontal light-shielding area 61 is greater than that shown in FIG. 6. In this case, black level dips 64 occur on the left and right of the light 63, resulting in streaking. Meanwhile, the greater distance A increases the distance from the power supply unit provided at the edge to the pixels on the light-receiving surface. This results in a phenomenon in which the black level dips in the effective area 60 on the left and right of the light 63 are greater than the black level dips in the horizontal light-shielding area 61.

[0048] Figure 8(B) shows the distribution of pixel values I in the y direction (vertical direction) at x-coordinate points x1, x2, x3, and x4 in Figure 8(A). The vertical direction of the drawing indicates the y-coordinate direction (vertical direction), and the horizontal direction of the drawing indicates pixel values, with higher pixel values indicating higher values further to the right. The point with x-coordinate x1 is the position of the horizontal shading area 61 on the left side of the effective area 60. The point with x-coordinate x2 is the position of the effective area 60 to the left of the light 63. The point with x-coordinate x3 is the position of the light 63. The point with x-coordinate x4 is the position of the effective area 60 to the right of the light 63.

[0049] As shown in FIG. 8(B), at the position x3 where the y coordinate is light 63, the pixel value becomes higher than the black level by D2 due to the incident light of light 63. Also, at the position x2 where the y coordinate corresponds to light 63, depression 64 occurs as described above, and the pixel value becomes lower than the black level of other areas by D3. The same is true for x4. Furthermore, in the horizontal shading region 61 at x1, at the position where the y coordinate corresponds to light 63, the pixel value becomes lower than the black level of other areas by D1. Here, the amount of depression of the black level due to D3 is greater than the amount of depression of the black level due to D1. This is because strong light 63 is incident at a position away from the horizontal shading region 61, as shown in FIG. 8(A).

[0050] Fig. 9 is a diagram for explaining correction by the imaging device of the present invention for the second example of Fig. 8, where (C-1) shows the state of the streaking signal before correction, (C-2) shows the state after correction without compensation, and (C-3) shows the state after correction with compensation. In Fig. 9, the horizontal direction of the drawing indicates the x-coordinate direction (horizontal direction), and the vertical direction of the drawing indicates the pixel value.

[0051] In the state before correction in Figure 9 (C-1), the black level depression is D3 in the effective areas 60 on the left and right of the light 63. On the other hand, the depression in the horizontal light-blocking area 61 shown in Figure 8 is D1, which is a value smaller than D3.

[0052] The correction value calculation unit 13 shown in FIG. 5 calculates a correction value taking into account the black level value output from the horizontal shading area smoothing unit 11 and the compensation value calculated by the compensation value calculation unit 12. At this time, if the compensation value calculated by the compensation value calculation unit 12 is not taken into account, correction is performed taking into account only the black level depression D1 in the horizontal shading area 61, as shown in FIG. 9(C-2). As a result, black level depressions of D3-D1 remain in the effective area 60 on the left and right of the light 63. In this case, the streaking phenomenon remains even after correction processing.

[0053] For this reason, the correction value calculation unit 13 calculates a correction value taking into account the compensation value calculated by the compensation value calculation unit 12. As a result, as shown in Fig. 9(C-3), the sinking of the black level in the effective area 60 on the left and right of the light 63 is eliminated, and the streaking phenomenon is also eliminated. A specific example taking into account compensation in this case will be described later.

[0054] <Third example of streaking> 10A and 10B are diagrams illustrating a third example of streaking occurring in the image sensor shown in Fig. 3, where (A) is a diagram showing the light receiving surface of the image sensor, and (B) is a diagram showing the relationship between the coordinate y and pixel value I at each position of the coordinate x. In Fig. 10A, the lateral direction (left-right direction) is the horizontal direction (x direction), and the longitudinal direction is the vertical direction (y direction).

[0055] In the example shown in FIG. 10, as shown in FIG. 10(A), a case occurs in which light 73 strong enough to be a predetermined level or more is input to the effective area 70 of the image sensor 1. The light 73 strong enough to be a predetermined level or more is, for example, light with a high brightness that far exceeds the rated level of the image processing system of the image sensor. On the other hand, as shown in FIG. 10(A), an example is shown in which the width B of the light 73 in the horizontal direction (x direction) is larger than that of the light 53 in FIG. 6. In this case, black level depressions 74 occur on the left and right of the light 73, resulting in streaking. On the other hand, because the width B is large, a phenomenon occurs in which the black level depressions in the effective area 70 on the left and right of the light 73 are greater than the black level depressions in the horizontal light-shielding area 71.

[0056] Figure 10(B) shows the distribution of pixel values I in the y direction (vertical direction) at x-coordinate points x1, x2, x3, and x4 in Figure 10(A). The vertical direction of the drawing indicates the y-coordinate direction (vertical direction), and the horizontal direction of the drawing indicates pixel values, with higher pixel values indicating higher values further to the right. The point with x-coordinate x1 is the position of the horizontal shading area 71 on the left side of the effective area 70. The point with x-coordinate x2 is the position of the effective area 70 to the left of the light 73. The point with x-coordinate x3 is the position of the light 73. The point with x-coordinate x4 is the position of the effective area 70 to the right of the light 73.

[0057] As shown in FIG. 10(B), at the position x3 whose y coordinate corresponds to light 73, the pixel value becomes higher than the black level by D4 due to the incident light of light 73. At the position x2 whose y coordinate corresponds to light 73, depression 74 occurs as described above, and the pixel value becomes lower than the black level of other areas by D5. The same is true for x4. Furthermore, in the horizontal light-shielded area 71 at x1, at the position whose y coordinate corresponds to light 73, the pixel value becomes lower than the black level of other areas by D1. Here, the amount of depression of the black level due to D5 is greater than the amount of depression of the black level due to D1. This is because light 73 with a width is input, as shown in FIG. 10(A).

[0058] Fig. 11 is a diagram for explaining correction by the imaging device of the present invention for the third example of Fig. 10, where (C) shows the state of the streaking signal before correction, (C-2) shows the state after correction without compensation, and (C-3) shows the state after correction with compensation. In Fig. 11, the horizontal direction of the drawing indicates the x-coordinate direction (horizontal direction), and the vertical direction of the drawing indicates the pixel value.

[0059] In the state before correction in Figure 11 (C-1), the black level depression is D5 in the effective areas 70 on the left and right of the light 73. On the other hand, the depression in the horizontal light-blocking area 71 shown in Figure 10 is D1, which is a value less than D5.

[0060] The correction value calculation unit 13 shown in FIG. 5 calculates a correction value taking into account the black level value output from the horizontal light-shielded area smoothing unit 11 and the compensation value calculated by the compensation value calculation unit 12. At this time, if the compensation value calculated by the compensation value calculation unit 12 is not taken into account, correction is performed taking into account only D1, as shown in FIG. 10(C-2). As a result, a dip in the black level by D5-D1 remains in the effective area 70 on the left and right of the light 73. In this case, the streaking phenomenon remains even after correction processing.

[0061] For this reason, the correction value calculation unit 13 calculates a correction value taking into account the compensation value calculated by the compensation value calculation unit 12. As a result, as shown in Fig. 10(C-3), the sinking of the black level in the effective area 70 on the left and right of the light 73 is eliminated, and the streaking phenomenon is also eliminated. A specific example taking into account compensation in this case will be described later.

[0062] <Examples of compensation> The compensation value calculation unit 12 obtains a histogram of one line (one horizontal line) of the video signal from the image sensor 1. This histogram can be a histogram of light intensity along the y direction. The compensation value calculation unit 12 uses this histogram to calculate the compensation value (compensation gain value). The light intensity compensation gain, position compensation gain, width compensation gain, and compensation streaking correction signal will be explained below.

[0063] <Light intensity compensation gain> This section explains how to calculate the compensation gain (light intensity compensation gain) according to the level of incident light. When the light intensity increases, a phenomenon similar to that shown in Figures 9 and 11 occurs. Specifically, when strong light is incident on the effective area 80 shown in Figure 3, the amount of black level reduction in the effective area 80 becomes greater than the amount of black level reduction in the horizontal light-shielded area 81. As a result, streaking remains even after the correction shown in Figure 7 is performed. Here, we focus on signals that exceed the rated video level (signals with high brightness exceeding the rated level) from a one-line histogram. The compensation gain is determined by determining the range in which the signal level falls between the maximum light intensity of the image sensor 1 and the rated level. This allows the light intensity compensation gain value to be expressed as follows: Light intensity compensation gain value = α × input signal level / maximum input signal level ...(Formula 1) Here, if the maximum rated level is 100%, the input signal level is 100% or higher. Also, α is a coefficient determined by the characteristics of the image sensor.

[0064] In Equation 1, when the input signal level exceeds the rated level, the greater the input signal level, the greater the value of the light intensity compensation gain. This means that the stronger the light (the higher the brightness), the greater the degree of compensation in the light intensity compensation gain.

[0065] <Position compensation gain> We will now explain how to calculate the compensation gain (position compensation gain) according to the location of the incident light. This corresponds to the compensation explained in Figures 8 and 9. Here, we focus on signals that exceed the rated level of the video signal (signals with high brightness that exceed the rated level) from a one-line histogram. The compensation gain is determined according to the location of the signal that exceeds the rated level, particularly the distance from the center position of the signal to the horizontal shading area 81. The distance here is the distance in the horizontal direction (x direction). As a result, the position compensation gain value can be expressed as follows: Position compensation gain value = β × distance to horizontal shading area (Equation 2) Here, β is a coefficient determined by the characteristics of the image sensor.

[0066] A specific method is to detect the left and right edges of the signal exceeding the rated level in one line of video signal, and then find the center of the signal exceeding the rated level from the results and apply this to the above formula 2. Note that a position other than the center of the signal exceeding the rated level may also be used. For example, the position of the edge of the horizontal shading region 81 side of the signal exceeding the rated level.

[0067] In equation 2, when the input signal level exceeds the rated level, the value of the position compensation gain increases as the distance from the center of the signal exceeding the rated level to horizontal shading region 81 increases. As a result, the degree of compensation increases in proportion to the distance from horizontal shading region 81 between the center of the signal exceeding the rated level and the position compensation gain.

[0068] <Width compensation gain> We will now explain how to calculate the compensation gain according to the width of the incident light. This corresponds to the compensation explained in Figures 10 and 11. Here, we focus on signals that exceed the rated level of the video signal (signals with high brightness that exceed the rated level) from a one-line histogram. The compensation gain is determined according to the width of the signal that exceeds the rated level. The width here is the distance in the horizontal direction (x direction). As a result, the width compensation gain value can be expressed as follows: Width compensation gain value = γ × optical signal width (Equation 3) Here, γ is a coefficient determined by the characteristics of the image sensor.

[0069] A specific method is to detect the left and right edges of the signal that exceed the rated level in one line of video signal, calculate the width, and then apply the result to Equation 3 above.

[0070] In Equation 3, when the input signal level exceeds the rated level, the larger the width of the signal exceeding the rated level, the larger the value of the position compensation gain. As a result, the greater the width of the signal exceeding the rated level, the greater the degree of compensation in the width compensation gain.

[0071] <Compensation streaking correction signal> The correction value calculation unit 13 shown in Fig. 5 can generate a compensated streaking correction signal, which is a compensated streaking correction signal, using the values of the light intensity compensation gain, position compensation gain, and width compensation gain calculated as described above. The compensated streaking correction signal can be expressed by the following equation. Compensation streaking correction signal = Black level sinking amount x light intensity compensation gain value x position compensation gain value x width compensation gain value ...(Formula 4) Here, the black level subsidence amount is a subsidence amount based on the black level calculated by the horizontal light-shielded area smoothing unit 11 illustrated in FIGS.

[0072] According to Equation 4, the value of each compensation gain is taken into consideration for the black level, and the correction signal can be compensated.

[0073] The video signal from the image sensor 1 is temporarily held in the memory unit 10 until signal processing is completed in the compensation value calculation unit 12 and the correction value calculation unit 13 shown in Fig. 5. The video signal is then read out in synchronization with the output timing of the streaking correction signal generated by the correction value calculation unit 13, and is subjected to subtraction in the subtractor 14 using the streaking correction signal. This makes it possible to obtain a good video signal without a dip in the black level, as shown in Fig. 9 (C-3) or Fig. 11 (C-3).

[0074] According to the above-described embodiment, not only the correction based on the black level in the horizontal shaded area smoothing unit 11 but also the correction taking into account the compensation value in the compensation value calculation unit 12 can prevent the sinking of the black level and suppress the occurrence of streaking. In this case, by using the light intensity compensation gain as the compensation value, more accurate correction can be made according to the light intensity. Furthermore, by using the position compensation gain as the compensation value, more accurate correction can be made according to the distance to the horizontal shaded area. Furthermore, by using the width compensation gain, more accurate correction can be made according to the width of light with high light intensity. Furthermore, these features eliminate the need to build a complex system and limit the load on the system.

[0075] The present invention is not limited to the above-described embodiments and includes various modifications. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the described configurations. Furthermore, it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment, or to add the configuration of another embodiment to the configuration of one embodiment. Furthermore, it is possible to add, delete, or replace part of the configuration of each embodiment with other configurations.

[0076] For example, in calculating the light intensity compensation gain value, position compensation gain value, and width compensation gain value, signals exceeding the rated level of the video signal were targeted, but in this case, signals exceeding a predetermined level equivalent to the rated level may also be targeted.

[0077] Furthermore, in the calculation of the compensation streaking correction signal in Equation 4, any of the light intensity compensation gain value, the position compensation gain value, and the width compensation gain value is used, but it is also possible to use at least one of these gain values. [Explanation of symbols]

[0078] 1...imaging element, 2...streaking correction unit, 4...output generation unit, 10...memory unit, 11...horizontal shading area smoothing unit, 12...compensation value calculation unit, 13...correction value calculation unit, 14...subtractor, 50...effective area, 51...horizontal shading area, 52...vertical shading area, 53...light, 54...sinking, 60...effective area, 61...horizontal shading area, 62...vertical shading area, 63...light, 64...sinking, 70...effective area, 71...horizontal shading area, 72...vertical shading area, 73...light, 74...sinking, 80...effective area, 81...horizontal shading area, 82...vertical shading area, 300...computer system 302...processor, 302A...processing device, 302B...processing device, 304...memory, 306...memory bus, 308...I / O bus, 309...bus interface unit, 310...I / O bus interface unit, 312...terminal interface unit, 314...storage interface, 316...I / O device interface, 318...network interface, 320...user I / O device, 324...display system, 326...display device, 330...network, 350...latent factor identification application

Claims

1. an image pickup element having an effective area for sensing light and a horizontal light-shielding area disposed at at least one of the left and right ends of the effective area and configured to output a video signal; a horizontal light-shielded area smoothing unit that detects a black level drop in the horizontal light-shielded area from the video signal; a compensation value calculation unit that detects compensation parameters from the video signal and calculates a compensation value; a correction value calculation unit that calculates a correction value for correcting the video signal to suppress a streaking phenomenon, using a black level sinking amount from the horizontal light-shielding area smoothing unit and the compensation value from the compensation value calculation unit, The imaging device according to claim 1, wherein the compensation parameter uses a value of the distance between the position of a signal having a light intensity equal to or greater than a predetermined value and the horizontal light-blocking area.

2. an image pickup element having an effective area for sensing light and a horizontal light-shielding area disposed at at least one of the left and right ends of the effective area and configured to output a video signal; a horizontal light-shielded area smoothing unit that detects a black level drop in the horizontal light-shielded area from the video signal; a compensation value calculation unit that detects compensation parameters from the video signal and calculates a compensation value; a correction value calculation unit that calculates a correction value for correcting the video signal to suppress a streaking phenomenon, using a black level sinking amount from the horizontal light-shielding area smoothing unit and the compensation value from the compensation value calculation unit, The compensation parameter uses a value of light intensity equal to or greater than a predetermined value, the compensation value calculation unit calculates, for each horizontal line of the video signal, a compensation value proportional to a level of a signal having a light intensity equal to or greater than a predetermined level from the video signal; The imaging device, wherein the correction value calculation unit calculates a correction value obtained by compensating a correction value based on an amount of black level sinking using the compensation value.

3. In the imaging device according to claim 1, the compensation value calculation unit calculates, for each horizontal line of the video signal, a compensation value proportional to a distance from a center position of a signal having a light intensity equal to or greater than a predetermined value to the horizontal light-blocking region; The imaging device, wherein the correction value calculation unit calculates a correction value obtained by compensating a correction value based on an amount of black level sinking using the compensation value.

Images