Image correction device and imaging apparatus

Abstract

To suitably reduce noise even in an image pick-up device that can switch the presence or absence of binning in a local area.SOLUTION: An image correction device comprises: a pixel information acquisition unit that acquires pixel information picked up by an image pick-up device that has a plurality of pixel blocks having a plurality of pixels and a floating diffusion shared by the plurality of pixels, and a control circuit capable of individually controlling the presence or absence of binning for each pixel block or for the plurality of pixel blocks; a scene information generation unit that generates scene information indicating the degree of change in the brightness or the pixel value for each pixel block, according to the pixel value of the pixel information; and a correction unit that corrects the pixel values according to the presence or absence of the binning based on the scene information and the pixel value of the pixel information.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an image correction device and an imaging device. [Background technology]

[0002] In conventional imaging systems, the resolution, frame rate, and dynamic range characteristics are generally set to be constant within the screen. Here, in wide-field-of-view images that are intended for display on head-mounted displays, LED (Light Emitting Diode) domes, etc., and that significantly exceed the conventional display angle of view, it is expected that subjects with various characteristics will be simultaneously captured within the same image. Examples of subjects with various characteristics include subjects with fine details, subjects that move quickly, and subjects with large differences in brightness, such as between sunlight and shade.

[0003] To capture images of all subjects captured in such wide-field images using conventional imaging systems while suppressing aliasing, motion blur, blown-out highlights, blocked-up shadows, and degradation of the signal-to-noise ratio (SNR), an image sensor that simultaneously satisfies performance requirements such as high resolution, high frame rate, and high dynamic range is required. However, achieving high resolution and high frame rate requires high-speed readout of pixel signals within the image sensor, but the faster the pixel signals are readout, the more likely the dynamic range and SNR are to degrade. Furthermore, increasing the pixel signal readout speed also leads to problems such as increased readout circuit size and power consumption. Therefore, it is not easy to achieve the high performance described above with conventional image sensors, such as complementary metal oxide semiconductor (CMOS) image sensors.

[0004] On the other hand, from the perspective of improving subjective image quality, the imaging parameters of the image sensor do not necessarily need to be constant within the screen. For example, there is no need to capture a still subject at a high frame rate, and there is no need to capture a fast-moving subject at high resolution because motion blur reduces spatial frequency components. Furthermore, for highly luminous subjects, ensuring dark gradation and SNR is given lower priority, while for low-luminance subjects, ensuring bright gradation is given lower priority.

[0005] As described above, since the required imaging parameter levels differ depending on the characteristics of each of the multiple subjects included in the same image, it is not necessary to keep the imaging parameters of the imaging element constant within the screen. For example, in an area where a stationary subject exists, it is not necessary to capture the image at a high frame rate, and it is desirable to capture the image at a high resolution. In addition, in an area where a fast-moving object exists, the spatial frequency decreases due to motion blur, so it is not necessary to capture the image at a high resolution, and it is desirable to capture the image at a high frame rate.

[0006] Furthermore, by partially shortening the exposure time in areas where high-brightness objects exist and partially extending the exposure time in areas where low-brightness objects exist, the dynamic range of the entire image can be improved. In this way, by appropriately controlling imaging parameters according to the characteristics of each of multiple objects contained in the same image, it is possible to obtain image quality that is essentially equivalent to that achieved when high resolution, frame rate, and dynamic range characteristics are all achieved, even if they are not all achieved simultaneously. An image sensor using a pixel structure described in Patent Document 1 is known as a technology for achieving such imaging.

[0007] Furthermore, the imaging element described in Non-Patent Document 1 has a pixel structure similar to that of Patent Document 1, and discloses a technique for shortening or extending the exposure time for each pixel group. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] Japanese Patent Publication No. 2022-123539 [Non-patent literature]

[0009] [Non-Patent Document 1] Kohei Tomioka, et.al., “Feedback control of Block-wise-controlled Image Sensor Based on Brightness Distribution Analysis”, 2023 International Image Sensor Workshop (IISW 2023), P25, 2023 Summary of the Invention [Problem to be solved by the invention]

[0010] In general, images captured using an image sensor may be degraded by random noise such as thermal noise and shot noise. Circuit-induced noise may also occur due to manufacturing variations in pixel characteristics and readout circuit characteristics, changes in characteristics during operation, and other factors. Specific examples of circuit-induced noise include pixel defects, in which a specific pixel does not output a normal signal; line noise, in which the black level on a horizontal or vertical line fluctuates due to fluctuations in the power supply voltage of a circuit depending on the image; and fixed pattern noise (FPN), in which the black level offset varies from pixel to pixel.

[0011] There are known techniques for reducing noise caused by these circuits. For example, for pixel defects, a known method is to perform interpolation using pixels surrounding the defective pixel. For line noise, a known method is to acquire black level offsets for each horizontal and vertical line from light-shielded pixels (OB pixels) outside the effective pixel area of ​​the sensor for each frame using averaging processing, and then subtract the offsets from the effective pixel area. For fixed pattern noise, a known method is to measure the black level of each pixel in advance by, for example, averaging the output of multiple frames with the camera shielded from light, and then subtract the measured value from the normal captured image.

[0012] However, these known techniques are methods used in imaging devices that use an image sensor in which the imaging mode is constant across the entire image. In imaging devices using an image sensor that can switch between binning and non-binning in local regions, as shown in Patent Document 1 and Non-Patent Document 1, the sampling positions and offset amounts of pixel defects, line noise, and fixed pattern noise change depending on whether binning is used or not. Therefore, imaging devices using an image sensor that can switch between binning and non-binning in local regions have a problem in that it is difficult to achieve appropriate noise reduction. For example, the fixed pattern noise in each pixel without binning may differ from the fixed pattern noise in each pixel with binning, and incorrect correction may result in residual noise or unwanted artifacts.

[0013] In view of the above problems, the present invention provides an image correction device and an imaging device that can suitably reduce noise even in an imaging element that can switch between binning and non-binning in local areas. [Means for solving the problem]

[0014] [1] In order to solve the above problem, one aspect of the present invention is an image correction device comprising: a pixel information acquisition unit that acquires pixel information captured by an image sensor having a plurality of pixel blocks, each pixel block having a plurality of pixels and a floating diffusion shared by the plurality of pixels, and a control circuit that can independently control a drive mode for each of the pixel blocks or for each of the plurality of pixel blocks; a scene information generation unit that generates scene information indicating the degree of change in brightness or pixel value for each of the pixel blocks according to the pixel values ​​of the pixel information; and a correction unit that corrects the pixel values ​​according to whether or not binning is performed based on the scene information and the pixel values ​​of the pixel information.

[0015] [2] In another aspect of the present invention, in the image correction device described in [1] above, the correction unit further includes a pixel defect correction unit that, when a pixel included in the pixel block is a defective pixel, corrects the pixel value in accordance with the pixel values ​​of other pixels included in the pixel block or the pixel values ​​of pixels included in the neighboring pixel blocks.

[0016] [3] Furthermore, in one aspect of the present invention, in the image correction device described in [1] or [2] above, the pixel defect correction unit corrects the pixel value of the defective pixel to a value obtained by statistically calculating the pixel values ​​of other pixels included in the pixel block when the pixel block to be corrected is not binned.

[0017] [4] Furthermore, in one aspect of the present invention, in the image correction device described in any one of [1] to [3] above, the pixel defect correction unit corrects the pixel values ​​of the pixel block to be corrected based on the pixel values ​​of pixels included in the pixel block that are not the defective pixels, when the pixel block to be corrected has binning.

[0018] [5] Furthermore, in one aspect of the present invention, in the image correction device described in [1] or [2] above, the pixel defect correction unit corrects the pixel value of the defective pixel to a value obtained by statistically calculating pixel values ​​read from four pixel blocks adjacent to the defective pixel block in the vertical and horizontal directions when the pixel block to be corrected is not binned.

[0019] [6] Furthermore, in one aspect of the present invention, in the image correction device described in [1], [2] or [5] above, the pixel defect correction unit, when the pixel block to be corrected is binned, sets the pixel value of the defective pixel to the pixel value read from the pixel block, which is a statistical calculation result of pixel values ​​read from four pixel blocks adjacent in the vertical and horizontal directions.

[0020] [7] Furthermore, in one aspect of the present invention, in the image correction device described in any one of [1] to [6] above, the correction unit further includes an OB clamp unit that corrects pixel values ​​by subtracting a correction coefficient corresponding to the presence or absence of binning based on the scene information, based on correction coefficients obtained from binning correction coefficient calculation pixels and no-binning correction coefficient calculation pixels provided in an OB (Optical Black) area.

[0021] [8] Furthermore, in one aspect of the present invention, in the image correction device described in any one of [1] to [7] above, the correction unit further includes an FPN cancellation unit that corrects pixel values ​​by canceling fixed pattern noise by subtracting FPN (Fixed Pattern Noise) data calculated depending on whether binning is performed or not for each pixel block, depending on whether binning is performed or not.

[0022] [9] Another aspect of the present invention is an imaging device comprising an imaging element having a plurality of pixel blocks, each having a plurality of pixels and a floating diffusion shared by the plurality of pixels, and an image correction device according to any one of [1] to [8] above, which corrects pixel values ​​of pixel information captured by the imaging element. [Effects of the Invention]

[0023] According to the present invention, it is possible to provide an image correction device, an imaging device, and an image correction method that can effectively reduce noise even in an imaging element that can switch between binning and non-binning in local areas. [Brief explanation of the drawings]

[0024] [Figure 1] FIG. 1 is a functional configuration diagram illustrating an example of a functional configuration of an imaging system according to an embodiment. [Figure 2] FIG. 2 is a functional configuration diagram showing an example of the detailed functional configuration of a scene information / image acquisition board and a signal processing board according to the present embodiment. [Figure 3] FIG. 2 is a diagram for explaining a first method for pixel defect correction according to the present embodiment. [Figure 4] FIG. 10 is a diagram for explaining a second method for pixel defect correction according to the embodiment. [Figure 5] FIG. 2 is a first diagram for explaining an OB clamp according to the present embodiment. [Figure 6] FIG. 10 is a second diagram for explaining the OB clamp according to the present embodiment. [Figure 7] FIG. 1 is a first diagram for explaining FPN cancellation according to the present embodiment. [Figure 8] FIG. 2 is a second diagram for explaining FPN cancellation according to the present embodiment. [Figure 9] FIG. 2 is a block diagram showing an example of the internal configuration of the imaging device according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0025] [Embodiment] A preferred embodiment of the present invention will be described in detail below with reference to the accompanying drawings. It should be noted that the present invention is not limited to these embodiments and includes various modifications and improvements. In other words, the components described below include those that would be easily conceivable to a person skilled in the art or that are substantially identical, and the components described below can be combined as appropriate. Furthermore, various omissions, substitutions, or modifications of the components can be made without departing from the spirit of the present invention. Furthermore, in the drawings, the scale and number of components may differ from the scale and number of the actual structures to make each configuration easier to understand.

[0026] 1 is a functional configuration diagram showing an example of the functional configuration of an imaging system according to an embodiment. First, with reference to the diagram, an imaging system 1 according to this embodiment will be described. The imaging system 1 includes an imaging device 3 and a video display device 5.

[0027] The imaging device 3 captures an image. A single image captured by the imaging device 3 may contain multiple subjects with different characteristics. An example of multiple subjects with different characteristics is a stationary object and a fast-moving object. That is, a single image captured by the imaging device 3 may contain a mixture of areas where stationary objects exist and areas where fast-moving objects exist. The imaging device 3 sets suitable imaging conditions for each of these multiple areas, thereby optimally capturing images of multiple subjects with different characteristics.

[0028] Specifically, the imaging device 3 includes a lens 31, a prism 32, a sub-sensor 33, an image sensor 34, a sensor drive board 35, a sensor power supply board 36, a scene information / image acquisition board 37, a signal processing board 38, and a power supply module 39. In the following description, the imaging device 3 captures a color image. In this case, the image sensor 34, the sensor drive board 35, the sensor power supply board 36, and the scene information / image acquisition board 37 are provided for each of the colors R (Red), G (Green), and B (Blue). This embodiment is not limited to this example, and can also be applied to monochrome images and grayscale images.

[0029] The lens 31 guides incident light to the prism 32. The figure shows the optical axis OA of the light incident on the lens 31. The lens 31 is preferably a wide-angle lens or an ultra-wide-angle lens in order to simultaneously capture images of multiple subjects with different characteristics. In this embodiment, lenses 31 with various angles of view can be used, and the angle of view is not limited in any way. A turret-type structure may also be provided to allow selection of multiple lenses with different angles of view.

[0030] The prism 32 splits the incident light according to the wavelength of the light. For example, the prism 32 may split the incident light into infrared light and send it to the sub-sensor 33, and visible light and send it to the image sensor 34. The prism 32 may also split the light into multiple image sensors 34 according to the wavelengths of each of the R, G, and B components. In the example shown in the figure, the prism 32 guides light having a wavelength corresponding to the G component to the image sensor 34G, guides light having a wavelength corresponding to the B component to the image sensor 34B, and guides light having a wavelength corresponding to the R component to the image sensor 34R.

[0031] The sub-sensor 33 is a sensor provided auxiliary to the image sensor 34 that generates an image. The sub-sensor 33 may be, for example, an event detection sensor that detects changes in luminance of each pixel arranged on a two-dimensional coordinate system. The sub-sensor 33 may be, for example, a distance sensor (Time-of-Flight Sensor: ToF sensor) that detects distance according to changes in the amount of infrared light emitted from the prism 32.

[0032] In the illustrated example, a sensor that detects infrared light is used as the sub-sensor 33, but this embodiment is not limited to this example. For example, the sub-sensor 33 may be configured to detect visible light. By separating (assigning) infrared light that is not necessary for detecting an image to the sub-sensor 33, it is possible to obtain the effect of reducing image degradation, so it is preferable that the sub-sensor 33 be a sensor that detects changes in the luminance of infrared light.

[0033] The image sensor 34 is an image sensor that detects the brightness of visible light emitted from the prism 32. In the example shown in the figure, an image sensor 34 for each of the R, B, and C colors is provided. Specifically, image sensor 34G is shown as a sensor that detects green (G), image sensor 34B is shown as a sensor that detects blue (B), and image sensor 34R is shown as a sensor that detects red (R).

[0034] The image sensor 34 has multiple pixel blocks. Each pixel block includes at least multiple pixels and one floating diffusion. A pixel block may include, for example, 2×2=4 pixels or 4×4=16 pixels. According to this embodiment, it is possible to determine whether or not to perform binning for each pixel block. When binning is performed, i.e., when pixel values ​​of multiple pixels are simultaneously extracted, the resolution in the three-dimensional direction can be improved (i.e., a higher frame rate can be achieved) at the expense of the resolution in the two-dimensional direction. When binning is not performed, i.e., when pixel values ​​of multiple pixels are individually extracted, the resolution in the two-dimensional direction can be improved at the expense of the resolution in the three-dimensional direction (i.e., the frame rate). The image sensor 34 does not perform binning in areas where stationary objects exist, but performs binning in areas where fast-moving objects exist, thereby enabling imaging with a mixture of areas where binning is performed and areas where binning is not performed.

[0035] The sensor drive boards 35 are provided corresponding to the image sensors 34. In the example shown, a sensor drive board 35G is provided corresponding to the image sensor 34G, a sensor drive board 35B is provided corresponding to the image sensor 34B, and a sensor drive board 35R is provided corresponding to the image sensor 34R. The sensor drive boards 35 control the image sensors 34 in accordance with the drive mode and read out pixel values ​​from the image sensors 34.

[0036] Sensor power supply board 36 supplies power to sensor drive board 35. Sensor power supply board 36 may be provided corresponding to sensor drive board 35. In the example shown in the figure, sensor power supply board 36G is provided corresponding to sensor drive board 35G, sensor power supply board 36B is provided corresponding to sensor drive board 35B, and sensor power supply board 36R is provided corresponding to sensor drive board 35R.

[0037] The scene information and image acquisition boards 37 are provided corresponding to the sensor drive boards 35. In the example shown in the figure, a scene information and image acquisition board 37G is provided corresponding to the sensor drive board 35G, a scene information and image acquisition board 37B is provided corresponding to the sensor drive board 35B, and a scene information and image acquisition board 37R is provided corresponding to the sensor drive board 35R.

[0038] The scene information / image acquisition board 37 generates scene information based on pixel values ​​read by the sensor drive board 35. The scene information is generated from a brightness map and a motion map. The brightness map may be generated based on an image captured by the image sensor 34. For example, the brightness map may indicate the degree of brightness for each pixel block. Examples of the degree of brightness include bright determination (Bright), dark determination (Dark), and neutral determination (Normal). The motion map may also be generated based on an image captured by the sub-sensor 33. For example, the brightness map may indicate the presence or degree of motion for each pixel corresponding to a pixel block of the image sensor 34. Examples of the presence or degree of motion include moving determination (Fast) and still determination (Slow). The scene information may indicate the degree of brightness or the degree of motion for each pixel block.

[0039] For example, as an example of scene information, pixel blocks determined to be moving may be determined to be moving, and pixel blocks determined to be still may be determined to be bright, dark, or neutral based on a brightness map.

[0040] Note that one piece of scene information may be generated in common for all RGB colors, or different scene information may be generated for each RGB color. When one piece of scene information is generated in common for all RGB colors, the result of G, which has the greatest influence on luminance among the RGB colors, may be prioritized, or one piece of scene information common to all RGB colors may be generated using other logic. The luminance map and scene information may be shared among the RGB colors.

[0041] The signal processing board 38 generates video data based on the pixel values ​​read out by the sensor driving board 35 and the scene information generated by the scene information / video acquisition board 37. Note that the image information captured by the image sensor 34 according to this embodiment includes areas that have been binned and areas that have not been binned, so in order to display the video information on the video display device 5, it is preferable to perform spatiotemporal interpolation processing to generate the video information. However, the video data generated by the signal processing board 38 may be information in a state before the spatiotemporal interpolation processing is performed.

[0042] The power supply module 39 supplies power to each component included in the imaging device 3 .

[0043] The video display device 5 displays the video captured by the imaging device 3. The video display device 5 includes a time-space interpolation processing unit 51, a signal processing unit 52, and a display unit 53.

[0044] The spatiotemporal interpolation processing unit 51 acquires video data and scene information from the imaging device 3. The spatiotemporal interpolation processing unit 51 performs interpolation processing in time and space based on the acquired video data and scene information. Specifically, the spatiotemporal interpolation processing unit 51 interpolates missing pixels in two dimensions or three dimensions based on the video data and scene information. An example of an interpolation method is linear interpolation.

[0045] The signal processing unit 52 generates a video signal based on the information interpolated by the spatio-temporal interpolation processing unit 51 .

[0046] The display unit 53 performs display based on the video signal generated by the signal processing unit 52. The display unit 53 may be, for example, a liquid crystal display, an organic EL (Electroluminescence) display, or the like.

[0047] The video display device 5 does not need to be a device that includes all of the components of the spatiotemporal interpolation processing unit 51, the signal processing unit 52, and the display unit 53, but may be a device in which the spatiotemporal interpolation processing unit 51 and the signal processing unit 52 are added to an existing display unit 53.

[0048] FIG. 2 is a functional configuration diagram showing an example of the detailed functional configuration of the scene information / image acquisition board and signal processing board according to this embodiment. An example of the detailed functional configuration of the scene information / image acquisition board 37 and the signal processing board 38 will be described with reference to the same figure. It should be noted that a scene information / image acquisition board 37 exists for each of the RGB colors. Hereinafter, the configuration that processes the image sensor 34G that receives green light will be referred to as the scene information / image acquisition board 37G, the configuration that processes the image sensor 34B that receives blue light will be referred to as the scene information / image acquisition board 37B, and the configuration that processes the image sensor 34R that receives red light will be referred to as the scene information / image acquisition board 37R. The scene information / image acquisition board 37G, the scene information / image acquisition board 37B, and the scene information / image acquisition board 37R may have similar configurations. In the description given with reference to the same figure, only the scene information / image acquisition board 37G will be described as a representative. It should be noted that when the colors are not to be distinguished, they may simply be referred to as the scene information / image acquisition board 37.

[0049] The scene information / image acquisition board 37 includes a sensor data receiving unit 311, a binarization unit 312, a filtering / expansion processing unit 313, an affine transformation unit 314, a motion map generation unit 315, a sensor data receiving unit 321, a pixel defect correction unit 3215, an OB clamp unit 322, an FPN cancellation unit 323, a dark mode correction unit 324, a linearity correction unit 325, a brightness map generation unit 331, a scene information generation unit 332, a drive mode transmission unit 333, and a drive mode out control unit 334.

[0050] The sensor data receiving unit 311, binarization unit 312, filtering and expansion processing unit 313, affine transformation unit 314, and motion map generation unit 315 may be provided in any one of the scene information and image acquisition boards 37R for each color. In the example shown in the figure, these components are provided in the scene information and image acquisition board 37G. Alternatively, these components may be provided on a board separate from the scene information and image acquisition board 37.

[0051] The sensor data receiving unit 311 receives sensor data from the sub-sensor 33. The sensor data acquired by the sensor data receiving unit 311 is, for example, information indicating changes in pixel values ​​on each two-dimensional coordinate. The sensor data includes information indicating which of three types of states exists for each coordinate, compared with the pixel values ​​of a past frame image: a change in the direction of increasing the pixel value (positive), a change in the direction of decreasing the pixel value (negative), or no change.

[0052] The binarization unit 312 converts the three types of states of the sensor data into two states: whether there has been a change or not. That is, the binarization unit 312 binarizes the sensor data. The binarization unit 312 may generate a binary event image, for example, in which pixels that have accumulated either positive or negative are colored white, and pixels that have accumulated neither are colored black.

[0053] The filtering and expansion processing unit 313 performs noise removal processing from the information binarized by the binarization unit 312. The filtering and expansion processing unit 313 may perform opening processing, median filter processing, and expansion processing on the event image, for example. The base and filter radius for each processing may be set from an external terminal or the like. Furthermore, the processing performed by the filtering and expansion processing unit 313 may be bypassable.

[0054] The affine transformation unit 314 performs a process of adjusting the size of the image captured by the sub-sensor 33 to the size of the image captured by the image sensor 34. The process performed by the affine transformation unit 314 can also be called an offset process or a trimming process. Furthermore, when the optical axis of the sub-sensor 33 and the optical axis of the image sensor 34 do not completely coincide with each other, the affine transformation unit 314 may adjust the position by performing an affine transformation in addition to or instead of adjusting the image size. The transformation parameters used in the affine transformation may be set from an external terminal or the like. Furthermore, the process performed by the affine transformation unit 314 may be bypassable.

[0055] The motion map generator 315 processes the sensor data captured by the sensor data receiver 311 and subjected to predetermined processing to generate a motion map. The motion map is a traveled map in which the number of events (the number of pixels with changes) is accumulated for each control block, and if the accumulated number of events is greater than a threshold, it is determined to be moving, and if it is less than a threshold, it is determined to be still. The motion map can also be said to be information indicating the degree of change from the immediately preceding frame image. The threshold used for the determination may be set from an external terminal or the like.

[0056] The sensor data receiving unit 321 receives sensor data from the image sensor 34. The sensor data acquired by the sensor data receiving unit 321 is, for example, information indicating changes in pixel values ​​on each two-dimensional coordinate system, i.e., image information. The sensor data includes pixel values ​​for each coordinate system. Here, the image sensor 34 has multiple pixel blocks, each including multiple pixels and a floating diffusion shared by the multiple pixels. Therefore, the sensor data may include information indicating the presence or absence of binning for each pixel block. In the following description, the sensor data receiving unit 321 may be referred to as a pixel information acquiring unit, and the process performed by the sensor data receiving unit 321 may be referred to as a pixel information acquiring process.

[0057] In the following description, the image sensor 34 may be referred to as the first imaging element, and the sub-sensor 33 may be referred to as the second imaging element. The first imaging element may be referred to as an image sensor that converts incident visible light into an electrical signal, and the second imaging element may be referred to as an infrared sensor that converts incident infrared light into an electrical signal. Furthermore, the second imaging element may be referred to as an event vision sensor (EVS) that detects whether or not there is a change in movement according to a change in the amount of incident light.

[0058] The pixel defect correction unit 3215 corrects pixel values ​​at coordinates where a pixel defect occurs among multiple pixels included in the image sensor 34. Information on whether or not a pixel defect occurs may be detected in advance using a predetermined method, and the detected information may be stored. Alternatively, the presence or absence of a pixel defect may be determined for each frame image. The pixel defect correction unit 3215 corrects the pixel value of a pixel with a pixel defect depending on whether or not binning is performed and the pixel value. More specifically, if a pixel included in a pixel block is a defective pixel, the pixel defect correction unit 3215 corrects the pixel value depending on at least one of the pixel values ​​of other pixels included in the pixel block and the pixel values ​​of pixels included in a neighboring pixel block.

[0059] The OB clamp unit 322 cancels an increase in dark current caused by a temperature rise or the like by subtracting pixel values ​​based on pixel values ​​in the OB (Optical Black) region. The OB region may be provided in either or both of the vertical and horizontal portions of the image sensor 34. Note that the processing performed by the OB clamp unit 322 may be bypassable.

[0060] The FPN cancellation unit 323 performs a process of canceling fixed pattern noise (FPN) caused by dark current components. The FPN cancellation unit 323 performs an FPN correction process and an update process of FPN data used in the FPN correction process. Note that the process performed by the FPN cancellation unit 323 may be bypassable.

[0061] The Dark mode correction unit 324 performs Dark mode correction. The correction value used for Dark mode correction may be stored in advance or may be acquired as needed. Note that the processing performed by the Dark mode correction unit 324 may be bypassable.

[0062] The linearity correction unit 325 corrects the linearity of the image. For the linearity correction, a known technique such as gamma correction may be used. Note that the processing performed by the linearity correction unit 325 may be bypassable.

[0063] In the following description, a configuration including any one of the pixel defect correction unit 3215, the OB clamp unit 322, and the FPN cancellation unit 323 may be referred to as a correction unit.

[0064] The luminance map generation unit 331 generates a luminance map. The luminance map is information indicating the degree of brightness of pixel values ​​included in each pixel block. Note that "each pixel block" includes each control block configured including multiple pixels. The degree of brightness indicated in the luminance map may be, for example, three degrees of brightness: bright determination, dark determination, and intermediate determination. The degree of brightness may be determined, for example, according to the number of pixels whose pixel values ​​are greater than a predetermined threshold value or the number of pixels whose pixel values ​​are smaller than a predetermined threshold value among the multiple pixels included in the pixel block to be determined. Note that in the following description, the process performed by the luminance map generation unit 331 may be referred to as a luminance map generation process.

[0065] Here, the degree of brightness in the luminance map may fluctuate from frame to frame when pixel values ​​are near a threshold value. When such fluctuations occur, problems such as flicker may occur in an image captured by the image sensor 34 controlled based on the fluctuating luminance map, resulting in degradation of subjective image quality. Therefore, the luminance map generation unit 331 may correct pixel information (specifically, pixel values) according to scene information of the immediately preceding frame and generate a luminance map according to the corrected pixel information. Specifically, the luminance map generation unit 331 may correct the acquired pixel information by multiplying it by a different coefficient depending on whether the luminance map of the corresponding pixel block in the immediately preceding frame is determined to be bright, dark, or neutral. For example, if the luminance map of the corresponding pixel block in the immediately preceding frame is determined to be bright, the correction coefficient may be multiplied by approximately 1.1 to 1.2; if the luminance map is determined to be dark, the correction coefficient may be multiplied by approximately 0.9 to 0.8; and if the luminance map is determined to be neutral, the correction coefficient may not be multiplied (or, alternatively, the correction coefficient may be multiplied by 1). The correction coefficient may be multiplied by a correction circuit (not shown), which can provide the effect of suppressing fluctuations in drive mode determination.

[0066] The scene information generation unit 332 generates scene information. Scene information is information indicating the brightness or degree of change for each pixel block. The brightness or degree of change included in the scene information may include the brightness levels included in the brightness map, i.e., light, dark, and neutral determinations, as well as a motion determination. The scene information generation unit 332 may generate scene information for each pixel block, for example, by employing the brightness level indicated in the brightness map generated by the brightness map generation unit 331 or the degree of change indicated in the motion map generated by the motion map generation unit 315. Note that in the following description, the process performed by the scene information generation unit 332 may be referred to as a scene information generation process.

[0067] The scene information generated by the scene information generation unit 332 is stored (buffered) in a predetermined temporary storage unit. The scene information generation unit 332 may generate scene information based additionally on scene information from one frame before. By performing correction based additionally on scene information from one frame before, the scene information generation unit 332 can prevent a situation in which the determination results differ from frame to frame when the degree of brightness or the degree of movement is near a threshold value.

[0068] The drive mode transmission unit 333 controls the drive of the image sensor 34 based on the scene information generated by the scene information generation unit 332. The drive mode transmission unit 333 may also control the drive of the image sensor 34 based on an instruction from the drive mode outside control unit 334.

[0069] The out-of-drive mode control unit 334 acquires an operation related to the control of the image sensor 34 from the user, transmits a control signal based on the acquired operation to the drive mode transmission unit 333, and controls the drive mode of the image sensor 34. Note that the out-of-drive mode control unit 334 is not limited to performing control based on user control, and may control the drive mode of the image sensor 34 by other methods.

[0070] It can also be said that the scene information used by the drive mode transmission unit 333 for drive control is information from one frame ago. In other words, it can also be said that the image sensor 34 captures an image of the (n+1)th frame in accordance with scene information generated in accordance with the frame image of the nth frame. It should be noted that, as described above, the motion map and brightness map are referenced to generate the scene information.

[0071] Since the scene information / image acquisition board 37 generates a luminance map, a device having this function can also be called a luminance map generation device. Also, since the scene information / image acquisition board 37 generates scene information, a device having this function can also be called a scene information generation device. Also, since the scene information / image acquisition board 37 has a configuration including a pixel defect correction unit 3215, a scene information generation unit 332, and an FPN cancellation unit 323, etc., and thereby corrects an image, a device having this function can also be called an image correction device.

[0072] The signal processing board 38 includes an event data output unit 381, a paint unit 382, ​​a linear matrix unit 383, a gamma correction unit 384, a detail unit 385, a video output unit 386, and a scene information output unit 387. By including these components, the signal processing board 38 adjusts the color tone, linearity, and contrast of the image.

[0073] The event data output unit 381 transmits the motion map generated by the motion map generation unit 315 to the video display device 5. The event data output unit 381 may convert the motion map generated by the motion map generation unit 315 into a predetermined format and transmit it. For example, the event data output unit 381 may transmit the motion map using a signal based on the 3G-SDI standard.

[0074] The paint unit 382 acquires pixel information for each of the RGB colors from the scene information and image acquisition board 37G, the scene information and image acquisition board 37B, and the scene information and image acquisition board 37R. The paint unit 382 performs painting processing based on the acquired color information for each color.

[0075] The linear matrix unit 383 performs linear matrix correction, which is a correction for reproducing colors that cannot be expressed when dividing into three primary colors using a prism, and may be a known method.

[0076] The gamma correction unit 384 performs gamma correction. Gamma correction is a correction that corrects the gamma characteristics (gamma curve) to bring out details in shadows and highlights while maintaining the overall image quality. A known method may be used for the processing performed by the gamma correction unit 384.

[0077] The detail section 385 performs correction to bring out further details. The processing performed by the detail section 385 may be performed using a known method.

[0078] In addition to the above-mentioned corrections, the signal processing board 38 may also perform aperture correction to correct high-frequency components lost due to the MTF (Modulation Transfer Function) of the optical system or an optical low-pass filter inserted in front of the sensor, and shading correction to make the level uniform across the entire screen.

[0079] The video output unit 386 outputs the video signal obtained as a result of the above-described correction processing. For example, the video output unit 386 may transmit the video signal as a signal based on the 12G-SDI standard.

[0080] The scene information output unit 387 transmits the scene information generated by the scene information generation unit 332 to the video display device 5. The scene information output unit 387 may convert the scene information generated by the scene information generation unit 332 into a predetermined format and transmit it. For example, the event data output unit 381 may convert a motion map into a video signal using a signal based on the 3G-SDI standard and transmit it.

[0081] Next, pixel defect correction will be described with reference to Figures 3 and 4. Pixel defect correction is performed by the pixel defect correction unit 3215 described above.

[0082] FIG. 3 is a diagram illustrating a first method of pixel defect correction according to this embodiment. First, the first method of pixel defect correction will be described. In the first method, correction is performed based on other pixels (pixels without defects) included in a pixel block that includes a pixel with a pixel defect. The diagram illustrates an example in which a pixel block includes pixels A, B, C, and D. It is assumed that pixel D is a pixel with a pixel defect, and pixels A, B, and C are pixels without a pixel defect. In the description that will be made with reference to the diagram, the output value of pixel A may be referred to as a, the output value of pixel B as b, the output value of pixel C as c, and the output value of pixel D as d. Furthermore, when binning is performed, the pixel values ​​output in each frame (subframe) with a high frame rate may be referred to as S1 to S4. For S1 to S4, the values ​​a+b+c+d are output during each subframe period.

[0083] First, in the first method, when binning is not performed, i.e., when pixel values ​​are read out for each pixel, the output value d' of a pixel D having a pixel defect is corrected to a value obtained by statistically calculating a, b, and c. Examples of statistical calculations include using an average value or a median value. When the pixel block to be corrected is not binned, the pixel defect correction unit 3215 can also correct the pixel value of the defective pixel to a value obtained by statistically calculating the pixel values ​​of the other pixels included in the pixel block. In this case, d' can also be expressed as d' = (a + b + c) / 3 or d' = median(a, b, c).

[0084] Furthermore, in the first method, when binning is performed, i.e., when pixel values ​​are read for each pixel block, the pixel values ​​of the pixel block are corrected based on the pixel values ​​of the non-defective pixels contained in the pixel block. Specifically, the pixel defect correction unit 3215 can also correct the pixel values ​​of the pixel block by subtracting the pixel value of d having a pixel defect from the values ​​read from the entire pixel block and multiplying the result by a coefficient for adjusting the number of pixels contained in the pixel block. In this case, sn' = (sn - d) × (4 / 3), where n is a natural number from 1 to 4.

[0085] In the above description, the first method was described as an example using a pixel block having 2 × 2 = 4 pixels. Instead of multiplying by 4 / 3, multiplying by n / n-1 (n is the number of pixels in the pixel block) allows the size of the pixel block to be generalized. This type of correction makes it possible to achieve appropriate pixel defect correction depending on whether binning is used or not. The first method is effective for defects in which the output value of a pixel having a pixel defect is sufficiently small.

[0086] FIG. 4 is a diagram illustrating a second method of pixel defect correction according to this embodiment. Next, the second method of pixel defect correction will be described. In the second method, correction is performed based on pixel values ​​of pixel blocks located near a pixel block containing a pixel with a pixel defect. The diagram illustrates an example in which each pixel block contains pixel A, pixel B, pixel C, and pixel D. Of the pixels contained in pixel block PB1 to be corrected, pixel D1 is a pixel with a pixel defect, while pixels A1, B1, and C1 are pixels without a pixel defect. Furthermore, none of pixel blocks PB2 to PB5 located near pixel block PB1 contain a pixel with a pixel defect. In the description that will be made with reference to the diagram, the output value of pixel An contained in pixel block PBn (n is a natural number from 1 to 5) may be referred to as an, the output value of pixel Bn as bn, the output value of pixel Cn as cn, and the output value of pixel Dn as dn. Furthermore, when binning is performed, the output value of a pixel block PBn in an mth subframe (m is a natural number from 1 to 4) may be written as smn.

[0087] In the second method, correction is performed using pixel values ​​of pixel blocks surrounding a pixel block containing a pixel with a pixel defect. The surrounding pixel blocks may be four pixel blocks vertically and horizontally adjacent to the pixel block to be corrected. The surrounding pixel blocks may include pixel blocks that undergo binning and pixel blocks that do not undergo binning. When referencing pixel values ​​of a pixel block that undergoes binning, pixel values ​​of a subframe having a scan timing corresponding to the pixel with the pixel defect may be referenced. When referencing pixel values ​​of a pixel block that does not undergo binning, pixel values ​​of a pixel at a position corresponding to the pixel with the pixel defect may be referenced. In other words, in the second method, correction can also be performed using information on pixel blocks in the surrounding pixel blocks that have the same scan timing as the pixel defect.

[0088] When pixel block PB1 does not use binning, i.e., when pixel values ​​are read out pixel by pixel, the output value d1' of pixel D with a pixel defect is corrected using the pixel value d of D at the corresponding position from the control blocks without binning surrounding the pixel block, and the output result of the fourth scan from the pixel blocks with binning. If none of the surrounding pixel blocks use binning, then d1' = d2 + d3 + d4 + d5 is calculated as a statistical value (for example, this can be divided by 4 and used as the average; similarly below). If all of the surrounding pixel blocks use binning, then d1' = s42 + s43 + s44 + s45 is calculated as a statistical value. Furthermore, if some of the surrounding pixel blocks do not undergo binning and others do, the value is obtained by statistically calculating d1'=d2+d3+s44+s45, etc. (In this case, pixel blocks pb2 and pb3 do not undergo binning, and pixel blocks pb4 and pb5 do undergo binning.) Examples of statistical calculations that can be used include median, bilinear, and bicubic.

[0089] When pixel block PB1 is binned, i.e., when pixel values ​​are read out for each pixel block, the read values ​​for each subframe may be the same as those for the non-binned case described above. That is, when all surrounding pixel blocks do not perform binning, the read values ​​are the values ​​obtained by statistically calculating s11'=a2+a3+a4+a5, s21'=b2+b3+b4+b5, s31'=c2+c3+c4+c5, and s41'=d2+d3+d4+d5. When all surrounding pixel blocks perform binning, the read values ​​are the values ​​obtained by statistically calculating s11'=s12+s13+s14+s15, s21'=s22+s23+s24+s25, s31'=s32+s33+s34+s35, and s41'=s42+s43+s44+s45. Furthermore, if there are pixel blocks that do not undergo binning and pixel blocks that do undergo binning, the value is obtained by statistically calculating s1'=a2+a3+s14+s15, etc. (In this case, pixel blocks pb2 and pb3 do not undergo binning, and pixel blocks pb4 and pb5 do undergo binning.) Examples of statistical calculations that can be used include median, bilinear, and bicubic.

[0090] Note that weighting may be applied depending on whether binning is used or not, and averaging may be performed. For example, the weighting may be large for no binning and small for binning. Furthermore, for pixel blocks that contain pixels with pixel defects and that have binning, the above correction may be performed on all outputs in accordance with the scan order. This type of correction makes it possible to achieve appropriate pixel defect correction depending on whether binning is used or not.

[0091] Next, OB clamping will be described with reference to Figures 5 and 6. OB clamping is performed by the OB clamping unit 322 described above.

[0092] 5 is a first diagram for explaining the OB clamp according to this embodiment. The diagram shows the effective pixel area and the OB (Optical Black) area of ​​the image sensor 34. An example of the OB area according to this embodiment will be described with reference to the diagram.

[0093] FIG. 5A shows an example of a case where an OB region is provided in the horizontal direction. As shown in the figure, the OB region according to this embodiment includes a Fast region (with binning) for calculating correction coefficients for correcting pixel blocks determined to be moving in the scene information, and a Normal region (without binning) for calculating correction coefficients for correcting pixel blocks determined to be other than moving in the scene information (i.e., still, or alternatively, bright, dark, or neutral). In the example shown in the figure, the Fast region is provided at both ends of the effective pixel region in the horizontal direction, and the Normal region is provided further outside the Fast region. The Fast region and the Normal region may each have 64 lines. In the following description, pixels included in the Fast region may be referred to as binned correction coefficient calculation pixels. Similarly, pixels included in the Normal region may be referred to as non-binned correction coefficient calculation pixels.

[0094] The arrangement of the fast region and the normal region is not limited to the example shown in the figure. For example, the normal region may be adjacent to the effective pixel region, with the fast region located outside of that. Furthermore, the OB region does not need to be symmetrical as in the example shown in the figure, and one effective pixel region may be adjacent to the fast region and the other to the normal region.

[0095] FIG. 5B shows an example of a case where an OB region is provided in the vertical direction. As shown in the figure, the OB region may be provided in the vertical direction. In the example shown in the figure, fast regions are provided at both vertical ends of the effective pixel region, and normal regions are provided further outside the fast regions. The fast regions and normal regions may each have 64 lines. The arrangement of the fast regions and normal regions is not limited to the example shown in the figure. For example, the normal region may be adjacent to the effective pixel region, and the fast region may be located outside of that. Furthermore, the OB regions do not need to be symmetrical in the vertical direction as in the example shown in the figure, and one side of the effective pixel region may be adjacent to the fast region and the other side may be adjacent to the normal region.

[0096] 5A and 5B show an example in which the OB area is provided in either the horizontal or vertical direction. However, this embodiment is not limited to this example, and various arrangements are possible. For example, the OB area may be provided in both the left-right and top-bottom directions, or the effective pixel area may be provided below and to the right of the effective pixel area in an L-shape.

[0097] FIG. 6 is a second diagram for explaining the OB clamp according to this embodiment. OB clamp correction will be described in detail with reference to this drawing. According to this embodiment, correction values ​​are calculated from the Fast region and the Normal region as described above. One example of a method for calculating the correction values ​​is to find the average value for each line.

[0098] In the illustrated example, the number of pixels in the image sensor 34 is H2084×V1132 pixels. Each pixel block is composed of 2×2 pixels. In the illustrated example, scene information is generated for each pixel block, with one pixel block serving as a control block, resulting in a H1040×V564 pixel block. Scene information is generated for each pixel block. For each pixel block, a correction coefficient is identified by referring to the scene information, and the identified correction coefficient is subtracted from the pixel value of the pixel block. In the illustrated example, a correction coefficient based on the normal region is subtracted from the pixel value of the pixel block.

[0099] Next, FPN cancellation will be described with reference to Figures 7 and 8. FPN cancellation is performed by the FPN cancellation unit 323 described above.

[0100] 7 is a first diagram for explaining FPN cancellation according to this embodiment. First, a method for acquiring FPN (Fixed Pattern Noise) data will be described with reference to the diagram.

[0101] Figure 7(A) shows the FPN data acquisition method for modes other than motion detection (Fast) (i.e., light detection, dark detection, and neutral detection). First, all pixel blocks in the effective pixel area are set to Normal mode (no binning), and pixel values ​​for four subframes are acquired by shading and frame addition.

[0102] Figure 7(B) shows the FPN data acquisition method for motion detection (Fast). First, all pixel blocks in the effective pixel area are set to Fast mode (with binning), and pixel values ​​for four subframes are acquired by shading and frame addition.

[0103] The FPN cancellation unit 323 calculates FPN data from the pixel values ​​acquired by the methods of Figures 7(A) and 7(B). This FPN data may be calculated, for example, by averaging pixel values ​​for four subframes in the time direction. The FPN cancellation unit 323 stores the calculated FPN data and uses it to correct image data.

[0104] 8 is a second diagram for explaining FPN cancellation according to this embodiment. Next, with reference to the same figure, a correction method using FPN data obtained by the method described above will be described. According to this embodiment, the FPN data calculated by the method described above depending on whether binning is present or not is subtracted for each pixel block depending on whether binning is present or not, thereby canceling fixed pattern noise and correcting pixel values.

[0105] In the illustrated example, the number of pixels in the image sensor 34 is H2084×V1132 pixels. Each pixel block is composed of 2×2 pixels. In the illustrated example, scene information is generated for each pixel block, with one pixel block serving as a control block, resulting in a H1040×V564 pixel block. Scene information is generated for each pixel block. For each pixel block, a correction coefficient is identified by referring to the scene information, and the identified correction coefficient is subtracted from the pixel value of the pixel block. In the illustrated example, FPN data for a mode other than motion determination (Fast) is subtracted from the pixel value of the pixel block.

[0106] That is, according to this embodiment, FPN data acquired in advance depending on whether or not binning is used is used, and for pixel blocks without binning, the FPN data without binning of the pixel block is subtracted. Also, for pixel blocks with binning, the FPN data with binning of the pixel block is subtracted. By adopting such a configuration, appropriate FPN cancellation can be achieved.

[0107] FIG. 9 is a block diagram showing an example of the internal configuration of an imaging device according to this embodiment. At least some of the functions of the configuration of the imaging device 3 can be implemented using a computer. As shown in the figure, the computer includes a central processing unit 901, a RAM 902, an input / output port 903, input / output devices 904 and 905, and a bus 906. The computer itself can be implemented using existing technology. The central processing unit 901 executes instructions contained in a program read from the RAM 902 or the like. In accordance with each instruction, the central processing unit 901 writes data to the RAM 902, reads data from the RAM 902, and performs arithmetic and logical operations. The RAM 902 stores data and programs. Each element included in the RAM 902 has an address and can be accessed using the address. RAM is an abbreviation for "random access memory." The input / output port 903 is a port through which the central processing unit 901 exchanges data with external input / output devices. The input / output devices 904 and 905 are input / output devices. The input / output devices 904 and 905 exchange data with the central processing unit 901 via the input / output port 903. The bus 906 is a common communication path used within the computer. For example, the central processing unit 901 reads and writes data from and to the RAM 902 via the bus 906. Also, for example, the central processing unit 901 accesses the input / output port via the bus 906. All or part of the functional units provided in the imaging device 3 may be realized using hardware such as an ASIC, a PLD, or an FPGA. All or part of the functional units may be realized by a combination of software and hardware.

[0108] [Summary of the embodiment] According to the above-described embodiment, the image correction device includes a sensor data receiving unit 311, a scene information generating unit 332, and a correction unit. The sensor data receiving unit 311 acquires pixel information captured by an image sensor 34 having a plurality of pixel blocks, each of which has a plurality of pixels and a floating diffusion shared by the plurality of pixels. The scene information generating unit 332 generates scene information indicating the degree of change in brightness or pixel value for each pixel block, according to the pixel values ​​of the pixel information. The correction unit corrects the pixel values ​​according to the presence or absence of binning based on the scene information and the pixel values ​​of the pixel information. According to this embodiment, in an image sensor capable of switching between the presence or absence of binning in local regions, noise caused by manufacturing variations and characteristic variations in circuits superimposed on the image sensor 34 can be suitably reduced using a simple method. Furthermore, according to this embodiment, the image quality of the image capture device can be improved.

[0109] It should be noted that the effects described in this specification are merely descriptive or exemplary and are not limiting. In other words, the technology according to the present disclosure may achieve other effects in addition to or in place of the above-described effects that would be apparent to those skilled in the art from the description of this specification. Furthermore, the present invention is not limited to these embodiments, and various modifications and substitutions can be made within the scope of the present invention. [Explanation of symbols]

[0110] 1. Imaging system 3. Imaging device 5. Video display devices 31 Lens 32 Prism 33 Sub-sensor 34 Image Sensor 35 Sensor drive board 36 Sensor power supply board 37 Scene information and image acquisition board 38 Signal Processing Board 39 Power Supply Module 51 Spatiotemporal Interpolation Processing Unit 52 Signal processing section 53 Display section

Claims

1. a pixel information acquisition unit that acquires pixel information captured by an image sensor having a plurality of pixel blocks, each pixel block having a plurality of pixels and a floating diffusion shared by the plurality of pixels, and a control circuit that can independently control whether or not binning is performed for each pixel block or for each of the plurality of pixel blocks; a scene information generating unit that generates scene information indicating a degree of change in brightness or pixel value for each pixel block according to the pixel values ​​of the pixel information; a correction unit that corrects pixel values ​​according to whether binning is performed based on the scene information and the pixel values ​​of the pixel information; An image correction device comprising:

2. the correction unit further includes a pixel defect correction unit that, when a pixel included in the pixel block is a defective pixel, corrects the pixel value in accordance with pixel values ​​of other pixels included in the pixel block or pixel values ​​of pixels included in neighboring pixel blocks; The image correction device according to claim 1 .

3. When the pixel block to be corrected is not binned, the pixel defect correction unit corrects the pixel value of the defective pixel to a value obtained by statistically calculating pixel values ​​of other pixels included in the pixel block. The image correction device according to claim 2 .

4. when the pixel block to be corrected is subjected to binning, the pixel defect correction unit corrects pixel values ​​of the pixel block based on pixel values ​​of pixels included in the pixel block that are not the defective pixels; The image correction device according to claim 2 .

5. when the pixel block to be corrected is not binned, the pixel defect correction unit corrects the pixel value of the defective pixel to a value obtained by statistically calculating pixel values ​​read from the four pixel blocks adjacent in the vertical and horizontal directions. The image correction device according to claim 2 .

6. when the pixel block to be corrected is subjected to binning, the pixel defect correction unit sets the pixel value of the defective pixel to a value obtained by statistically calculating pixel values ​​read from the four pixel blocks adjacent in the vertical and horizontal directions, as the pixel value read from the pixel block. The image correction device according to claim 2 .

7. the correction unit further includes an OB (Optical Black) clamp unit that corrects pixel values ​​by subtracting a correction coefficient according to the presence or absence of binning based on the scene information, based on correction coefficients obtained from binning-enabled correction coefficient calculation pixels and no-binning correction coefficient calculation pixels provided in an OB (Optical Black) region. The image correction device according to claim 1 .

8. the correction unit further includes an FPN cancellation unit that corrects pixel values ​​by subtracting, for each pixel block, FPN (Fixed Pattern Noise) data calculated depending on whether binning is performed or not, depending on whether binning is performed or not, thereby canceling fixed pattern noise. The image correction device according to claim 1 .

9. an imaging element having a plurality of pixel blocks each having a plurality of pixels and a floating diffusion shared by the plurality of pixels; and an image correction device according to claim 1 , which corrects pixel values ​​of pixel information captured by the image sensor. Imaging device.

Images