Signal processing device, photoelectric conversion device and equipment

Abstract

To provide a configuration advantageous for improving the accuracy of detecting a lateral smear and preventing the occurrence of lateral stripes when the lateral smear is corrected.SOLUTION: A signal processing device comprises a line memory, a detection unit, a correction value generation unit, and a data correction unit. The line memory holds image data of a selected row. The detection unit generates a selection signal for selecting a correction coefficient on the basis of at least the image data of the selected row. The correction value generation unit has a correction value holding unit that holds a correction value, and generates and holds the correction value on the basis of the correction coefficient selected with the selection signal and light shielding data of the selected row. The data correction unit corrects the image data of the selected row held in the line memory on the basis of the correction value.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to a signal processing device, a photoelectric conversion device, and equipment.

Background Art

[0002] There is a method of performing black level correction based on data from a light-shielding region, which has a light-receiving region including a plurality of pixels and a light-shielded light-shielding region (which may also be called an optical black (OB) region) arranged around the light-receiving region. Patent Document 1 describes an apparatus for correcting streaking (also referred to as "horizontal smear") that occurs on the left and right sides of a high-brightness subject when imaging the high-brightness subject.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Patent Document 1 describes an apparatus that detects the occurrence of streaking from the data of light-shielding pixels in the same row as the light-receiving pixels when there is a high-brightness subject in the light-receiving region, and performs streaking correction based on the offset values detected on the left and right sides of the high-brightness subject, respectively. In Patent Document 1, if random noise or the like is included in the signal obtained from the pixels in the optical black region, the detection accuracy of streaking may be reduced. Also, horizontal stripes may occur due to correction in the region where streaking occurs.

[0005] An object of the present invention is to provide a configuration that is advantageous for improving the detection accuracy of horizontal smear and suppressing the occurrence of horizontal stripes when correcting horizontal smear.

Means for Solving the Problems

[0006] A signal processing device according to one aspect of the present invention is a signal processing device that corrects image data based on signals from a plurality of light-receiving pixels using light-shielding data based on signals from a plurality of light-shielding pixels, comprising a line memory, a detection unit, a correction value generation unit, and a data correction unit, wherein the line memory holds image data of a selected row, and the detection unit has at least the selected row Light-receiving pixel Image data The average value is compared with the average value of the pixel data of light-receiving pixels in rows different from the selected row. Based on this, a selection signal is generated to select a correction coefficient, and the correction value generation unit has a correction value holding unit that holds the correction value, and generates and holds the correction value based on the correction coefficient selected by the selection signal and the light-shielding data of the selected row, and the data correction unit corrects the image data of the selected row held in the line memory based on the correction value. [Effects of the Invention]

[0007] According to the present invention, it is possible to provide a configuration that is advantageous for improving the detection accuracy of transverse smear and for suppressing the occurrence of transverse stripes when transverse smear is corrected. [Brief explanation of the drawing]

[0008] [Figure 1] An example configuration of a photoelectric converter equipped with a signal processing device. [Figure 2] A diagram illustrating the photoelectric conversion unit. [Figure 3] Block diagram of a signal processing unit. [Figure 4] Block diagram of the black level correction unit. [Figure 5] A diagram illustrating the signal level when lateral smear occurs. [Figure 6] Block diagram of the detection unit. [Figure 7] Block diagram of the correction value generation unit of the first embodiment. [Figure 8] This diagram illustrates the signal level before black level correction. [Figure 9] This diagram illustrates the signal level after black level correction. [Figure 10] A flowchart showing the black level correction processing method in the first embodiment. [Figure 11] Block diagram of the black level correction unit in the signal processing device of the second embodiment. [Figure 12] A block diagram of the correction value generation unit in the black level correction unit of the second embodiment. [Figure 13] A flowchart showing the black level correction processing method in the second embodiment. [Figure 14] A block diagram of the detection unit in the black level correction unit of the third embodiment. [Figure 15] A diagram illustrating an example of the application of a signal processing device to equipment. [Modes for carrying out the invention]

[0009] The embodiments will be described in detail below with reference to the attached drawings. Note that the following embodiments do not limit the invention as defined in the claims. While the embodiments describe multiple features, not all of these features are essential to the invention, and the features may be combined in any way. Furthermore, in the attached drawings, identical or similar configurations are given the same reference numerals, and redundant descriptions are omitted.

[0010] In the embodiments described below, the focus will be on imaging devices (imaging devices) as examples of photoelectric conversion devices. However, each embodiment is not limited to imaging devices and can be applied to other examples of photoelectric conversion devices. For example, rangefinders (devices for distance measurement using focus detection or TOF (Time of Flight)), photometers (devices for measuring the amount of incident light, etc.).

[0011] (First Embodiment) A configuration example of the photoelectric conversion device 100 including the signal processing device 109 according to the present embodiment will be described with reference to FIG. 1. The photoelectric conversion device 100 includes a signal processing device 109 and a photoelectric conversion unit 101 that has a light receiving region and a light shielding region to be described later and outputs image data processed by the signal processing device 109. The photoelectric conversion device 100 may further include a vertical scanning unit 103, a control unit 104, a reading unit 105, an AD conversion unit 106, a memory unit 107, and a horizontal scanning unit 108. In the present embodiment, an example where the signal processing device 109 is within the photoelectric conversion device 100 is shown, but it is not limited thereto. The signal processing device 109 may be separate from the photoelectric conversion device 100.

[0012] In the photoelectric conversion unit 101, m×n pixels P(0,0) to P(m−1,n−1) are arranged in a matrix. Each pixel includes a photoelectric conversion element, and charges corresponding to the incident light are generated in the pixel. The vertical scanning unit 103 is connected to the m pixels 102 arranged in that row by row selection lines 110 (in FIG. 1, there are n lines V(0) to V(n−1)), and can select the row from which the signal is to be read. Usually, the vertical scanning unit 103 selects rows from the 0th row to the (n−1)th row. The signals of the pixels in the row selected by the vertical scanning unit 103 are read out to the reading unit 105 via vertical output lines 111 (in FIG. 1, there are m lines H(0) to H(m−1)). The reading unit 105 may include an amplifier or the like and amplify the signal output from the pixel 102.

[0013] In the following description, the direction in which the row selection line 110 extends is referred to as the row direction (the horizontal direction in FIG. 1), and the direction in which the vertical output line 111 extends is referred to as the column direction (the vertical direction in FIG. 1).

[0014] The signal output from the readout unit 105 is converted from an analog signal to a digital signal (AD conversion) by the analog-to-digital (AD) conversion unit 106 and temporarily held in the memory unit 107. Subsequently, the digital signals addressed by the horizontal scanning unit 108 are sequentially read out to the signal processing unit 109, where digital signal processing is performed. The control unit 104 acquires setting information such as the shooting conditions when the photoelectric converter 100 is imaging, and can supply control signals according to the conditions to each component included in the photoelectric converter 100. The control unit 104 can control the vertical scanning unit 103, the readout unit 105, the AD conversion unit 106, the memory unit 107, the horizontal scanning unit 108, and the signal processing unit 109.

[0015] The signal processing device 109 performs a process to reduce reset noise generated by the switching elements (e.g., MOS transistors) contained in each pixel 102 of the photoelectric conversion unit 101. In addition, the image data output from the photoelectric conversion unit 101 may contain dark current generated from the photodiodes contained in the pixels 102, and variations (fixed pattern noise (FPN)) caused by circuit-related differences such as power supply impedance and signal delay. In this specification, the state in which this FPN changes from row to row and from column to column will be referred to as shading.

[0016] The signal processing unit 109 reduces the reset noise component and then averages the light-shielding data row by row and column by column to generate correction data that includes FPN components and shading components. Subsequently, the signal processing unit 109 can perform processing to correct variations between columns and processing to correct the dark current component of pixels. It can also perform black level correction.

[0017] Figure 2 illustrates the arrangement of pixels in the photoelectric conversion unit 101. Pixels in the photoelectric conversion unit 101 are arranged in a matrix. The photoelectric conversion unit 101 includes a light-receiving region 201 where light-receiving pixels that receive incident light passing through an optical system such as a lens are located, and a light-shielding region 202 (also called an optical black (OB) region) where light-shielding pixels that optically block the incident light are located. Image data can be obtained from the light-receiving pixels in the light-receiving region 201. Light-shielding data can also be obtained from the light-shielding pixels in the light-shielding region 202. The light-shielding region 202 is a reference region for determining the black level reference in the data obtained by the photoelectric conversion unit 101.

[0018] The light-shielding region 202 is divided into a vertical OB (VOB) region 203 and a horizontal OB (HOB) region 204. In the VOB region, optically shielded pixels 102 are arranged in the row direction across all columns. In the horizontal OB region, optically shielded pixels 102 are arranged in the column direction across all rows. The VOB region is located at the column end of the photoelectric conversion unit 101 and is shown on the upper side in Figure 2. The HOB region is located at the row end of the photoelectric conversion unit and is shown on the left side in Figure 2.

[0019] Figure 3 illustrates the signal processing device 109 according to this embodiment. The signal processing device 109 includes a scan conversion line memory 301, a column FPN correction unit 302, and a black level correction unit 303. The scan conversion line memory 301 is a storage unit capable of holding at least one row of data input from the memory unit 107. The column FPN correction unit 302 is a circuit that corrects the FPN for each column, including horizontal shading.

[0020] The black level correction unit 303 corrects the dark current component of a pixel using shading data from shading pixels. This process is also called OB clamping. Here, it is referred to as black level correction. The area from which shading data is acquired can be set to any area within the shading area 202. Similarly, the area from which the average value of each row and column of shading data is acquired can also be set to any area within the shading area 202. This area can be said to be the area from which correction values ​​for black level correction are generated. The correction values ​​for black level correction calculated from the shading data can also be called clamp values.

[0021] The configuration of the black level correction unit 303 in the signal processing device 109 of this embodiment will be explained with reference to Figure 4. The black level correction unit 303 may include a line memory 401, a detection unit 402, a correction value generation unit 403, and a data correction unit 404. Image data and light-shielding data of the row selected as the target of black level correction are sequentially input to the black level correction unit 303. The image data and light-shielding data input to the black level correction unit 303 are branched and input to the line memory 401 and the detection unit 402. The line memory 401 is a storage unit that can hold at least one row of data from the selected row. The data from the selected row is input to the detection unit 402 to detect lateral smearing.

[0022] The detection unit 402 detects events that may cause horizontal band-like smearing from the data input to the black level correction unit 303 and generates a correction coefficient selection signal. For example, in a CMOS image sensor, when strong light hits the light-receiving area 201, the signal levels of both the light-receiving area 201 and the HOB area 204 rise, and a phenomenon occurs in which a horizontal band-like area with a different brightness from the rest of the image is generated. This phenomenon is called horizontal smearing.

[0023] The correction value generation unit 403 selects a correction coefficient based on the correction coefficient selection signal output from the detection unit 402 and generates a correction value for black level correction using the light-shielding data output from the line memory 401. The data correction unit 404 can perform black level correction of the data by subtracting the correction value generated by the correction value generation unit 403 from the output data of the line memory 401.

[0024] Lateral smearing is explained with reference to Figures 5(A) and (B). Figure 5(A) shows image data of the light-receiving region 201 and the HOB region 204 when strong light (shown as the white area in Figure 5) hits the center of the screen, causing lateral smearing. 501A shows the position of the rows within the HOB region 204. 502A shows the position of the rows within the light-receiving region 201 that are not hit by strong light. 503A shows the position of the rows within the light-receiving region 201 that are hit by strong light.

[0025] Figure 5(B) shows the signal levels of pixels in the column direction in each row of columns 501A, 502A, and 503A in Figure 5(A). The signal levels of each row in column 501A within the HOB area 204 are shown in 501B of Figure 5(B). The signal levels of each row in column 502A, which is not exposed to strong light within the light-receiving area 201, are shown in 502B. The signal levels of each row in column 503A, which is exposed to strong light within the light-receiving area 201, are shown in 503B.

[0026] To the left and right of the brightly lit area in the center of the screen, not only the signal levels of column 503A but also those of columns 501A and 502A become elevated, which can result in a horizontal band of brightness in the row indicated by T502. This corresponds to the position in Figure 5(B) where the signal level is convex horizontally. Such horizontal bands are called lateral smears.

[0027] The detection unit 402 in the signal processing device 109 of this embodiment will be explained with reference to Figure 6. The detection unit 402 includes a row average value calculation unit 601, a row average value holding unit 602, a difference calculation unit 603, and a correction coefficient selection signal generation unit 604. The detection unit 402 can detect lateral smear and generate a correction coefficient selection signal.

[0028] The row average value calculation unit 601 calculates and outputs the average value of the pixel data of the row to be corrected for black level. The row selected for black level correction is the same row as the row of data held in the line memory 401. The row average value holding unit 602 holds the output of the row average value calculation unit 601. The row average value calculation unit 601 calculates the average of the data from the light-receiving pixels of the light-receiving area 201 of the selected row. The average value may be calculated using data from pixels in both the light-receiving area 201 and the light-shielding area 202.

[0029] The difference calculation unit 603 calculates the difference between the average value of the data calculated by the row average value calculation unit 601 and the average value of the pixel data of a predetermined row held in the row average value holding unit 602. By taking the difference, changes caused by lateral smearing can be detected. The detection unit 402 can detect lateral smearing based on a sufficient number of data by using the data of the light-receiving pixels in the light-receiving area.

[0030] Here, the designated row is the reference row for detecting lateral smearing. If the row to be corrected for black level is the Nth row, then the average value of the pixel data in the Mth row (where M≠N and M≧0) may also be used, and in this case, the Mth row may be the N-1th row. When M=N-1, the values ​​of adjacent rows can be compared to detect changes in data due to lateral smearing. Furthermore, whether the row average value holding unit 602 uses data from the light-receiving region or data from both the light-receiving region and the light-shielding region can be selected according to the operation of the average value calculation unit 601.

[0031] The correction coefficient selection signal generation unit 604 compares the difference calculated by the difference calculation unit 603 with the detection level threshold and generates and outputs a correction coefficient selection signal. If the difference is equal to or greater than the detection level threshold, a correction coefficient selection signal is generated and output that selects a correction coefficient with high black level correction tracking performance. If the difference is smaller than the detection level threshold, a correction coefficient selection signal is generated and output that selects a correction coefficient with low black level correction tracking performance. As shown in Figure 5, the difference can be large at the boundary between the row indicated by T501 and the row indicated by T502, and at the boundary between the row indicated by T502 and the row indicated by T503, so it is desirable to improve the black level correction tracking performance.

[0032] Furthermore, the correction coefficient selection signal generation unit 604 can stop generating the correction coefficient selection signal over multiple rows when the correction coefficient selection signal changes. If the difference changes to a value greater than the detection level threshold, the same correction coefficient selection signal may be output over a predetermined number of rows, including the row targeted for correction and subsequent rows. Instead of generating and outputting a correction coefficient selection signal to select a correction coefficient with high tracking performance for black level correction only for the row targeted for correction, the tracking performance of the correction is improved for the surrounding rows where lateral smearing occurred, including that row. This allows for appropriate correction in the vicinity where lateral smearing occurred.

[0033] The data correction unit 404 corrects the black level of the data of the row held in the line memory 401 based on the correction value generated by the correction value generation unit 403. In this way, the black level of the data of the row held in the line memory 401 is corrected using the correction value generated based on the same row data as the row held in the line memory. The series of operations, from writing to and reading from the line memory 401, detection by the detection unit 402, and generation and correction of the correction value, are performed with timing control by the control unit 104.

[0034] The correction value generation unit 403 in the signal processing device 109 of this embodiment will be explained with reference to Figure 7. The correction value generation unit 403 includes a correction coefficient selection unit 701, an attenuation unit 702, and a correction value holding unit 703. The correction value generation unit 403 generates and updates correction values ​​by filtering. The generation and updating of correction values ​​involves subtracting the correction value held in the correction value holding unit 703 from the light-shielding data of the light-shielding pixels. Next, the attenuation result is attenuated by the attenuation unit 702 using the correction coefficient selected by the correction coefficient selection unit 701. Finally, the attenuation result and the correction value held in the correction value holding unit 703 are added to generate a new correction value, which is then held in the correction value holding unit 703. By sequentially performing this operation for the light-shielding pixels of the row selected as the target of black level correction, a correction value based on the light-shielding pixels of the VOB of the same row as the row written to the line memory 401 is obtained.

[0035] In this embodiment, the filtering of the correction value generation unit 403 can use a filter of the type that smooths the input data, such as a low-pass filter. The filtering in this embodiment is performed by inputting the light-shielding data to the filter pixel by pixel and filtering it. The filter can be configured as an infinite impulse response (IIR) type low-pass filter (LPF). Here, the correction coefficient corresponds to the attenuation coefficient K used in the multiplication of the IIR filter. Note that the filter configuration is not limited to this, as long as the response characteristics of the filter can be changed.

[0036] The correction coefficient selection unit 701 selects either a first correction coefficient or a second correction coefficient based on the correction coefficient selection signal. In this specification, an example is shown in which the first correction coefficient is selected when the correction coefficient selection signal is 0, and the second correction coefficient is selected when the correction coefficient selection signal is 1.

[0037] For example, if the second correction coefficient is chosen to have higher tracking capabilities for the IIR-type LPF than the first correction coefficient, then under normal circumstances, the first correction coefficient is used to generate the correction value, and when lateral smear is detected, the second correction coefficient is used to generate the correction value. By selecting the second correction coefficient, which has higher tracking capabilities when lateral smear is detected, it becomes possible to track the sharp fluctuations in black levels caused by the occurrence of lateral smear, and thus effectively correct the lateral smear.

[0038] The attenuation unit 702 sets the attenuation amount of the IIR type LPF using the correction coefficient selected in the correction coefficient selection unit 701. Therefore, the correction coefficient corresponds to the attenuation coefficient set in the IIR type filter. A larger attenuation coefficient improves the response characteristics of the filter, and appropriate correction is performed at the edge portions where the change in lateral smear is large.

[0039] In this embodiment, an example is shown where there are two correction coefficients to select in the correction coefficient selection unit 701, but there is no limit to the number of correction coefficients. Also, in this embodiment, an example is shown where the correction coefficient selection unit 701 switches the correction coefficient based on the correction coefficient selection signal, but depending on the shooting conditions and the type of imaging device, the control unit 104 can control the signal processing device so that the correction coefficient is not switched.

[0040] The value of the correction value Y by the IIR filter shown in Figure 7 is given by the following equation. Here, Y ―1 is the correction value held in the correction value holding unit 703, K is the attenuation coefficient, and X is the light shielding data.

[0041] Y = K × X + (1 - K) × Y -1 ...Formula 1 Figures 8 and 9 will be used to illustrate the comparison between the image after black level correction in the comparative example and the image after black level correction according to this embodiment. The input data for the black level correction unit 303 is the same in Figures 8 and 9. The black level correction in the comparative example in Figure 8 is a process with low filter processing responsiveness. The black level correction in this embodiment shown in Figure 9 is a process that increases the filter processing responsiveness only for a few rows after detecting lateral smear, and decreases the filter processing responsiveness for the other rows.

[0042] First, we will explain the comparative example of black level correction using Figure 8. Figure 8(A) shows the image of the light-receiving area 201 and the HOB area 204 when strong light (the white area in Figure 8) hits the center of the screen, causing lateral smearing. 801A, 802A, and 803A indicate the column positions which will be explained in Figure 8(C). Figure 8(B) shows the signal level of the correction value in each row, and Figure 8(C) shows the image signal levels in columns 801A, 802A, and 803A after black level correction in the comparative example.

[0043] In the row indicated by T801, no lateral smear occurs, so even if the correction value changes, the magnitude of the change is small. Therefore, even if the tracking performance of the filter processing is low, the change in signal level is small, and good correction results can be obtained.

[0044] The row indicated by T802 represents the region immediately after the start of lateral smearing. Due to the effect of lateral smearing, the signal level in column 801A is high, and because the filtering process has low tracking ability, the correction value increases only gradually, as shown in Figure 8(B). Therefore, it takes time for the correction to take effect. In the comparative example, because the black level correction was performed based on a correction coefficient with low tracking ability, as shown in column 801A of Figure 8(C), the floating is noticeable at the top of the row in T802, and it can be seen that the correction takes effect as you move down, and the floating is eliminated.

[0045] In row T803, although lateral smearing occurs, the change in the correction value is small. Therefore, good correction results can be obtained even with low filtering responsiveness. Row T804 is the region immediately after the end of lateral smearing. Due to the low filtering responsiveness, the correction value decreases only gradually. As a result of the black level correction, as shown in column 801A of Figure 8(C), the black level is noticeably lowered at the top of row T804, and the black level is reduced towards the bottom.

[0046] In the T805 row, since no lateral smearing occurs, the range of change in the correction value is small. Therefore, good correction results can be obtained even with low tracking performance of the filter processing. As mentioned above, in the black level correction of the comparative example, residual correction occurs in the areas immediately after the start and end of the lateral smearing, and the image appears to be emphasized in a band-like manner above and below the lateral smearing due to the correction.

[0047] Next, the black level correction of this embodiment will be explained using Figure 9. Figure 9(A) shows an image of the light-receiving area 201 and the HOB area 204 when strong light hits the center of the screen and causes lateral smearing. Figure 9(B) shows the signal level of the correction value in each row, and Figure 9(C) shows the signal level in each row of columns 901A, 902A, and 903A after the black level correction of this embodiment.

[0048] In the row indicated by T901, no lateral smear occurs, so the range of change in the correction value is small. Therefore, the tracking capability of the filter processing is reduced, making the correction value less susceptible to the influence of random noise in the HOB region 204.

[0049] The row indicated by T902 is the region immediately after the start of lateral smearing. The signal level shown in column 903A becomes high, but in the black level correction of this embodiment, the change in the signal level is detected by the detection unit 402, and a correction coefficient is selected to improve the tracking ability of the filter processing. As a result, the correction value increases instantaneously as shown in Figure 9(B), and a good correction result can be obtained from the top of T902 as shown in Figure 9(C).

[0050] Although lateral smearing occurs in the row indicated by T903, the change in brightness within this range is small, and the range of change in the correction value is small. Therefore, the tracking ability of the filter processing can be reduced, making the correction value less susceptible to the influence of random noise in the HOB region 204. In other words, during this period, the detection unit 402 should select a correction coefficient that reduces tracking ability.

[0051] The row indicated by T904 is the region immediately after the end of lateral smearing. The signal level in column 901A becomes low, but in the black level correction of this embodiment, the change in the signal level is detected by the detection unit 402, and a correction coefficient is selected to improve the tracking ability of the filter processing. As a result, the correction value can be lowered in a short time, so a good correction result can be obtained from the top of T902 as shown in Figure 9(C).

[0052] In T905, since no lateral smear occurs, the change in brightness is small, and the range of change in the correction value is also small. Therefore, the tracking capability of the filter processing is reduced, and the correction value is less affected by the random noise in the HOB region 204.

[0053] As described above, the black level correction in this embodiment improves the tracking ability of the correction in the region immediately after the start and immediately after the end of lateral smearing, thus suppressing the emphasis on the remaining parts of the correction compared to Figure 8.

[0054] The signal processing of the black level correction unit 303 in the signal processing device 109 of this embodiment will be explained with reference to the flowchart in Figure 10. The black level correction unit 303 corrects lateral smear in steps S1001 to S1011 shown in Figure 10.

[0055] First, in step S1001, the data of the row selected for black level correction is input to the line memory 401 and the detection unit 402. At this time, the data of the selected row may be input to the line memory 401 and the detection unit 402 via branching. In step S1002, the row data is stored in the line memory 401.

[0056] In step S1003, the detection unit 402 detects lateral smear by referring to the row data input. In step S1004, it is determined whether the input of row data for one row has been completed. If the input is completed (Yes in S1004), the process proceeds to step S1005, where the correction coefficient selection signal generation unit 604 generates a correction coefficient selection signal. If the input is not completed (No in S1004), the process returns to inputting row data.

[0057] In step S1005, if lateral smear is not detected, the correction coefficient selection signal is generated as 0; if lateral smear is detected, the correction coefficient selection signal is generated as 1. In step S1006, it is determined whether the correction coefficient selection signal is 0 or not. If the correction coefficient selection signal is 0 (Yes in S1006), the process proceeds to step S1007 and the first correction coefficient is selected. If the correction coefficient selection signal is not 0 (No in S1006), the process proceeds to step S1008 and the second correction coefficient is selected.

[0058] In step S1009, the system starts reading the data of the selected row that was stored in line memory. In step S1010, the correction value generation unit 403 performs filtering on the data of each pixel using the light-shielding data for each pixel, and updates the correction value. In step S1011, the system performs black level correction on the data of the selected row using the updated correction value.

[0059] By repeating the above operation for each row, horizontal stripes can be suppressed under normal circumstances by performing a less responsive correction, while horizontal smear can be corrected by performing a more responsive correction when horizontal smear is detected.

[0060] (Second embodiment) The signal processing device according to this embodiment will be explained with reference to Figures 11 to 13. This embodiment differs from the first embodiment in the data flow relationship within the black level correction unit and the configuration of the correction value generation unit. Other configurations are the same as in the first embodiment and may be omitted from the explanation.

[0061] The black level correction unit 303 in the signal processing device 109 of this embodiment will be explained with reference to the block diagram shown in Figure 11. The black level correction unit 303 includes a line memory 1101, a detection unit 1102, a correction value generation unit 1103, and a data correction unit 1104.

[0062] The line memory 1101 is a storage unit that holds at least one line of data input to the black level correction unit 303. The light-shielding data of the HOB area 204 does not necessarily need to be stored in the line memory 1101; only the image data of the light-receiving area 201 may be stored in the line memory 1101.

[0063] The detection unit 1102 detects events that may cause horizontal bands from the data input to the black level correction unit 303 and generates a correction coefficient selection signal.

[0064] The correction value generation unit 1103 selects a correction coefficient based on the correction coefficient selection signal output from the detection unit 1102 and generates a correction value using the light shielding data input to the black level correction unit 303.

[0065] The data correction unit 1104 subtracts the correction value output by the correction value generation unit 1103 from the output data of the line memory 1101.

[0066] The correction value generation unit 1103 in the signal processing device 109 of this embodiment will be explained with reference to the block diagram shown in Figure 12. The correction value generation unit 1103 includes an HOB average value calculation unit 1201, a correction coefficient selection unit 1202, an attenuation unit 1203, and a correction value holding unit 1204.

[0067] The correction value generation unit 1103 updates the correction value through filtering. First, the HOB average value calculation unit 1201 calculates the average value of the light-shielding data for the row selected as the correction target. Next, the correction value held in the correction value holding unit 1204 is subtracted from the calculated average value. Next, the subtraction result is attenuated in the attenuation unit 1203 using the correction coefficient selected in the correction coefficient selection unit 1202. Finally, the attenuation result and the correction value held in the correction value holding unit 1204 are added to generate a new correction value, which is then held in the correction value holding unit 1204.

[0068] The signal processing of the black level correction unit 303 in the signal processing device 109 of this embodiment will be explained with reference to the flowchart shown in Figure 13. The black level correction unit 303 corrects lateral smear in steps S1301 to S1312 shown in Figure 13.

[0069] First, in step S1301, the row data is branched and input to the line memory 1101 and the detection unit 1102. At this time, the light-shielding data of the selected row is input to the correction value generation unit 1103. In step S1302, the row data is stored in the line memory 1101. In step S1303, the HOB average value calculation unit calculates the average value of the light-shielding data.

[0070] In step S1304, the detection unit 1102 detects lateral smear by referring to the row data input. In step S1305, it is determined whether the input of one row of row data has been completed. If the input is completed (Yes in S1305), the process proceeds to step S1306, where the correction coefficient selection signal generation unit 604 generates a correction coefficient selection signal. If the input is not completed (No in S1305), the process returns to inputting row data.

[0071] In step S1306, if lateral smear is not detected, the correction coefficient selection signal is generated as 0; if lateral smear is detected, the correction coefficient selection signal is generated as 1. In step S1307, it is determined whether the correction coefficient selection signal is 0 or not. If the correction coefficient selection signal is 0 (Yes in S1307), the process proceeds to step S1308 and the first correction coefficient is selected. If the correction coefficient selection signal is not 0 (No in S1307), the process proceeds to step S1309 and the second correction coefficient is selected.

[0072] In step S1310, the correction value is updated by filtering in the correction value generation unit 1103. In step S1311, the line data held in line memory is read out. In step S1312, the black level is corrected using the updated correction value.

[0073] By repeating the above operation, horizontal stripes can be suppressed under normal conditions by performing a less responsive correction, while horizontal smear can be corrected by performing a more responsive correction when horizontal smear is detected.

[0074] In the first embodiment, the correction value is updated for each pixel, so the influence of the shading data in each column on the correction value differs between the first and last columns of shading data. However, in the second embodiment, the correction value is updated using the HOB average value, so the influence of the shading data in each column on the correction value becomes the same.

[0075] (Third embodiment) The signal processing device according to this embodiment will be explained with reference to Figure 14. This embodiment differs from the second embodiment in the configuration of the detection unit. The other configurations are the same as those of the first and second embodiments and will not be explained.

[0076] The detection unit 402 in the signal processing device 109 of this embodiment will be explained with reference to the block diagram shown in Figure 14. The detection unit 402 includes a pixel count unit 1401, a pixel count holding unit 1402, a difference calculation unit 1403, and a correction coefficient selection signal generation unit 1404. The pixel count unit 1401 counts the number of pixels that exceed a predetermined threshold level from the N (N≧0)th row of received light data.

[0077] The pixel count holding unit 1402 holds the output of the pixel count count unit 1401. The difference calculation unit 1403 calculates the difference between the number of pixels with a level exceeding the threshold level of the Nth row held in the pixel count holding unit 1402 and the number of pixels with a level exceeding the threshold level of the M (M≠N, M≧0) row calculated by the pixel count count unit.

[0078] The correction coefficient selection signal generation unit 1404 compares the difference calculated by the difference calculation unit 1403 with a specified detection level threshold and generates a correction coefficient selection signal. For example, if the difference is greater than the detection level threshold, it generates a correction coefficient selection signal that improves the tracking performance of the black level correction, and if the difference is smaller than the detection level threshold, it generates a correction coefficient selection signal that reduces the tracking performance of the black level correction.

[0079] Furthermore, when the correction coefficient selection signal generation unit 1404 changes the correction coefficient selection signal, it is possible to stop updating the correction coefficient selection signal for a certain period of time across multiple rows. For example, if the difference is greater than the detection level threshold, instead of changing the correction coefficient for only that row from the normal state, the correction coefficient is changed over a predetermined number of rows, including that row.

[0080] In the first embodiment, a correction coefficient selection signal is generated from the difference between the row average value of row N and row M. Therefore, when imaging a subject that significantly increases the average value, the lateral smear detection accuracy is easily affected by the brightness of the subject pattern. However, in the third embodiment, a correction coefficient selection signal is generated from the difference between the number of pixels with a signal level exceeding a predetermined threshold level in row N and the number of pixels with a signal level exceeding the threshold level in row M. As a result, the lateral smear detection accuracy is less affected by the brightness of the subject pattern.

[0081] <Application of signal processing devices to equipment> Hereinafter, with reference to Figure 15, a device 1500 comprising a semiconductor device 1600 including a package 1520 on which a semiconductor chip 1610, which includes a signal processing device according to this embodiment, is mounted will be described. The semiconductor chip 1610 is housed in the package 1520 and mounted on the device 1500. In the configuration shown in Figure 15, the semiconductor chip 1610 includes the signal processing device according to the above embodiment. The semiconductor device 1600 may include a package 1520 which includes a base 1510 on which the semiconductor chip 1610 is fixed, and, if the semiconductor chip 1610 includes an image sensor, a light-transmitting member 1530 such as glass. The package 1520 may be provided with connecting members such as wires and bumps that connect inner leads provided on the base 1510 and terminals such as pad electrodes provided on the semiconductor chip 1610.

[0082] The device 1500 may include at least one of the following: an optical device 1540, a control device 1550, a processing device 1560, a display device 1570, a storage device 1580, and a mechanical device 1590. The optical device 1540 is, for example, a lens, a shutter, or a mirror. The control device 1550 controls the semiconductor chip 1610. The control device 1550 is, for example, a semiconductor device such as an ASIC.

[0083] The processing unit 1560 processes the output signals from the semiconductor integrated circuit contained in the semiconductor chip 1610. The processing unit 1560 is a semiconductor device such as a CPU or ASIC for configuring an analog front-end AFE or a digital front-end DFE. If the semiconductor chip is equipped with an image sensor, it may generate an image based on an event signal E, for example. The display device 1570 is an EL display device or liquid crystal display device that displays the information image obtained from the semiconductor chip 1610. The storage device 1580 is a magnetic device or semiconductor device that stores the information image obtained from the semiconductor chip 1610. The storage device 1580 is a volatile memory such as SRAM or DRAM, or a non-volatile memory such as flash memory or a hard disk drive.

[0084] The mechanical device 1590 has movable parts or propulsion parts such as a motor or engine. In the device 1500, signals output from the semiconductor chip 1610 are displayed on the display device 1570 or transmitted to the outside by a communication device (not shown) provided by the device 1500. For this purpose, the device 1500 may further include a storage device 1580 and a processing device 1560, separate from the memory circuits and arithmetic circuits of the semiconductor chip 1610. The mechanical device 1590 may be controlled based on signals output from the semiconductor chip 1610.

[0085] Furthermore, the device 1500 is suitable for electronic devices such as information terminals with shooting capabilities, such as smartphones and wearable devices, and cameras, such as interchangeable lens cameras, compact cameras, video cameras, and surveillance cameras. In a camera, the mechanical device 1590 can drive components of the optical device 1540 for zooming, focusing, and shutter operation. Alternatively, the mechanical device 1590 in a camera can move the optical device 1540 for vibration damping.

[0086] Furthermore, the device 1500 may be a transport device such as a vehicle, ship, or aircraft. The mechanical device 1590 in the transport device may be used as a mobile device. The device 1500 as a transport device is suitable for transporting semiconductor chips 1610 or for assisting and / or automating driving operations through its imaging function. The processing device 1560 for assisting and / or automating driving operations can perform processing to operate the mechanical device 1590 as a mobile device based on information obtained from the semiconductor chip 1610. Alternatively, the device 1500 may be a medical device such as an endoscope, a measuring instrument such as a distance sensor, an analytical instrument such as an electron microscope, office equipment such as a copier, or industrial equipment such as a robot.

[0087] (Other embodiments) The disclosures herein include the following signal processing devices, photoelectric converters, and equipment. (Item 1) A signal processing device that corrects image data based on signals from multiple light-receiving pixels using light-shielding data based on signals from multiple light-shielding pixels, It comprises a line memory, a detection unit, a correction value generation unit, and a data correction unit. The line memory holds the image data of the selected row, The detection unit generates a selection signal for selecting a correction coefficient based on the image data of at least the selected row. The correction value generation unit has a correction value holding unit that holds the correction value, and generates and holds the correction value based on the correction coefficient selected by the selection signal and the light shielding data of the selected row. The data correction unit corrects the image data of the selected row held in the line memory based on the correction value. Signal processing device. (Item 2) The signal processing apparatus according to item 1, characterized in that the correction value generation unit updates the held correction value based on the correction coefficient selected by the selection signal and the light-shielding data of the selected row. (Item 3) The signal processing device according to item 1 or 2, characterized in that the image data and the light-shielding data are branched and input to the line memory and the detection unit. (Item 4) The signal processing device according to any one of items 1 to 3, characterized in that the detection unit generates the selection signal based on comparing the average value of the image data of the light-receiving pixels in the selected row with the average value of the pixel data of light-receiving pixels in rows other than the selected row. (Item 5) The signal processing device according to any one of items 1 to 3, characterized in that the detection unit generates the selection signal based on a comparison between the number of image data exceeding a predetermined threshold among the image data of light-receiving pixels in the selected row and the number of image data exceeding the predetermined threshold among the image data of light-receiving pixels in rows other than the selected row. (Item 6) The signal processing device according to any one of items 1 to 5, characterized in that the correction value generation unit includes a filter, and the correction coefficient includes the attenuation coefficient of the filter. (Item 7) The signal processing device according to item 6, characterized in that the filter is an infinite impulse response filter. (Item 8) The signal processing apparatus according to item 6 or 7, characterized in that the correction value is generated based on filtering the light-shielding data from each pixel of the plurality of light-shielding pixels in the selected row using the filter for each pixel. (Item 9) The signal processing apparatus according to item 6 or 7, characterized in that the correction value is generated based on filtering the average value of the light-shielding data from the plurality of light-shielding pixels of the selected row using the filter. (Item 10) The signal processing device according to any one of items 1 to 9, characterized in that the data correction unit corrects image data of a predetermined number of rows, including the selected row, based on the correction value. (Item 11) The line memory further stores the light-shielding data of the selected row, The signal processing apparatus according to any one of items 1 to 10, characterized in that the data correction unit corrects the light-shielding data of the selected row held in the line memory based on the correction value. (Item 12) A photoelectric conversion unit is provided, wherein multiple light-receiving pixels and multiple light-shielding pixels are arranged in a matrix, and each of the multiple light-receiving pixels and the multiple light-shielding pixels is equipped with a photoelectric conversion element. A photoelectric converter comprising a signal processing device described in any one of items 1 to 11. (Item 13) A signal processing device described in any one of items 1 to 11, A device having a processing unit that processes the signal output from the signal processing unit.

[0088] The invention is not limited to the embodiments described above, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, claims are attached to disclose the scope of the invention. [Explanation of Symbols]

[0089] 303 Black level correction unit, 401 Line memory, 402 Detection unit, 403 Correction value generation unit, 404 Data correction unit

Claims

1. A signal processing device that corrects image data based on signals from multiple light-receiving pixels using light-shielding data based on signals from multiple light-shielding pixels, It comprises a line memory, a detection unit, a correction value generation unit, and a data correction unit. The line memory holds the image data of the selected row, The detection unit generates a selection signal for selecting a correction coefficient based on comparing the average value of the image data of at least the selected row's light-receiving pixels with the average value of the pixel data of light-receiving pixels in rows other than the selected row. The correction value generation unit has a correction value holding unit that holds the correction value, and generates and holds the correction value based on the correction coefficient selected by the selection signal and the light shielding data of the selected row. The data correction unit is characterized by correcting the image data of the selected row held in the line memory based on the correction value. Signal processing device.

2. The signal processing apparatus according to claim 1, wherein the correction value generation unit updates the held correction value based on the correction coefficient selected by the selection signal and the light-shielding data of the selected row.

3. The signal processing apparatus according to claim 1, characterized in that the image data and the light-shielding data are branched and input to the line memory and the detection unit.

4. The signal processing apparatus according to claim 1, characterized in that the row different from the selected row is a row selected before the selected row.

5. A signal processing device that corrects image data based on signals from multiple light-receiving pixels using light-shielding data based on signals from multiple light-shielding pixels, It comprises a line memory, a detection unit, a correction value generation unit, and a data correction unit. The line memory holds the image data of the selected row, The detection unit generates a selection signal for selecting a correction coefficient based on a comparison between the number of image data exceeding a predetermined threshold among the image data of the light-receiving pixels in the selected row and the number of image data exceeding the predetermined threshold among the image data of light-receiving pixels in rows other than the selected row. The correction value generation unit has a correction value holding unit that holds the correction value, and generates and holds the correction value based on the correction coefficient selected by the selection signal and the light shielding data of the selected row. The data correction unit is characterized by correcting the image data of the selected row held in the line memory based on the correction value. Signal processing device.

6. The signal processing apparatus according to claim 5, characterized in that the row different from the selected row is a row selected before the selected row.

7. The signal processing apparatus according to claim 1, characterized in that the correction value generation unit includes a filter, and the correction coefficient includes the attenuation coefficient of the filter.

8. The signal processing apparatus according to claim 7, characterized in that the filter is an infinite impulse response filter.

9. The signal processing apparatus according to claim 7, characterized in that the correction value is generated based on filtering the light-shielding data from each pixel of the plurality of light-shielding pixels in the selected row using the filter for each pixel.

10. The signal processing apparatus according to claim 7, characterized in that the correction value is generated based on filtering the average value of the light-shielding data from the plurality of light-shielding pixels of the selected row using the filter.

11. The signal processing apparatus according to claim 1, characterized in that the data correction unit corrects image data of a predetermined number of rows, including the selected row, based on the correction value.

12. A photoelectric conversion unit is provided, wherein multiple light-receiving pixels and multiple light-shielding pixels are arranged in a matrix, and each of the multiple light-receiving pixels and the multiple light-shielding pixels is equipped with a photoelectric conversion element. A photoelectric converter comprising a signal processing device according to any one of claims 1 to 11.

13. A signal processing device according to any one of claims 1 to 11, A device having a processing unit that processes signals output from the signal processing device.

Images