Signal processing device, photoelectric conversion device, imaging system, and method for controlling signal processing device

Abstract

To suppress deterioration of image quality when OB floating occurs.SOLUTION: A portion, in which a pixel value exceeding a first threshold value continues for a predetermined number, of continuous image data indicated by pixel values output from a continuous plurality of light shielding pixels is identified as floating data. Reference data that is obtained using residual image data, in which the floating data is removed from the continuous image data, and alternate image data having a pixel value which does not exceed the first threshold value substituting the floating data, and that indicates a reference of a specific color is output.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present disclosure relates to a signal processing device, a photoelectric conversion device, an imaging system, and a method for controlling a signal processing device. [Background technology]

[0002] There is known a photoelectric conversion device having light receiving pixels arranged in a light receiving region that receives light from a subject, and light shielding pixels arranged in a light shielding region that is shielded by a light shielding film such as metal. The light shielding pixels may be used to detect the black level of image data output from the light receiving pixels, and the light shielding region is also called an optical black (OB) region. Here, the output of the light shielding pixels usually does not change, but as a result of light entering the OB region, a gradual change called "OB floating" may occur in the output of the light shielding pixels. Patent Document 1 discloses a method of determining that OB floating has occurred when the output of the light shielding pixels is greater than a preset threshold value, and not using image data of a row in which OB floating has occurred to generate an OB level correction value. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2006-157242 A Summary of the Invention [Problem to be solved by the invention]

[0004] In a row that is determined to have OB floating, the output of the light-blocking data may peak at the pixel closest to the target subject, and gradually decrease to the normal OB level as it gets farther away from the target subject. If the threshold value for determining OB floating is made large, overcorrection occurs, while if the threshold value for determining OB floating is made small, normal data is determined to be OB floating data. As a result, it was not possible to output appropriate reference data for a specific color, such as the OB level. [Means for solving the problem]

[0005] A signal processing device according to one embodiment of the present disclosure is a signal processing device that processes image data from a solid-state imaging element having a plurality of light-receiving pixels arranged in a light-receiving region and a plurality of light-shielding pixels arranged in a light-shielding region, and is characterized in that the signal processing device has a first determination means that identifies as floating data a portion of continuous image data indicated by pixel values ​​output from a plurality of consecutive light-shielding pixels, in which a predetermined number of pixel values ​​exceed a first threshold value are present, and an output means that outputs reference data indicating a standard for a specific color, obtained using the remaining image data obtained by removing the floating data identified by the first determination means from the continuous image data, and alternative image data having pixel values ​​not exceeding the first threshold value that are used to replace the floating data identified by the first determination means. Effect of the Invention

[0006] According to the present disclosure, it is possible to suppress deterioration of image quality when OB floating occurs. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of a photoelectric conversion device including a signal processing device. [Diagram 2] 2 is a conceptual diagram of image data output from a photoelectric conversion unit of the photoelectric conversion device of FIG. 1. [Diagram 3] FIG. 13 is a diagram showing an example of image data before OB clamp processing. [Figure 4] 13A and 13B are diagrams illustrating an example of image data after OB clamp processing. [Diagram 5] 4 is a block diagram showing an example of the configuration of a clamp value generating unit; [Figure 6] 13 is a flowchart showing the flow of processing executed by a second determination unit. [Figure 7] 11 is a diagram illustrating a process executed by a second determination unit. FIG. [Figure 8] FIG. 1 is a diagram illustrating an example of the configuration of a photoelectric conversion device including a signal processing device. [Figure 9] 4 is a block diagram showing an example of the configuration of a clamp value generating unit; [Figure 10] 4 is a block diagram showing an example of the configuration of a clamp value generating unit; [Figure 11] FIG. 2 is a diagram illustrating an example of an arrangement of photoelectric conversion devices. [Figure 12] FIG. 1 is a diagram illustrating an example of a schematic configuration of an imaging device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. Note that the following embodiments do not limit the present disclosure according to the claims, and not all of the combinations of features described in the present embodiments are essential to the solution of the present disclosure. Note that the same components are given the same reference numbers and their description will be omitted.

[0009] (Embodiment 1) The signal processing device according to the present embodiment will be described with reference to the drawings. FIG. 1 is a diagram showing a configuration example of a photoelectric conversion device including a signal processing device according to the present embodiment. As shown in FIG. 1, the photoelectric conversion device 100 has a photoelectric conversion unit 101, a vertical scanning unit 103, a control unit 104, a readout circuit unit 105, an AD conversion unit 106, a memory unit 107, and a horizontal scanning unit 108. The photoelectric conversion unit 101 has a plurality of pixels 102 arranged in a plurality of rows and a plurality of columns. The photoelectric conversion unit 101 has a light receiving region and a light blocking region. The light receiving region is a region in which a plurality of pixels that receive light from a subject are arranged. The light blocking region is a region in which a plurality of pixels that are shielded by a light blocking film such as metal around the light receiving region and are used to detect a black level are arranged. The photoelectric conversion unit 101 outputs image data processed by a signal processing device 109. In the present embodiment, an example in which the signal processing device 109 is mounted on the photoelectric conversion device 100 is shown, but is not limited thereto. The signal processing device 109 may be arranged separately from the photoelectric conversion device 100 and the photoelectric conversion section 101, for example.

[0010] The photoelectric conversion unit 101 has 2m×2n pixels 102 arranged in a row direction, 2m pixels 102 arranged in a column direction. The vertical scanning unit 103 is connected to the 2m pixels 102 arranged in the row by a row selection line 110 (V(2n)), and selects a row from which a signal is to be read out. In the selected row, signals output from the 2m pixels 102 included in the selected row are simultaneously read out to the readout circuit unit 105 via a vertical output line 111 (H(2m)). The readout circuit unit 105 may include an amplifier or the like and amplify the signals output from the pixels 102. The signals output from the readout circuit unit 105 are converted from analog signals to digital signals (AD conversion) by the AD conversion unit 106 and temporarily held in the memory unit 107. Thereafter, the digital signals addressed by the horizontal scanning unit 108 are sequentially read out to the signal processing device 109, where digital signal processing is performed. The control unit 104 acquires setting information such as shooting conditions for capturing an image of the photoelectric conversion device 100 from an externally input signal, and supplies control signals according to the conditions to each component included in the photoelectric conversion device 100. The control unit 104 controls the vertical scanning unit 103, the readout circuit unit 105, the AD conversion unit 106, the memory unit 107, the horizontal scanning unit 108, and the signal processing device 109 based on the acquired setting information.

[0011] The signal processing device 109 performs a process to reduce reset noise generated in a switch element (e.g., a MOS transistor) included in each pixel 102 of the photoelectric conversion unit 101. In addition, the image data output from the photoelectric conversion unit 101 may contain a dark current generated from a photodiode included in the pixel 102, and a variation (FPN: fixed pattern noise) generated due to a difference caused by a circuit such as a power supply impedance or a signal delay. In this embodiment, a state in which this FPN changes with a certain rule for each row and column is called shading. After reducing the reset noise component, the signal processing device 109 averages the light-shielded data for each row and column to generate correction data including the FPN component and the shading component. Next, the signal processing device 109 performs a process to reduce the FPN component and the shading component of the image data using the correction data generated from the light-shielded data. In this embodiment, the process in which the signal processing device 109 reduces the FPN component and the shading component of the image data using the light-shielded data is called OB clamping. Here, the region for acquiring the average value of the light-shielded data for each row and each column can be set to any region within the light-shielded region 201. This region is called a clamp value generation region. Also, the correction data acquired by averaging the light-shielded data is called a clamp value. That is, the above-mentioned signal processing device 109 can be said to be a device for processing image data from a solid-state imaging element having a plurality of light-receiving pixels arranged in a light-receiving region and a plurality of light-shielded pixels arranged in a light-shielded region. Note that the signal processing device 109 may be a computer such as a personal computer including a processor (e.g., a CPU or an MPU) that is separate from the photoelectric conversion device 100. Also, the signal processing device 109 may be, for example, a circuit such as an ASIC that realizes the functions described below.

[0012] <Image data output from the photoelectric conversion unit> The image data output from the photoelectric conversion unit will be described with reference to the drawings. FIG. 2 is a conceptual diagram when the image data output from the photoelectric conversion unit 101 is arranged according to the pixel arrangement of the photoelectric conversion unit 101. The photoelectric conversion unit 101 includes a light receiving region 202 that receives light incident through an optical system such as a lens, and a light shielding region 201 that optically shields the incident light with a light shielding film such as metal. The light shielding region 201 is a reference region for determining the so-called black level reference in the image data obtained by the photoelectric conversion unit 101. Here, among the light shielding regions 201, as shown in FIG. 2, the region where the pixel 102 is optically shielded over all columns at the upper part of the screen is referred to as the VOB region 203. Also, the region located on the left side of the light receiving region 202 and optically shielding the pixel 102 over all rows is referred to as the HOB region 204.

[0013] <Image data before and after OB clamp processing> First, the image data before the OB clamp processing will be described with reference to the drawings. FIG. 3 is a diagram showing an example of the image data before the OB clamp processing. FIG. 3(a) shows an example of the signal levels corresponding to the shading components of each row between A-A' in FIG. 2, and FIG. 3(b) shows an example of the signal levels corresponding to the shading components of each row between B-B' in FIG. 2. In FIGS. 3(a) and 3(b), the vertical axis represents the number of rows, and the horizontal axis represents the signal level. The signal level has a pixel value of 0 at the origin, and the pixel value increases as it goes to the right. When the image data is 8 bits, the maximum pixel value is 256. Note that the explanation of the signal level is the same in FIG. 4 described later.

[0014] FIG. 3(a) shows a conceptual diagram where a uniform FPN component increases for each row, and shows the clamp value 301 when the entire light shielding region 201 is used as the clamp value generation region. The signal level L1 indicates the reference value of the black level. The difference between the clamp value 301 and the signal level L1 is the shading component to be reduced by the OB clamp processing. In FIG. 3(b), from row V1 onwards (the region shown in white below row V1 in FIG. 3(b) including row V1), the light receiving region is arranged, showing a state where the signal level has increased by the amount of the received light data.

[0015] Next, the image data after the OB clamp process will be described with reference to the drawings. FIG. 4 is a diagram showing an example of image data after the OB clamp process. FIG. 4(a) shows an example of the result (image data) of subtracting the clamp value 301 (shading component) from the signal level of each row between A-A' in FIG. 2. FIG. 4(b) shows an example of the result (image data) of subtracting the clamp value 301 (shading component) from the signal level of each row between B-B' in FIG. 2. In FIG. 4(a) and FIG. 4(b), the vertical axis represents the number of rows, and the horizontal axis represents the signal level. L2 indicates the light reception data after the shading component has been reduced.

[0016] 4(a) and 4(b) show a state in which shading components have been removed from the image data obtained by the photoelectric conversion unit 101, including the light receiving area 202, by subtracting the clamp value 301. That is, this shows an example in which the OB clamp process by the signal processing device 109 has been appropriately performed.

[0017] Here, the clamp value 301 may be obtained by, for example, integrating and averaging the light-shielded data for each row. However, in the OB clamp process, if the integrated average value of the light-shielded data for each row is used for subtraction as is, the integrated average value may vary due to the influence of random noise that differs for each pixel. For this reason, the FPN component and the shading component may not be appropriately reduced in the OB clamp process, and the accuracy of the OB clamp process may decrease. Therefore, for example, by using the following filter for the light-shielded data, the clamp value may be made resistant to row variations, thereby improving the accuracy of the OB clamp process. As the filter, for example, an IIR type LPF (Infinite Impulse Response type Low Pass Filter) may be used.

[0018] In addition, the light-shielded data may contain image data with a uniquely large or small pixel value (hereinafter referred to as "defective data") due to defects that occur during the manufacturing process. If defective data is mixed into the process of generating the clamp value, the clamp value will deviate from the ideal value, thereby reducing the accuracy of the OB clamp process. Therefore, a method can be considered in which a predetermined threshold is set in advance, and when the difference between this threshold and the light-shielded data is large, the clamp value acquired in the previous row, for example, is input instead of the light-shielded data. In this way, defective data with a large difference from the desired FPN component can be complemented with the clamp value acquired in the previous row, thereby improving the accuracy of the OB clamp process.

[0019] <Configuration of the clamp value generator> The configuration of the clamp value generating unit included in the signal processing device 109 will be described with reference to the drawings. Fig. 5 is a block diagram showing an example of the configuration of the clamp value generating unit 500 included in the signal processing device 109. Note that this embodiment shows a signal processing device capable of preventing black sunkenness due to OB floating and appropriately reducing shading components.

[0020] 5, the clamp value generation unit 500 has a first determination unit 501, a second determination unit 502, and an averaging processing unit 503. Shaded light data is input to the clamp value generation unit 500. The clamp value generation unit 500 performs OB clamp processing on the input shaded light data, and outputs a clamp value generated by this processing.

[0021] The averaging processor 503 complements the input light-shielded data with defective data, obtains an average value by filtering, and holds the obtained average value as a clamp value. Note that the complementation of the input light-shielded data with defective data is performed according to the judgment results by the first judgment unit 501 and the second judgment unit 502, which will be described in detail later. From the viewpoint of reducing row variations, it is desirable to use the IIR type LPF described above for the filtering process in the averaging processor 503, but an integral average may be used, and this is not limited as long as the average value of the light-shielded data can be obtained as the clamp value.

[0022] The first determination unit 501 determines whether the input light-shielded data is defective data. For example, the first determination unit 501 compares a pixel value of the light-shielded data with a predetermined first threshold value set in advance, and determines whether the input data is defective data according to the comparison result. Then, the first determination unit 501 outputs a defective data flag indicating whether the input data is defective data to the averaging processing unit 503 according to the determination result. When the comparison result indicates that the pixel value of the light-shielded data is greater than the first threshold value, the first determination unit 501 outputs a defective data flag "1" indicating that the input data is defective data to the averaging processing unit 503. On the other hand, when the comparison result indicates that the pixel value of the light-shielded data is equal to or less than the first threshold value, the first determination unit 501 outputs a defective data flag "0" indicating that the input data is not defective data to the averaging processing unit 503. The first threshold value is determined based on the signal level of a defective pixel generated in the photoelectric conversion unit 101. Since the signal level of the defective pixel is, for example, about five times the reference value of the black level, the first threshold value may be, for example, a value that is five times the signal level L1. The upper limit of the first threshold value may be, for example, the maximum value of the data processed by the signal processing device 109. When the signal processing device 109 processes 14-bit data, the upper limit of the first threshold value may be, for example, 16383. The first threshold value can be, for example, data for identifying defective data that occurs randomly among a plurality of image data. When the averaging processing unit 503 receives the defective data flag "1" from the first judgment unit 501, the averaging processing unit 503 uses the clamp value held in the averaging processing unit 503 or the first threshold value used in the judgment of the first judgment unit 501 for filtering. That is, it can be said that the first judgment unit 501 identifies image data having a pixel value exceeding the first threshold value among the continuous image data as defective data.

[0023] The second determination unit 502 determines whether the input light-shielded data is OB-floor data. For example, the second determination unit 502 counts the number of outliers determined to be greater than a predetermined second threshold value of the pixel value of the light-shielded data, and compares the outlier data number counter with the number of comparisons. Then, the second determination unit 502 determines whether the light-shielded data is OB-floor data according to the comparison result. However, the outlier data number counter is reset if the position of the light-shielded data or light-shielded data greater than the second threshold value is not continuous. Then, the first determination unit 501 outputs an OB-floor data flag indicating whether the input data is OB-floor data to the averaging processing unit 503 according to the determination result. If the comparison result indicates that the outlier data number counter is greater than the number of comparisons, the second determination unit 502 outputs an OB-floor data flag "1" indicating that the input data is OB-floor data to the averaging processing unit 503. On the other hand, when the comparison result indicates that the outlier data counter is equal to or smaller than the number of comparisons, the second determination unit 502 outputs an OB outlier data flag "0" indicating that the input data is not OB outlier data to the averaging processing unit 503. The second threshold is a threshold for identifying outlier data in which the pixel value of a specific color changes gradually, and is determined based on the signal level of a defective pixel generated in the photoelectric conversion unit 101. The specific color is, for example, black or white. The second threshold is a threshold for identifying outlier data in which the pixel value of a specific color changes gradually, and is determined based on the reference value of the black level. The specific color is, for example, black or white. The second threshold may be, for example, a numerical value equivalent to the signal level L1. In addition, the upper limit value of the second threshold may be, for example, the maximum value of data processed by the signal processing device 109. When the signal processing device 109 processes 14-bit data, the upper limit value of the second threshold may be, for example, 16383. When the averaging processor 503 receives the OB floating data flag "1" from the second determination unit 502, it performs filtering using the following data. That is, the averaging processor 503 outputs reference data indicating a reference for a specific color, which is obtained based on the remaining image data obtained by removing the floating data identified by the second determination unit 502 from the continuous image data, and on the alternative image data.The substitute image data is data having pixel values that do not exceed a second threshold value to be substituted for the floating data specified by the second determination unit 502. As the substitute image data, for example, the clamp value previously held in the averaging processing unit 503, the clamp value output from the previous target by the averaging processing unit 503, or the image data having the second threshold value used for the determination by the second determination unit 502 is used. In the averaging processing unit 503, it can be said that the light-shielding data input to the clamp value generation unit 500 corresponding to the OB floating data flag is not used. Note that the light-shielding data input to the clamp value generation unit 500 corresponding to the defect data flag is not used.

[0024] <Determination Process of OB Floating Data> The determination process of OB floating data executed by the second determination unit 502 will be described with reference to the drawings. FIG. 6 is a flowchart showing the flow of the determination process of OB floating data executed by the second determination unit 502. Hereinafter, steps are denoted as "S". When determining OB floating, data is sequentially fetched from the data group output from the photoelectric conversion unit 101 to the second determination unit 502. Then, for each data, the process according to FIG. 6 is performed. The determination as to whether OB floating has occurred is carried out according to S601 to S609 shown in FIG. 6. In the present embodiment, it is determined whether OB floating has occurred in the target row or pixel.

[0025] First, in S601, it is determined whether the target data is data within the clamp value generation region (the light-shielding region 201 in FIG. 2). For this determination, the pixel (position information) included in the target data or the pixel (position information) corresponding to the target data sent from the control unit 104 is used. As a result of the determination, if the target data is data within the clamp value generation region (YES in S601), the process proceeds to S602. On the other hand, if the target data is data within the light-receiving region 202 in FIG. 2 and not data within the clamp value generation region (NO in S601), the flow shown in FIG. 6 ends.

[0026] In S602, it is determined whether the target data is the first data. Here, the first data refers to the data to be processed first in each row in the clamp value generation area, which is represented by, for example, the light-shielded data P(1, y). If the result of the determination is that the target data is the first data (YES in S602), the process proceeds to S603. On the other hand, if the target data is not the first data (NO in S602), the process proceeds to S604.

[0027] In S603, a process is executed to reset the current outlier data number counter N to "0" (N=0).

[0028] In S604, the signal level of the target data is compared with a second threshold. The second threshold is the sum of the signal level L1 and a random variation component depending on the manufacturing process, and may be, for example, 100 LSB. In this embodiment, light-shielded data having a large difference from the second threshold is called "outlier data". Here, the second threshold is a threshold for determining outlier data, and is controlled independently of the first threshold described above. If the comparison result shows that the signal level (input data) of the target data is greater than the second threshold (YES in S604), the process proceeds to S605. If the signal level (input data) of the target data is equal to or less than the second threshold (NO in S604), the process proceeds to S606.

[0029] In S605, a process of adding 1 to the current outlier data number counter N and setting (updating) the outlier data number counter N to "N+1" is executed.

[0030] In S606, a process is executed to reset the current outlier data counter N to "0" (N=0). As in the processes of S603 and S606, by resetting the outlier data counter N when the target data is not outlier data, it becomes possible to count the number of consecutive outlier data. Since the number of consecutive outlier data is counted in this way, the outlier data counter N can also be called a consecutive number counter.

[0031] In S607, the value of the outlier data counter N is compared with a preset number of comparisons. Here, the number of comparisons is a threshold value for determining how many consecutive outlier data counter N is required to determine that an OB has been detected, and is a threshold value determined based on the characteristics of defective pixels generated in the photoelectric conversion unit 101, and may be, for example, 5 to 10 times. If the comparison result shows that the value of the outlier data counter N is greater than the number of comparisons (YES in S607), the process proceeds to S608. On the other hand, if the comparison result shows that the value of the outlier data counter N is equal to or less than the number of comparisons (NO in S607), the process proceeds to S609.

[0032] In S608, it is determined that OB floating has occurred, and the process of setting the OB floating flag to “1” is executed. After the process of S608, the flow shown in FIG.

[0033] In S609, it is determined that no OB float has occurred, and the process of setting the OB float flag to “0” is executed. After the process of S609, the flow shown in FIG.

[0034] As a result of the determination according to the flowchart shown in FIG. 6, if it is determined that OB floating has occurred in the target row (if the OB floating flag is "1"), the clamp value generation unit 500 outputs the clamp value already held in the averaging processing unit 503, for example, the clamp value acquired in the previous row. In other words, in the row where it is determined that OB floating has occurred, the average value acquired from the data of the target row is not used for clamp value generation. In this way, it is possible to prevent blackening caused by OB floating due to OB clamp processing. Also, in the above flow, it is determined whether or not there are a predetermined number of consecutive image data whose pixel values ​​exceed the second threshold value.

[0035] In the above flow, the second determination unit 502 identifies a portion of continuous image data represented by pixel values ​​output from a plurality of continuous light-shielding pixels, in which a predetermined number of pixel values ​​exceeding the second threshold value are consecutive, as floating data. In addition, the above flow may be performed for each horizontal line output from a plurality of light-shielding pixels, or may be performed for each vertical line.

[0036] Here, an example of data processing by the above-mentioned OB floating data determination process will be described with reference to the drawings. FIG. 7 is a diagram for explaining an example of processing of light-shielded data. FIG. 7(a) shows the concept of the output signal of the photoelectric conversion unit when OB floating occurs at the boundary between the HOB region and the opening region. FIG. 7(b) shows an example of processing by the second determination unit 502 in the i-th row 701 where OB floating does not occur. FIG. 7(c) shows an example of processing by the second determination unit 502 in the j-th row 702 where OB floating occurs. FIG. 7(b) and FIG. 7(c) show an example of light-shielded data of a light-shielded region, where the second threshold is set to 100 LSB and the number of comparisons is set to 9.

[0037] As shown in FIG. 7(a), the light-shielded data located on the right side of the HOB region 204 is in a state of being floated due to strong light hitting the opening region adjacent to the HOB region 204. As shown in FIG. 7(b), in the i-th row 701 where the OB floating does not occur, it was confirmed that the maximum value of the outlier data number counter N is 2 when the second threshold is set to 100 LSB for the light-shielded data P(1,i), P(2,i), . . . , P(24,i). When the number of comparisons is set to 9, it was confirmed that the i-th row is determined to have no OB floating, and the OB floating flag is set to "0". As shown in FIG. 7(b), the light-shielded data exceeding the second threshold is not continuous for 10 pixels or more, so the OB floating flag is set to "0". When it is confirmed that the OB floating flag is "0", the averaging processing unit 503 updates the clamp value based on the light-shielded data of the i-th row.

[0038] On the other hand, as shown in FIG. 7(c), in the jth row 702 where the OB floating occurs, it was confirmed that the maximum value of the outlier data number counter N is 10 in the shaded data P(1,j), P(2,j), . . . , P(24,j) when the second threshold is set to 100LSB. When the number of comparisons is set to 9, it was confirmed that the jth row is determined to have the OB floating, and the OB floating flag is set to "1". As shown in FIG. 7(c), since there are 10 or more pixels of shaded data exceeding the second threshold, the OB floating flag is set to "1". When it is confirmed that the OB floating flag is "1", the averaging processing unit 503 uses, for example, the clamp value acquired in the previous row (j-1) as the clamp value to be applied to the jth row.

[0039] As an application example of this embodiment, it is possible to set the second threshold to a value smaller than the first threshold. For example, the second threshold is set to about 1 / 5 of the first threshold. By counting consecutive outlier data, even if the second threshold is made smaller, random variations in the light-shielded data are not erroneously detected as outlier data, and it is possible to detect only the OB floating. Therefore, even if a gently changing OB floating occurs, black sunkenness due to the OB clamping process can be prevented, and suitable OB clamping process can be performed.

[0040] In the above embodiment, when it is determined that OB floating occurs in the target line, the clamp value used in the OB clamp process is complemented with the clamp value already held for each target line to prevent black sunkenness. Therefore, even if, for example, OB floating data occurs only in a part of the VOB area 203 and the majority of the light-shielded data in the target line is normal, the update of the clamp value of the target line is canceled. In such a case, the average parameter used to generate the clamp value is reduced, and the accuracy of the OB clamp process may be reduced. Therefore, when it is determined that OB floating occurs in the target data, the averaging processor 503 may use the clamp value held in the averaging processor 503 for the filter process, instead of the input light-shielded data. That is, the averaging processor 503 may complement the data determined to have OB floating with the clamp value held in the averaging processor 503, obtain an average value by filter process, and hold the obtained average value as the clamp value. Here, an example is shown in which the clamp value held in the averaging processor 503 is used for complementation, but the present invention is not limited to this. For example, any data indicating the level of normal light-blocking data, such as the second threshold value used to determine the outlier data, may be used.

[0041] In this way, when it is determined that OB floating has occurred, it is possible to complement the input data for obtaining the clamp value on a single data basis, rather than complementing the clamp value used in the OB clamp process on a target line basis with the already stored clamp value. Therefore, it is possible to prevent blackening caused by OB floating due to OB clamp process, and further increase the average parameter, thereby appropriately reducing shading components and enabling suitable OB clamp process. This makes it possible to suppress deterioration of image quality when OB floating occurs.

[0042] In addition, in this embodiment, preset fixed values ​​are used as the first threshold and the second threshold, but instead of the fixed values, for example, a value obtained by adding a preset fixed value to the clamp value acquired in the previous row may be used as the threshold. By doing so, it becomes possible to use a relative value from the clamp value acquired in the previous row as the threshold.

[0043] Furthermore, in this embodiment, an example has been shown of using a counter as a method for detecting consecutive outlier data, but for example, a Schmitt trigger may be used instead of a counter, and this is not limited as long as it is possible to detect consecutive outlier data.

[0044] (Embodiment 2) The signal processing device according to the present embodiment will be described with reference to the drawings.

[0045] In recent years, in order to further increase the read speed of pixel data, a technology for increasing the number of vertical output lines has been attracting attention. This is a technology for increasing the number of vertical output lines to increase the amount of data read at one time and reduce the number of reads per frame. When using this technology for increasing the number of vertical output lines, the signal path of the image data is different for each vertical output line, so that there is a possibility that each vertical line has a different FPN component. In addition, in consideration of wiring, it is also possible to arrange multiple signal processing devices for each color. In this case, the signal path of the image data is also different for each color, so that there is a possibility that each color has a different FPN component. In this embodiment, the smallest unit in which the FPN component differs due to the data path is called a "channel". On the other hand, the signal level floating in the white direction due to OB floating is constant regardless of the channel. Therefore, when the difference in FPN component between channels is large, the signal level to be determined as OB floating will change for each channel.

[0046] In this embodiment, a signal processing device that can generate a more appropriate clamp value while taking into consideration the case where the FPN component for each channel is large at the same time as the occurrence of OB floating will be described.

[0047] FIG. 8 is a diagram showing an example of the configuration of a photoelectric conversion device 800 according to this embodiment. As shown in FIG. 8, the photoelectric conversion device 800 further includes a vertical scanning unit 103′, a control unit 104′, a readout circuit unit 105′, an AD conversion unit 106′, a memory unit 107′, a horizontal scanning unit 108′, and a signal processing device 109′ in addition to the components of the photoelectric conversion device 100 shown in FIG. 1. That is, the photoelectric conversion device 800 includes two vertical scanning units, two control units, two readout circuit units, two AD conversion units, two memory units, two horizontal scanning units, and two signal processing devices. In FIG. 8, the functional units and devices included in the photoelectric conversion device 800 are arranged as follows. That is, the photoelectric conversion unit 101 and the vertical scanning units 103 and 103′ are arranged in the center. The control unit 104, the readout circuit unit 105, the AD conversion unit 106, the memory unit 107, the horizontal scanning unit 108, and the signal processing device 109 are disposed below the photoelectric conversion unit 101 and the vertical scanning units 103 and 103'. The control unit 104', the readout circuit unit 105', the AD conversion unit 106', the memory unit 107', the horizontal scanning unit 108', and the signal processing device 109' are disposed above the photoelectric conversion unit 101 and the vertical scanning units 103 and 103'.

[0048] The control unit 104' obtains setting information such as the shooting conditions for capturing images of the photoelectric conversion device 800 from a signal directly input from the outside or from a signal sent from the control unit 104, and supplies control signals according to the conditions to each component included in the photoelectric conversion device 800.

[0049] The signals output from the signal processing device 109 and the signal processing device 109' may be output from the corresponding ports, or may be output from a single port.

[0050] Of the pixels 102 in the photoelectric conversion unit 101, image data obtained from pixels belonging to odd-numbered columns is output using a path 821 arranged on the lower side. Of the pixels 102 in the photoelectric conversion unit 101, image data obtained from pixels belonging to even-numbered columns is output using a path 822 arranged on the upper side. That is, this embodiment shows a configuration example having two channels, and is an example in which signal processing devices 109, 109' are arranged for each channel. Note that the functional units and devices of the photoelectric conversion device 800 are not limited to the above-mentioned arrangement.

[0051] By adopting such a configuration, it becomes possible to perform the judgment of OB floating for each channel. Furthermore, considering that each channel has its own unique FPN component, the second threshold value used for OB floating judgment is set for each channel. In this embodiment, a photoelectric conversion device having each functional unit and each device unit for each channel is configured as described above, and the judgment process of OB floating data shown in the first embodiment is performed.

[0052] As described above, according to this embodiment, even if a unique FPN component occurs for each channel, it is possible to appropriately reduce the FPN component and the shading component by preventing blackening caused by OB floating due to OB clamp processing. This makes it possible to perform even more suitable OB clamp processing.

[0053] In addition, in this embodiment, the photoelectric conversion device 800 having the signal processing device 109, 109' for each channel has been described, but the configuration of the photoelectric conversion device 800 is not limited to this. For example, the photoelectric conversion device may have a signal processing device in which the clamp value generating unit 500 and the second determining unit 502 are provided for each channel, as long as the OB lift determination can be performed for each channel.

[0054] (Embodiment 3) The signal processing device according to this embodiment will be described with reference to the drawings. The same components as those in the signal processing devices according to the first and second embodiments are given the same reference numerals, and the description thereof will be omitted or simplified.

[0055] In this embodiment, a mode will be described in which a more appropriate clamp value can be generated, taking into consideration the case where a shading component in the vertical direction occurs simultaneously with the OB floating.

[0056] FIG. 9 is a block diagram showing an example of the configuration of the clamp value generating unit included in the signal processing device according to the present embodiment. As shown in FIG. 9, the clamp value generating unit 900 includes a holding unit 901 in addition to the elements 501, 502, and 503 included in the clamp value generating unit 500. The holding unit 901 is connected to the averaging processing unit 503 so that the clamp value can be transmitted and received. The holding unit 901 is, for example, a line memory, and holds in advance the clamp value of each row in which the OB floating does not occur, such as image data of pixel values ​​not exceeding a first threshold value. The clamp value held in the holding unit 901 may be set according to the shooting conditions at the time of shipment, for example. In the present embodiment, the embodiment of the signal processing device including the clamp value generating unit 900 equipped with the holding unit 901 has been described, but the mounting location of the holding unit 901 is not limited thereto, and the holding unit 901 may be mounted, for example, in a photoelectric conversion device that is separate from the signal processing device.

[0057] 6, if it is determined that OB floating has occurred in the target row (if OB floating flag=1 is received), the clamp value received by the averaging processor 503 from the storage unit 901 is output from the clamp value generator 900. That is, for the row determined to have OB floating, the OB clamp process is performed using the clamp value previously stored in the storage unit 901.

[0058] As described above, according to this embodiment, the FPN components can be appropriately reduced by preventing the blackening caused by the OB floating due to the OB clamping process, and at the same time, the shading components in the vertical direction can be reduced. This enables more suitable OB clamping process.

[0059] (Embodiment 4) The signal processing device according to this embodiment will be described with reference to the drawings. The same components as those in the signal processing devices according to the first to third embodiments are given the same reference numerals, and the description thereof will be omitted or simplified.

[0060] In this embodiment, a more appropriate clamp value is generated by taking into consideration not only the last data determined to be OB floating, but all OB floating data.

[0061] 10 is a block diagram showing an example of the configuration of a clamp value generation unit included in the signal processing device according to this embodiment. As shown in FIG. 10, the clamp value generation unit 1000 includes a delay unit 1001 in addition to the elements 501, 502, and 503 included in the clamp value generation unit 500.

[0062] The delay unit 1001 is arranged on the transmission path of the light-shielded data input to the clamp value generating unit 1000, but not on the transmission path to the second determining unit 502, and on the transmission path to the first determining unit 501 and the averaging processing unit 503. The delay unit 1001 is, for example, a delay buffer for delaying the processing for the number of comparisons for the second determining unit 502 to determine OB floating, and outputs the input data to the first determining unit 501 and the averaging processing unit 503 after delaying the input data for a predetermined time. This allows the averaging processing unit 503 to receive the OB floating data flag, the defect data flag, and the light-shielded data at an appropriate timing. Note that, as the predetermined time, for example, the same time may be set in the first determining unit 501 and the averaging processing unit 503, which are the output destinations, or different times may be set for each of the first determining unit 501 and the averaging processing unit 503.

[0063] In this embodiment, when it is determined that OB floating occurs in the target data as a result of the determination using the flowchart in Fig. 6, the averaging processor 503 uses a clamp value held by the averaging processor 503 itself for filtering, rather than the input light-blocking data. In this embodiment, the delay unit 1001 delays the processing of the first determination unit 501 and the averaging processor 503 by the processing time consumed by the second determination unit 502, thereby making it possible to complement all OB floating data.

[0064] As described above, according to this embodiment, the blackening caused by OB floating due to OB clamp processing can be prevented, and the average parameter can be increased, so that the FPN component and the shading component can be appropriately reduced. This makes it possible to perform suitable OB clamp processing.

[0065] (Embodiment 5) The photoelectric conversion device according to this embodiment will be described with reference to the drawings. FIG. 11 is a schematic diagram showing an example of the configuration of the photoelectric conversion device according to this embodiment. As shown in FIG. 11, the photoelectric conversion device according to this embodiment can be configured by forming each element shown in FIG. 1 separately on two semiconductor substrates 1101 and 1102 and bonding these semiconductor substrates 1101 and 1102. The semiconductor substrates 1101 and 1102 can be electrically connected to each other via a conductive member such as a bump electrode or a through electrode. The semiconductor substrates 1101 and 1102 are substrates using a semiconductor such as silicon.

[0066] 1, only the photoelectric conversion unit 101 including a plurality of pixels 102 is arranged on the upper semiconductor substrate 1102. That is, the photoelectric conversion unit 101 in which the pixels 102 each having a light receiving region 202 and a light blocking region 201 are arranged in an array is arranged on the upper substrate 1102. The elements other than the photoelectric conversion unit 101 are arranged on the lower semiconductor substrate 1101, among the elements shown in FIG. That is, the elements such as the vertical scanning unit 103, the control unit 104, the readout circuit unit 105, the AD conversion unit 106, the memory unit 107, the horizontal scanning unit 108, and the signal processing device 109 are arranged on the lower semiconductor substrate 1101.

[0067] 11, at least a part of the substrate 1101 and the substrate 1102 may be laminated. By configuring the photoelectric conversion device 100 in this manner, it becomes possible to select a manufacturing process for each of the analog section including the photoelectric conversion unit 101 and the logic section including the signal processing device 109, so that it becomes possible to obtain good characteristics for each. As a result, the photoelectric conversion device according to this embodiment can obtain image data (image data) with improved image quality.

[0068] (Embodiment 6) The imaging system according to this embodiment will be described with reference to Fig. 12. Fig. 12 is a diagram showing an example of the schematic configuration of the imaging system according to this embodiment.

[0069] The photoelectric conversion device 100 described above is applicable to various imaging systems. Examples of applicable imaging systems include digital still cameras, digital camcorders, security cameras, copiers, fax machines, mobile phones, vehicle-mounted cameras, and observation satellites. Camera modules equipped with an optical system such as a lens and an imaging device are also included in the imaging system. FIG. 12 illustrates a block diagram of a digital still camera as an example of these.

[0070] 12, the imaging system 1200 includes a photoelectric conversion device 100, a CPU 1203, an external input unit 1204, an optical system 1205, an image display unit 1206, a recording unit 1207, and a drive system 1208. The photoelectric conversion device 100 includes a signal generation unit 1201 and a signal correction unit 1202.

[0071] The signal generating unit 1201 includes the photoelectric conversion unit 101 including the above-mentioned pixels 102, the vertical scanning unit 103, the control unit 104, the readout circuit unit 105, the AD conversion unit 106, the memory unit 107, and the horizontal scanning unit 108. The signal correcting unit 1202 includes the above-mentioned signal processing device 109. Therefore, a configuration including the signal generating unit 1201 and the signal correcting unit 1202 corresponds to the above-mentioned photoelectric conversion device 100. In the imaging system 1200 configured as above, the signal generating unit 1201 alone may be called a photoelectric conversion device. In other words, the signal processing device 109 may be arranged separately from the photoelectric conversion device, as described above.

[0072] The signal generating unit 1201 performs photoelectric conversion in response to light incident on the photoelectric conversion unit 101 of the signal generating unit 1201 by the optical system 1205 to generate an analog image signal, and outputs image data by AD converting this image signal.

[0073] The signal correction unit 1202 performs a predetermined process on the image signal output from the pixel 102 of the photoelectric conversion device 100. For example, the signal correction unit 1202 performs various correction processes and compression processes on the image signal, and outputs the processed image signal. The image signal processed by the signal correction unit 1202 can be displayed on the video display unit 1206 or recorded in the recording unit 1207. The video display unit 1206 is, for example, a liquid crystal monitor that displays an image, and may be provided with a touch panel that accepts user operations and perform part of the function of the external input unit 1204. The recording unit 1207 may be built into the imaging system 1200, or may be detachable from the imaging system 1200 like a recording medium.

[0074] The CPU 1203 is responsible for the overall control of the imaging system 1200. The external input unit 1204 is various buttons through which the user performs input / operation, and receives information input from the outside, such as information regarding operations performed by the user on the imaging system 1200. Examples of the user's input / operation include input of information regarding imaging conditions, pressing of a shutter button, etc. The drive system 1208 drives, for example, a lens and an aperture (neither of which are shown) that constitute the optical system 1205. The imaging system 1200 may further include a memory unit for temporarily storing image data, an external interface unit for communicating with an external computer, etc.

[0075] [Other embodiments] The present disclosure can also be realized by a process in which a program for implementing one or more functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that implements one or more functions.

[0076] In the above, a mode in which a clamp value (reference data) indicating a reference for black as a specific color is output has been described, but it is also possible to apply this to a mode in which reference data indicating a reference for white as a specific color is output. In this case, if the image data is 8 bits, the pixel value of white is 255, so the pixel value of the first threshold value is greater than the pixel value of the second threshold value. The pixel value of black is 0.

[0077] In the above description, the reference data is described by taking the clamp value, which is data indicating the black level when processing the image data output from the light-receiving pixels, as an example, but is not limited to this. For example, data indicating the white level when processing the image data output from the light-receiving pixels can be used as the reference data. [Explanation of symbols]

[0078] 500 Clamp value generator 502 Second Judgment Section 503 Averaging processing unit

Claims

1. A signal processing device that processes image data from a solid-state imaging device having a plurality of light-receiving pixels arranged in a light-receiving region and a plurality of light-shielding pixels arranged in a light-shielding region, a first determination means for determining, as floating data, a portion of continuous image data represented by pixel values ​​output from a plurality of continuous light-shielded pixels, where a predetermined number of pixel values ​​exceeding a first threshold value are consecutive; an output means for outputting reference data indicating a reference for a specific color, the reference data being obtained by using remaining image data obtained by removing the floating data identified by the first determination means from the continuous image data and substitute image data having a pixel value not exceeding the first threshold value to be substituted for the floating data identified by the first determination means; A signal processing device comprising:

2. The output means outputs the reference data obtained by integrating and averaging the remaining image data excluding the floating data and the alternative image data.

2. The signal processing device according to claim 1 .

3. The substitute image data is image data stored in advance, reference data output from a previous object by the output means, or image data having the first threshold value.

3. The signal processing device according to claim 1, wherein the first and second signals are input to the first and second input terminals.

4. A storage unit for storing the alternative image data 4. The signal processing device according to claim 1, wherein the signal processing device is a digital signal processing device.

5. a second determination means for determining, from among the continuous image data, image data having a pixel value exceeding a second threshold value as defective data; The output means does not use the defective data identified by the second determination means when outputting the reference data.

5. The signal processing device according to claim 1, wherein the signal processing device is a digital signal processing device.

6. the specific color is black, The first threshold pixel value is smaller than the second threshold pixel value.

6. The signal processing device according to claim 5.

7. the specific color is white, The first threshold pixel value is greater than the second threshold pixel value.

6. The signal processing device according to claim 5,

8. The image data is input to the output means.

5. The signal processing device according to claim 1, wherein the signal processing device is a digital signal processing device.

9. 9. The signal processing device according to claim 8, wherein the delay means delays the input of the predetermined number of consecutive image data to the output means so as to be at the same timing as the input of the floating data identified by the first determination means to the output means.

10. The image data is input to the output means.

8. A signal processing device according to claim 5, wherein the signal processing device comprises:

11. 11. The signal processing device according to claim 10, wherein the delay means delays the input of the predetermined number of consecutive image data to the output means so as to be at the same timing as the input of the floating data identified by the first determination means to the output means.

12. The delay means delays the input of the defect data identified by the second determination means to the output means so as to be at the same timing as the input of the floating data identified by the first determination means to the output means.

12. The signal processing device according to claim 10 or 11.

13. 13. The signal processing device according to claim 1, wherein the reference data is data indicating a black level when processing image data output from the light receiving pixels.

14. The first determination means identifies the floating data for each horizontal line of the plurality of light-shielded pixels, The output means outputs the reference data for each horizontal line of the plurality of light-shielded pixels.

14. A signal processing device according to any one of claims 1 to 13.

15. The first determination means identifies the floating data for each vertical line of the plurality of light-shielded pixels, The output means outputs the reference data for each vertical line of the plurality of light-shielded pixels.

14. A signal processing device according to any one of claims 1 to 13.

16. A signal processing device according to any one of claims 1 to 15; a photoelectric conversion unit having the light receiving pixels and the light blocking pixels, and outputting the image data to be processed by the signal processing device; A photoelectric conversion device comprising:

17. a first substrate on which the signal processing device is disposed, and a second substrate on which the photoelectric conversion unit is disposed; The photoelectric conversion device according to claim 16 , further comprising:

18. At least a portion of the first substrate and the second substrate are laminated.

18. The photoelectric conversion device according to claim 17.

19. A signal processing device according to any one of claims 1 to 15; a photoelectric conversion unit having the light receiving pixels and the light blocking pixels, and outputting the image data to be processed by the signal processing device; An imaging system comprising:

20. A method for controlling a signal processing device that processes image data from a solid-state imaging device having a plurality of light-receiving pixels arranged in a light-receiving region and a plurality of light-shielding pixels arranged in a light-shielding region, the method comprising the steps of: a first determination step of identifying a portion of continuous image data represented by pixel values ​​output from a plurality of continuous light-shielded pixels, in which a predetermined number of pixel values ​​exceeding a first threshold value are consecutive, as floating data; an output step of outputting reference data indicating a reference for a specific color, the reference data being obtained by using remaining image data obtained by removing the floating data identified in the first determination step from the continuous image data and substitute image data having a pixel value not exceeding the first threshold value to be substituted for the floating data identified in the first determination step; 23. A method for controlling a signal processing device, comprising:

Images