Imaging device
The imaging device addresses moiré suppression in high-frequency regions through a pixel arrangement with varied exposure times and blending ratio adjustments, improving image quality by reducing moiré patterns.
Patent Information
- Application Number
- PCT/JP2025/026226
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2024-08-09
- Filing Date
- 2025-07-24
- Publication Date
- 2026-02-12
AI Technical Summary
Existing imaging devices struggle to effectively suppress moiré patterns, particularly in high-frequency regions where brightness differences change abruptly, using conventional signal processing methods.
The imaging device employs a pixel arrangement with different exposure times and pixel group configurations in a two-dimensional array, along with a processing unit that adjusts blending ratios to minimize moiré effects.
This approach enhances image quality by reducing moiré patterns, especially in high-frequency subjects, by stabilizing pixel group ratios and adjusting blending ratios for improved image generation.
Smart Images

Figure JP2025026226_12022026_PF_FP_ABST
Abstract
Description
Imaging device
[0001] The present disclosure relates to an imaging device.
[0002] In an imaging device, when an image of a subject including a high frequency region with abrupt changes is captured, moire may occur.
[0003] JP 2016-201733 A International Publication No. 2022 / 130888 International Publication No. 2021 / 002213
[0004] For example, Patent Document 1 discloses a method for suppressing moiré by signal processing based on the detection results of moving objects and aliasing components of an image. However, there is still room for further study of techniques for suppressing moiré, in addition to such a method.
[0005] One aspect of the present disclosure suppresses moire.
[0006] An imaging device according to one aspect of the present disclosure comprises a plurality of pixel blocks arranged in a two-dimensional array, the plurality of pixel blocks including a first pixel block and a second pixel block located in different rows, each of the first pixel block and the second pixel block including a first pixel group and a second pixel group, and the pixel arrangement direction of the first pixel group in the first pixel block and the pixel arrangement direction of the first pixel group in the second pixel block are different from each other.
[0007] An imaging device according to one aspect of the present disclosure includes: a plurality of pixel blocks arranged in a two-dimensional array, each including a first pixel group and a second pixel group; and a processing unit that calculates, for each of the plurality of pixel blocks, a signal value of the pixel block based on the signal value of the first pixel group of the pixel block, the signal value of the second pixel group of the pixel block, and an adjusted blending ratio, wherein the processing unit adjusts the blending ratio so that the blending ratio approaches 0.5 when the pixel block is used for imaging a high-frequency region of a subject.
[0008] 1 is a diagram illustrating an example of a schematic configuration of an imaging device 100 according to a first embodiment. FIG. 1 is a diagram illustrating an example of a pixel circuit. FIG. 2 is a diagram illustrating an example of a schematic configuration of a pixel array section 1. FIG. 3 is a diagram illustrating an example of an exposure time. FIG. 4 is a diagram illustrating an example of a center position. FIG. 5 is a diagram illustrating an example of processing (imaging method, image generation method) executed by a processing section 11. FIG. 6 is a diagram illustrating an example of a first technique. FIG. 7 is a diagram illustrating an example of a second technique. FIG. 8 is a diagram illustrating an example of a pixel circuit. FIG. 9 is a diagram illustrating an example of a pixel circuit. FIG. 10 is a diagram illustrating an example of a pixel 2 used for imaging a high frequency region 19a. FIG. 11 is a diagram illustrating an example of a pixel 2 used for imaging a high frequency region 19a. FIG. 12 is a diagram illustrating an example of a pixel 2 used for imaging a high frequency region 19a. FIG. 13 is a diagram illustrating a comparative example. FIG. 14 is a diagram illustrating a comparative example. FIG. 15 is a diagram illustrating a comparative example. FIG. 16 is a diagram illustrating a comparative example. FIG. 17 is a diagram illustrating a comparative example. FIG. 18 is a diagram illustrating a comparative example. FIG. 1 is a diagram illustrating an example of processing (imaging method, image generation method) executed by a processing unit 11. FIG. 2 is a diagram illustrating an example of adjustment of a blending ratio α. FIG. 3 is a diagram illustrating an example of suppression of moiré. FIG. 4 is a diagram illustrating an example of a schematic configuration of a pixel array unit 1. FIG. 5 is a diagram illustrating an example of a schematic configuration of a pixel array unit 1. FIG. 6 is a diagram illustrating an example of a schematic configuration of a pixel array unit 1.
[0009] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In the following embodiments, the same elements are designated by the same reference numerals, and redundant description will be omitted.
[0010] The present disclosure will be described in the following order: 0. Introduction 1. First embodiment 2. Modification of first embodiment 3. Second embodiment 4. Modification of second embodiment 5. Other embodiments 6. Conclusion
[0011] 0. A subject having a high-frequency region where the color, shape, etc. change abruptly is also called a high-frequency subject. It is known that capturing an image of a high-frequency subject can result in false colors called moiré. One way to address this issue is to detect the high-frequency region and remove colors from that region to make the false colors less noticeable. However, this method is not effective for high-frequency regions where the brightness difference changes abruptly. In other words, brightness moiré remains. The disclosed technology makes it possible to suppress moiré, including brightness moiré, by devising a pixel arrangement or a binning processing algorithm.
[0012] 1 is a diagram showing an example of the schematic configuration of an imaging device 100 according to the first embodiment. The illustrated imaging device 100 is a solid-state imaging device (also called an image sensor, etc.), and each element is provided on a substrate 12. The substrate 12 is, for example, a silicon semiconductor substrate.
[0013] The object to be imaged by the imaging device 100 is referred to as a subject 19 and is illustrated schematically. Light from the subject 19 is detected by the imaging device 100, and an image 13 including the subject 19 is generated and output. Note that the term "image" may be interpreted as meaning a video, and "imaging" may be interpreted as meaning "shooting." To the extent that there is no contradiction, the terms "image" and "shooting" may be appropriately interpreted as meaning a video and "shooting."
[0014] The imaging device 100 includes a pixel array section 1, as well as peripheral circuits and wiring, etc. The peripheral circuits are exemplified by a control circuit 3, a vertical drive circuit 4, a column signal processing circuit 5, a horizontal drive circuit 6, and an output circuit 7, all of which are designated by reference numerals. The wirings are exemplified by signal lines 8, 9, and 10, all of which are designated by reference numerals.
[0015] The pixel array unit 1 includes a plurality of pixel blocks 20 arranged in a two-dimensional array. An XYZ coordinate system for the pixel array unit 1 is also shown. The X-axis direction corresponds to the row direction of the array, for example, the lateral direction (horizontal direction) of the imaging device 100. The Y-axis direction corresponds to the column direction of the array, for example, the longitudinal direction (vertical direction) of the imaging device 100. The Z-axis direction corresponds to the front-to-rear direction of the imaging device 100.
[0016] The pixel block 20 includes two or more pixels 2. The pixels 2 are arranged in a two-dimensional array across the entire pixel array section 1. Each pixel 2 includes a photoelectric conversion unit such as a photodiode (PD). The pixel 2 also includes a circuit for generating and outputting a signal (voltage signal) corresponding to the amount of light incident on the photoelectric conversion unit. A pixel 2 including such a circuit can also be called a pixel circuit.
[0017] Each pixel 2 is configured so that light of a color corresponding to that pixel 2 is incident on the photoelectric conversion unit. Examples of colors include red (R), green (G), and blue (B). For example, a color filter that transmits light of a color corresponding to that pixel 2 may be provided in that pixel 2. Further details of the pixel array unit 1 will be described later.
[0018] The control circuit 3 receives data instructing the input clock, operation mode, etc., and outputs data such as internal information of the imaging device 100. The control circuit 3 generates clock signals and control signals that serve as references for the operations of the vertical drive circuit 4, column signal processing circuit 5, horizontal drive circuit 6, etc., based on a vertical synchronization signal, a horizontal synchronization signal, and a master clock. The control circuit 3 supplies these generated signals to the vertical drive circuit 4, column signal processing circuit 5, horizontal drive circuit 6, etc.
[0019] The vertical drive circuit 4 includes, for example, a shift register. The vertical drive circuit 4 is connected to the pixel array unit 1 via a plurality of signal lines 8 (horizontal signal lines) extending in the row direction of the pixels 2. Each signal line 8 extends, for example, for each pixel row, and each signal line 8 may include a plurality of signal lines. The vertical drive circuit 4 supplies a drive signal (for example, a pulse signal) for driving the pixels 2 to a selected signal line 8.
[0020] The driving of the pixels 2 by the vertical drive circuit 4 includes driving of pixel transistors (such as transistor 22 in FIG. 2 ) described below. The pixel transistors are driven to output voltage signals corresponding to the amount of charge generated in the photoelectric conversion units in the pixels 2 to corresponding signal lines 9 among a plurality of signal lines 9 (vertical signal lines) extending in the column direction of the pixels 2.
[0021] The column signal processing circuits 5 are connected to the pixel array unit 1 via signal lines 9. Each signal line 9 may include a plurality of signal lines. The column signal processing circuits 5 are arranged, for example, for each pixel column or for each pixel group (described later), and perform signal processing such as noise removal on signals from one row of pixels 2 for each pixel column or pixel group. The column signal processing circuits 5 perform signal processing such as CDS (Correlated Double Sampling) for removing fixed pattern noise specific to the pixels 2, signal amplification, and AD (Analog to Digital) conversion.
[0022] The horizontal drive circuit 6 includes, for example, a shift register. The horizontal drive circuit 6 sequentially outputs horizontal scanning pulses to select each of the column signal processing circuits 5 in turn, and causes each of the column signal processing circuits 5 to output a signal to the signal line 10.
[0023] The output circuit 7 processes and outputs the signals sequentially supplied from each of the column signal processing circuits 5 through the signal line 10. For example, buffering, black level adjustment, column variation correction, various digital signal processing, etc. are performed. By such processing, a signal (image signal, image data) of an image 13 is generated and output to the outside of the imaging device 100.
[0024] Of the above-mentioned circuits, the column signal processing circuit 5 and the output circuit 7 in particular generate an image 13 based on signals from the pixel array unit 1. The column signal processing circuit 5 and the output circuit 7 are also collectively referred to as a processing unit 11. The image 13 generated by the processing unit 11 may be an HDR (High Dynamic Range) image, as will be described in detail later.
[0025] 2 is a diagram showing an example of a pixel circuit. The photoelectric conversion unit included in pixel 2 is illustrated as a photoelectric conversion unit 21. As described above, the photoelectric conversion unit 21 generates electric charges according to the amount of incident light. In FIG. 2, the photoelectric conversion unit 21 is shown as a photodiode with an anode connected to GND (ground).
[0026] Examples of circuit elements provided around the photoelectric conversion unit 21 include a transistor 22, a charge storage unit 23, a transistor 24, a transistor 25, and a transistor 26. In the following description, when a transistor is connected between two elements, it means that one of the source and drain of the transistor is connected to one element, and the other of the source and drain is connected to the other element.
[0027] The transistor 22 is connected between the photoelectric conversion unit 21 and the charge accumulation unit 23, and transfers the charges generated in the photoelectric conversion unit 21 and accumulated in the photoelectric conversion unit 21 to the charge accumulation unit 23. The transistor 22 can also be called a transfer transistor.
[0028] The gate electrode of the transistor 22 is connected to the corresponding signal line 8. The on / off (conduction / non-conduction) of the transistor 22 is controlled by a control signal from the signal line 8. When the transistor 22 is on, the charge in the photoelectric conversion unit 21 is transferred to the charge accumulation unit 23 via the transistor 22.
[0029] The charge storage unit 23 stores the charge of the photoelectric conversion unit 21 transferred via the transistor 22, and generates a voltage according to the stored charge. The charge storage unit 23 includes, for example, a floating diffusion (FD).
[0030] The transistor 24 is connected between the charge storage unit 23 and the power supply node VDD, and discharges to the power supply node VDD the charge stored in the charge storage unit 23. The transistor 24 can also be called a reset transistor.
[0031] The gate electrode of the transistor 24 is connected to the corresponding signal line 8. The transistor 24 is turned on and off by a control signal from the signal line 8. When the transistor 24 is on, the charge in the charge storage unit 23 is discharged to the power supply node VDD via the transistor 24. When the transistor 22 is on, the charge in the photoelectric conversion unit 21 is also discharged.
[0032] The transistor 25 is connected between the power supply node VDD and the transistor 26. The gate of the transistor 25 is connected to the charge storage unit 23. The transistor 25 amplifies and outputs the voltage generated in the charge storage unit 23. The transistor 25 can also be called an amplifying transistor.
[0033] The transistor 26 is connected between the transistor 25 and the signal line 9. The transistor 26 selectively outputs the output voltage of the transistor 25 to the signal line 9. The transistor 26 can also be called a selection transistor.
[0034] The gate electrode of the transistor 26 is connected to the corresponding signal line 8. The transistor 26 is turned on and off by a control signal from the signal line 8. When the transistor 26 is turned on, the output voltage of the transistor 25 is output to a signal line 9 via the transistor 26. This signal is supplied to the processing unit 11 (more specifically, the column signal processing circuit 5) described above via the signal line 9 and processed.
[0035] Note that the configuration shown in Fig. 2 is merely one example of a pixel circuit configuration. Various known circuit configurations may be adopted. Other examples of pixel circuit configurations will be described later with reference to Figs. 9 and 10. The pixel array unit 1 (Fig. 1) will be further described with reference to Fig. 3 and subsequent figures.
[0036] FIG. 3 is a diagram showing an example of the schematic configuration of the pixel array unit 1. One pixel block 20 corresponds to one type of color. The pixel block 20 corresponding to red is also referred to as pixel block 20R. The pixel 2 included in pixel block 20R is the pixel 2 corresponding to red and is also referred to as pixel 2R. The pixel block 20 corresponding to green is also referred to as pixel block 20G. The pixel 2 included in pixel block 20G is the pixel 2 corresponding to green and is also referred to as pixel 2G. The pixel block 20 corresponding to blue is also referred to as pixel block 20B. The pixel 2 included in pixel block 20B is the pixel 2 corresponding to blue and is also referred to as pixel 2B.
[0037] 3 , pixel block 20R includes eight pixels 2R. Pixel block 20G includes ten pixels 2G. Pixel block 20B includes eight pixels 2B. These pixel blocks 20R, 20G, and 20B are arranged in a Bayer array. That is, one pixel block 20R, two pixel blocks 20G, and one pixel block 20B form one unit, and multiple units are arranged in a square.
[0038] In this example, the plurality of pixel blocks 20 includes two types of pixel blocks 20G. The first (type) pixel block is referred to and illustrated as pixel block 20G-1. The second (type) pixel block is referred to and illustrated as pixel block 20G-2. When there is no particular distinction between these, they are simply referred to as pixel blocks 20G.
[0039] Pixel block 20G-1 and pixel block 20G-2 are located in different rows. The row in which pixel block 20G-1 is located is referred to as pixel block row L1 and is illustrated. The row in which pixel block 20G-2 is located is referred to as pixel block row L2 and is illustrated. In the example shown in Figure 3, pixel block row L1 and pixel block row L2 are located in order in the column direction (Y-axis direction).
[0040] Each pixel block 20R includes a first pixel group and a second pixel group. Each of the first pixel group and the second pixel group includes two or more pixels 2. The pixel 2 of the first pixel group of the pixel block 20R is referred to as pixel 2R. L Pixel 2 of the second pixel group is also referred to as pixel 2R. S The pixel 2 of the first pixel group of the pixel block 20G is also referred to as pixel 2G. L Pixel 2 of the second pixel group is also referred to as pixel 2G. S Pixel 2 of the first pixel group in pixel block 20B is also referred to as pixel 2B. L Pixel 2 of the second pixel group is also referred to as pixel 2B. S It is also called.
[0041] The first pixel group and the second pixel group have different exposure times for the pixels 2. In terms of the circuit of FIG. 2 described above, for example, the exposure time for the pixels 2 corresponds to the length of the period during which the transistor 22 is turned off and electric charges are generated and accumulated in the photoelectric conversion unit 21.
[0042] 4 is a diagram showing an example of exposure time. The exposure time of pixel 2 of the first pixel group is set as exposure time T L The exposure time of pixel 2 of the second pixel group is referred to as exposure time T S The exposure time T L is the exposure time T S longer than (T L >T S In other words, the exposure time T S is the exposure time T L shorter than (T S <T L In the example shown in FIG. L and exposure time T S In the example shown in FIG. 4B, the exposure times T L and exposure time T S The start times of each are the same.
[0043] Returning to FIG. 3, in this example, the first pixel group of pixel block 20R is made up of four pixels 2R L The second pixel group includes four pixels 2R S The first pixel row includes two pixels 2R S In the second pixel row, four pixels 2R L are arranged side by side. In the third pixel row, two pixels 2R S are arranged side by side.
[0044] The first pixel group of pixel block 20G is made up of six pixels 2G L The second pixel group includes four pixels 2G S The first pixel row contains two pixels 2G L and two 2G S In the second pixel row, two pixels 2G L In the third pixel row, two pixels 2GS and two pixels 2G L are arranged side by side.
[0045] The first pixel group of pixel block 20B is four pixels 2B L The second pixel group includes four pixels 2B S The first pixel row includes two pixels 2B S In the second pixel row, four pixels 2B L In the third pixel row, two pixels 2B S are arranged side by side.
[0046] In this pixel array unit 1, the center position (center of gravity) of the first pixel group and the center position of the second pixel group are the same within the same pixel block 20. The following description will also refer to FIG.
[0047] 5 is a diagram showing examples of center positions. The center position of pixel block 20R is illustrated as center position 200R. The center position of pixel block 20G is illustrated as center position 200G. The center position of pixel block 20B is illustrated as center position 200B.
[0048] Four pixels 2R in pixel block 20R S and the center position of the four pixels 2R L The center positions of the four pixels 2G in the pixel block 20G are the same and overlap at the center position 200R. S The center position of the six pixels 2G L The four pixels 2B of the pixel block 20B overlap at the center position 200G. S and the center position of the four pixels 2B L The center positions of the two lines 100A and 100B are the same and overlap at the center position 200B.
[0049] 3, the pixel block 20G will be further described. Between the pixel block 20G-1 and the pixel block 20G-2, the pixel 2G L , that is, the pixel arrangement directions of the first pixel group are different from each other.
[0050] Specifically, pixel 2GL The arrangement direction of the pixel 2G in the pixel block 20G-1 includes a diagonal direction relative to the row direction (X-axis direction) and the column direction (Y-axis direction). L The arrangement direction of the pixel 2G in the pixel block 20G-2 includes the direction from the upper left to the lower right in the drawing (or the direction from the lower right to the upper left). L The arrangement direction of the pixel 2G in the pixel block 20G-1 includes the direction from the top right to the bottom left in the drawing (or the direction from the bottom left to the top right). L and the pixel 2G of the pixel block 20G-2. L The arrangement directions of the two are opposite to each other.
[0051] Returning to Fig. 1 , for example, the processing unit 11 generates an image 13 (Fig. 1) based on a signal from the pixel array unit 1 as described above. Because each pixel block 20 of the pixel array unit 1 contains a first pixel group and a second pixel group with different exposure times, the dynamic range for the amount of light from the subject 19 can be expanded compared to when only one pixel group is present. In other words, an HDR image can be generated. Unless otherwise specified, the image 13 is assumed to be an HDR image.
[0052] Specifically, the processing unit 11 calculates a signal value for each of the plurality of pixel blocks 20. The signal value of a pixel block 20 is one signal value corresponding to one pixel block 20, and may indicate, for example, the color level corresponding to that pixel block 20. The processing by the processing unit 11 will be described with reference to FIG. 6 as well.
[0053] 6 is a diagram showing an example of processing (imaging method, image generation method) executed by the processing unit 11. In step S1, the processing unit 11 acquires signal values of a first pixel group and a second pixel group for each of the plurality of pixel blocks 20. In step S2, the processing unit 11 calculates a signal value for each of the plurality of pixel blocks 20 based on the signal values of the first pixel group and the signal values of the second pixel group of that pixel block 20.
[0054] As described above, the signal value of each pixel block 20 is calculated. The signal value of pixel block 20R may indicate the amount of incident light (red light) at its center position 200R, in other words, the level of red. The signal value of pixel block 20G may indicate the amount of incident light (green light) at its center position 200G, in other words, the level of green. The signal value of pixel block 20B may indicate the amount of incident light (blue light) at its center position 200B, in other words, the level of blue. An image 13 with a resolution corresponding to the number of pixel blocks 20 is obtained.
[0055] As mentioned above, the first pixel group and the second pixel group within the same pixel block 20 have the same center position. This allows the position corresponding to the signal value of the first pixel group and the position corresponding to the signal value of the second pixel group to be aligned. This makes it possible to obtain signal values that correspond to more accurate positions than when the center positions of the first pixel group and the second pixel group are different (shifted). This can contribute to improving the image quality of the image 13.
[0056] There are two possible methods for the process of step S1 described above, that is, for calculating the signal values of the first pixel group and the second pixel group. These methods will be explained in order.
[0057] <First Method> Fig. 7 is a diagram showing an example of the first method. For simplicity of explanation, only the processing for pixel block 20G is shown.
[0058] In step S1, the processing unit 11 performs AD conversion on the signal from each pixel 2. In this example, each pixel 2G of the first pixel group in the pixel block 20G L The signal from each pixel is converted to a digital signal. L The signal value of each pixel 2G of the second pixel group of the pixel block 20G is obtained. S The signal from each pixel is converted to a digital signal. S The AD conversion is performed by, for example, the column signal processing circuit 5 (FIG. 1) described above.
[0059] The processing unit 11 performs the processing for each pixel 2G LFor example, the signal value of each pixel 2G is calculated based on the signal value of the first pixel group. L Similarly, the signal values of each pixel 2G are added or averaged. S The signal values of the second pixel group are calculated based on the signal values of the first pixel group and the second pixel group. The calculation of the signal values here is performed by, for example, the output circuit 7 (FIG. 1) described above.
[0060] In step S2, the processing unit 11 calculates the signal value of the pixel block 20G based on the signal values of the first pixel group and the second pixel group. For example, the signal values of the first pixel group and the second pixel group are added or averaged to calculate the signal value of the pixel block 20G. At this time, the signal values of the first pixel group and the second pixel group may be arbitrarily weighted (blending ratio adjustment). Various known calculation methods for obtaining an HDR image may be used. The signal value calculation here is performed, for example, by the output circuit 7 ( FIG. 1 ) described above.
[0061] For example, the signal value of pixel block 20G is calculated as described above. Similarly, the signal values of pixel block 20R and pixel block 20B are calculated.
[0062] <Second Method> In the first method described above, it is necessary to AD convert all signals from each pixel 2. To obtain the signal value of one pixel group, AD conversion is required the same number of times as the number of pixels 2 included in that pixel group. In the second method described next, AD conversion is performed only once to obtain the signal value of one pixel group. This will be described with reference to Figures 8 to 10.
[0063] 8 shows an example of the second technique. For simplicity, only the processing for pixel block 20G is shown. The specific principles will be explained later in FIGS. 9 and 10, but one signal corresponding to one pixel group is obtained from the beginning.
[0064] For the first pixel group, all the pixels 2G contained therein LA signal corresponding to the first pixel group is obtained as a signal of the first pixel group. S A signal corresponding to the second pixel group is obtained as the signal of the second pixel group.
[0065] In step S1, the processing unit 11 performs AD conversion on the signals of the first pixel group to obtain the signal values of the first pixel group. The processing unit 11 also performs AD conversion on the signals of the second pixel group to obtain the signal values of the second pixel group. The processing in step S2 is the same as that in FIG. 7 , and therefore will not be described further.
[0066] The pixel circuit is designed so that the signals of the first pixel group and the second pixel group are available from the beginning. This will be explained with reference to FIGS.
[0067] 9 and 10 are diagrams showing examples of pixel circuits. Fig. 9 shows the pixel circuit of pixel block 20G-1. Fig. 10 shows the pixel circuit of pixel block 20G-2. When there is no need to distinguish between the two, they will simply be referred to as pixel block 20G, as before.
[0068] 2 described above, the photoelectric conversion unit 21 and the transistor 22 are provided for each pixel 2. On the other hand, the charge accumulation unit 23, the transistor 24, the transistor 25, the transistor 26, and the signal line 9 are provided for each pixel group. It can be said that multiple pixels 2 in the same pixel group share the charge accumulation unit 23, the transistor 24, the transistor 25, the transistor 26, and the signal line 9.
[0069] Two charge accumulation units 23, two transistors 24, two transistors 25, two transistors 26, two wirings 27, and two signal lines 9 are provided corresponding to the two pixel groups, the first pixel group and the second pixel group. The first charge accumulation unit, the transistors, the wirings, and the signal lines are referred to as the charge accumulation unit 23. L , transistor 24 L , transistor 25 L , transistor 26 L , wiring 27 L and signal line 9 LThese are pixels 2G of the first pixel group. L The second charge storage unit, the transistor, the wiring, and the signal line are provided in common for the charge storage unit 23. S , transistor 24 S , transistor 25 S , transistor 26 S , wiring 27 S and signal line 9 S These are pixels 2G of the second pixel group. S It is provided in common for
[0070] Wiring 27 L is each pixel 2G of the first pixel group L The photoelectric conversion unit 21 is connected to the charge storage unit 23 L Specifically, in this example, the wiring 27 L is each pixel 2G L The transistor 22 and the charge storage unit 23 L Each pixel 2G is connected between L The charge of the photoelectric conversion unit 21 of the pixel 2G L The transistor 22 and the wiring 27 L via the charge storage unit 23 L will be forwarded to.
[0071] Charge storage section 23 L is each pixel 2G L A voltage corresponding to the total charge of the photoelectric conversion unit 21 is generated. A signal corresponding to this voltage is output from the transistor 25 L and transistor 26 L via signal line 9 L This signal corresponds to the signal of the first pixel group described above with reference to FIG. 8. The signal value of the first pixel group obtained by AD converting this signal is output to the charge storage unit 23. L It can be said that the signal value corresponds to the charge accumulated in the
[0072] Wiring 27 S is each pixel 2G of the second pixel group S The photoelectric conversion unit 21 is connected to the charge storage unit 23 S Specifically, in this example, the wiring 27S is each pixel 2G S The transistor 22 and the charge storage unit 23 S Each pixel 2G is connected between S The charge of the photoelectric conversion unit 21 of the pixel 2G S The charge storage unit 23 S will be forwarded to.
[0073] Charge storage section 23 S is each pixel 2G L A voltage corresponding to the total charge of the photoelectric conversion unit 21 is generated. A signal corresponding to this voltage is output from the transistor 25 S and transistor 26 S via signal line 9 S This signal corresponds to the signal of the second pixel group described above with reference to FIG. 8. The signal value of the second pixel group obtained by AD converting this signal is output to the charge storage unit 23. S It can be said that the signal value corresponds to the charge accumulated in the
[0074] As can be seen from FIGS. 9 and 10, the pixel block 20G-1 and the pixel block 20G-2 have a pixel 2G L and pixel 2G S Since the position of the wire 27 is different, L and wiring 27 S The patterns, etc. may be different. It can also be said that the wiring structure is different.
[0075] For example, as described above, the first pixel group (each pixel 2G L ) to wire 27 L etc., and the second pixel group (each pixel 2G S ) to wire 27 S etc., signals of the first pixel group and signals of the second pixel group are obtained. Similarly, signals of the first pixel group and signals of the second pixel group are obtained for pixel block 20R and pixel block 20B. The signals from the first pixel group and signals from the second pixel group obtained in this manner are AD converted in step S1 of FIG. 8 described above, and signal values of the first pixel group and signal values of the second pixel group are obtained.
[0076] As described above, the pixel 2G is located between the pixel block 20G-1 and the pixel block 20G-2. L By varying the arrangement direction of the pixels, moiré can be suppressed even when the subject 19 is a high-frequency subject. The high-frequency region of the subject 19 will be referred to as the high-frequency region 19a in the following description. One of the causes of moiré is large fluctuations in the number and combination of pixels in the first pixel group and the second pixel group used to capture the high-frequency region 19a in the pixel array unit 1. The imaging device 100 according to the embodiment reduces this fluctuation, resulting in moiré suppression. This will be described with reference to FIGS. 11 to 14.
[0077] 11 to 14 are diagrams showing examples of pixels 2 used to capture an image of a high-frequency region 19a. The extent of the high-frequency region 19a of the subject 19 in the pixel array section 1 is schematically shown by a dashed line. Although multiple high-frequency regions 19a may exist, only one high-frequency region 19a is shown for ease of understanding. In this example, the high-frequency region 19a extends from the upper left to the lower right of the diagram.
[0078] For ease of understanding, the following description will focus on only the pixel block 20G-1 and the pixel block 20G-2 marked with reference numerals in the figure. GL and pixel 2 GS Among them, the pixel 2G overlapping with the high frequency region 19a L and pixel 2G S is used to image the high frequency region 19a.
[0079] During actual imaging, the high-frequency region 19a can be located at any position in the pixel array section 1. In order to confirm the influence of this difference in position, the positions of the high-frequency region 19a in the pixel array section 1 when shifted by one pixel (one column) are shown in order in Figures 11 to 14. As can be seen, depending on the position of the high-frequency region 19a in the pixel array section 1, the pixel 2G used to image the high-frequency region 19a will be L and pixel 2G S The ratio of the numbers may be different.
[0080] Specifically, in the example shown in FIG. 11, six pixels 2G L and one pixel 2G S are used to image the high frequency region 19a. In the pixel block 20G-2, three pixels 2G L and four pixels 2G S are used to image the high frequency region 19a. In total, nine pixels 2G L and 5 pixels 2G S is used to image the high frequency region 19a. L and pixel 2G S The ratio of the numbers is 9 to 5.
[0081] In the example shown in FIG. 12, six pixels 2G L and one pixel 2G S are used to image the high frequency region 19a. In the pixel block 20G-2, three pixels 2G L and four pixels 2G S are used to image the high frequency region 19a. In total, nine pixels 2G L and 5 pixels 2G S The pixel 2G used to capture the high frequency region 19a L and pixel 2G S The ratio of the numbers is 9 to 5.
[0082] In the example shown in FIG. 13, in the pixel block 20G-1, three pixels 2G L and two pixels 2G S are used to image the high frequency region 19a. In the pixel block 20G-2, three pixels 2G L and two pixels 2G S are used to image the high frequency region 19a. In total, six pixels 2G L and four pixels 2G S is used to image the high frequency region 19a. L and pixel 2G S The ratio of the numbers is 6 to 4.
[0083] In the example shown in FIG. 14, two pixels 2G S are used to image the high frequency region 19a. In the pixel block 20G-2, two pixels 2G L are used to image the high frequency region 19a. L and two pixels 2G S is used to image the high frequency region 19a. L and pixel 2G S The ratio of the numbers is 2:2.
[0084] In this way, depending on the position of the high frequency region 19a in the pixel array unit 1, the pixel 2G used for imaging the high frequency region 19a L and pixel 2G S The ratio of the number of pixels fluctuates slightly. However, in the above example, the fluctuation is within a range between 9:5 and 1:1. This fluctuation is smaller than the fluctuation when the pixel arrangement of all pixel blocks 20G is the same. A comparative example will also be used for explanation.
[0085] 15 to 19 are diagrams illustrating a comparative example. As shown in FIG. 15, a pixel array unit 1E according to the comparative example differs from the pixel array unit 1 (FIG. 3) according to the embodiment described above in that it does not include pixel block 20G-2, and that this portion is replaced by pixel block 20G-1. In the pixel array unit 1E, the pixel arrangement direction of all pixel blocks 20G is the same. FIGS. 16 to 19 sequentially show the positions of the high-frequency region 19a in the pixel array unit 1 when the position is shifted by one pixel (one column).
[0086] In the example shown in FIG. 16, in the upper left pixel block 20G-1, six pixels 2G L and one pixel 2G S are used to image the high frequency region 19a. In the pixel block 20G-1 at the bottom right, six pixels 2G L and one pixel 2G S are used to image the high frequency region 19a. In total, 12 pixels 2G L and two pixels 2G Sis used to image the high frequency region 19a. L and pixel 2G S The ratio of the numbers is 12 to 2.
[0087] In the example shown in FIG. 17, in the upper left pixel block 20G-1, six pixels 2G L and one pixel 2G S are used to image the high frequency region 19a. In the pixel block 20G-1 at the bottom right, six pixels 2G L and one pixel 2G S are used to image the high frequency region 19a. In total, 12 pixels 2G L and two pixels 2G S The pixel 2G used to capture the high frequency region 19a L and pixel 2G S The ratio of the numbers is 12 to 2.
[0088] In the example shown in FIG. 18, in the upper left pixel block 20G-1, three pixels 2G L and two pixels 2G S are used to image the high frequency region 19a. In the pixel block 20G-1 at the bottom right, three pixels 2G L and two pixels 2G S are used to image the high frequency region 19a. In total, six pixels 2G L and four pixels 2G S is used to image the high frequency region 19a. L and pixel 2G S The ratio of the numbers is 6 to 4.
[0089] In the example shown in FIG. 19, in the upper left pixel block 20G-1, two pixels 2G S are used to image the high frequency region 19a. In the pixel block 20G-1 at the bottom right, two pixels 2G S are used to image the high frequency region 19a. In total, four pixels 2G S is used to image the high frequency region 19a. Land pixel 2G S The ratio of the numbers is 0 to 4.
[0090] In this way, in the pixel array section 1E according to the comparative example, the pixel 2G used for imaging the high frequency region 19a is determined depending on the position of the high frequency region 19a in the pixel array section 1E. L and pixel 2G S The ratio of the number of pixels fluctuates greatly. In the above example, the fluctuations range over a wide range, from 12:2 to 0:4. The larger the fluctuations, the more obvious the moiré becomes. As explained above, such fluctuations are reduced in the imaging device 100 according to the first embodiment, and moiré is suppressed.
[0091] 20 and 21 are diagrams showing examples of moiré suppression. FIG. 20 shows an example of an image 13 obtained when the pixel array unit 1 according to the embodiment is used. FIG. 21 shows an example of an image obtained when the pixel array unit 1E according to the comparative example is used, as image 13E. As can be seen, moiré is significantly observed in image 13E. Compared to image 13E, moiré is suppressed in image 13.
[0092] 2. Modifications of the First Embodiment Several modifications based on the technology of the above-described first embodiment will now be described.
[0093] For example, in a pixel block 20 of the pixel array unit 1, some of the pixels 2 in the first pixel group are shielded from light, and some of the pixels 2 in the second pixel group are shielded from light, so that the pixel block 20 can be used as a phase difference pixel unit (see, for example, Patent Document 3). This allows phase difference information to be obtained, which can be used for focus adjustment, distance measurement, and the like when capturing an image of a subject 19.
[0094] In the above embodiment, a case has been described in which a first pixel block and a second pixel block (pixel block 20G-1 and pixel block 20G-2) exist for pixel block 20G among pixel block 20R, pixel block 20G, and pixel block 20B. However, a first pixel block and a second pixel block may exist for any one or more types of pixel blocks among pixel block 20R, pixel block 20G, and pixel block 20B.
[0095] In the above embodiment, when the exposure time of the first pixel group is relatively long and the exposure time of the second pixel group is relatively short (exposure time T L > Exposure time T S However, the relationship between the exposure times may be reversed. That is, the exposure time of the first pixel group may be relatively short and the exposure time of the second pixel group may be relatively long (exposure time T S > Exposure time T L ).
[0096] In the above embodiment, an example was described in which the pixel block 20R of the pixel array unit 1 includes eight pixels 2R, the pixel block 20G includes ten pixels 2G, and the pixel block 20B includes eight pixels 2B. However, the configuration of the pixel array unit 1 is not limited to this. Any configuration may be employed in which the pixel array direction of the first pixel group is anisotropic and the direction can be made different between the first pixel block and the second pixel block. Some examples will be described with reference to FIGS. 22 to 25 .
[0097] 22 to 25 are diagrams showing an example of the schematic configuration of the pixel array unit 1. In the example shown in Fig. 22, pixel block 20R includes four pixels 2R arranged in a square. Pixel block 20G includes four pixels 2G arranged in a square. Pixel block 20B includes four pixels 2B arranged in a square. Pixel blocks 20R, 20G, and 20B are arranged in a Bayer array.
[0098] A plurality of units are arranged in a square, with one pixel block 20R, two pixel blocks 20G, and one pixel block 20B as one unit. Here, the row in which one unit is located corresponds to the pixel block row (pixel block row L1 or pixel block row L2).
[0099] Pixel blocks 20R, 20G, and 20B located in pixel block row L1 are illustrated and designated pixel block 20R-1, pixel block 20G-1, and pixel block 20B-1, respectively. Pixel blocks 20R, 20G, and 20B located in pixel block row L2 are illustrated and designated pixel block 20R-2, pixel block 20G-2, and pixel block 20B-2, respectively.
[0100] The pixel block 20R is made up of two pixels 2R L and two pixels 2R S In the pixel block 20R-1, the first pixel row includes one pixel 2R S and one pixel 2R L In the second pixel row, one pixel 2R L and one pixel 2R S In the pixel block 20R-2, one pixel 2R is arranged in the first pixel row. L and one pixel 2R S In the second pixel row, one pixel 2R S and one pixel 2R L are arranged side by side.
[0101] The pixel block 20G is made up of two pixels 2G L and two pixels 2G S In the pixel block 20G-1, the first pixel row includes one pixel 2G S and one pixel 2G L In the second pixel row, one pixel 2G L and one pixel 2G S In the pixel block 20G-2, one pixel 2G is arranged in the first pixel row. L and one pixel 2G S In the second pixel row, one pixel 2G S and one pixel 2G L are arranged side by side.
[0102] The pixel block 20B is made up of two pixels 2B L and two pixels 2B SIn the pixel block 20B-1, the first pixel row includes one pixel 2B S and one pixel 2B L In the second pixel row, one pixel 2B L and one pixel 2B S In the pixel block 20B-2, one pixel 2B is arranged in the first pixel row. L and one pixel 2B S In the second pixel row, one pixel 2B S and one pixel 2B L are arranged side by side.
[0103] 23 , pixel block 20R includes nine pixels 2R arranged in a square. Pixel block 20G includes nine pixels 2G arranged in a square. Pixel block 20B includes nine pixels 2B arranged in a square. Pixel blocks 20R, 20G, and 20B are arranged in a Bayer array.
[0104] The pixel block 20R is made up of five pixels 2R L and four pixels 2R S In the pixel block 20R-1, the first pixel row includes three pixels 2R L In the second pixel row, two pixels 2R L and one pixel 2R S In the third pixel row, three pixels 2R S In the pixel block 20R-2, three pixels 2R are arranged in the first pixel row. S In the second pixel row, one pixel 2R S and two pixels 2R L In the third pixel row, three pixels 2R L are arranged side by side.
[0105] The pixel block 20G is made up of five pixels 2G L and 4 pixels 2G S In the pixel block 20G-1, the first pixel row includes three pixels 2G LIn the second pixel row, two pixels 2G L and one pixel 2G S In the third pixel row, three pixels 2G S In the pixel block 20G-2, three pixels 2G are arranged in the first pixel row. S In the second pixel row, one pixel 2G S and two pixels 2G L In the third pixel row, three pixels 2G L are arranged side by side.
[0106] The pixel block 20B is made up of five pixels 2B L and four pixels 2B S In the pixel block 20B-1, the first pixel row includes three pixels 2B L In the second pixel row, two pixels 2B L and one pixel 2B S are arranged side by side. In the third pixel row, three pixels 2B S In the pixel block 20B-2, three pixels 2B are arranged in the first pixel row. S In the second pixel row, one pixel 2B S and two pixels 2B L are arranged side by side. In the third pixel row, three pixels 2B L are arranged side by side.
[0107] 24 , pixel block 20R includes 16 pixels 2R arranged in a square. Pixel block 20G includes 16 pixels 2G arranged in a square. Pixel block 20B includes 16 pixels 2B arranged in a square. Pixel blocks 20R, 20G, and 20B are arranged in a Bayer array.
[0108] The pixel block 20R is made up of eight pixels 2R L and 8 pixels 2R S In the pixel block 20R-1, two pixels 2R are included in each of the first pixel row and the second pixel row. L and two pixels 2R Sare arranged side by side. Two pixels 2R are arranged in each of the third and fourth pixel rows. S and two pixels 2R L In the pixel block 20R-2, two pixels 2R are arranged side by side in each of the first pixel row and the second pixel row. S and two pixels 2R L are arranged side by side. Two pixels 2R are arranged in each of the third and fourth pixel rows. L and two pixels 2R S are arranged side by side.
[0109] The pixel block 20G is made up of eight pixels 2G L and 8 pixels 2G S In the pixel block 20G-1, two pixels 2G are included in each of the first pixel row and the second pixel row. L and two pixels 2G S are arranged side by side. Two pixels 2G are arranged in each of the third and fourth pixel rows. S and two pixels 2G L In the pixel block 20G-2, two pixels 2G are arranged side by side in each of the first pixel row and the second pixel row. S and two pixels 2G L are arranged side by side. Two pixels 2G are arranged in each of the third and fourth pixel rows. L and two pixels 2G S are arranged side by side.
[0110] The pixel block 20B is made up of eight pixels 2B L and eight pixels 2B S In the pixel block 20B-1, two pixels 2B are included in each of the first pixel row and the second pixel row. L and two pixels 2B S are arranged side by side. Two pixels 2B are arranged in each of the third pixel row and the fourth pixel row. S and two pixels 2B L In the pixel block 20B-2, two pixels 2B are arranged side by side in each of the first pixel row and the second pixel row. S and two pixels 2B Lare arranged side by side. Two pixels 2B are arranged in each of the third pixel row and the fourth pixel row. L and two pixels 2B S are arranged side by side.
[0111] 25, pixel block 20R includes eight pixels 2R. Pixel block 20G includes eight pixels 2G. Pixel block 20B includes eight pixels 2B. Pixel blocks 20R, 20G, and 20B are arranged in a Bayer array.
[0112] The pixel block 20R is made up of four pixels 2R L and four pixels 2R S In the pixel block 20R-1, the first pixel row includes two pixels 2R S and two pixels 2R L In the second pixel row, two pixels 2R L and two pixels 2R S In the pixel block 20R-2, two pixels 2R L and two pixels 2R S In the second pixel row, two pixels 2R S and two pixels 2R L are arranged side by side.
[0113] The pixel block 20G is made up of four pixels 2G L and 4 pixels 2G S In the pixel block 20G-1, the first pixel row includes two pixels 2G S and two pixels 2G L In the second pixel row, two pixels 2G L and two pixels 2G S In the pixel block 20G-2, two pixels 2G are arranged in the first pixel row. L and two pixels 2G S In the second pixel row, two pixels 2G S and two pixels 2G L are arranged side by side.
[0114] The pixel block 20B is made up of four pixels 2B L and four pixels 2B S In the pixel block 20B-1, the first pixel row includes two pixels 2B S and two pixels 2B L In the second pixel row, two pixels 2B L and two pixels 2B S In the pixel block 20B-2, two pixels 2B are arranged in the first pixel row. L and two pixels 2B S In the second pixel row, two pixels 2B S and two pixels 2B L are arranged side by side.
[0115] In the various configurations of the pixel array unit 1 as described above, the pixel block 20R-1 and the pixel block 20R-2 form a pixel 2R L The pixel block 20G-1 and the pixel block 20G-2 have different arrangement directions (pixel arrangement directions of the first pixel group). L The pixel block 20B-1 and the pixel block 20B-2 have different arrangement directions (pixel arrangement directions of the first pixel group). L The arrangement direction of the first pixel group (the arrangement direction of the pixels of the first pixel group) is different. Moire can be suppressed by the principles described above.
[0116] 3. Second Embodiment In the second embodiment, moire is suppressed by signal processing. The basic configuration of the image pickup device 100 may be the same as that shown in Fig. 1. The pixel array section 1 will be described with reference to Fig. 26.
[0117] 26 is a diagram showing an example of a schematic configuration of the pixel array unit 1. The multiple pixel blocks 20 include pixel block 20R, pixel block 20G, and pixel block 20B. In the second embodiment, the second pixel blocks (pixel block 20R-2, pixel block 20G-2, pixel block 20B-2) described in the first embodiment may be omitted.
[0118] 27 is a diagram showing an example of processing (imaging method, image generation method) executed by the processing unit 11. In step S11, the processing unit 11 acquires signal values of the first pixel group and the second pixel group for each of the multiple pixel blocks 20. The processing unit 11 also acquires a detection value D based on the signal values of each pixel 2.
[0119] The detection value D is an index indicating whether or not the pixel 2 of the pixel block 20 is used to capture an image of the high frequency region 19a of the subject 19. For example, a filter for detecting high frequency regions (bandpass filter) is applied to the signal value of each pixel 2 of the pixel array unit 1, thereby obtaining (calculating) the detection value D. Examples of filters include a filter for detecting high frequency components in the vertical and horizontal directions of the pixel array unit 1, and a filter for detecting high frequency components in the diagonal direction. The higher the possibility that the pixel is in the high frequency region 19a, the larger the detection value D.
[0120] In step S12, for each of the multiple pixel blocks 20, the processing unit 11 calculates the signal value of that pixel block 20 based on the signal value of the first pixel group of that pixel block 20, the signal value of the second pixel group of that pixel block 20, and the adjusted blend ratio α.
[0121] The blend ratio α is used to weight the signal values of the first pixel group and the second pixel group when they are added together. V1 and the signal value of the second pixel group is S V2 and the signal value of the pixel block 20 is S V3 Then, as an example, S V3 The constants A and B in the formula (1) are calculated as follows: V1 and S V2 and may be determined arbitrarily. V3 = (1-α) × A × S V1 +α×B×S V2 (1)
[0122] When pixel block 20 is used to capture an image of high frequency region 19 a of subject 19, processing unit 11 adjusts blending ratio α so that blending ratio α approaches 0.5 (including setting blending ratio α to 0.5). This will be described with reference to FIG. 28 as well.
[0123] 28 is a diagram showing an example of adjusting the blending ratio α. The horizontal axis of the graph represents the detected value D. The vertical axis of the graph represents the blending ratio α. In this example, a threshold value Dth is set for the detected value D. When the detected value D is equal to or greater than the threshold value Dth, the processing unit 11 adjusts the blending ratio α so that the blending ratio α approaches 0.5.
[0124] The detected value D is the threshold D th When the blending ratio α is less than 0, the blending ratio α may be adjusted to any value. For example, by default, the blending ratio α may be adjusted to 0, in which case the first pixel group (pixel 2R L , pixel 2G L , pixel 2B L The signal value of pixel block 20 is calculated using only the signal values of pixel 2R. When the signal value of the first pixel group approaches a maximum value (e.g., the upper limit of the non-saturation range), the blending ratio α may be adjusted to 1, in which case the signal value of pixel 2R is used to calculate the signal value of pixel block 20. S , pixel 2G S , pixel 2B S ) are used to calculate the signal value of pixel block 20.
[0125] The signal value of each pixel block 20 is calculated using the blending ratio α adjusted as described above. An HDR image with a resolution according to the number of pixel blocks 20 is obtained as the image 13. Note that information related to the blending ratio α (such as the value of the blending ratio α) may be added to, for example, a register (not shown) in the imaging device 100.
[0126] By setting the blending ratio α close to 0.5, moire is suppressed. That is, as described above with reference to FIGS. 16 to 19, depending on the position of the high-frequency region 19a in the pixel array unit 1, the pixel 2G used for capturing the high-frequency region 19a is L and pixel 2G SIn this case, if the blending ratio α is close to 1 or 0 (pixel 2G L or pixel 2G S If only one of the signal values is used, the fluctuation in the signal value of the pixel block 20 calculated will be large, and moiré may become apparent. By setting the blending ratio α closer to 0.5, the fluctuation in the signal value of the pixel block 20 can be reduced, and moiré can be suppressed.
[0127] 29 is a diagram showing an example of moiré suppression. The image shown is an example of image 13 obtained when the blending ratio α is adjusted as described above. For example, moiré is suppressed compared to image 13E (comparison example) in FIG. 21 described above.
[0128] 4. Modifications of the Second Embodiment Several modifications of the configuration of the pixel array section 1 based on the technology of the second embodiment will be described with reference to FIGS.
[0129] 30 to 33 are diagrams showing an example of the schematic configuration of the pixel array unit 1. In the example shown in Fig. 30, pixel block 20R includes four pixels 2R arranged in a square. Pixel block 20G includes four pixels 2G arranged in a square. Pixel block 20B includes four pixels 2B arranged in a square. Pixel blocks 20R, 20G, and 20B are arranged in a Bayer array.
[0130] The pixel block 20R is made up of two pixels 2R L and two pixels 2R S The first pixel row includes one pixel 2R S and one pixel 2R L In the second pixel row, one pixel 2R L and one pixel 2R S are arranged side by side.
[0131] The pixel block 20G is made up of two pixels 2G L and two pixels 2G S The first pixel row includes one pixel 2G. S and one pixel 2G L In the second pixel row, one pixel 2G Land one pixel 2G S are arranged side by side.
[0132] The pixel block 20B is made up of two pixels 2B L and two pixels 2B S The first pixel row includes one pixel 2B. S and one pixel 2B L In the second pixel row, one pixel 2B L and one pixel 2B S are arranged side by side.
[0133] 31 , pixel block 20R includes nine pixels 2R arranged in a square. Pixel block 20G includes nine pixels 2G arranged in a square. Pixel block 20B includes nine pixels 2B arranged in a square. Pixel blocks 20R, 20G, and 20B are arranged in a Bayer array.
[0134] The pixel block 20R is made up of five pixels 2R L and four pixels 2R S The first pixel row includes one pixel 2R L , one pixel 2R S and one pixel 2R L In the second pixel row, one pixel 2R S , one pixel 2R L and one pixel 2R S are arranged side by side. In the third pixel row, one pixel 2R L , one pixel 2R S and one pixel 2R L are arranged side by side.
[0135] The pixel block 20G is made up of five pixels 2G L and 4 pixels 2G S The first pixel row includes one pixel 2G. L , one pixel 2G S and one pixel 2G L In the second pixel row, one pixel 2G S , one pixel 2G L and one pixel 2G SIn the third pixel row, one pixel 2G L , one pixel 2G S and one pixel 2G L are arranged side by side.
[0136] The pixel block 20B is made up of five pixels 2B L and four pixels 2B S The first pixel row includes one pixel 2B. L , one pixel 2B S and one pixel 2B L In the second pixel row, one pixel 2B S , one pixel 2B L and one pixel 2B S are arranged side by side. In the third pixel row, one pixel 2B L , one pixel 2B S and one pixel 2B L are arranged side by side.
[0137] 32 , pixel block 20R includes 16 pixels 2R arranged in a square. Pixel block 20G includes 16 pixels 2G arranged in a square. Pixel block 20B includes 16 pixels 2B arranged in a square. Pixel blocks 20R, 20G, and 20B are arranged in a Bayer array.
[0138] The pixel block 20R is made up of eight pixels 2R L and 8 pixels 2R S Each of the first pixel row and the second pixel row includes two pixels 2R L and two pixels 2R S are arranged side by side. Two pixels 2R are arranged in each of the third and fourth pixel rows. S and two pixels 2R L are arranged side by side.
[0139] The pixel block 20G is made up of eight pixels 2G L and 8 pixels 2G S Each of the first pixel row and the second pixel row includes two pixels 2G L and two pixels 2G Sare arranged side by side. Two pixels 2G are arranged in each of the third and fourth pixel rows. S and two pixels 2G L are arranged side by side.
[0140] The pixel block 20B is made up of eight pixels 2B L and eight pixels 2B S Each of the first pixel row and the second pixel row includes two pixels 2B L and two pixels 2B S are arranged side by side. Two pixels 2B are arranged in each of the third pixel row and the fourth pixel row. S and two pixels 2B L are arranged side by side.
[0141] 33, pixel block 20R includes eight pixels 2R. Pixel block 20G includes eight pixels 2G. Pixel block 20B includes eight pixels 2B. Pixel blocks 20R, 20G, and 20B are arranged in a Bayer array.
[0142] The pixel block 20R is made up of four pixels 2R L and four pixels 2R S The first pixel row includes two pixels 2R S and two pixels 2R L In the second pixel row, two pixels 2R L and two pixels 2R S are arranged side by side.
[0143] The pixel block 20G is made up of four pixels 2G L and 4 pixels 2G S The first pixel row includes two pixels 2G. S and two pixels 2G L In the second pixel row, two pixels 2G L and two pixels 2G S are arranged side by side.
[0144] The pixel block 20B is made up of four pixels 2B L and four pixels 2B S The first pixel row includes two pixels 2B Sand two pixels 2B L In the second pixel row, two pixels 2B L and two pixels 2B S are arranged side by side.
[0145] For example, even when the pixels 2 are arranged in various ways as described above, moire can be suppressed by adjusting the blend ratio α as previously described with reference to FIG.
[0146] 5. Other Embodiments It is also possible to combine the technology of the first embodiment described above with the technology of the second embodiment. For example, the pixel array unit 1 as shown in Figures 3 and 22 to 25 described above may be combined with the process shown in Figure 27 described above. This can further enhance the moire suppression effect.
[0147] 6. Summary The techniques described above can be specified, for example, as follows. One of the techniques disclosed is an imaging device 100. As described with reference to FIGS. 1 to 10 and 22 to 25, the imaging device 100 includes a plurality of pixel blocks 20 arranged in a two-dimensional array. The plurality of pixel blocks 20 includes a first pixel block (e.g., pixel block 20G-1) and a second pixel block (e.g., pixel block 20G-2) located in different rows (pixel block row L1 and pixel block row L2). Each of the first pixel block and the second pixel block is a first pixel group (e.g., pixel block 20G-1). L ) and a second pixel group (e.g., pixel 2G S ) in the pixel array direction of the first pixel group of the first pixel block (for example, pixel 2G of pixel block 20G-1). L the pixel array direction of the first pixel group of the second pixel block (for example, pixel 2G of pixel block 20G-2) L As described with reference to FIGS. 3 and 4, the arrangement direction of the pixel 2 (for example, pixel 2R) of the first pixel group is different from that of the pixel 2 of the second pixel group. L , pixel 2G L , pixel 2B L ) exposure time T L and pixel 2 of the second pixel group (e.g., pixel 2R S , pixel 2GS , pixel 2B S ) exposure time T S 3 and 22 to 25, the first pixel blocks (e.g., pixel block 20R-1, pixel block 20G-1, pixel block 20B-1) and the second pixel blocks (e.g., pixel block 20R-2, pixel block 20G-2, pixel block 20B-2) may correspond to the same color (e.g., red, green, blue).
[0148] According to the imaging device 100, the pixel arrangement directions of the first pixel groups are different between the first pixel block and the second pixel block. As a result, as previously described with reference to FIGS. 11 to 19, the number of pixels 2 of the first pixel group used to image the high frequency region 19 a (for example, pixel 2G L ) and the number of pixels 2 in the second pixel group (e.g., pixel 2G S As a result, when generating an HDR image (an example of image 13), moire can be suppressed.
[0149] As described with reference to FIGS. 3 and 22 to 25, the pixel array direction of the first pixel group (for example, pixel 2R L Array direction of pixel 2G L Array direction of pixel 2B L The pixel arrangement direction of the first pixel group in the first pixel block may include a direction diagonal to the row direction (X-axis direction) and the column direction (Y-axis direction). The pixel arrangement direction of the first pixel group in the first pixel block and the pixel arrangement direction of the first pixel group in the second pixel block may be opposite directions. For example, by making the pixel arrangement directions of the first pixel groups in the first pixel block and the second pixel block different from each other in this way, moiré can be suppressed.
[0150] As described with reference to Figure 5 and other figures, the center position of the first pixel group and the center position of the second pixel group within the same pixel block may be the same (e.g., center position 200R, center position 200G, center position 200B). This makes it possible to obtain signal values of the first pixel group and the second pixel group that correspond to more accurate positions than when the center positions of the first pixel group and the second pixel group are different. This can contribute to improving the image quality of the image 13.
[0151] 1 and 6 to 8, the imaging device 100 may include a processing unit 11 that calculates (step S2) a signal value for each of a plurality of pixel blocks 20 based on the signal values of the first pixel group and the signal values of the second pixel group of the pixel block 20. Moiré can be suppressed by generating an image 13 based on the signal values of the pixel blocks 20 obtained by such calculation.
[0152] Various methods may be used to obtain the signal values of the first pixel group and the second pixel group. For example, when the processing unit 11 performs AD conversion on all of the signals from each pixel 2, the method described with reference to FIG. 7 etc. may be used. That is, the processing unit 11 may calculate the signal values of the first pixel group based on the signal values of each pixel 2 in the first pixel group, and may calculate the signal values of the second pixel group based on the signal values of each pixel 2 in the second pixel group (step S1). On the other hand, according to the method described with reference to FIGS. 8 to 10 etc., the signal values of one pixel group can be obtained with one AD conversion, so the number of AD conversions can be reduced. That is, each of the first pixel block and the second pixel block performs AD conversion on each pixel 2 in the first pixel group (for example, pixel 2G L ) photoelectric conversion unit 21 to the first charge accumulation unit (charge accumulation unit 23 L ) for common connection to the first wiring (wiring 27 L ), and each pixel 2 of the second pixel group (for example, pixel 2G S ) photoelectric conversion unit 21 to the second charge accumulation unit (charge accumulation unit 23 S ) for common connection to the second wiring (wiring 27S The signal value of the first pixel group may be stored in a first charge storage unit (charge storage unit 23 L ), and the signal value of the second pixel group is a signal value corresponding to the charge accumulated in the second charge accumulation unit (charge accumulation unit 23 S ) (step S1).
[0153] Various pixel arrangements are possible. For example, as described with reference to FIGS. 3 and 4, each of the first pixel block (pixel block 20G-1) and the second pixel block (pixel block 20G-2) includes 10 pixels 2G, and the first pixel group includes 6 pixels 2G. L the second pixel group includes four pixels 2G S and the exposure time T L is the exposure time T of the second pixel group S The first pixel block and the second pixel block may correspond to green. The plurality of pixel blocks 20 include a pixel block 20R corresponding to red and a pixel block 20B corresponding to blue, and each of the pixel block 20R corresponding to red and the pixel block 20B corresponding to blue includes eight pixels 2 (eight pixels 2R, eight pixels 2B), and four of the eight pixels 2 (four pixels 2R, L , four pixels 2B L ) exposure time T L The remaining four pixels 2 (four pixels 2R S , four pixels 2B S ) exposure time T S It could be longer than that.
[0154] Alternatively, as described with reference to FIGS. 4 and 22, each of the first pixel blocks (pixel block 20R-1, pixel block 20G-1, pixel block 20B-1) and the second pixel blocks (pixel block 20R-2, pixel block 20G-2, pixel block 20B-2) includes four pixels 2 (four pixels 2R, four pixels 2G, four pixels 2B), and each of the first pixel group and the second pixel group includes two pixels (two pixels 2R, L and two pixels 2R S, two pixels 2G L and two pixels 2G S , two pixels 2B L and two pixels 2R S ), and the first pixel group (pixel 2R L , pixel 2G L , pixel 2B L ) exposure time T L is the second pixel group (pixel 2R S , pixel 2G S , pixel 2B S ) exposure time T S It could be longer than that.
[0155] Alternatively, as described with reference to FIGS. 4 and 23, each of the first pixel blocks (pixel block 20R-1, pixel block 20G-1, pixel block 20B-1) and the second pixel blocks (pixel block 20R-2, pixel block 20G-2, pixel block 20B-2) includes nine pixels 2 (nine pixels 2R, nine pixels 2G, nine pixels 2B), and the first pixel group includes five pixels 2 (five pixels 2R L , 5 pixels 2G L , five pixels 2B L ), and the second pixel group includes four pixels 2 (four pixels 2R S , 4 pixels 2G S , four pixels 2B S ), and the exposure time T L is the exposure time T of the second pixel group S It could be longer than that.
[0156] Alternatively, as described with reference to FIGS. 4 and 24, each of the first pixel blocks (pixel block 20R-1, pixel block 20G-1, pixel block 20B-1) and the second pixel blocks (pixel block 20R-2, pixel block 20G-2, pixel block 20B-2) includes 16 pixels 2 (16 pixels 2R, 16 pixels 2G, 16 pixels 2B), and each of the first pixel group and the second pixel group includes 8 pixels (8 pixels 2R L and 8 pixels 2R S , 8 pixels 2G L and 8 pixels 2G S, 8 pixels 2B L and eight pixels 2B S ), and the exposure time T L is the exposure time T of the second pixel group S It could be longer than that.
[0157] Alternatively, as described with reference to FIGS. 4 and 25, each of the first pixel blocks (pixel block 20R-1, pixel block 20G-1, pixel block 20B-1) and the second pixel blocks (pixel block 20R-2, pixel block 20G-2, pixel block 20B-2) includes eight pixels 2 (eight pixels 2R, eight pixels 2G, eight pixels 2B), and each of the first pixel group and the second pixel group includes four pixels 2 (four pixels 2R, L and four pixels 2R S , 4 pixels 2G L and four pixels 2G S , four pixels 2B L and four pixels 2B S ), and the exposure time T L is the exposure time T of the second pixel group S It could be longer than that.
[0158] The imaging device 100 described with reference to FIGS. 1, 26 to 28, 30 to 33, etc., is also one of the techniques disclosed. The imaging device 100 is arranged in a two-dimensional array, and each pixel is a first pixel group (pixel 2R L , pixel 2G L , pixel 2B L ) and the second pixel group (pixel 2R S , pixel 2G S , pixel 2B SThe image processing apparatus includes a plurality of pixel blocks 20 including a first pixel group and a second pixel group, and a processing unit 11 that calculates (step S12) a signal value for each of the plurality of pixel blocks 20 based on the signal values of the first pixel group, the signal values of the second pixel group, and the adjusted blending ratio α of the pixel block 20. When the pixel block 20 is used to capture an image of a high-frequency region 19a of a subject 19, the processing unit 11 adjusts the blending ratio α so that the blending ratio approaches 0.5. As described above, when the blending ratio α is close to 1 or 0, the fluctuation in the calculated signal value of the pixel block 20 increases, which may result in moiré becoming apparent. By bringing the blending ratio α closer to 0.5, the fluctuation in the signal value of the pixel block 20 is reduced, thereby suppressing moiré.
[0159] 27 and 28 , the processing unit 11 may adjust the blending ratio α so that the blending ratio α approaches 0.5 when the detection value D, which is an index indicating whether or not the pixel block 20 is used to capture an image of the high frequency region 19 a of the subject 19, is equal to or greater than a predetermined threshold value Dth. For example, the blending ratio α can be adjusted based on such a threshold determination.
[0160] The effects described in this disclosure are merely examples and are not limited to the disclosed contents. Other effects may also be obtained.
[0161] Although the embodiments of the present disclosure have been described above, the technical scope of the present disclosure is not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of the present disclosure. Furthermore, components of different embodiments and modifications may be combined as appropriate.
[0162] Note that the present technology may also be configured as follows. (1) An imaging device including a plurality of pixel blocks arranged in a two-dimensional array, the plurality of pixel blocks including a first pixel block and a second pixel block located in different rows, the first pixel block and the second pixel block each including a first pixel group and a second pixel group, and a pixel array direction of the first pixel group in the first pixel block and a pixel array direction of the first pixel group in the second pixel block differ from each other. (2) The imaging device according to (1), in which the pixel array direction of the first pixel group includes a direction diagonal to the row direction and the column direction. (3) The imaging device according to (1) or (2), in which the pixel array direction of the first pixel group in the first pixel block and the pixel array direction of the first pixel group in the second pixel block are opposite to each other. (4) The imaging device according to any of (1) to (3), in which an exposure time of pixels of the first pixel group and an exposure time of pixels of the second pixel group differ from each other. (5) The imaging device according to any one of (1) to (4), wherein the first pixel block and the second pixel block correspond to the same color. (6) The imaging device according to any one of (1) to (5), wherein a center position of the first pixel group and a center position of the second pixel group within the same pixel block are the same. (7) The imaging device according to any one of (1) to (6), comprising: a processing unit that calculates, for each of the plurality of pixel blocks, a signal value of the pixel block based on a signal value of the first pixel group and a signal value of the second pixel group of the pixel block. (8) The imaging device according to (7), wherein the processing unit calculates the signal value of the first pixel group based on the signal value of each pixel of the first pixel group, and calculates the signal value of the second pixel group based on the signal value of each pixel of the second pixel group.(9) The imaging device according to (7), wherein the first pixel block and the second pixel block each include: a first wiring for commonly connecting a photoelectric conversion unit of each pixel of the first pixel group to a first charge accumulation unit; and a second wiring for commonly connecting a photoelectric conversion unit of each pixel of the second pixel group to a second charge accumulation unit. (10) The imaging device according to (9), wherein a signal value of the first pixel group is a signal value corresponding to a charge accumulated in the first charge accumulation unit, and a signal value of the second pixel group is a signal value corresponding to a charge accumulated in the second charge accumulation unit. (11) The imaging device according to any of (1) to (10), wherein the first pixel block and the second pixel block each include 10 pixels, the first pixel group includes 6 pixels, the second pixel group includes 4 pixels, and an exposure time of the first pixel group is longer than an exposure time of the second pixel group. (12) The imaging device according to (11), wherein the first pixel block and the second pixel block correspond to green. (13) The imaging device according to (12), wherein the plurality of pixel blocks include a pixel block corresponding to red and a pixel block corresponding to blue, wherein the pixel block corresponding to red and the pixel block corresponding to blue each include eight pixels, and an exposure time of four of the eight pixels is longer than an exposure time of the remaining four pixels. (14) The imaging device according to any of (1) to (10), wherein the first pixel block and the second pixel block each include four pixels, wherein the first pixel group and the second pixel group each include two pixels, and an exposure time of the first pixel group is longer than an exposure time of the second pixel group. (15) The imaging device according to any one of (1) to (10), wherein each of the first pixel block and the second pixel block includes nine pixels, the first pixel group includes five pixels, the second pixel group includes four pixels, and an exposure time of the first pixel group is longer than an exposure time of the second pixel group.(16) The imaging device according to any one of (1) to (10), wherein the first pixel block and the second pixel block each include 16 pixels, the first pixel group and the second pixel group each include 8 pixels, and an exposure time of the first pixel group is longer than an exposure time of the second pixel group. (17) The imaging device according to any one of (1) to (10), wherein the first pixel block and the second pixel block each include 8 pixels, the first pixel group and the second pixel group each include 4 pixels, and an exposure time of the first pixel group is longer than an exposure time of the second pixel group. (18) An imaging device comprising: a plurality of pixel blocks arranged in a two-dimensional array, each including a first pixel group and a second pixel group; and a processing unit that calculates, for each of the plurality of pixel blocks, a signal value of the pixel block based on the signal value of the first pixel group of the pixel block, the signal value of the second pixel group of the pixel block, and an adjusted blending ratio, wherein the processing unit adjusts the blending ratio so that the blending ratio approaches 0.5 when the pixel block is used for imaging a high frequency region of a subject. (19) The imaging device described in (18), wherein the processing unit adjusts the blending ratio so that the blending ratio approaches 0.5 when a detection value that is an index indicating whether the pixel block is used for imaging a high frequency region of a subject is equal to or greater than a predetermined threshold.
[0163] 1 Pixel array section 2 Pixel 2R L Pixel 2R S Pixel 2G Pixel 2G L Pixel 2G S Pixel 2B Pixel 2B L Pixel 2B S Pixel 3 Control circuit 4 Vertical drive circuit 5 Column signal processing circuit 6 Horizontal drive circuit 7 Output circuit 8 Signal line 9 Signal line 9 L Signal line 9S Signal line 10 Signal line 11 Processing unit 12 Substrate 13 Image 19 Object 19a High frequency region 20 Pixel block 20R Pixel block 20R-1 Pixel block 20R-2 Pixel block 20G Pixel block 20G-1 Pixel block 20G-2 Pixel block 20B Pixel block 20B-1 Pixel block 20B-2 Pixel block 21 Photoelectric conversion unit 22 Transistor 23 Charge storage unit 23 L Charge storage section 23 S Charge storage section 24 Transistor 24 L Transistor 24 S Transistor 25 Transistor 25 L Transistor 25 S Transistor 26 Transistor 26 L Transistor 26 S Transistor 27 Wiring 27 L Wiring 27 S Wiring 100 Imaging device 200R Center position 200G Center position 200B Center position L1 Pixel block row L2 Pixel block row T L Exposure time T S Exposure time VDD Power supply node α Blend ratio
Claims
1. An imaging device comprising a plurality of pixel blocks arranged in a two-dimensional array, the plurality of pixel blocks including a first pixel block and a second pixel block located in different rows, the first pixel block and the second pixel block each including a first pixel group and a second pixel group, and the pixel arrangement direction of the first pixel group in the first pixel block and the pixel arrangement direction of the first pixel group in the second pixel block are different from each other.
2. The imaging device according to claim 1, wherein the pixel array direction of the first pixel group includes a direction diagonal to the row direction and the column direction.
3. The imaging device according to claim 1, wherein the pixel arrangement direction of the first pixel group in the first pixel block and the pixel arrangement direction of the first pixel group in the second pixel block are opposite to each other.
4. The imaging device according to claim 1, wherein the exposure time of the pixels of the first pixel group and the exposure time of the pixels of the second pixel group are different from each other.
5. The imaging device according to claim 1, wherein the first pixel block and the second pixel block correspond to the same color.
6. The imaging device according to claim 1, wherein the center position of the first pixel group and the center position of the second pixel group within the same pixel block are the same.
7. The imaging device according to claim 1, further comprising a processing unit that calculates, for each of the plurality of pixel blocks, a signal value of the pixel block based on a signal value of a first pixel group and a signal value of a second pixel group of the pixel block.
8. The imaging device according to claim 7, wherein the processing unit calculates a signal value of the first pixel group based on the signal value of each pixel of the first pixel group, and calculates a signal value of the second pixel group based on the signal value of each pixel of the second pixel group.
9. The imaging device of claim 7, wherein each of the first pixel block and the second pixel block includes: a first wiring for commonly connecting the photoelectric conversion units of each pixel of the first pixel group to a first charge accumulation unit; and a second wiring for commonly connecting the photoelectric conversion units of each pixel of the second pixel group to a second charge accumulation unit.
10. An imaging device as described in claim 9, wherein the signal value of the first pixel group is a signal value corresponding to the charge accumulated in the first charge accumulation section, and the signal value of the second pixel group is a signal value corresponding to the charge accumulated in the second charge accumulation section.
11. The imaging device of claim 1, wherein each of the first pixel block and the second pixel block includes 10 pixels, the first pixel group includes 6 pixels, the second pixel group includes 4 pixels, and the exposure time of the first pixel group is longer than the exposure time of the second pixel group.
12. The imaging device according to claim 11, wherein the first pixel block and the second pixel block correspond to green.
13. The imaging device of claim 12, wherein the plurality of pixel blocks include a pixel block corresponding to red and a pixel block corresponding to blue, each of the pixel block corresponding to red and the pixel block corresponding to blue includes eight pixels, and the exposure time of four of the eight pixels is longer than the exposure time of the remaining four pixels.
14. The imaging device of claim 1, wherein the first pixel block and the second pixel block each include four pixels, the first pixel group and the second pixel group each include two pixels, and the exposure time of the first pixel group is longer than the exposure time of the second pixel group.
15. The imaging device of claim 1, wherein each of the first pixel block and the second pixel block includes nine pixels, the first pixel group includes five pixels, the second pixel group includes four pixels, and the exposure time of the first pixel group is longer than the exposure time of the second pixel group.
16. The imaging device of claim 1, wherein the first pixel block and the second pixel block each include 16 pixels, the first pixel group and the second pixel group each include 8 pixels, and the exposure time of the first pixel group is longer than the exposure time of the second pixel group.
17. The imaging device of claim 1, wherein the first pixel block and the second pixel block each include eight pixels, the first pixel group and the second pixel group each include four pixels, and the exposure time of the first pixel group is longer than the exposure time of the second pixel group.
18. An imaging device comprising: a plurality of pixel blocks arranged in a two-dimensional array, each including a first pixel group and a second pixel group; and a processing unit that calculates, for each of the plurality of pixel blocks, a signal value of the pixel block based on the signal value of the first pixel group of the pixel block, the signal value of the second pixel group of the pixel block, and an adjusted blending ratio; wherein the processing unit adjusts the blending ratio so that the blending ratio approaches 0.5 when the pixel block is used to image a high-frequency region of a subject.
19. The imaging device according to claim 18, wherein the processing unit adjusts the blending ratio so that the blending ratio approaches 0.5 when a detection value, which is an index indicating whether or not the pixel block is used to image a high-frequency region of a subject, is equal to or greater than a predetermined threshold value.
Citation Information
Patent Citations
Signal processing device, and signal processing method
JP2014179939A
Imaging device, imaging method and program
JP2016201733A
Imaging apparatus and control method thereof, and program
JP2018148271A
Solid-state imaging element, imaging device, and method for controlling solid-state imaging element
WO2023105916A1