Imaging element and electronic device

Abstract

A camera element includes a plurality of pixel blocks each having a plurality of pixels, and a pixel control section that controls the plurality of pixels. The plurality of pixels of each of the plurality of pixel blocks includes image pixels for forming an image and determination pixels for at least determination of the pixel block. The pixel control section can independently control the determination pixels and the image pixels of the pixel block. The plurality of pixel blocks can each have a plurality of determination pixels. The pixel control section has a pixel selection section that selects and controls any of the determination pixels of the plurality of determination pixels for each of the plurality of pixel blocks.

Description

Technical Field

[0001] This invention relates to camera elements and electronic devices. Background Technology

[0002] Imaging elements with multiple pixels arranged in a two-dimensional configuration along both row and column directions are known (e.g., Patent Document 1). The increasing power consumption of imaging elements has long been a problem.

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent Application Publication No. 2013-162148 Summary of the Invention

[0006] Regarding a first aspect of the present invention, there is a camera element comprising: a plurality of pixel blocks having a plurality of pixels; and a pixel control unit for controlling the plurality of pixels, wherein each of the plurality of pixels in the plurality of pixel blocks has an image pixel for forming an image and at least a determination pixel for determining the pixel block, and the pixel control unit is capable of independently controlling the determination pixel and the image pixel of the pixel block.

[0007] Multiple pixel blocks can each have multiple decision pixels. The pixel control unit can have a pixel selection unit, which selects and controls any one of the multiple decision pixels for each pixel block.

[0008] The pixel selection unit can be set corresponding to each pixel block of multiple pixel blocks.

[0009] Multiple pixel blocks can be disposed on the first semiconductor substrate. The pixel selection unit can be disposed on the second semiconductor substrate in the region overlapping with the corresponding pixel blocks, and the second semiconductor substrate is stacked with the first semiconductor substrate.

[0010] The pixel selection unit can add two or more decision pixels from a set of decision pixels together and then read the result.

[0011] It may also include a determination unit, which determines, based on the determination pixel, which controls the image pixel block using either a high frame rate or a low frame rate.

[0012] The pixel control unit can control the pixels to be determined at a high frame rate within a pixel block that is controlled at a low frame rate.

[0013] It may also include a power cut-off unit that cuts off the power supply to the image pixels in a pixel block controlled at a low frame rate during frames in which it is not necessary to read from the image pixels.

[0014] It may also include a read signal output unit, which enables the read signal to the image pixels in frames where the image pixels are read from, and disables it in frames where no reading is performed, in pixel blocks controlled at low frame rates.

[0015] It may also include an invalid signal output unit, which outputs an invalid signal indicating that the image pixel is invalid to the outside of the pixel block in a pixel block controlled at a low frame rate in a frame in which it is not necessary to read from the image pixel.

[0016] It may also include a determination unit, which performs determinations related to multiple pixel blocks based on the determination pixels of multiple pixel blocks.

[0017] It may also include a determination unit that is set corresponding to each of the multiple pixel blocks, and the determination unit performs a determination related to the corresponding pixel block based on the determination pixel of the corresponding pixel block.

[0018] It may also include a determination unit, which determines the focus level of the subject captured by the pixel block based on the determination pixels.

[0019] It may also have a determination unit that determines the exposure of pixel blocks based on the determination of the pixels used.

[0020] Regarding a second aspect of the present invention, there is an electronic device comprising the aforementioned camera element.

[0021] It should be noted that the above description of the invention does not list all the features of the invention. Furthermore, sub-combinations of these feature groups can also constitute an invention. Attached Figure Description

[0022] Figure 1 This is a block diagram of a camera device 10, which is an example of an electronic device in this embodiment.

[0023] Figure 2 This is a conceptual diagram representing the overall functional modules of the camera element 100.

[0024] Figure 3 The diagram schematically illustrates the positional relationship between the area where the pixel section 50 is arranged in the imaging element 100 and the area where the processing circuit section 160 is arranged.

[0025] Figure 4 The schematic diagram shows one of the multiple unit blocks 200 and the peripheral circuit section 500.

[0026] Figure 5 An example of an image constructed using the circuitry of pixel 202 is shown.

[0027] Figure 6This is an explanatory diagram illustrating the pixel blocks of the ROI and non-ROI used in the camera element 100.

[0028] Figure 7 The diagram illustrates the relationship between a high frame rate for pixel blocks of ROI in the imaging element 100 and a low frame rate for pixel blocks of non-ROI.

[0029] Figure 8 It is a block diagram of unit block 200 and peripheral circuit section 500 that focuses on the determination of ROI and the control related to the determination.

[0030] Figure 9 This is a block diagram representing an example of the ROI determination unit 600.

[0031] Figure 10 This is a block diagram of the unit block 200 and the peripheral circuit section 500 that focus on the operation of the ROI control unit 420.

[0032] Figure 11 It is a timeline of the actions of the ROI control unit 420.

[0033] Figure 12 The diagram schematically shows the pixel signal output to horizontal line 502 and the data enable signal output to horizontal line 504.

[0034] Figure 13 Other examples of control using pixel 204 are shown.

[0035] Figure 14 This is a block diagram representing other unit blocks 206 and peripheral circuit sections 506.

[0036] Figure 15 This is a block diagram of the ROI determination unit 440.

[0037] Figure 16 Other pixel blocks 148 are shown schematically.

[0038] Figure 17 This is another example of a block diagram of a unit block 200 and a peripheral circuit section 500 that focuses on the determination of ROI and the control based on that determination.

[0039] Figure 18 schematically showing the direction Figure 17 The pixel signals output from horizontal lines 502 and 503 and the data enable signal output to horizontal line 504. Detailed Implementation

[0040] The present invention will now be described through embodiments thereof, but these embodiments do not limit the invention as defined in the claims. Furthermore, the combinations of features described in the embodiments are not necessarily all necessary for the solutions provided by the invention.

[0041] Figure 1 This is a block diagram of a camera device 10, which is an example of an electronic device according to this embodiment. The camera device 10 includes an optical system 20, an image sensor 100, an image processing unit 24, a control unit 26, a recording unit 28, an operation unit 30, a display unit 32, and a power supply unit 34. The image sensor 100, the image processing unit 24, the control unit 26, the recording unit 28, the operation unit 30, the display unit 32, and the power supply unit 34 are interconnected via a bus 36.

[0042] The optical system 20 has a lens group that causes incident light from the subject, i.e. image light, to be imaged along the optical axis 22 on the imaging surface of the imaging element 100.

[0043] The imaging element 100 converts the amount of incident light that is imaged on the imaging surface by the optical system 20 into an electrical signal for each of the plurality of pixels, thereby obtaining an analog pixel signal. The imaging element 100 further outputs the pixel signal obtained after processing such as analog-to-digital conversion. It should be noted that, unless otherwise stated, the "pixel signal" can be either analog or digital.

[0044] The image processing unit 24, for example, is an ISP (Image Signal Processor), which receives pixel signals from the imaging element 100 and performs processing as a whole image. This processing includes color interpolation, grayscale conversion, compression, etc.

[0045] The control unit 26 controls the entire camera device 10 and controls each of the camera element 100, image processing unit 24, recording unit 28, operation unit 30, display unit 32 and power supply unit 34 via the bus 36.

[0046] The recording unit 28 includes a recording medium such as a hard disk or semiconductor memory for storing video or still images captured by the imaging element 100. The operation unit 30 issues operation commands for various functions of the imaging device 10 under user operation. The display unit 32 is a panel-type display device, such as a liquid crystal panel or an organic EL (Electro-luminescence) panel, that displays the video or still images captured by the imaging element 100. The power supply unit 34 supplies various power sources to the image processing unit 24, control unit 26, recording unit 28, operation unit 30, and display unit 32.

[0047] Figure 2This is a conceptual diagram representing the overall functional modules of the camera element 100. Figure 2 This shows the functional modules and their signal transmission and reception relationships, but does not show the spatial configuration of the camera element 100.

[0048] The camera element 100 includes a pixel unit 50, a drive control unit 60, a vertical drive unit 62, a signal processing unit 64, a horizontal drive unit 68, and an output unit 70.

[0049] The pixel unit 50 has a plurality of pixels arranged in a two-dimensional shape. These pixels perform photoelectric conversion on the image light from the optical system 20.

[0050] The drive control unit 60 receives input clock and indication signals such as operating mode from the control unit 26 to drive the camera element 100. For example, the drive control unit 60 generates clock signals and control signals that serve as references for the operation of the vertical drive unit 62, signal processing unit 64, and horizontal drive unit 68 based on the vertical synchronization signal, horizontal synchronization signal, and master clock, and outputs them to the vertical drive unit 62, signal processing unit 64, and horizontal drive unit 68.

[0051] The vertical drive unit 62 drives pixels, for example, row by row, based on the aforementioned control signal. In other words, the vertical drive unit 62 controls the exposure of the pixels using this control signal. The signal processing unit 64 reads pixel signals from the pixels selected by the vertical drive unit 62 column by column and processes these pixel signals. For example, the signal processing unit 64 converts analog pixel signals from the pixels into digital pixel signals, removes noise, or temporarily stores the processed pixel signals. The horizontal drive unit 68 sequentially reads the pixel signals stored in the signal processing unit 64 and outputs them to the output unit 70.

[0052] The output unit 70 processes the pixel signals sequentially supplied from the signal processing unit 64 and outputs them to the image processing unit 24. For example, buffering may be performed as part of the processing by the output unit 70.

[0053] Figure 3 The diagram schematically illustrates the positional relationship between the area where the pixel portion 50 is disposed and the area where the processing circuit portion 160 is disposed in the imaging element 100. The imaging element 100 has a first semiconductor substrate 140 and a second semiconductor substrate 150, which are stacked in the z-direction in the figure.

[0054] The pixel unit 50 is disposed on the first semiconductor substrate 140. It should be noted that the first semiconductor substrate 140 is sometimes referred to as a pixel chip. The pixel unit 50 has a plurality of pixel blocks 142 arranged in a two-dimensional shape. Each pixel block 142 includes a plurality of pixels arranged in a two-dimensional shape.

[0055] The processing circuit section 160, the drive control section 60, the output section 58, the global vertical drive section 152, and the global horizontal drive section 154 are disposed on the second semiconductor substrate 150. It should be noted that the second semiconductor substrate 150 is sometimes referred to as a signal processing chip.

[0056] Corresponding to the pixel unit 50 having a plurality of pixel blocks 142 arranged in a two-dimensional shape, the processing circuit unit 160 has a plurality of signal processing blocks 162 arranged in a two-dimensional shape. Each signal processing block 162 includes circuitry for driving and processing signals to the pixel blocks 142, and may include, for example, [missing information - likely related to signal processing]. Figure 2 The vertical drive unit 62, the signal processing unit 64, and the horizontal drive unit 68 are each a part or all of each other.

[0057] In this example, the processing circuit section 160 is disposed on the second semiconductor substrate 150 at a position opposite to the pixel section 50. That is, the processing circuit section 160 is configured to overlap at least partially with the pixel section 50 in the optical axis direction. In this case, they may also overlap in the optical axis direction such that one of the pixel section 50 and the processing circuit section 160 includes the other.

[0058] Furthermore, the signal processing block 162 is disposed in the second semiconductor substrate 150 at a position opposite to the corresponding pixel block 142. That is, the signal processing block 162 that drives and processes signals for a pixel block 142 at a certain position is configured to at least partially overlap with the pixel block 142 in the optical axis direction. In this case, they may also overlap in the optical axis direction such that one of the pixel block 142 and the corresponding signal processing block 162 includes the other.

[0059] The drive control unit 60, output unit 58, global vertical drive unit 152, and global horizontal drive unit 154 are disposed around the processing circuit unit 160 in the second semiconductor substrate. Here, the global vertical drive unit 152, global horizontal drive unit 154, and output unit 58 perform control shared by multiple pixel blocks 142, and may be, for example, parts of the vertical drive unit 62, horizontal drive unit 68, and output unit 70 other than those included in the signal processing block 162.

[0060] The imaging element 100 may also have a third semiconductor substrate stacked on top of the first semiconductor substrate 140 and the second semiconductor substrate 150. For example, the third semiconductor substrate may be a memory chip that performs image processing corresponding to the signal output from the second semiconductor substrate 150. In addition, the imaging element 100 may be either a back-illuminated type or a front-illuminated type.

[0061] Figure 4The diagram schematically shows one of the multiple unit blocks 200 and its peripheral circuit section 500. In this or the following figures, for ease of observation, functional modules not shown in this figure and signal lines connected to these functional modules are sometimes omitted from the functional modules included in the unit block 200 and the peripheral circuit section 500.

[0062] Unit block 200 includes one of a plurality of pixel blocks 142, and a so-called local circuit portion disposed corresponding to that pixel block 142. Therefore, unit block 200 has a number corresponding to the plurality of pixel blocks 142, for example, the same number. It should be further noted that all or part of the portion of unit block 200 other than pixel blocks 142 corresponds to... Figure 3 The signal processing block 162. On the other hand, the peripheral circuit section 500 is a circuit section that is shared by multiple unit blocks 200 and can be called a global circuit section.

[0063] Figure 4 The pixel block 142 shown includes image pixels 202 and decision pixels 204. As an example, image pixels 202 are shown in 3 rows and 3 columns, totaling 9 pixels. Image pixels 202 are pixels used to form an image. To simplify the reference numerals, image pixels are sometimes represented by Aij (where i is the row number and j is the column number).

[0064] As an example, the decision pixels 204 are also shown in 3 rows and 3 columns, totaling 9 pixels. Each decision pixel 204 is arranged to the left of each image pixel 202. The decision pixels 204 are images used at least for making decisions on pixel blocks 142. The decision pixels 204 can also be used to form an image. To simplify the reference numerals, the decision pixels 204 are sometimes represented by Bij (i is the row number, j is the column number).

[0065] The number of rows, columns, and total number of pixels 202 for image processing and pixels 204 for decision processing are not limited to those mentioned above. For example, pixels 202 for image processing and pixels 204 for decision processing can also be 16 rows and 16 columns or 32 rows and 32 columns, etc.

[0066] Unit block 200 has a block vertical drive unit 210 and a signal processing circuit 222 for image pixels 202. The block vertical drive unit 210 drives the image pixels 202 line by line based on control signals, such as drive pulses, from the peripheral circuit unit 500. Therefore, it can also be said that the block vertical drive unit 210 constitutes Figure 2 At least a portion of the vertical drive section 62.

[0067] The signal processing circuit 222 reads and processes the pixel signals of the image pixels 202 column by column based on the control signals from the peripheral circuit unit 500. As the signal processing circuit 222, in Figure 4The diagram shows an ADC 224 that converts analog pixel signals to digital pixel signals, and an SRAM 226 that stores the digital pixel signals. The ADC 224 can be, for example, a single-slope configuration, but other configurations are also possible. The signal processing circuitry 222 may also include other processing circuitry such as digital and / or analog CDS (correlated double sampling) circuitry. Figure 4 In this circuit, three signal processing circuits 222 are provided corresponding to three columns of image pixels 202. Pixel signals from the signal processing circuits 222 are output to the peripheral circuit section 500 via the horizontal line 502.

[0068] There is a case where the signal processing circuits 222 contained in a single unit block 200 are collectively referred to as signal processing circuit group 220. Alternatively, it can be said that signal processing circuit group 220 constitutes... Figure 2 At least a portion of the signal processing unit 64 and the horizontal drive unit 68.

[0069] Unit block 200 further includes a horizontal selection unit 230, a vertical selection unit 240, and an ROI signal processing unit 400 regarding the decision pixel 204. The horizontal selection unit 230 selects and drives the column of the decision pixel 204. The vertical selection unit 240 selects the row of the decision pixel 204 and reads the pixel signal. The output of the vertical selection unit 240 is connected to switches 242, 244, and 246, which are connected to the output lines of the decision pixel 204.

[0070] The horizontal selection unit 230 and the vertical selection unit 240, based on control signals from the peripheral circuit unit 500, can select any position and number of determination pixels 204 contained in the pixel block 142 for exposure and / or readout control. The pixel signal of the selected determination pixel 204 is output to the ROI signal processing unit 400. Details about the ROI signal processing unit 400 and the horizontal line 504 from which the signal is output will be described later. It should be noted that the horizontal selection unit 230 and the vertical selection unit 240 are provided corresponding to each of the plurality of pixel blocks 142, and can independently control the determination pixel 204 of the corresponding pixel block 142, separate from the image pixel 202.

[0071] Figure 5 An example of the circuit configuration for an image pixel 202 is shown. The image pixel 202 includes a photoelectric conversion unit 120, a transmission unit 123, an output unit 124, a reset unit 126, and a pixel output unit 127. The pixel output unit 127 includes an amplification unit 128 and a selection unit 129. In this example, the transmission unit 123, the output unit 124, the reset unit 126, the amplification unit 128, and the selection unit 129 are described as N-channel FETs, but the type of transistor is not limited to this.

[0072] The photoelectric conversion unit 120 has a photoelectric conversion function that converts light into electrical charge. The photoelectric conversion unit 120 stores the charge obtained from the photoelectric conversion. The photoelectric conversion unit 120 is, for example, a photodiode.

[0073] The transfer section 123 transfers the charge stored in the photoelectric conversion section 120 to the storage section 125. The transfer section 123 is an example of a transfer gate for transferring the charge from the photoelectric conversion section 120. In other words, by using the transfer section 123 as the gate, the photoelectric conversion section 120 as the source, and the storage section 125 as the drain, a so-called transfer transistor is constructed. A control signal is input from the block vertical drive section 210 to the gate terminal of the transfer section 123. TX1.

[0074] The discharge section 124 discharges the charge accumulated in the photoelectric conversion section 104 to the power supply wiring supplied with power supply voltage VDD. A discharge control signal is input from the block vertical drive section 210 to the gate terminal of the discharge section 124. TX2. It should be noted that, in this example, regarding the discharge section 124, it is described that the charge of the photoelectric conversion section 120 is discharged to the power supply wiring supplied with power supply voltage VDD, but it can also be discharged to the power supply wiring supplied with a power supply voltage different from the power supply voltage VDD.

[0075] The accumulation section 125 transmits the charge from the photoelectric conversion section 120 through the transmission section 123. The accumulation section 125 is an example of floating diffusion (FD).

[0076] The reset unit 126 discharges the charge from the storage unit 125 to the power supply wiring supplied with a predetermined power supply voltage VDD. A reset control signal shared by multiple pixel blocks 142 is input from the vertical drive unit 62 to the gate terminal of the reset unit 126. RST.

[0077] The pixel output unit 127 outputs a signal based on the potential of the storage unit 125 to the signal line 122. The pixel output unit 127 has an amplification unit 128 and a selection unit 129. The gate terminal of the amplification unit 128 is connected to the storage unit 125, the drain terminal is connected to the power supply wiring supplied with the power supply voltage VDD, and the source terminal is connected to the drain terminal of the selection unit 129.

[0078] The selection unit 129 controls the electrical connection between the image pixel 202 and the signal line 122. When the image pixel 202 is electrically connected to the signal line 122 via the selection unit 129, a pixel signal is output from the image pixel 202 to the signal line 122. A selection control signal shared by multiple pixel blocks 142 is input from the vertical drive unit 62 to the gate terminal of the selection unit 129. SEL. The source terminal of the selector 129 is connected to the load current source 121.

[0079] The load current source 121 supplies current to the signal line 122. Based on this configuration, it can also be said that the main control signal... TX1 and control signals TX2 controls the exposure time primarily by selecting the control signal. SEL controls the timing of reading.

[0080] Additionally, the image pixel 202 includes at least one photoelectric conversion unit 120 and a pixel output unit 127, which reads the image signal from the at least one photoelectric conversion unit 120 onto the signal line 122. In other words, the image pixel 202 is the smallest unit of the circuit that outputs the pixel signals constituting the image to the signal line 122.

[0081] Furthermore, the image pixel 202 is sometimes referred to as a 5-transistor type. Alternatively, a 4-transistor type without an exhaust section 124 can be used instead of the image pixel 202.

[0082] The circuit configuration of the determination pixel 204 can be the same as that of the image pixel 202, but they differ in the following aspects: the control signal for the determination pixel 204. TX1, Discharge control signal TX2, Reset Control Signal RST and selection control signals SEL is input from the horizontal selection section 230.

[0083] Regarding the use of pixel 204 for judgment, it can also be said that the exposure time is mainly determined by the control signal. TX1 and control signals TX2 control. It can be said that the reading of the pixel signal of pixel 204 is mainly determined by the selection control signal. The switch 242, 244, 246 of the SEL and vertical selection unit 240 is used for control.

[0084] Figure 6 This is an explanatory diagram illustrating the pixel blocks of the Region of Interest (ROI) and non-ROI pixel blocks used by the imaging element 100. Figure 6 The image shows a subject 55 captured using a 4x4 pixel block. Furthermore, examples are used to illustrate the ROI, where the subject 55 moves quickly, and other subjects move slowly or not at all compared to the subject 55.

[0085] Subject 55 moves to the right from state 52 to state 54. Here, if we focus on... Figure 6The pixel block 144, surrounded by a medium-thick line, does not contain the subject 55 in state 52, but it does in state 54. Therefore, in state 52, pixel block 144 can be a region of interest (ROI) (sometimes called a non-ROI), allowing for a lower frame rate in video recording. On the other hand, in state 54, pixel block 144 is considered an ROI and therefore a higher frame rate is preferred.

[0086] Figure 7 The diagram schematically illustrates the relationship between a high frame rate for pixel blocks of regions of interest (ROIs) and a low frame rate for pixel blocks of non-ROIs in the imaging element 100. The imaging element 100 controls the image pixels 202 at, for example, a frame rate of up to 1000 fps. Therefore, a frame rate of 1000 fps is used for the pixel block 142 of the ROI.

[0087] exist Figure 7 In this process, the pixel block 142 of the ROI reads the pixel signal from the image using pixel 202 in all frames and performs AD conversion. In each frame, as according to... Figure 4 and Figure 5 As the composition clearly indicates, the image of pixel block 142 is read row by row using pixel 202 (also known as column-level scrolling read). Figure 7 In, corresponding to Figure 4 The pixel block 142 has 3 rows and 3 columns of image pixels 202. The timing of the AD conversion for rows 1 to 3 is shown in the first frame. The same timing is shown in frames 2 to 16. In addition, for the sake of simplicity, the timing of the AD conversion is illustrated to represent the exposure, pixel signal reading and subsequent processing in each pixel.

[0088] On the other hand, from the perspective of simplicity in motion timing, using every 16 frames of 1000fps, i.e., (1000 / 16) fps (approximately 60 fps), as the frame rate for non-ROI pixel blocks. In this case, it is arbitrary to read the low frame rate frame in which frame corresponds to the high frame rate of 16 frames, but... Figure 7 The image shows the situation in frame 16.

[0089] In the non-ROI pixel block 142, no reading from the image pixel 202 and AD conversion (denoted as NA in the figure) are performed from frame 1 to frame 15. On the other hand, in frame 16, similar to the ROI pixel block, the image pixel 202 within the pixel block 142 is read row by row in a column-to-row manner.

[0090] In the above configuration, assuming as follows Figure 6As shown, the frames in which the subject 55 begins to appear from pixel block 144, which was originally determined to be a non-ROI, are the 2nd and 3rd frames. In this case, the frame after the dividing point of 16 frames, where pixel block 144 is determined to be an ROI and switched to a high frame rate, may cause a substantial delay in the switching response. Therefore, in the imaging element 100, a determination pixel 204 is used as described below to reduce the delay between low and high frame rates.

[0091] Figure 8 This is a block diagram of unit block 200 and peripheral circuit section 500 focusing on modules related to ROI determination and control based on that determination. Figure 8 In China, for the sake of Figures 1 to 7 The same components are labeled with the same reference numerals, and the descriptions are omitted.

[0092] Figure 8 In this circuit, the ROI signal processing unit 400 includes a determination pixel processing unit 410 and an ROI control unit 420. The peripheral circuit unit 500 includes a determination pixel control unit 510, an image pixel control unit 520, an ROI determination unit 600, and an output unit 70. The output unit 70 includes a data deletion unit 710, an image memory 720, and an output IF unit 730. The determination pixel control unit 510 and the image pixel control unit 520 perform... Figure 2 The vertical drive unit 62 performs at least a portion of its functions. The pixel processing unit 410, ROI determination unit 600, ROI control unit 420, and output unit 70 are described sequentially based on the processing flow.

[0093] The decision pixel processing unit 410 processes the pixel signals of the decision pixels 204 selected by the horizontal selection unit 230 and the vertical selection unit 240, and outputs them to the horizontal line 502. The processing includes converting analog pixel signals into digital pixel signals. This conversion can be, for example, a single-slope method, but other methods are also possible. The decision pixel processing unit 410 may also include other processing methods such as CDS. Furthermore, when multiple decision pixels 204 within the pixel block 142 are selected by the horizontal selection unit 230 and the vertical selection unit 240, the pixel signals of the multiple decision pixels 204 are added analogally. Specifically, the sum of the pixel signals of the selected multiple decision pixels 204 is converted into a digital value.

[0094] The pixel signal of the determination pixel 204, which is output from the determination pixel processing unit 410 to the horizontal line 502, is input to the ROI determination unit 600 via the output unit 70. Based on the pixel signal of the determination pixel 204, the ROI determination unit 600 determines whether the pixel block 142 is an ROI or a non-ROI. The ROI determination unit 600 outputs a signal indicating the determination result to the ROI control unit 420.

[0095] Figure 9 This is a block diagram representing an example of the ROI determination unit 600. Figure 9 The ROI determination unit 600 determines each of the multiple pixel blocks 142 as either an ROI or a non-ROI based on the pixel signals of the determination pixels 204 of the multiple pixel blocks 142 through object recognition.

[0096] The ROI determination unit 600 includes an object recognition unit 610, an ROI determination unit 620, and an ROI address memory 630. The object recognition unit 610 acquires pixel signals of determination pixels 204 of a plurality of pixel blocks 142 from the image memory 720. The object recognition unit 610 identifies objects based on the pixel signals of these determination pixels 204 and determines the position information of the objects.

[0097] The object recognition unit 610 generates a determination image, for example, based on the pixel signals of the determination pixels 204 of the plurality of pixel blocks 142. When a determination image is generated by acquiring the pixel signal of a determination pixel 204 for each pixel block 142, it becomes an image with a number of pixels corresponding to the number of rows and columns of the pixel blocks 142 included in the pixel unit 50.

[0098] The object recognition unit 610 uses known image recognition methods to identify objects within an image for determination. For example, whether the object is dynamic or static, it first identifies the entire object and then extracts dynamic objects from it using pattern matching or similar methods. Alternatively, it can pre-store the previous frame's image for determination in the image memory 720 and identify dynamic objects based on the difference between the current frame and the previous frame. The object recognition unit 610 outputs the position of the identified dynamic object on the image to the ROI determination unit 620.

[0099] The ROI determination unit 620 determines the ROI based on the position of the image that reflects the moving object. For example, the ROI determination unit 620 may pre-establish a relationship based on the pixel signals of the determination pixel 204 of which pixel block 142 each pixel corresponds to when the image for determination is generated, and determine which pixel block 142 the determination pixel 204 of which the pixel that reflects the moving object is based on the relationship, thereby determining the pixel block 142 as the ROI.

[0100] The ROI determination unit 620 stores the information of the pixel block 142 determined as an ROI, i.e., the ROI address, in the ROI address memory 630. In this case, the ROI determination unit 620 may also assign an ROI ON signal to the address of the pixel block 142 determined to be an ROI for each of the plurality of pixel blocks 142, and assign an ROI OFF signal to the address of the pixel block 142 not determined to be an ROI, i.e., not an ROI. There is a case where the ROI ON signal and the ROI OFF signal are collectively referred to as ROI ON / OFF signals.

[0101] The ROI control unit 420 of each unit block 200 reads the ROI address from the ROI address memory 630 and controls the pixel blocks 142, etc. contained in that unit block 200.

[0102] Figure 10 This is a block diagram focusing on the unit block 200 and peripheral circuit section 500 of the ROI control unit 420. Figure 10 In China, for the sake of Figures 1 to 9 The same components are labeled with the same reference numerals, and explanations are omitted. It should be noted that... Figure 10 The text combines multiple ADC224 and multiple SRAM226 and shows one of each.

[0103] The ROI control unit 420 includes a memory 422, a counter 424, and an ROI control signal generation unit 426. The ROI control signal generation unit 426 is connected to AND gates 430, 432, and 434 within the unit block 200. The ROI control signal generation unit 426 is also connected to the load current source 121 of the pixel block 142 and the power supply 235 of the ADC 224 within the unit block 200.

[0104] Figure 11 This is a timeline of the actions of the ROI control unit 420. Figure 11 In China, the use of, such as Figure 6 As shown, pixel block 142 (in) Figure 6 The value is 144, but due to the use of Figure 10 To illustrate this, we will use case 142) to illustrate the switch from a non-ROI to an ROI.

[0105] <Image using pixels>

[0106] exist Figure 11 First, assume that pixel block 142 is determined to be a non-ROI. During the period when pixel block 142 is determined to be a non-ROI, an ROI OFF signal indicating non-ROI is output from ROI address memory 630 to memory 422. ROI ON / OFF signals are read from ROI address memory 630 to memory 422 synchronously with frame switching.

[0107] When the ROI OFF signal is stored in the memory 422 and the counter 424 counts frames 1 to 15, the ROI control signal generation unit 426 generates a gate signal "L". Additionally, in this case, the ROI control signal generation unit 426 generates a power-down signal "H". Figure 11 The diagram shows the "L" and "H" of the gating signals. Unless otherwise stated, the "L" and "H" of the gating signals and the power-down signals are in a completely opposite relationship.

[0108] When a gating signal "L" is input from the ROI control signal generation unit 426 to the AND gate 430, even if a control signal "H" is input from the image pixel control unit 520 to the AND gate 430, the AND gate 430 will still output "L" as a control signal. Therefore, pixels in the non-ROI pixel block 142 are not driven in frames 1 to 15.

[0109] Similarly, when a gating signal "L" is input from the ROI control signal generation unit 426 to AND gates 432 and 434, even if a control signal "H" is input from the image pixel control unit 520 to AND gates 432 and 434, AND gates 432 and 434 will still output "L" as a control signal. Therefore, in frames 1 to 15, the ADC224 and SRAM corresponding to the non-ROI pixel blocks 142 are not driven.

[0110] Furthermore, when the power-down signal "H" is output from the ROI control signal generation unit 426, the power supply to the load current source 121 and the power supply 235 is cut off.

[0111] Therefore, in frames 1 to 15 of the non-ROI pixel block 142, the image pixel 202 is not driven, that is, the pixel signal is not read for AD conversion and other processing. Figure 11 The term "NA" is used to represent this situation (see reference). Figure 11 Frame 2).

[0112] Even if the ROI OFF signal is stored in memory 422, the ROI control signal generation unit 426 will still generate a gate signal "H" and a power-down signal "L" when the counter 424 counts to frame 16. Therefore, when the control signal "H" is input from the image pixel control unit 520 to AND gates 430, 432, and 434, AND gates 430, 432, and 434 output "H" as a control signal. Additionally, power is supplied to the load current source 121 and the power supply 235.

[0113] Therefore, in frame 16 of the non-ROI pixel block 142, image pixel 202 is driven, i.e., pixel signals are read for processing such as AD conversion. Regarding the processing of the pixel signals of image pixel 202 in this case, such as using... Figure 7 As explained.

[0114] Therefore, non-ROI pixel blocks 142 are controlled at a low frame rate. Here, the ROI control signal generation unit 426 functions as a power cut-off unit, which cuts off the power supply to the image pixels in frames where reading from the image pixels 202 is not required in the pixel blocks 142 controlled at a low frame rate. Furthermore, the ROI control signal generation unit 426 and AND gates 432 and 434 function as read signal output units, which enable the read signal for the image pixels 202 in frames where reading of the image pixels 202 is required in the pixel blocks 142 controlled at a low frame rate, and disable it in frames where reading is not required. As a result, power consumption and processing load for non-ROI pixel blocks 142 can be reduced.

[0115] <Pixels used for judgment>

[0116] On the other hand, regardless of whether it is a non-ROI or an ROI, the determination is controlled by pixel 204 at a high frame rate. Figure 11 Showing all (if) the multiple determination pixels 204 of pixel block 142. Figure 4 The example is the sum of 9 pixel signals.

[0117] At a timing corresponding to the control signal of the image pixel control unit 520, the determination pixel control unit 510 inputs a control signal to the horizontal selection unit 230 and the vertical selection unit 240. Based on the input control signal, the horizontal selection unit 230 and the vertical selection unit 240 select the determination pixel 204 of the pixel block 142 according to the position, number and timing preset for each pixel block 142, and process the pixel signal.

[0118] exist Figure 11 In the example, throughout all 16 frames, the pixel signals of all decision pixels 204 are read synchronously with the timing of the reading of the first row of pixels 202 in the image of pixel block 142. For example, as in Figure 4 As explained, all the pixel signals of pixel 204 are added together in an analog manner, converted into digital signals, and output to the horizontal line 502 (see reference). Figure 11 (Frames 2 and 3).

[0119] Mainly as in Figure 8 and Figure 9 As explained, the pixel signals of the decision pixels 204, which are output from multiple pixel blocks 142 to the horizontal line 502, are used to determine the ROI for each of the multiple pixel blocks 142. Here, since the reading processing of the decision pixels 204 is not performed during the period when the rows of image pixels 202 from the second row onwards are read, a time margin for processing the pixel signals of the decision pixels 204 can be obtained, especially a time margin for obtaining the decision ROI.

[0120] <Frame rate switching>

[0121] As described above, since pixel 204 is controlled at a high frame rate regardless of whether pixel block 142 is at a high or low frame rate, the ROI determination is also performed at a high frequency. Therefore, the frame rate of pixel block 142 can be switched even in the middle of 16 frames.

[0122] exist Figure 11 The example shown is that, based on the pixel signal of the determination pixel 204 read in frame 2, pixel block 142 is switched from a non-ROI to an ROI in frame 3. Specifically, before the start of frame 3, when the ROI determination unit 600 determines pixel block 142 as an ROI based on the pixel signal of the determination pixel 204 read in frame 2, it reads an ROI signal from the ROI determination unit 600 into the memory 422. Therefore, at the start of frame 3, the ROI control signal generation unit 426 switches the gate signal from "L" to "H" based on the ROI ON signal read into the memory 422, and also switches the power-down signal from "L" to "H". Thus, pixel block 142 is controlled at a high frame rate after frame 3.

[0123] exist Figure 11 The example shown is a switch from non-ROI to ROI, but the operation is the same when switching from ROI to non-ROI, that is, from high frame rate to low frame rate. It should be noted that the ROI determination unit 600 determines whether to set pixel block 142 as ROI or non-ROI. Regarding the result of this determination, if it is determined to be ROI, then pixel block 142 is controlled at a high frame rate; if it is determined to be non-ROI, then it is controlled at a low frame rate. Therefore, it can also be said that the ROI determination unit 600 essentially determines whether to control pixel block 142 at a high frame rate or a low frame rate.

[0124] Figure 12 The diagram schematically illustrates the pixel signals output to horizontal line 502 and the data enable signals output to horizontal line 504. Horizontal lines 502 and 504 are shared by multiple pixel blocks 142. Based on the control signals of the horizontal drive unit 68, the image from the multiple pixel blocks 142 is output pixel by pixel signal of pixel 202 to horizontal line 502 and input to output unit 70 in a preset order.

[0125] Therefore, regardless of whether it is a ROI or a non-ROI, the horizontal drive unit 68 inputs the output of the horizontal line 502 as a pixel signal to the output unit 70 at a preset timing in each frame. Here, as Figure 12Therefore, consider the case where pixel blocks 142 of the ROI (referred to as block 1 for explanation) and pixel blocks 142 of non-ROI (referred to as block 2 for explanation) coexist. In this case, although the image pixels 202 are not controlled for frames 1 to 15 in block 2, it is still possible to obtain signals that are originally invalid, such as noise that is output to the output unit 70 in accordance with the control signal of the horizontal drive unit 68.

[0126] Therefore, with the ROI OFF signal stored in memory 422, Figure 8 The ROI control unit 420 outputs a data enable signal that is "L" during frames 1 to 15 and "H" in frame 16 to the horizontal line 504 in sync with the control signal of the horizontal drive unit 68. It should be noted that when the ROI ON signal is stored in the memory 422, the ROI control unit 420 outputs the data enable signal "H" for frames 1 to 16.

[0127] Output unit 70 has Figure 8 The data deletion unit 710 shown saves the signal output to the horizontal line 502 to the horizontal line 504 based on the inputs from the horizontal line 502 and the horizontal line 504, if the data enable signal is "H". Figure 8 The image memory 720 shown. On the other hand, if the data enable signal is "L", the data deletion unit 710 does not save the signal output to the horizontal line 502 into the image memory 720, but deletes it (or simply ignores it).

[0128] As needed, the image data stored in the image memory 720 is output to the outside of the camera element 100 via the output IF section.

[0129] Additionally, the ROI control unit 420 functions as an invalid signal output unit. In the pixel block 142 controlled at a low frame rate, for frames that do not require reading from the image pixel 202, the invalid signal output unit outputs the aforementioned data enable signal, which indicates that the image pixel is invalid, to the outside of the pixel block 142 as an invalid signal. This prevents the formation of the image from using unexpected signals.

[0130] exist Figure 12 The following situation is shown, that is, corresponding to the direction Figure 8 The horizontal line 502 outputs not only image pixel 202 but also decision pixel 204; the output of image pixel 202 and decision pixel 204 are sequential. Figure 12In the example, after the data of pixel 202 for the first row of the image of a specific block is output to the horizontal line 502, the data of pixel 204 for determining the block is output to the same horizontal line 502. This timing is the same regardless of whether the pixel block is a ROI or not. In addition, since the data of pixel 204 is used for ROI determination, the data enable signal is set to "H" and saved to the image memory 720 regardless of whether the pixel block is an ROI or not.

[0131] Figure 13 Other examples of control using pixel 204 are shown. Figure 13 In, replacing Figure 11 The pixel signals are read from the decision pixels 204 of each row in a time-division manner. For example, synchronously with the reading of the pixel signals of the image pixels 202 of the first row of pixel block 142, the decision pixels 204 of all columns of pixel block 142 are selected by the horizontal driving unit, and the pixel signals of the decision pixels 204 of the first row are selected by the vertical driving unit. Thus, the decision pixels 204 of the first row ( Figure 4 The pixel signals (3 in total) are added together in an analog manner and then converted into digital.

[0132] Next, in sync with the reading of the pixel signal of the second row of image pixels 202 of pixel block 142, the pixel signal of the second row of determination pixels 204 of pixel block 142 is added analogously and converted into a digital value. Furthermore, in sync with the reading of the pixel signal of the third row of image pixels 202 of pixel block 142, the pixel signal of the third row of determination pixels 204 of pixel block 142 is added analogically and converted into a digital value.

[0133] according to Figure 13 For example, pixel signals of multiple determination pixels 204 are output from a pixel block 142 to the output unit 70. Therefore, high-resolution images can be used in the determination of the ROI determination unit 600, and ROI determination can be performed more appropriately.

[0134] The control of pixel 204 is not limited to the determination. Figure 11 and Figure 13 For example, pixels 204 can be selected to determine any position and number of pixels contained in pixel block 142, and control functions such as exposure, readout, and summation can be performed. For example, in Figure 11 In the example, instead of synchronizing with the first row (row 1) of image pixels 202, the timing of reading the decision pixel 204 can be synchronized with any row after the second row. It should be noted that, for each pixel block 142, the number of times the decision pixel 204 is read can be less than or the same as the number of times the image pixels 202 are read (e.g., the number of columns if it is a column read).

[0135] exist Figure 8 The ROI determination unit 600 is located in the peripheral circuit unit 500, but alternatively, the ROI address can be specified from outside the imaging element 100. In this case, the output IF unit 730 can output the pixel signal of the determination pixel 204 to the outside of the imaging element 100, and determine the ROI from outside the imaging element 100.

[0136] Figure 14 This is a block diagram representing other unit blocks 206 and peripheral circuit sections 506. Unless otherwise specified, for... Figures 1 to 13 The same components are labeled with the same reference numerals, and the descriptions are omitted.

[0137] The main difference between unit block 206 and peripheral circuit section 506 and unit block 200 and peripheral circuit section 500 is that ROI determination is performed in each unit block 206. Specifically, in addition to the pixel processing section 410 and ROI control section 420, the ROI signal processing section 400 of unit block 206 also has an ROI determination section 440.

[0138] Figure 15 This is a block diagram of the ROI determination unit 440. The ROI determination unit 440 includes an average value memory 442, a differential processing unit 444, and a brightness threshold determination unit 446.

[0139] The average value memory 442 stores the pixel signal of the determination pixel 204 after processing by the determination pixel processing unit 410. In particular, it stores the brightness value of the frame preceding the frame to be determined for ROI. In this case, the brightness value of the previous frame can be stored, or the average value of a predetermined number of frames can be stored.

[0140] The differential processing unit 444 acquires the brightness value of the determination pixel 204 of the frame to be determined for ROI determination. Further, the differential processing unit 444 calculates the difference between this brightness value and the brightness value stored in the average value memory 442.

[0141] The brightness threshold determination unit 446 determines whether the absolute value of the difference calculated by the difference processing unit 444 exceeds a predetermined threshold. If the absolute value of the difference exceeds the predetermined threshold, the brightness threshold determination unit 446 determines the pixel block 142 as a Region of Interest (ROI). On the other hand, if the absolute value of the difference does not exceed the predetermined threshold, the brightness threshold determination unit 446 determines the pixel block 142 as a non-ROI. The brightness threshold determination unit 446 outputs the ROI / OF signal obtained based on the determination result to the ROI control unit 420.

[0142] Figure 15The ROI determination unit 440 is configured correspondingly to each of the plurality of pixel blocks 142, and performs ROI determination for the corresponding pixel block 142 based on the determination of the pixel block 142 using the pixel 204. Since the ROI determination is performed inside the unit block 206, the signal lines do not become complicated, and the delay caused by transmission is minimal.

[0143] exist Figure 15 In the example, with Figure 11 Similarly, the pixel processing unit 410 adds the pixel signals for all the decision pixels 204 of the pixel block 142. Alternatively, it can also utilize... Figure 13 And other methods described in association with the accompanying drawings for using the pixel signal of the determination pixel 204. Additionally, in cases such as Figure 13 In the case of using pixel signals from multiple decision pixels 204, representative values ​​such as the maximum, minimum, and average values ​​of these pixel signals can be used to compare brightness values. Furthermore, in Figure 8 The ROI determination unit 600 shown can also be determined using the same method as the ROI determination unit 440.

[0144] Figure 16 Other pixel blocks 148 are schematically shown. Pixel blocks 148 include image pixels 314, 316, 322, 324 and decision pixels 312, 318, 320, 326, which are arranged in a two-dimensional repeating configuration as repeating units 328.

[0145] Pixel block 148 differs from pixel block 142 in three aspects. The first aspect is that the sizes of image pixels 314, 316, 322, and 324 are different from those of determination pixels 312, 318, 320, and 326. Specifically, the area of ​​the photoelectric conversion unit of image pixels 314, 316, 322, and 324 is larger than the area of ​​the photoelectric conversion unit of determination pixels 312, 318, 320, and 326. Conversely, the area of ​​the photoelectric conversion unit of image pixels 314, 316, 322, and 324 may be smaller than the area of ​​the photoelectric conversion unit of determination pixels 312, 318, 320, and 326.

[0146] Secondly, the positional relationship between image pixels 314, 316, 322, 324 and decision pixels 312, 318, 320, 326 varies depending on the location. In particular, decision pixels 312, 320 are positioned to the left of image pixels 314, 322, while decision pixels 318, 326 are positioned to the right of image pixels 316, 324.

[0147] Thirdly, image pixels 314, 316, 322, and 324, and decision pixels 312, 318, 320, and 326 are color pixels. Specifically, image pixels 314 and decision pixels 312 are R pixels, image pixels 316 and decision pixels 318 are Gr pixels, image pixels 322 and decision pixels 320 are Gb pixels, and image pixels 324 and decision pixels 326 are B pixels. These pixels can be implemented by placing a color filter upstream of the photoelectric conversion unit. Furthermore, decision pixels 312, 318, 320, and 324 can be selected and read for each color. This allows for detailed ROI determination.

[0148] The above three points can be used without combination, or any one or any two points can be used. As another example, the number of image pixels 202 and decision pixels 204 in pixel block 142 can also be different. For example, by placing a decision pixel 204 in the center of four image pixels 202, the number of decision pixels 204 can be less than the number of image pixels 202. Conversely, by placing an image pixel 202 in the center of four decision pixels 204, the number of decision pixels 204 can be more than the number of image pixels 202.

[0149] Figure 17 This is another example of a block diagram focusing on the unit block 200 and peripheral circuit section 500 related to the determination of ROI and control based on that determination. In addition to the configuration described below, Figure 17 With Figure 8 Since they have the same structure, the explanation of the same structure is omitted.

[0150] Figure 17 It has a horizontal line 503 that outputs the pixel signal of the determination pixel 204 after processing by the determination pixel processing unit 410. The horizontal line 503 is shared by multiple pixel blocks. The horizontal line 503 can be connected to the image memory 720. Figure 8 The difference does not occur through the data deletion section 710.

[0151] Figure 18 schematically showing the direction Figure 17 The pixel signals output from horizontal lines 502 and 503 and the data enable signal output to horizontal line 504. Except for the timing described below, Figure 18 Showing with Figure 12 Since the timing is the same, the description of the same timing is omitted.

[0152] exist Figure 18 In this process, the output of image pixel 202 to horizontal line 502 and the output of decision pixel 204 to horizontal line 503 can be parallel. Figure 18In the example, the pixel signal of the determination pixel 204 is output in a timing that matches the timing of outputting the pixel signal of the initial image of the corresponding pixel block using pixel 202. Alternatively, it can be output in a timing other than the timing of outputting the pixel signal of the initial image using pixel 202. This timing is the same regardless of whether the pixel block is a ROI or not. Corresponding to the horizontal line 503, without passing through the data deletion unit 710, the pixel signal of the determination pixel 204 is saved to the image memory 720 regardless of the data enable signal output to the horizontal line 504.

[0153] Alternatively, microlenses can be provided on the upstream side of the photoelectric conversion unit, above the light source, for both the image pixel 202 and the determination pixel 204. In this case, one microlens can be provided for adjacent image pixels 202 and determination pixels 204, or a separate microlens can be provided for each of the image pixels 202 and determination pixels 204.

[0154] In addition, regarding the spatial layout of each block, part or all of the horizontal selection unit 230, vertical selection unit 240, and ROI signal processing unit 400 can be set in Figure 3 The signal processing block 162 shown.

[0155] The above example illustrates a low frame rate that is one-sixteenth of a high frame rate, but it is not limited to this. For example, it could also be one-eighth, one-quarter, or one-half of a high frame rate. Furthermore, multiple low frame rates can be used, such as one-sixteenth and one-quarter of a frame rate. In this case, the ROI determination can also be differentiated from ROI to non-ROI, instead of being classified as high ROI, medium ROI, and low ROI, and then assigned to high, one-quarter, and one-sixteenth frame rates respectively. When there are three or more types of ROI, the determination can be made for each object identified by the object recognition unit 610. For example, if it is a car, it can be classified as a high ROI; if it is a person, it can be classified as a medium ROI; and others can be classified as low ROI. Alternatively, two thresholds can be set in the brightness threshold determination unit 446. If the difference exceeds the larger threshold, it is classified as a high ROI; if it is between the larger and smaller thresholds, it is classified as a medium ROI; and others are classified as low ROI.

[0156] The example given is that the ROI determination units 440 and 600 use the determination pixel 204 in ROI determination, but the determination pixel 204 can also be used in other determinations. For example, a determination unit could be included that determines the focus level of the subject captured by pixel block 142 based on the determination pixel 204. The focus level determination could be a contrast method that compares the brightness values ​​of the surrounding area, or it could be based on... Figure 16The phase difference method with the configuration shown.

[0157] As another example, a determination unit may be included to determine the exposure of pixel block 142 based on determination pixel 204. In this case, the block vertical drive unit 210 and the like may be operated to make the exposure time of image pixel 202 for pixel block 142 determined to have low exposure a long time, and / or the exposure time of image pixel 202 for pixel block 142 determined to have high exposure a short time.

[0158] The above describes an example of using the pixel signal of the determination pixel 204 for determination related to pixel block 142. However, it can also be used to form an image from the pixel signal of the determination pixel 204. In this case, for example, by selecting the determination pixels 204 in pixel block 142 one by one, processing them, and outputting them to the horizontal line 502, an image with a resolution having the number of pixels that combine the number of image pixels 202 and the number of determination pixels 204 for that pixel block 142 can be formed. Furthermore, the use of the determination pixel 204 in image formation or its exclusion from image formation can be switched. In this case, when the determination pixel 204 is not used in image formation, a low-resolution image can be formed by reading the image pixels 202 at a high frame rate. On the other hand, when the determination pixel 204 is used in image formation, a high-resolution image can be formed by reading at a low frame rate corresponding to the number of determination pixels 204 used in image formation.

[0159] The present invention has been described above using embodiments, but the technical scope of the present invention is not limited to the scope described in the above embodiments. Various modifications or improvements can be made to the above embodiments, which will be apparent to those skilled in the art. It is clear from the claims that such modifications or improvements are also included within the technical scope of the present invention.

[0160] It should be noted that the execution order of actions, processes, steps, and stages in the apparatus, systems, programs, and methods shown in the claims, description, and drawings may be implemented in any order unless specifically stated as "before," "before," etc., and the output of the preprocessing is used in the postprocessing. Even if the flow of actions in the claims, description, and drawings is described using terms such as "firstly" or "next" for convenience, this does not mean that they must be implemented in this order.

[0161] Explanation of reference numerals in the attached figures

[0162] 10 Imaging device, 20 Optical system, 22 Optical axis, 24 Image processing unit, 26 Control unit, 28 Recording unit, 30 Operation unit, 32 Display unit, 34 Power supply unit, 36 Bus, 50 Pixel unit, 52, 54 Status, 55 Subject, 58, 70 Output unit, 60 Drive control unit, 62 Vertical drive unit, 64 Signal processing unit, 68 Horizontal drive unit, 100 Imaging element, 120 Photoelectric conversion unit, 121 Load current source, 122 Signal line, 123 Transmission unit, 124 Output unit, 126 Reset unit, 127 Pixel output unit, 128 Magnifying unit, 129 Selection unit, 140 First semiconductor substrate, 142, 144, 148 Pixel block, 150 Second semiconductor substrate, 152 Global vertical drive unit, 154 Global horizontal drive unit, 162 Signal processing block, 200, 206 unit blocks, 202, 314, 316, 322, 324 image pixels, 204, 312, 318, 320, 326 decision pixels, 210 vertical drive unit, 220 signal processing circuit group, 222 signal processing circuit, 230 horizontal selection unit, 224 ADC, 235 power supply, 226 SRAM, 240 vertical selection unit, 242, 244, 246 switch, 400 ROI signal processing unit, 410 decision pixel processing unit, 420 ROI control unit, 422 memory, 424 counter, 426 ROI control signal generation unit, 430, 432, 434 gates, 440 ROI decision unit, 442 average value memory, 444 differential processing unit, 446 brightness threshold decision unit, 500, 506 Peripheral circuit section, 502 horizontal line, 503 horizontal line, 504 horizontal line, 510 pixel control section for determination, 520 pixel control section for image, 600 ROI determination section, 610 object recognition section, 620 ROI determination section, 630 ROI address memory, 70 output section, 710 data deletion section, 720 image memory, 730 output IF section.

Claims

1. A camera element, comprising: Multiple pixel blocks having multiple pixels; and A pixel control unit that controls the plurality of pixels. Each of the plurality of pixel blocks has image pixels for forming an image and at least determination pixels for determining the pixel block. The pixel control unit can independently control the determination pixels of the pixel block and the image pixels.

2. The imaging element according to claim 1, wherein, Each of the plurality of pixel blocks has a plurality of the determination pixels. The pixel control unit has a pixel selection unit, which selects and controls any one of the multiple determination pixels for each of the multiple pixel blocks.

3. The imaging element according to claim 2, wherein, The pixel selection unit is configured corresponding to each of the plurality of pixel blocks.

4. The imaging element according to claim 3, wherein, The plurality of pixel blocks are disposed on the first semiconductor substrate. The pixel selection unit is disposed in the region of the second semiconductor substrate that overlaps with the corresponding pixel block, and the second semiconductor substrate is stacked with the first semiconductor substrate.

5. The imaging element according to claim 2, wherein, The pixel selection unit adds two or more of the decision pixels from a plurality of decision pixels together and then reads the result.

6. The imaging element according to claim 1, wherein, It also includes a determination unit that, based on the determination pixel, determines whether the image pixel of the pixel block is controlled by a high frame rate or a low frame rate.

7. The imaging element according to claim 6, wherein, The pixel control unit controls the determination pixel at the high frame rate within the pixel block controlled at the low frame rate.

8. The imaging element according to claim 6, wherein, It also includes a power cut-off unit that, in the pixel block controlled by the low frame rate, cuts off the power supply to the image pixels at frames where it is not necessary to read from the image pixels.

9. The imaging element according to claim 6, wherein, It also includes a read signal output unit, which, in the pixel block controlled by the low frame rate, makes the read signal to the image pixel valid in a frame in which the image pixel is read, and invalidates the read signal in a frame in which the image pixel is not read.

10. The imaging element according to claim 6, wherein, It also includes an invalid signal output unit, which outputs an invalid signal indicating that the image pixel is invalid to the outside of the pixel block in the low frame rate controlled pixel block during frames in which it is not necessary to read from the image pixel.

11. The imaging element according to claim 1, wherein, It also includes a determination unit that performs determinations related to the plurality of pixel blocks based on the determination pixels of the plurality of pixel blocks.

12. The imaging element according to claim 1, wherein, It also includes a determination unit provided corresponding to each of the plurality of pixel blocks, the determination unit performing a determination related to the corresponding pixel block based on the determination pixel of the corresponding pixel block.

13. The imaging element according to claim 1, wherein, It also includes a determination unit that determines the focus level of the subject captured by the pixel block based on the determination pixels.

14. The imaging element according to claim 1, wherein, It also includes a determination unit that determines the exposure of the pixel block based on the determination pixels.

15. An electronic device comprising a camera element according to any one of claims 1 to 14.

Images