Driver monitoring system and associated image capture method

Abstract

An image capture method includes capturing a first image of a scene by generating a first plurality of pixel signals using an image sensor having an array of pixels. The first plurality of pixel signals includes (i) a target pixel signal group T1, (ii) a remaining pixel signal group P11, and (iii) a remaining pixel signal group P21. The method further includes transmitting the first plurality of pixel signals to a storage area of the image sensor; reading a target pixel signal group T1 and a residual pixel signal group P11 from the storage area; capturing a second image of the scene by generating a second plurality of pixel signals, the second plurality of pixel signals comprising a set of target pixel signals T2 corresponding to the set of target pixels; transmitting a second plurality of pixel signals to the storage area; and reading the target pixel signal group T2 and the remaining pixel signal group P21 from the storage area.

Description

Technical Field

[0001] This disclosure relates to the field of image processing, and more particularly to driver detection systems and associated image capture methods. Background Technology

[0002] More and more vehicles are incorporating Driver Monitoring Systems (DMS), which capture and process video of the driver's face to detect when the driver is fatigued. DMS can monitor the driver's eye status (open or closed, blink rate, pupil dilation) and gaze (direction, head posture). Accurate driver monitoring requires high frame rates for video capture. However, this requirement places demands on the DMS's power consumption and the data rate between the DMS camera and the processor. Summary of the Invention

[0003] The embodiments disclosed herein enable DMS to capture high frame rate video with reduced power and data rate requirements.

[0004] In a first aspect, an image capture method is disclosed. The method includes: capturing a first image of a scene by generating a first plurality of pixel signals using an image sensor having a pixel array. The first plurality of pixel signals include (i) a target pixel signal group T1 generated from a target pixel group having pixel coordinates of a target region of the pixel array, and (ii) a pixel signal group P generated from a first remaining pixel group. 11 The first remaining pixel group has the pixel coordinates of the first remaining pixel array region of the pixel array, and (iii) the pixel signal group P generated by the second remaining pixel group. 21 The second remaining pixel group has the pixel coordinates of the second remaining pixel array region of the pixel array. The method further includes: transmitting a first plurality of pixel signals to a storage area of ​​the image sensor; and reading the target pixel signal group T1 and the pixel signal group P from the storage area. 11 The method further includes: capturing a second image of the scene by generating a second plurality of pixel signals, the second plurality of pixel signals including a target pixel signal group T2 corresponding to a target pixel group; transmitting the second plurality of pixel signals to a storage area; and reading the target pixel signal group T2 and the pixel signal group P from the storage area. 21 . Attached Figure Description

[0005] Figure 1 This is a schematic diagram of a vehicle equipped with a driver monitoring system (DMS) for monitoring the driver, as described in the embodiment.

[0006] Figure 2 In the embodiments Figure 1 Functional block diagram of the camera, image sensor, and pixel array of the DMS.

[0007] Figure 3 In the embodiments Figure 2 Functional block diagram of the image sensor.

[0008] Figure 4 It is indicated in the embodiment. Figure 2 A schematic diagram of multiple pixel groups and storage areas of a pixel array.

[0009] Figure 5 In the embodiments Figure 2 A schematic diagram of the pixel array of an image sensor generating groups of pixel signals during a time-series period.

[0010] Figure 6 Show Figure 4 The storage area in Figure 5 Stored during the time period Figure 5 Pixel signal group.

[0011] Figure 7 The illustrated embodiment can be made by Figure 3 The flowchart shows the image capture method implemented by the image sensor.

[0012] Figure 8 This is a circuit diagram of the pixels and the sampling and holding circuitry in the embodiment.

[0013] Figure 9 This is a circuit diagram of the pixels and the sampling and holding circuitry in the embodiment. Detailed Implementation

[0014] Figure 1 This is a schematic diagram of a vehicle 192 equipped with a driver monitoring system (DMS) 100 that monitors the driver 194. Figure 2 This is the function block diagram of the DMS100. It's best to review it along with the description below. Figure 1 and Figure 2 .

[0015] DMS100 includes a camera 200, which includes an image sensor 300. Camera 200 captures an image sequence including image 180. Image 180 includes a target region 185 and multiple remaining image regions 181, 182, 183, and 184. Although Figure 1 Five image regions are shown; however, without departing from the scope of this application, image 180 may have a different number of image regions. For example... Figure 1 As shown, target area 185 may include the eyes of driver 194.

[0016] DMS100 may also include circuitry 102 that implements the functions of DMS100. Circuitry 102 may include at least one of processor 286 and memory 104. In embodiments, circuitry 102 is an integrated circuit or includes (e.g., as processor 286) integrated circuits, such as application-specific integrated circuits and field-programmable gate arrays. Part or all of circuitry 102 may be part of image sensor 300.

[0017] Memory 104 may be transient and / or non-transient, and may include one or both of volatile memory (e.g., SRAM, DRAM, computational RAM, other volatile memory, or any combination thereof) and non-volatile memory (e.g., FLASH, ROM, magnetic media, optical media, other non-volatile memory, or any combination thereof). Part or all of memory 104 may be integrated into processor 286.

[0018] Figure 3 This is a functional block diagram of an image sensor 300. The image sensor 300 includes a pixel array 310, which includes pixels 320 in a two-dimensional array. In an embodiment, the image sensor 300 includes... Figure 2 The pixel die 302 includes a pixel array 310. The image sensor 300 may also include a sample and hold circuit array 367A, which includes a plurality of sample and hold circuits 367. Each circuit 367 may be electrically connected to a corresponding pixel 320 or an adjacent pixel 320 of a corresponding group.

[0019] Each pixel 320 includes, for example: Figure 2 The storage region 330 shown can be a complementary metal-oxide-semiconductor (CMOS) pixel, a charge-coupled device (CCD) pixel, or other pixel types. Storage region 330 can be an analog storage element of pixel 320, such as a floating diffusion node. Each pixel 320 may include an additional storage region 331. For example, pixel 320 may be a charge-domain global-shutter (CDGS) pixel or a voltage-domain global-shutter (VDGS) pixel. Storage region 330 may include a target portion 339.

[0020] The pixel array 310 has M pixel rows 307 (1-M) and N pixel columns 308 (1-N), where in Figure 3 The numbers R1, R2, ..., R3 represent pixel rows respectively. M And pixel columns C1, C2...C N Each pixel, 320, is represented as p. mn, where the indices m and n of the pixel coordinates (m,n) represent the row and column of the pixel in the pixel array 310, respectively.

[0021] After each pixel 320 has acquired its image charge, the image charge is read out by the readout circuit 340 and transferred to the functional logic 360. The image sensor 300 may further include control circuitry 350 coupled to the array 310 for generating various signals to control the operation of each pixel 320. The control circuitry may include a row driver 352.

[0022] Figure 4 This is a functional block diagram of a pixel array 310, which identifies multiple regions. These regions include a target region 315 and remaining pixel array regions 311-314, each defined by pixel coordinates (m, n) of a corresponding group. The target region 315 includes a target pixel group 425. The remaining pixel array regions 311-314 include corresponding remaining pixel groups 421-424. Pixel groups 421-425 generate pixel signals for image regions 181-185 of image 180, which are processed by the image sensor 300 to generate image 180 respectively. Although Figure 4 This refers to five pixel array regions 311-315 defined by corresponding pixel groups 421-425. However, without departing from the scope of this application, pixel array 310 can be divided into any number of pixel array regions, up to N. R For example, the total number of pixel array regions can be equal to 3 (N). R =3), where one of the regions is the target region 315.

[0023] The pixel array regions of pixel array 310 can be non-overlapping and continuous. Each pixel of pixel array 310 can be located in one of the pixel array regions, such that the pixel array region (e.g., region 311-315) includes all pixels in pixel array 310.

[0024] exist Figure 4 In the example, each pixel group 421-425 includes multiple adjacent pixel rows 307. Any two pixels in pixel groups 421-425 can be non-overlapping, so that no pixel 320 is part of multiple pixel groups. The target pixel group 425 can be between two remaining pixel groups 421-424, such as... Figure 4 As shown.

[0025] The pixel group of pixel array 310 may include a portion of pixel row 307 and / or a portion of pixel column 308. In embodiments, pixel array 310 may include at least one of the following: (i) a first pixel group surrounded by a second pixel group and (ii) a pixel group comprising two or more groups (“islands”) of consecutive pixels separated by one or more different pixel groups. The shape of the pixel array region may be polygonal, for example... Figure 4 Examples include rectangles, or polygons with different numbers of sides. The shape of the pixel array region can be a convex polygon or a concave polygon.

[0026] Pixel groups 421-425 have corresponding storage regions 431-435, which include storage region 330 of pixel 320 (which is a part of the pixel group). For example, storage region 431 includes storage region 330 of pixel 320 (which is a part of pixel group 421). When pixel 320 includes both storage region 330 and storage region 331, storage regions 431-435 may include only storage region 330 of pixel 320 (which is a part of the pixel group), or both storage regions 330 and 331.

[0027] Figure 5 This is a schematic diagram of a pixel signal array 580 (1-4) generated by pixel array 310, image sensor 300, and / or camera 200 to generate time-series images 180. Pixel signal array 580 includes multiple pixel signals, each generated by a corresponding pixel 320 of image sensor 300.

[0028] Figure 5 The time intervals 502(1)-502(4) represent the time series during which the pixel array 310 generates the corresponding pixel signal arrays 580(1-4). The generation of the pixel signal arrays 580(k) can occur during a time interval shorter than the time interval 502(k), where the index k is a positive integer less than or equal to 4.

[0029] Time period 502(1) begins before time period 502(2), time period 502(2) begins before time period 502(3), and time period 502(3) begins before time period 502(4). Consecutive time periods 502 can be non-overlapping in time. For example, time period 502(1) can end before time period 502(2).

[0030] Each pixel signal array 580 includes pixel signal groups 521-525 generated from corresponding pixel groups 421-425. Figure 5 The pixel signal groups 521-524 are referred to as "remaining" because the pixel signals in each pixel signal group 521-524 are generated by the pixels 320 of the corresponding remaining pixel groups 421-424.

[0031] Figure 6Pixel signal groups 521-525 are shown stored by storage regions 431-435 during at least a portion of time period 502. During at least a portion of each of time periods 502(1)-502(4), the remaining storage regions 431-434 store pixel signal groups 521(1)-524(1), respectively. That is, each of the remaining storage regions 431-434 stores the same pixel signal group during each time period 502(1)-502(4). In contrast, storage region 435 stores different pixel signal groups during each time period 502(1)-502(4). During at least a portion of time periods 502(1)-502(4), storage region 435 stores target pixel signal groups 525(1)-525(4), respectively.

[0032] When pixel 320 is a dual-storage pixel and therefore includes an additional storage area 331, storage area 435 can store multiple target pixel signal groups, such as Figure 6 As shown. For example, storage area 435 may store at least one of the following: (i) pixel signal group 525(1) during time periods 502(1) and 502(2), (ii) pixel signal group 525(2) during time periods 502(2) and 502(3), and (iii) pixel signal group 525(3) during time periods 502(3) and 502(4). Similarly, any one of storage areas 431-434 may store multiple target pixel signal groups, as shown in the diagram. Figure 6 As shown. For example, storage area 431 may: (i) store pixel signal group 521(2) or 521(3) during time period 502(3), and (ii) store pixel signal group 521(2), 521(3) or 521(4) during time period 502(4).

[0033] During time period 502(3), in addition to target pixel signal group 525(3), storage area 435 may also store one of target pixel signal groups 525(1) and 525(2). During time period 502(4), in addition to target pixel signal group 525(4), storage area 435 may also store one of target pixel signal groups 525(1), 525(2), and 525(3). In the embodiment, each pixel signal in target pixel signal group 525(1) is stored in the corresponding storage area 330, while each pixel signal in subsequent pixel signal groups 525 (2≤k≤R) is stored in the corresponding additional storage area 331. Simultaneously storing earlier target pixel signal groups (such as target pixel signal group 525(1)) and subsequently captured target pixel signal groups (e.g., 525(2), 525(3), or 525(4)) allows for comparison of two target pixel signal groups.

[0034] Figure 7 This is a flowchart illustrating an image capture method 700, which can be implemented by a DMS 100 or one or more of its components. For example, method 700 can be implemented by the image sensor 300 itself, or by a processor 286 and the image sensor 300. Method 700 includes at least one of steps 710, 720, 730, 740, 750, 760, 770, 780, and 790.

[0035] Step 710 includes capturing a first image of the scene by generating a first plurality of pixel signals using an image sensor having a pixel array. The first plurality of pixel signals include (i) a target pixel signal group T1 generated from a target pixel group having pixel coordinates of a target region of the pixel array, and (ii) a pixel signal group P generated from a first remaining pixel group. 11 The first remaining pixel group has the pixel coordinates of the first remaining pixel array region of the pixel array, and (iii) the pixel signal group P generated by the second remaining pixel group. 21 The second remaining pixel group has the pixel coordinates of the second remaining pixel array region of the pixel array.

[0036] In the example of step 710, the image sensor 300 captures an image of the driver 194 during time period 502 (1) by generating a pixel signal array 580 (1). The target pixel signal group 525 (1), the remaining pixel signal group 521 (1), and the remaining pixel signal combination 522 (1) are the target pixel signal group T1 and the pixel signal group P. 11 and pixel signal group P 21 The corresponding example.

[0037] Step 720 includes transmitting a first plurality of pixel signals to a storage area of ​​an image sensor. In an example of step 720, for each pixel 320, applying a pulse to the transmission gate of pixel 320 causes the generation of its pixel signal to be transmitted to the storage area 330 of pixel 320. The pixel signal is part of one of the pixel signal groups 521(1)-525(1).

[0038] In one embodiment, the image sensor has a plurality of pixels, and the storage region includes a plurality of in-pixel analog storage elements, each of the plurality of in-pixel analog storage elements belonging to a corresponding one of the plurality of pixels. Each storage region 330 is an example of an in-pixel analog storage element. In such an embodiment, step 720 may include transmitting each of the first plurality of pixel signals to a corresponding one of the plurality of in-pixel analog storage elements.

[0039] Step 730 includes reading the target pixel signal group T1 and the pixel signal group P from the storage area. 11In the example of step 730, the readout circuit 340: (i) reads the target pixel signal group 525(1) from storage region 330, which is one of storage regions 435, and (ii) reads the remaining pixel signal group 521(1) from storage region 330, which is one of storage regions 431. Step 730 may include abandoning the reading of pixel signal group P. 21 Therefore, an example of step 730 may include abandoning the reading of the remaining pixel signal group 522(1).

[0040] Step 740 includes capturing a second image of the scene by generating a second plurality of pixel signals, which includes a target pixel signal group T2 corresponding to a target pixel group. In an example of step 740, image sensor 300 captures an image of driver 194 during time period 502(2) by generating a pixel signal array 580(2). Target pixel signal group 525(2) is an example of target pixel signal group T2. Image capture in step 740 can begin before signal readout in step 730 is complete, which may help improve the video frame rate.

[0041] Step 750 includes transmitting a second plurality of pixel signals to a storage region. In an example of step 750, for each pixel 320 in the target pixel group 425, applying a pulse to the transmission gate of pixel 320 causes the generation of its pixel signal to be transmitted to the storage region 330 of pixel 320. The pixel signal is part of the pixel signal group 525(2).

[0042] Steps 720 and 750 may respectively include steps 722 and 752. Step 722 includes transmitting a target pixel signal group T1 to a target portion of the storage area when transmitting the first plurality of pixel signals. In an example of step 722, the generated pixel signals transmitted in step 720 are transmitted to a target portion 339 of the storage area 330. Step 752 includes overwriting the target pixel signal group T1 to the target portion using the target pixel signal group T2 when transmitting the target pixel signal group T2. In an example of step 752, transmitting the generated pixel signals in step 750 includes overwriting the target pixel signal group T1 stored in the target portion 339 using the target pixel signal group T2.

[0043] In an embodiment, each of the plurality of pixels has a corresponding one of a plurality of additional in-pixel analog storage elements. A storage region includes a plurality of additional in-pixel analog storage elements. Each storage region 331 is an example of an additional in-pixel analog storage element. In such an embodiment, step 750 may include transmitting each of the second plurality of pixel signals to a corresponding one of the plurality of additional in-pixel analog storage elements.

[0044] The second plurality of pixel signals may include (i) the target pixel signal group T2 (e.g., pixel signal group 525(2)), and (ii) the pixel signal group P corresponding to the first remaining pixel group defining the first remaining pixel array region. 12 (e.g., the remaining pixel signal group 521(2)), and (iii) the pixel signal group P corresponding to the second remaining pixel group. 22 (For example, the remaining pixel signal group 522(2)). In such an embodiment, step 750 may include step 754. Step 754 includes: transmitting the target pixel signal group T2, pixel signal group P 12 and pixel signal group P 22 The target pixel signal group T2 is used so that each pixel signal in the target pixel signal group T2 is transmitted to a corresponding one of the analog storage elements in a plurality of additional pixels.

[0045] In the embodiment, the second plurality of pixel signals include (i) target pixel signal group T2, and (ii) pixel signal group P. 12 and (iii) pixel signal group P 22 In such an embodiment, step 750 may include step 752. Step 752 includes: transmitting the target pixel signal group T2 and the pixel signal group P. 12 and pixel signal group P 22 Only the target pixel signal group T2 is stored in the storage area. The target pixel signal group 525(2), the remaining pixel signal group 521(2), and the remaining pixel signal group 522(2) are the target pixel signal group T2 and the pixel signal group P. 12 and pixel signal group P 22 Corresponding examples.

[0046] In the example of step 752, the row driver 352 addresses only the target pixel group 425 during the execution of step 750. When each pixel 320 includes storage areas 330 and 331, the storage area receiving the pixel signal group transmitted in step 752 can belong to either storage area 330 or additional storage area 331.

[0047] Step 760 includes reading the target pixel signal group T2 and pixel signal group P from the storage area. 21 In the example of step 760, the readout circuit 340: (i) reads the target pixel signal group 525 (2) from storage region 330, which is one of storage regions 435, and (ii) reads the remaining pixel signal group 522 (1) from storage region 330, which is one of storage regions 432. Step 760 may include abandoning the reading of pixel signal group P. 11 Therefore, an example of step 760 may include abandoning the reading of the remaining pixel signal group 521(1).

[0048] When method 700 includes step 750, step 760 may include step 764. Step 764 includes (i) reading pixel signal group P from analog storage elements within a plurality of pixels during a first time period. 21 (ii) during a second time period that overlaps at least partially with the first time period, the target pixel signal group T2 is read from analog storage elements within a plurality of additional pixels.

[0049] Step 770 is a decision. When the first plurality of pixel signals includes an additional group of pixel signals, method 700 repeats steps 740, 750, and 760 for the second image, replacing the second plurality of pixel signals with subsequent images and subsequent plurality of pixel signals. In an embodiment, the first plurality of pixel signals further includes a pixel signal group P corresponding to a third remaining pixel group. 31 The third remaining pixel group has the pixel coordinates of the third remaining pixel array region. Remaining pixel group 423 ( Figure 4 ) and remaining pixel signal group 523(1)( Figure 5 ) is the pixel signal group P 31 And the corresponding example of the third remaining pixel group.

[0050] In such an embodiment, repeating step 740 includes capturing a third image of the scene by generating a third plurality of pixel signals, the third plurality of pixel signals including a target pixel signal group T3 corresponding to the target pixel group. In an example of repeating step 740, image sensor 300 captures an image of driver 194 by generating a pixel signal array 580 (3) during time period 502 (3). Target pixel signal group 525 (3) is an example of target pixel signal group T3.

[0051] The second image is captured during a first duration following the capture of the first image. The third image is captured during a second duration following the capture of the second image. The second duration can be substantially equal to the first duration. For example, the relative difference between the second and third durations can be less than 10%.

[0052] The second repetition step 740 includes capturing a fourth image of the scene by generating a fourth plurality of pixel signals, which includes a target pixel signal group T4 corresponding to the target pixel group. In an example of the second repetition step 740, the image sensor 300 captures an image of the driver 194 during time period 502 (4) by generating a pixel signal array 580 (4). The target pixel signal group 525 (4) is an example of the target pixel signal group T4.

[0053] Repeating step 750 includes transmitting a third plurality of pixel signals to the storage region. In an example of repeating step 750, for each pixel 320 in the target pixel group 425, applying a pulse to the transmission gate of pixel 320 causes the generation of its pixel signal to be transmitted to the storage region 330 of pixel 320. The pixel signal is part of the pixel signal group 525(3).

[0054] The second repetition step 750 includes transmitting a fourth plurality of pixel signals to the storage region. In an example of the second repetition step 750, for each pixel 320 in the target pixel group 425, a pulse is applied to the transmission gate of pixel 320, causing the pixel signal it generates to be transmitted to the storage region 330 of pixel 320. The pixel signal is part of the pixel signal group 525(4).

[0055] Repeating step 760 includes reading the target pixel signal group T3 and pixel signal group P from the storage area. 31 In the example of repeating step 760, the readout circuit 340: (i) reads the target pixel signal group 525 (3) from the storage area 330, which is one of the storage areas 435, and (ii) reads the remaining pixel signal group 523 (1) from the storage area 330, which is one of the storage areas 432.

[0056] The second repetition step 760 includes reading the target pixel signal group T4 and pixel signal group P from the storage area. 41 In the example of repeating step 760 for the second time, the readout circuit 340: (i) reads the target pixel signal group 525 (4) from the storage area 330, which is one of the storage areas 435, and (ii) reads the remaining pixel signal group 524 (1) from the storage area 330, which is one of the storage areas 432.

[0057] In an embodiment, repeating step 760 includes reading only one remaining pixel signal group. That is, reading the target pixel signal group T4 and the pixel signal group P. 41 This includes reading the target pixel signal group T4 and pixel signal group P. 41 Instead of reading pixel signal group P 11 Pixel signal group P 21 and pixel signal group P 31 Each of them.

[0058] Following the second repetition described above, method 700 may include repeating each of steps 740, 750, and 760 an additional number of times. The number of repetitions may exceed the number of storage areas per pixel 320.

[0059] Step 780 includes reading the target pixel signal group T2 and the pixel signal group P. 21The storage area is then cleared. In the example of step 780, for example, the storage area 330 of pixel 320, which is part of the target pixel group 425 or the remaining pixel group 421, is cleared by the reset transistor of pixel 320.

[0060] Step 790 includes determining the difference between (i) a first ROI image generated at least partially from target pixel signal group T1 and (ii) a second ROI image generated at least partially from target pixel signal group T2. Step 790 may be performed at least partially by circuit 102, the processor of image sensor 300, or a combination thereof. Step 780 may be performed before, after, or simultaneously with step 790.

[0061] Figure 8 This is a circuit diagram of pixel 820 and sample-and-hold circuit 867, which are corresponding examples of pixel 320 and sample-and-hold circuit 367. Pixel 820 includes a photodiode 816 coupled to generate an image charge in response to incident light. A transfer transistor 818 is coupled to transfer the light-generated image charge from photodiode 816 to floating diffusion region (FD) 820 in response to a transfer signal TX. Pixel 820 may include one or both of a dual floating diffusion (DFD) transistor 821 and a second floating diffusion region (FD2) 823.

[0062] Reset transistor 822 is coupled to the pixel voltage power supply PIXVD to reset FD2823 in response to the reset signal RST, and further to reset FD 820 in response to the double floating diffusion signal DFD. In an embodiment, pixel 820 includes capacitor C. LOFIC 827 is a lateral overflow integration capacitor (LOFIC). Capacitor C LOFIC 827 is coupled to receive the bias voltage CAP, as shown in the figure. C LOFIC The 827 is coupled between the FD2823 and a voltage source held at the bias voltage CAP, as shown in the figure.

[0063] The gate of the source follower (SF) transistor 824 is coupled to convert the charge in the floating diffusion region 820 into an image voltage signal. The gate of the SF transistor is coupled to be output to the pixel-level connection 806 via the select transistor 826 in response to the select signal SEL. The pixel-level connection 806 may be a hybrid bond.

[0064] In an imaging system utilizing CDS, responding simultaneously to both RST and DFD signals, after a floating diffusion reset operation, the charge on the floating diffusion region 820 is read out via pixel-level connection 806 to obtain a reset level. Furthermore, after the image charge is transferred to the FD 820, the charge on the FD 820 is read out via pixel-level connection 806 to obtain a signal voltage.

[0065] Continuing with the illustrated example, the sample-and-hold circuit 867 includes a first storage transistor 832 coupled to a pixel-level connection 806 to sample a first reset voltage from pixel 820 and hold it to a first storage device C1 834 in response to a first reset storage signal SW1 852. In an embodiment, storage region 330 includes storage devices C1 and C2, while additional storage region 331 includes storage devices C3 and C4.

[0066] In this example, the first storage device C1 834 of the sample-and-hold circuit 867 is a capacitor. Furthermore, the sample-and-hold circuit 867 also includes a second storage transistor 836 coupled to the pixel-level connection 806 to sample and hold a first signal voltage from pixel 820 to the second storage device C2 838 in response to a first signal storage signal SW2 856. In this example, the second storage device C2 838 of the sample-and-hold circuit 867 is a capacitor. The sample-and-hold circuit 867 also includes a third storage transistor 840 coupled to the pixel-level connection 806 to sample and hold a second reset voltage from pixel 820 to the third storage device C3 842 in response to a second reset storage signal SW3 860. In this example, the third storage device C3 842 of the sample-and-hold circuit 867 is a capacitor. In the above-described extension, the sample-and-hold circuit 867 further includes a fourth storage transistor 844 coupled to the pixel-level connection 806 to sample and hold a second signal voltage from pixel 820 to a fourth storage device C4 846 in response to the second signal storage signal SW4 864. In this example, the fourth storage device C4 846 of the sample-and-hold circuit 867 is a capacitor.

[0067] The first reset storage signal SW1 852, the first signal storage signal SW2 856, the second reset storage signal SW3 860 (due to the extension), and the second signal storage signal SW4 864 (due to the extension) are generated by the sample and hold switch driver circuit 358 of the control circuit 350.

[0068] A reset row transistor 854 is coupled between the power supply voltage SVD and the pixel-level connection 806. The reset row transistor 854 responds to the reset row signal RST_ROW. The reset row transistor 854 is an example of the reset transistor for pixel 320. In the depicted example, a source follower row transistor 870 with a gate is coupled to the pixel-level connection 806. A row select row transistor 872 is coupled between the source follower row transistor 870 and the bit line 880. The row select row transistor 872 responds to the row select row signal RS_ROW. In the depicted example, a bias transistor 874 biased with a bias voltage VB is coupled between the pixel-level connection 806 and ground. The bias transistor 874 serves as a sample-and-hold (SH) current source. The SH current source supplies current to the SF transistor 824 and the pixel-level connection 806 at a typical value of approximately 20 nA.

[0069] Figure 9 This is a circuit diagram of pixel 920, which is an example of pixel 320. Pixel 920 has a global shutter, which can be controlled using a global shutter control signal generator 990. In this example, the global shutter control signal generator 990 is coupled to receive a negative NVDD voltage 930 from a power supply circuit 928, which may have variable bandwidth. Pixel 920 may be one of multiple pixel units in pixel array 310. As shown in the depicted example, pixel 920 includes a global shutter transistor 902, a photodiode 904, a transfer transistor 906, a storage transistor 908, an output transistor 911, a readout node 914, a reset transistor 912, an amplifier transistor 916, and a row select transistor 918 coupled to bit line 978. In one example, the readout node 914 is a floating diffusion region disposed in the semiconductor material of pixel 920. In one example, a source follower coupled transistor is used to implement the amplifier transistor 916. Figure 9 As shown in the example, the global shutter transistor 902 is coupled between the AVDD voltage and the photodiode 904.

[0070] In operation, the global shutter transistor 902 is coupled in response to the global shutter control signal GS CTRL 926 generated by the global shutter control signal generator 990, selectively depleting the image charge accumulated in the photodiode 904 prior to normal exposure operation by selectively coupling the photodiode 904 to the voltage AVDD. In this example, all pixel units 900 included in the pixel array of the image sensor share the global shutter control signal GS CTRL 926 to implement the global shutter. After the image charge in the photodiode 904 has been depleted by the global shutter transistor 902, the global shutter control signal GS CTRL 926 is converted to AVDD to switch the global shutter transistor 902 to a low-leakage cutoff mode. As illustrated in the example, the negative AVDD voltage 930 is provided by the power supply circuit 928. As will be discussed in more detail below, during the first portion of the cutoff time of the global shutter transistor 902, the bandwidth of the global shutter control signal GS CTRL 926 is set to have a large bandwidth to provide a fast settling time. According to the teachings of the present invention, during the second part of the cutoff time of the global shutter transistor 902, the bandwidth of the global shutter control signal GS CTRL 926 is set to have a small bandwidth to provide low noise and surge.

[0071] After the global shutter switch 902 has been closed in response to the global shutter control signal GS CTRL 926, the photodiode 904 disposed in the semiconductor material of pixel 920 then begins to accumulate image charge during normal exposure operation in response to incident light 922 directed toward the photodiode 904. In one example, the incident light 922 may be directed through the front side of the semiconductor material of pixel 920. In another example, it should be understood that the incident light 922 may be directed through the back side of the semiconductor material of pixel 920. After normal exposure operation, the image charge accumulated in the photodiode 904 is transferred to the input of the storage transistor 908 via the transfer transistor 906. The storage transistor 908 is an example of the storage region 330 of pixel 320.

[0072] Figure 9The example also shows an output transistor 911 coupled to the output of a storage transistor 908 to selectively transfer image charge from the storage transistor 908 to a readout node 914, which in the illustrated example is a floating diffusion region FD. A reset transistor 912 is coupled between a reset voltage VRESET and the readout node 914 to selectively reset the charge in the readout node 914 in response to a reset signal RST. In the example, an amplifier transistor 916 includes an amplifier gate coupled to the readout node 914 to amplify the signal on the readout node 914 to output image data from pixel 920. A row select transistor 918 is coupled between bit line 978 and amplifier transistor 916 to output image data to bit line 978.

[0073] In one embodiment, pixel 920 includes circuitry 950 connected in parallel with transfer transistor 906, storage transistor 908, and output transistor 911. Circuitry 950 includes transistor 956, storage transistor 958, and output transistor 951, which are similar to transistors 906, 908, and 911, respectively. Storage transistor 958 is an example of storage region 331.

[0074] Modifications to the methods and systems described above may be made without departing from the scope of these embodiments. Therefore, it should be noted that the content contained in the above description or shown in the accompanying drawings should be interpreted as illustrative rather than restrictive. Here, unless otherwise stated, the phrase "in some embodiments" is equivalent to the phrase "in some embodiments" and does not refer to all embodiments.

[0075] Regarding the terms "and / or" and "at least one," for example, in the cases of "A and / or B" and "at least one of A and B," this wording includes selecting (i) only A, or (ii) only B, or (iii) both A and B. In the cases of "A, B and / or C" and "at least one of A, B and C," this wording includes selecting (i) only A, or (ii) only B, or (iii) only C, or (iv) only A and B, or (v) only A and C, or (vi) only B and C, or (vii) each of A, B and C. This can be extended to many of the listed items.

[0076] The following claims are intended to cover all the general and specific features described herein, as well as all statements regarding the scope of the method and system, which, in linguistic terms, may be said to be interposed therein.

Claims

1. An image capture method, comprising: Using an image sensor with a pixel array, a first image of a scene is captured by generating a first plurality of pixel signals, the first plurality of pixel signals including: (i) a target pixel signal group T1 generated by a target pixel group having pixel coordinates of a target region of the pixel array, and (ii) a pixel signal group P generated by a first remaining pixel group. 11 The first remaining pixel group has the pixel coordinates of the first remaining pixel array region of the pixel array, and (iii) the pixel signal group P generated by the second remaining pixel group. 21 The second remaining pixel group has the pixel coordinates of the second remaining pixel array region of the pixel array; Transmit the first plurality of pixel signals to the storage area of ​​the image sensor; Read the target pixel signal group T1 and the pixel signal group P from the storage area. 11 ; A second image of the scene is captured by generating a second plurality of pixel signals, the second plurality of pixel signals including a target pixel signal group T2 corresponding to the target pixel group; Transmit the second plurality of pixel signals to the storage area; and Read the target pixel signal group T2 and the pixel signal group P from the storage area. 21 .

2. The method according to claim 1, wherein the second plurality of pixel signals comprises: (i) the target pixel signal group T2, (ii) the pixel signal group P corresponding to the first remaining pixel group 12 The first remaining pixel group defines the first remaining pixel array region, and (iii) the pixel signal group P corresponding to the second remaining pixel group. 22 The transmission of the second plurality of pixel signals includes: Transmit the target pixel signal group T2 and the pixel signal group P 12 and the pixel signal group P 22 Only the target pixel signal group T2 is sent to the storage area.

3. The method according to claim 1, The reading of the target pixel signal group T1 and the pixel signal group P 11 include: Read the target pixel signal group T1 and the pixel signal group P 11 Instead, the reading of the pixel signal group P is abandoned. 21 ;as well as The reading of the target pixel signal group T2 and the pixel signal group P 21 Includes: reading the target pixel signal group T2 and the pixel signal group P 21 Instead, the reading of the pixel signal group P is abandoned. 11 .

4. The method according to claim 1, further comprising: Reading the target pixel signal group T2 and the pixel signal group P 21 Then clear the storage area.

5. The method according to claim 1, The transmission of the first plurality of pixel signals includes: Transmit the target pixel signal group T1 to the target portion of the storage area; as well as The transmission of the target pixel signal group T2 includes: overwriting the target pixel signal group T1 stored in the target portion with the target pixel signal group T2.

6. The method according to claim 1, wherein the target pixel group, the first remaining pixel group, and the second remaining pixel group are non-overlapping and continuous.

7. The method of claim 1, wherein each of the first plurality of pixel signals is mapped to a corresponding pixel coordinate in a pixel coordinate array, and the target pixel group is located between the first remaining pixel group and the second remaining pixel group.

8. The method according to claim 1, wherein the first plurality of pixel signals further comprises: (iv) Pixel signal group P corresponding to the third remaining pixel group 31 The third remaining pixel group has pixel coordinates of the third remaining pixel array region, and the method further includes: A third image of the scene is captured by generating a third plurality of pixel signals, the third plurality of pixel signals including a target pixel signal group T3 corresponding to the target pixel group; Transmit the third plurality of pixel signals to the storage area; and Read the target pixel signal group T3 and the pixel signal group P from the storage area. 31 .

9. The method according to claim 8, wherein reading the target pixel signal group T3 and the pixel signal group P 31 include: Read the target pixel signal group T3 and the pixel signal group P 31 Instead, the reading of the pixel signal group P is abandoned. 11 and the pixel signal group P 21 Each of them.

10. The method of claim 8, wherein capturing the second image occurs for a first duration after capturing the first image, and capturing the third image occurs for a second duration after capturing the second image, the second duration being substantially equal to the first duration.

11. The method of claim 8, further comprising: Reading the target pixel signal group T3 and the pixel signal group P 31 Then clear the storage area.

12. The method according to claim 8, wherein the target pixel group, the first remaining pixel group, the second remaining pixel group, and the third remaining pixel group are non-overlapping and continuous.

13. The method according to claim 8, wherein the first plurality of pixel signals further comprises: (v) Pixel signal group P corresponding to the fourth remaining pixel group 41 The fourth remaining pixel group has pixel coordinates of the fourth remaining pixel array region, and the method further includes: A fourth image of the scene is captured by generating a fourth plurality of pixel signals, the fourth plurality of pixel signals including a target pixel signal group T4 corresponding to the target pixel group; Transmit the fourth plurality of pixel signals to the storage area; and Read the target pixel signal group T4 and the pixel signal group P from the storage area. 41 .

14. The method according to claim 13, wherein reading the target pixel signal group T4 and the pixel signal group P 41 include: Read the target pixel signal group T4 and the pixel signal group P 41 Instead, the reading of the pixel signal group P is abandoned. 11 The pixel signal group P 21 and the pixel signal group P 31 Each of them.

15. The method according to claim 13, The capture of the second image occurs during a first duration following the capture of the first image; The capture of the third image occurs during a second duration following the capture of the second image; The fourth image is captured during a third duration following the capture of the third image; The second duration is substantially equal to the first duration; and The third duration is substantially equal to the second duration.

16. The method of claim 13, further comprising: Reading the target pixel signal group T4 and the pixel signal group P 41 Then clear the storage area.

17. The method of claim 13, wherein the target pixel group, the first remaining pixel group, the second remaining pixel group, the third remaining pixel group, and the fourth remaining pixel group are non-overlapping and continuous.

18. The method of claim 1, wherein the image sensor has a plurality of pixels, and the storage region includes a plurality of in-pixel analog storage elements, each of the plurality of in-pixel analog storage elements belonging to a corresponding one of the plurality of pixels. The transmission of the first plurality of pixel signals includes: Transmit each of the first plurality of pixel signals to a corresponding one of the analog storage elements within the plurality of pixels.

19. The method of claim 18, wherein each of the plurality of pixels has a corresponding one of a plurality of additional in-pixel analog storage elements, the storage region including the plurality of additional in-pixel analog storage elements, and the second plurality of pixel signals comprising: (i) the target pixel signal group T2, (ii) the pixel signal group P corresponding to the first remaining pixel group 12 The first remaining pixel group defines the first remaining pixel array region, and (iii) the pixel signal group P corresponding to the second remaining pixel group. 22 , The transmission of the second plurality of pixel signals further includes: transmitting each pixel signal in the target pixel signal group T2 to a corresponding one of the analog storage elements within the plurality of additional pixels.

20. The method according to claim 19, wherein, The transmission of the second plurality of pixel signals includes: Transmit the target pixel signal group T2 and the pixel signal group P 12 and the pixel signal group P 22 Only the target pixel signal group T2 is used, so that each pixel signal in the target pixel signal group T2 is transmitted to a corresponding one of the analog storage elements in the plurality of additional pixels.

21. The method according to claim 19, wherein, The transmission of the second plurality of pixel signals includes: transmitting each of the second plurality of pixel signals to a corresponding one of the analog storage elements within the plurality of additional pixels.

22. The method according to claim 19, wherein reading the target pixel signal group T2 and the pixel signal group P 21 include: During the first time period, the pixel signal group P is read from the analog storage element within the plurality of pixels. 21 ; as well as The target pixel signal group T2 is read from the analog storage element within the plurality of additional pixels during a second time period that overlaps at least partially with the first time period.

23. The method of claim 1, wherein the image sensor comprises a plurality of pixels, and the first image of the captured scene comprises: Each of the plurality of pixels is used to generate a corresponding one of the first plurality of pixel signals.

24. The method according to claim 1, wherein, After completing the reading of the target pixel signal group T1 and the pixel signal group P 11 The second image was captured earlier.

25. A driver monitoring system comprising circuitry performing the method of claim 1.

26. The driver monitoring system according to claim 25, further comprising: According to claim 1, the image sensor's pixel array comprises each of the target pixel group, the first remaining pixel group, and the second remaining pixel group.

27. The method of claim 1, wherein the image sensor has a plurality of pixels, each pixel having a corresponding one of a plurality of intra-pixel analog storage elements and a corresponding one of a plurality of additional intra-pixel analog storage elements, the storage region comprising each of the plurality of intra-pixel analog storage elements and the plurality of additional intra-pixel analog storage elements. The transmission of the first plurality of pixel signals includes: Transmit each of the first plurality of pixel signals to a corresponding one of the analog storage elements within the plurality of pixels.

28. The method of claim 27, wherein the second plurality of pixel signals is the target pixel signal group T2, and the transmission of the second plurality of pixel signals comprises: The target pixel signal group T2 is transmitted to the analog storage element within the plurality of additional pixels.

29. The method according to claim 28, wherein, The target pixel signal group T2 and the pixel signal group P are read from the storage area. 21 include: Read the target pixel signal group T2 from the analog storage elements within the plurality of additional pixels; and Read the pixel signal group P from the analog storage elements within the plurality of pixels. 21 .

30. The method of claim 29, further comprising: Reading the target pixel signal group T2 and the pixel signal group P 21 Then clear the storage area.

31. The method of claim 29, wherein the plurality of in-pixel analog storage elements comprises a first portion and a second portion, wherein, There are no additional in-pixel analog storage elements that are present in both the first and second parts. The step of reading the target pixel signal group T2 includes: reading the target pixel signal group T2 from the first part; The first plurality of pixel signals further include: (iv) a pixel signal group P corresponding to the third remaining pixel group. 31 The third remaining pixel group has pixel coordinates of the third remaining pixel array region, and the method further includes: A third image of the scene is captured by generating a third plurality of pixel signals, the third plurality of pixel signals including a target pixel signal group T3 corresponding to the target pixel group; and The target pixel signal group T3 is transmitted to the second part.

32. The method of claim 31, wherein the third plurality of pixel signals comprises: (i) the target pixel signal group T3, (ii) the pixel signal group P corresponding to the first remaining pixel group 13 The first remaining pixel group defines the first remaining pixel array region, and (iii) the pixel signal group P corresponding to the second remaining pixel group. 23 The transmission of the third plurality of pixel signals includes: Transmit the target pixel signal group T3 and the pixel signal group P 13 and the pixel signal group P 23 Only the target pixel signal group T3 is included, so that each pixel signal in the target pixel signal group T3 is transmitted to the analog storage element within the plurality of additional pixels; and Read the target pixel signal group T3 and the pixel signal group P from the storage area. 31 .

33. The method of claim 31, wherein the third plurality of pixel signals comprises: (i) the target pixel signal group T3, (ii) the pixel signal group P corresponding to the first remaining pixel group 13 The first remaining pixel group defines the first remaining pixel array region, and (iii) the pixel signal group P corresponding to the second remaining pixel group. 23 The transmission of the third plurality of pixel signals includes: transmitting each of the third plurality of pixel signals to a corresponding one of the analog storage elements within the plurality of additional pixels.

34. The method according to claim 31, wherein, The target pixel signal group T3 and the pixel signal group P are read from the storage area. 31 include: Read the target pixel signal group T3 from the analog storage elements within the plurality of additional pixels; and Read the pixel signal group P from the analog storage elements within the plurality of pixels. 31 .

35. The method of claim 34, further comprising: Reading the target pixel signal group T3 and the pixel signal group P 31 Then clear the storage area.

36. The method of claim 31, wherein the first plurality of pixel signals further comprises: (v) Pixel signal group P corresponding to the fourth remaining pixel group 41 The fourth remaining pixel group has the pixel coordinates of the fourth remaining pixel array region, and A fourth image of the scene is captured by generating a fourth plurality of pixel signals, the fourth plurality of pixel signals including a target pixel signal group T4 corresponding to the target pixel group; as well as Transmit the target pixel signal group T4 to one of the first part and the second part. The transmission of the target pixel signal group T4 includes: (i) overwriting the target pixel signal group T2 stored in the first part with the target pixel signal group T4, or (ii) overwriting the target pixel signal group T3 stored in the second part with the target pixel dataset T4.

37. The method according to claim 36, wherein, The target pixel signal group T4 and the pixel signal group P are read from the storage area. 41 include: Read the target pixel signal group T4 from the analog storage elements within the plurality of additional pixels; and Read the pixel signal group P from the analog storage elements within the plurality of pixels. 41 .

38. The method of claim 37, further comprising: Reading the target pixel signal group T4 and the pixel signal group P 41 Then clear the storage area.

Images