Video capture method and apparatus

Abstract

A video capture method includes controlling an image sensor to capture a plurality of first sensor output frames at a first frame rate for a first time period; during the first time period, checking whether a motion blur condition is satisfied; in response to the motion blur condition being satisfied during the first time period, controlling the image sensor to capture a plurality of second sensor output frames at a second frame rate for a second time period after the first time period, wherein the second frame rate is higher than the first frame rate; processing consecutive sensor output frames captured by the image sensor during the first time period and the second time period to generate a plurality of output frames.

Description

[0001] RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Application No. 63 / 211,031, filed on June 16, 2021, the entire contents of which are incorporated herein by reference. TECHNICAL FIELD

[0003] The present application relates to video recording, and more particularly, to a method and apparatus for dynamically changing frame rate of sensor output frames depending on whether motion blur condition is satisfied. BACKGROUND

[0004] In video recording, camera motion and / or object motion can introduce unwanted image blur. To address the motion blur issue, a traditional image deblurring scheme can be employed to restore sharpness. However, the traditional image deblurring scheme has very high computational complexity and can degrade image quality due to the introduction of processing artifacts. An alternative solution to reduce motion blur is to trade off exposure time of an image sensor. Specifically, less motion blur can be reduced due to less motion captured at a shorter exposure time. However, there is a trade-off between image noise and motion blur. That is, when an image sensor is configured to use a longer exposure time, the captured frame has a lower image noise level but a higher motion blur level; when an image sensor is configured to use a shorter exposure time, the captured frame has a higher image noise level but a lower motion blur level. Motion blur can be reduced at the cost of higher image noise. Image noise level is roughly proportional to sensor exposure time. Avoiding motion blur by reducing exposure time means not fully utilizing the capability of an image sensor.

[0005] Recently, some image sensors are equipped with high frame rate features. Having an image sensor work at high frame rate and short exposure time can achieve reduction of motion blur. Unfortunately, having an image sensor always work at high frame rate results in high power consumption. SUMMARY

[0006] It is one of the objectives of the present application to provide a method and apparatus for dynamically changing frame rate of sensor output frames depending on whether motion blur condition is satisfied.

[0007] According to a first aspect of the present disclosure, an exemplary video capturing method is disclosed. The exemplary video capturing method comprises: controlling an image sensor to capture a plurality of first sensor output frames at a first frame rate for a first time period; checking, during the first time period, whether a motion blur condition is satisfied; in response to the motion blur condition being satisfied during the first time period, controlling the image sensor to capture a plurality of second sensor output frames at a second frame rate for a second time period after the first time period, wherein the second frame rate is higher than the first frame rate; and processing consecutive sensor output frames captured by the image sensor during the first time period and the second time period to generate a plurality of output frames.

[0008] According to a second aspect of the present disclosure, an exemplary video capturing apparatus is disclosed. The exemplary video capturing apparatus comprises a frame rate control circuit, a motion analysis circuit and a processing circuit. The frame rate control circuit is configured to control an image sensor to capture a plurality of first sensor output frames at a first frame rate for a first time period. The motion analysis circuit is configured to check, during the first time period, whether a motion blur condition is satisfied. When the motion analysis circuit determines that the motion blur condition is satisfied during the first time period, the frame rate control circuit is further configured to control the image sensor to capture a plurality of second sensor output frames at a second frame rate for a second time period after the first time period. The second frame rate is higher than the first frame rate. The processing circuit is configured to process consecutive sensor output frames captured by the image sensor during the first time period and the second time period to generate a plurality of output frames.

[0009] These and other objects of the present application will no doubt become apparent to those skilled in the art after having read the following detailed description of the preferred embodiments illustrated in the several figures attached. BRIEF DESCRIPTION OF DRAWINGS

[0010] Figure 1 A camera system according to an embodiment of the present application is shown.

[0011] Figure 2 A flowchart of a video capturing method according to an embodiment of the present application is shown.

[0012] Figure 3 A diagram showing a video recording example using the proposed video capturing method shown in Figure 2

[0013] Figure 4 A diagram showing a first noise suppression circuit according to an embodiment of the present application is shown.

[0014] Figure 5 A diagram showing a second noise suppression circuit according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0015] ​Certain terminology is used throughout the following description and claims to refer to particular elements. As one skilled in the art will appreciate, electronic device manufacturers can refer to a component by different names. This document does not intend to distinguish between components that differ in name but not in function. In the following description and in the claims, the terms "include" and "comprise" are used in an open-ended fashion, and thus should be interpreted to mean "including, but not limited to...." Also, the term "couple" is intended to mean either an indirect or direct electrical connection. Accordingly, if one device is coupled to another device, that connection can be through a direct electrical connection, or through an indirect electrical connection via other devices and connections.

[0016] Figure 1 A camera system according to an embodiment of the present application is shown. The camera system 100 can be part of a portable device such as a cellular phone or a tablet computer. The camera system 100 can include an image sensor 102, a motion sensor 104, and a video acquisition apparatus 106. It should be noted that only elements relevant to the present application are shown in Figure 1 In practice, the video acquisition apparatus 106 can include additional elements for other designated functions. The image sensor 102 is arranged to acquire a video sequence having a plurality of sensor output frames Fl. For example, the image sensor 102 can be implemented by a complementary metal oxide semiconductor (CMOS) image sensor. In the present embodiment, the image sensor 102 is capable of high frame rate. The motion sensor 104 is arranged to provide motion information. For example, the motion sensor 104 can be implemented by a gyro sensor, and the motion information provided by the motion sensor 104 can be indicative of camera motion.

[0017] The video acquisition apparatus 106 has a plurality of circuits with designated functions, including an auto exposure circuit (labeled "AE") 112, a motion analysis circuit (labeled "MA") 114, and a processing circuit 116. In the present embodiment, the auto exposure circuit 112 includes a frame rate control circuit (labeled "FR control") 122 to enable dynamic frame rate of the image sensor 102. The processing circuit 116 is arranged to receive the output frames Fl from the image sensor 102, and to process the received sensor output frames Fl to produce a recorded video stream D OUT. In the present embodiment, the processing circuit 116 includes an image signal processor (ISP) 124, a noise suppression circuit (labeled "NS") 126, and a display encoder 128.

[0018] The auto exposure circuit 112 is configured to determine an exposure setting according to the ambient brightness, and then provide the exposure setting to the image sensor 102 through an inter-integrated circuit (I2C) bus 108. The motion analysis circuit 114 is arranged to obtain motion information INF M from the noise suppression circuit 126, and further arranged to determine whether a motion blur condition is satisfied. For example, the motion analysis circuit 114 can obtain the motion information INF M from image content analysis performed on consecutive frames derived from the sensor output frames F1 of the image sensor 102, where the motion information INF M can be indicative of camera motion and / or object motion. For another example, the motion analysis circuit 114 can obtain the motion information INF M from the motion sensor 104, where the motion information INF M can be indicative of camera motion. In addition, the motion analysis circuit 114 can obtain auto exposure (AE) statistics INF AE from the auto exposure circuit 112. When the motion information INF M and the AE statistics INF AE are available, the motion analysis circuit 114 checks whether the motion blur condition is satisfied by referring to the motion information INF M and the AE statistics INF AE. For example, the motion analysis circuit 114 refers to the AE statistics INF AE to learn the ambient brightness and the exposure time, and determines whether motion blur is involved according to the estimated amount of motion based on the motion information INF M.

[0019] The frame rate control circuit 122 is configured to control the frame rate of the image sensor 102. Therefore, the dynamic frame rate feature of the image sensor 102 can be implemented by the frame rate control circuit 122. For example, when the motion blur condition is not satisfied, the rate control circuit 122 controls the image sensor 102 to capture the sensor output frames F1 at a first frame rate, and when the motion blur condition is satisfied, controls the image sensor 102 to capture other sensor output frames F1 at a second frame rate. Wherein the second frame rate is higher than the first frame rate. That is, when motion blur is not involved at the estimated amount of motion (i.e. the motion blur condition is not satisfied), the image sensor 102 can capture the sensor output frames F1 at a low frame rate (e.g. 30 frames per second (FPS)), and when motion blur is involved at the estimated amount of motion (i.e. the motion blur condition is satisfied), the image sensor 102 can capture other sensor output frames F1 at a high frame rate (e.g. 90 FPS).

[0020] Further, the auto exposure circuit 112 can control the image sensor 102 to capture the sensor output frames F1 at a first exposure setting when the motion blur condition is not satisfied, and to capture other sensor output frames F1 at a second exposure setting when the motion blur condition is satisfied, where the second exposure setting specifies a shorter exposure time than the first exposure setting. Thus, in some embodiments of the present disclosure, the image sensor 102 can capture the sensor output frames F1 at a high frame rate and a short exposure time for one period of time, and can capture the sensor output frames F1 at a low frame rate and a long exposure time for another period of time. Further, the ratio of the second frame rate to the first frame rate is N: 1 (N > 1), and the ratio of the second exposure time to the first exposure time is 1:N (N > 1). In this way, the image quality can be enhanced by taking full advantage of the capability of the image sensor.

[0021] The processing circuit 116 processes the consecutive sensor output frames F1 captured by the image sensor 102 to generate a plurality of output frames F3, and encodes the output frames F3 into a recorded video stream D_OUT. The image signal processor 124 is configured to process the consecutive sensor output frames F1 to generate a plurality of processed frames F2. The noise suppression circuit 126 can be configured to apply temporal noise reduction (TNR) to the processed frames F2 to generate the output frames F3. In order to relax the buffer requirement and the bandwidth requirement, the output frames F3 undergo an encoding process. Specifically, the display encoder 128 is configured to encode the output frames F3 to generate the recorded video stream D_OUT, which can be stored in a storage device or transmitted via a wireless / wired communication link.

[0022] As described above, when no motion blur is involved under the estimated motion (i.e., the motion blur condition is not met), the image sensor 102 is controlled to acquire video frames at a lower frame rate and a longer exposure time; when motion blur is involved under the estimated motion (i.e., the motion blur condition is met), the image sensor 102 is controlled to acquire video frames at a higher frame rate and a shorter exposure time. There is a trade-off between image noise and motion blur. When the image sensor 102 is configured to use a shorter exposure time, the acquired frames have a higher level of image noise but a lower level of motion blur. Therefore, the motion blur problem can be solved by reducing the exposure time. For the image noise problem caused by reducing the exposure time, noise suppression can be performed using the additional frames acquired by the image sensor 102 at a higher frame rate. For example, when motion blur is involved under the estimated motion (i.e., the motion blur condition is met), the noise suppression circuit 126 can apply noise reduction (e.g., TNR) with frame rate conversion to a higher frame rate (e.g., 90 FPS) to generate output frames with a fixed output frame rate (e.g., 30 FPS). Further details of the proposed video acquisition device 106 are described below with reference to the accompanying drawings.

[0023] Will Figure 2 and Figure 3 and Figure 1 Combine, Figure 2 A flowchart illustrating a video acquisition method according to an embodiment of the present invention is shown. Figure 3 Showing the use Figure 2 The figure shows an example of video recording using the proposed video capture method. Figure 2 The video capture device 106 shown can be used Figure 1 The video capture device 106 shown. If the results are essentially the same, these steps do not need to be followed. Figure 2 The exact order of execution is shown.

[0024] In step 202, the automatic exposure circuit 112 determines the exposure settings of the image sensor 102 based on the ambient brightness. For example, the automatic exposure function can control the image sensor 102 to use a long exposure time to capture dim scenes (i.e., scenes with low ambient brightness) and can control the image sensor 102 to use a short exposure time to capture light scenes (i.e., scenes with high ambient brightness).

[0025] In step 204, the motion analysis circuit 114 uses the motion sensor 104 and / or image content to obtain motion information INF_M for the TNR performed at the noise suppression circuit 126. Furthermore, the motion analysis circuit 114 can refer to the obtained motion information INF_M to estimate the amount of motion (e.g., camera motion and / or object motion).

[0026] At step 206, the image sensor 102 is instructed to capture sensor output frames F1 at a first frame rate and a first exposure time. For example, the first exposure time is set by the auto exposure circuit 112 at step 202, and the first frame rate is set by the frame rate control circuit 122 according to the desired output frame rate (e.g. 30 FPS) for a particular application.

[0027] At step 208, the motion analysis circuit 114 checks whether the motion blur condition is satisfied. If the motion analysis circuit 114 determines that the motion blur condition is not satisfied, the flow proceeds to step 210. At step 210, the image sensor 102 continues to capture sensor output frames F1 at the first frame rate and the first exposure time. At step 212, the image signal processor 124 processes the sensor output frames F1 to generate processed frames F2. For example, but not limited to, the number of processed frames F2 output by the image signal processor 124 is equal to the number of sensor output frames F1 received by the image signal processor 124. That is, there is no frame rate change by the image signal processor 124. At step 214, the noise suppression circuit 126 applies noise reduction to the processed frames F2 without frame rate conversion to generate output frames F3 at a constant frame rate (e.g. 30 FPS).

[0028] With reference to Figure 3 , the frame rate control circuit 122 initially controls the image sensor 102 to capture sensor output frames F1_0-F1_3 at a first frame rate for a period of time P1, where each of the sensor output frames F1_0-F1_3 is captured at a first exposure time T1 determined by the auto exposure circuit 112. Accordingly, the image signal processor 124 processes the sensor output frames F1_0-F1_3 to generate processed frames F2_0-F2_3, respectively. Specifically, the processed frame F2_0 is derived from image signal processing operations on the sensor output frame F1_0, the processed frame F2_1 is derived from image signal processing operations on the sensor output frame F1_1, the processed frame F2_2 is derived from image signal processing operations on the sensor output frame F1_2, and the processed frame F2_3 is derived from image signal processing operations on the sensor output frame F1_3. The functions involved in the image signal processing operations depend on actual design considerations. Since the present invention does not focus on the design of the image signal processor, further details are omitted for brevity. The noise suppression circuit 126 applies noise reduction to the processed frames F2_0-F2_3 without frame rate conversion to generate output frames F3_0-F3_3 at a constant frame rate (e.g. 30 FPS). The first frame rate employed by the image sensor 102 can be set by the constant frame rate of the output frames F3.

[0029] If the motion analysis circuit 114 determines that the motion blur condition is met due to large motion (step 208), the process proceeds to step 216. In step 216, the image sensor 102 is instructed to acquire a sensing output frame F1 at a second frame rate and a second exposure time, wherein the second frame rate (e.g., 90 FPS) is higher than the first frame rate (e.g., 30 FPS), and the second exposure time is shorter than the first exposure time. In step 218, the image signal processor 124 processes the sensing output frame F1 to generate a processed frame F2. For example, but not limited to, the number of processed frames F2 output by the image signal processor 124 is equal to the number of sensing output frames F1 received by the image signal processor 124. That is, no frame rate change occurs in the image signal processing 124. In step 220, the noise suppression circuit 126 applies noise reduction with frame rate conversion to the processed frame F2 to generate an output frame F3 at a constant frame rate (e.g., 30 FPS).

[0030] like Figure 3 As shown, when image sensor 102 acquires sensor output frame F1_2 and begins acquiring sensor output frame F1_3, motion analysis circuit 114 detects that the motion blur condition is met. In response to the motion blur condition being met, frame rate control circuit 122 controls image sensor 102 to acquire sensor output frames F1_4-F1_15 at a second frame rate in the next time period P2 immediately following time period P1. Each of sensor output frames F1_4-F1_15 is acquired at a second exposure time T2, which is set by automatic exposure circuit 112 to reduce motion blur. In this embodiment, the first exposure time T1 is evenly divided into three second exposure times T2, and consecutive sensor output frames are acquired within a time period equal to the first exposure time T1, each consecutive sensor output frame having one of the second exposure times T2. Therefore, the capabilities of the image sensor are fully utilized when the exposure time is reduced to reduce motion blur. Specifically, during a time period equal to the first exposure time T1, continuous sensor output frames F1_4-F1_6 are acquired; during a time period equal to the first exposure time T1, continuous sensor output frames F1_7-F1_9 are acquired; during a time period equal to the first exposure time T1, continuous sensor output frames F1_10-F1_12 are acquired; and during a time period equal to the first exposure time T1, continuous sensor output frames F1_13-F1_15 are acquired.

[0031] The image signal processor 124 processes the sensor output frames F1_4-F1_15 to generate processed frames F2_4-F2_15, respectively. Specifically, the processed frame F2_4 is derived by performing image signal processing operations on the sensor output frame F1_4, the processed frame F2_5 is derived by performing image signal processing operations on the sensor output frame F1_5, and so on. The noise suppression circuit 126 applies noise reduction with frame rate conversion to the processed frames F2_4-F2_15 for generating output frames F3_4-F3_7 at a constant frame rate (e.g., 30 FPS).

[0032] It should be noted that the noise suppression circuit 126 can employ any device capable of achieving noise reduction through frame rate conversion. Figure 4 A diagram showing a first noise suppression circuit according to an embodiment of the present application. Figure 1 The illustrated noise suppression circuit 126 can be implemented by a noise suppression circuit 400. Referring to Figure 4 , the noise suppression circuit 400 includes an infinite impulse response (IIR) type TNR filter (labeled "IIR TNR") 402. The IIR type TNR filter 402 is arranged to work at a 1:1 ratio (1 frame input, 1 frame output) in normal mode, and is arranged to work at a higher frame rate to use the collected frame data to suppress noise. Once the noise is suppressed by the additional frames received, it can be down-sampled by a ratio of N:1 per frame, where N is the increased frame rate ratio in the frame rate control.

[0033] Figure 5 A diagram showing a second noise suppression circuit according to an embodiment of the present application. Figure 1 The illustrated noise suppression circuit 126 can be implemented by a noise suppression circuit 500. Referring to Figure 5 , the noise suppression circuit 500 includes a finite impulse response (FIR) filter (labeled "FIR") 502 and an infinite impulse response (IIR) type TNR filter (labeled "IIR TNR") 504. The FIR filter 502 can be an N-tap FIR filter, and the IIR type TNR filter 504 can be a 1:1 IIR TNR filter, where N is the increased frame rate ratio from the frame rate control. Thus, the N-to-1 noise suppression and frame rate conversion are performed at the FIR filter 502, and then the output of the FIR filter 502 is further processed by the IIR type TNR filter 504 without frame rate conversion.

[0034] After the image sensor 102 is instructed to operate at the second frame rate higher than the first frame rate, the motion analysis circuit 114 continuously monitors the motion blur condition (step 208) to determine whether to instruct the image sensor 102 to exit the high sensor frame rate mode and enter the normal sensor frame rate mode. If the motion analysis circuit 114 determines that the motion blur condition is not satisfied due to steady state video capture, the flow proceeds to step 210 to change the frame rate and the exposure time. At step 210, the image sensor 102 is instructed to capture sensor output frames Fl at the first frame rate and the first exposure time. At step 212, the image signal processor 124 processes the sensor output frames Fl to generate processed frames F2. At step 214, the noise suppression circuit 126 applies the noise reduction without frame rate conversion to the processed frames F2 for generating output frames F3 at a constant frame rate (e.g., 30 FPS).

[0035] As Figure 3 When the sensor output frames Fl_12 have been captured by the image sensor 102 and the image sensor 102 starts to capture the sensor output frames Fl_13, the motion analysis circuit 114 detects that the motion blur condition is not satisfied. In response to the motion blur condition not being satisfied, the frame rate control circuit 122 controls the image sensor 102 to capture sensor output frames Fl_16 and Fl_17 at the first frame rate for a next time period P3 immediately after the time period P2, where each of the sensor output frames Fl_16 and Fl_17 is captured at the first exposure time Tl determined by the auto exposure circuit 112. The image signal processor 124 processes the sensor output frames Fl_16 and Fl_17 to generate processed frames F2_16 and F2_17, respectively. Specifically, the processed frame F2_16 is derived by performing image signal processing operations on the sensor output frame Fl_16, and the processed frame F2_17 is derived by performing image signal processing operations on the sensor output frame Fl_17. The noise suppression circuit 126 applies the noise reduction without frame rate conversion to the processed frames F2_16 and F2_17 for generating output frames F3_8 and F3_9 at a constant frame rate (e.g., 30 FPS). It is noted that, Figure 3 The frame rates of the output frames F3_0-F3_9 are fixed, regardless of the frame rate of the image sensor 102.

[0036] In summary, the video capture device 106 and associated video capture method can break the trade-off between image noise and motion blur by jointly using a dynamic sensor frame rate control scheme and a noise reduction scheme. Furthermore, the video capture device 106 and associated video capture method employ a motion blur detection scheme such that the image sensor 102 and image signal processor 124 only process high frame rate data when the motion blur condition is satisfied. In this way, image quality can be improved without significantly increasing power consumption. When motion blur is not involved at the estimated amount of motion (i.e., the motion blur condition is not satisfied), the image sensor 102 generates a sensor output frame at a long exposure time. In this way, a low noise output frame with low motion blur is provided when the estimated amount of motion does not exceed a predetermined threshold. When motion blur is involved at the estimated amount of motion (i.e., the motion blur condition is satisfied), the image sensor 102 generates a sensor output frame at a short exposure time, thereby achieving motion blur reduction. The additional frames obtained due to the increased frame rate can be used by the noise suppression circuit 126 for noise suppression. In this way, a low noise frame with low motion blur is provided when the estimated amount of motion exceeds a predetermined threshold. Furthermore, to stabilize the frame rate of the recorded video stream, the noise suppression circuit 126 employs a noise reduction scheme with frame rate conversion.

[0037] Those skilled in the art will readily observe that numerous modifications and alterations of the device and method can be made without departing from the teaching of the present application. Accordingly, the above disclosure is intended to be illustrative only and not limiting. The scope of the present application is limited only by the claims that follow.

Claims

1. A video capture method, comprising: The image sensor is controlled to acquire multiple first sensor output frames at a first frame rate within a first time period. During the first time period, check whether the motion blur condition is met; In response to the motion blur condition being met during the first time period, the image sensor is controlled to acquire a plurality of second sensor output frames at a second frame rate during a second time period after the first time period, wherein the second frame rate is higher than the first frame rate. The image sensor processes the plurality of first sensor output frames acquired during the first time period to generate a plurality of output frames at a constant frame rate, and when the motion blur condition is satisfied during the first time period, the image sensor processes the plurality of continuous sensor output frames acquired during the second time period to generate a plurality of output frames at the constant frame rate. During the second time period, check whether the motion blur condition is not met; as well as In response to the motion blur condition not being met during the second time period, the image sensor is controlled to acquire multiple third sensor output frames at the first frame rate during a third time period following the second time period.

2. The video acquisition method as described in claim 1, characterized in that, Each of the plurality of first sensor output frames is acquired at a first exposure time, and each of the plurality of second sensor output frames is acquired at a second exposure time, wherein the second exposure time is shorter than the first exposure time.

3. The video acquisition method as described in claim 2, characterized in that, The first exposure time is divided into multiple second exposure times; And a plurality of consecutive second sensor output frames are acquired within a time period equal to the first exposure time, each of the plurality of consecutive second sensor output frames having one of the plurality of second exposure times.

4. The video acquisition method as described in claim 1, characterized in that, The constant frame rate is equal to the first frame rate.

5. The video acquisition method as described in claim 4, characterized in that, The plurality of continuous sensor output frames includes the plurality of first sensor output frames and the plurality of second sensor output frames. Processing the plurality of continuous sensor output frames acquired within the first time period and the second time period to generate the plurality of output frames includes: Image signal processing operations are performed on the plurality of second sensor output frames at the second frame rate to generate a plurality of processed frames; and Noise suppression with frame rate conversion operation is performed on the plurality of processed frames to generate a portion of the plurality of output frames at the first frame rate.

6. The video acquisition method as described in claim 5, characterized in that, The noise suppression with frame rate conversion operation includes infinite impulse response type time noise reduction.

7. The video acquisition method as described in claim 5, characterized in that, The noise suppression with frame rate conversion operation includes finite impulse response filtering, followed by infinite impulse response time noise reduction.

8. The video acquisition method as described in claim 1, characterized in that, Checking whether the motion blur condition is satisfied includes: Motion information is obtained from image content analysis; Obtain automatic exposure statistics; and The motion blur condition is checked by referring to the motion information and the automatic exposure statistics.

9. The video acquisition method as described in claim 1, characterized in that, Checking whether the motion blur condition is satisfied includes: Obtain motion information from motion sensors; Obtain automatic exposure statistics; and The motion blur condition is checked by referring to the motion information and the automatic exposure statistics.

10. A video capture device, comprising: A frame rate control circuit is used to control the image sensor to acquire multiple first sensor output frames at a first frame rate within a first time period. A motion analysis circuit is provided, wherein during the first time period, the motion analysis circuit is used to check whether a motion blur condition is met; when the motion analysis circuit determines that the motion blur condition is met during the first time period, the frame rate control circuit is further used to control the image sensor to acquire a plurality of second sensor output frames at a second frame rate during a second time period after the first time period, wherein the second frame rate is higher than the first frame rate. During the second time period, it is checked whether the motion blur condition is not met; and when the motion analysis circuit determines that the motion blur condition is not met during the second time period, the frame rate control circuit is further used to control the image sensor to acquire multiple third sensor output frames at the first frame rate during a third time period after the second time period. as well as The processing circuit is used to process the plurality of first sensing output frames acquired by the image sensor during the first time period, and when the motion blur condition is satisfied during the first time period, to process the plurality of continuous sensing output frames acquired by the image sensor during the second time period, and to generate a plurality of output frames at a constant frame rate.

11. The video acquisition device as described in claim 10, characterized in that, Each of the plurality of first sensor output frames is acquired at a first exposure time, and each of the plurality of second sensor output frames is acquired at a second exposure time, wherein the second exposure time is shorter than the first exposure time.

12. The video acquisition device as described in claim 11, characterized in that, The first exposure time is divided into multiple second exposure times; And a plurality of consecutive second sensor output frames are acquired within a time period equal to the first exposure time, each of the plurality of consecutive second sensor output frames having one of the plurality of second exposure times.

13. The video acquisition device as described in claim 10, characterized in that, The constant frame rate is equal to the first frame rate.

14. The video acquisition device as described in claim 13, characterized in that, The plurality of consecutive sensor output frames include the plurality of first sensor output frames and the plurality of second sensor output frames, and the processing circuit includes: an image signal processor, configured to process the plurality of second sensor output frames at the second frame rate to generate a plurality of processed frames; and A noise suppression circuit is used to perform noise suppression on the plurality of processed frames with a frame rate conversion operation, and to generate a portion of the plurality of output frames at the first frame rate.

15. The video acquisition device as described in claim 14, characterized in that, The noise suppression circuit includes an infinite impulse response time noise reduction filter.

16. The video acquisition device as described in claim 14, characterized in that, The noise suppression circuit includes: Finite impulse response filter; and An infinite impulse response time noise reduction filter is used to process one output of the finite impulse response filter.

17. The video acquisition device as described in claim 10, characterized in that, The motion analysis circuit is used for: Motion information is obtained from image content analysis; Obtain automatic exposure statistics; and The motion blur condition is checked by referring to the motion information and the automatic exposure statistics.

18. The video acquisition device as described in claim 10, characterized in that, The motion analysis circuit is used for: Obtain motion information from motion sensors; Obtain automatic exposure statistics; and The motion blur condition is checked by referring to the motion information and the automatic exposure statistics.

Images