Image sensing system control method and image sensing system

Abstract

The invention discloses an image sensing system control method and an image sensing system, the method is used in the image sensing system comprising an image sensor and a control circuit, and the method comprises the following steps: (a) defining a first return picture frame sensed at a first return picture frame time and a second return picture frame sensed at a second return picture frame time, wherein the image sensor returns the movement calculated according to the first return frame after the first return frame time and returns the movement calculated according to the second return frame after the second return frame time; and (b) adjusting a first return time interval between the first return frame time and the second return frame time to adjust the second return frame time to a second target time, wherein the second picture frame return time is after the first picture frame return time.

Description

Technical Field

[0001] The invention relates to an image sensor system control method and an image sensor system, and in particular to an image sensor system control method and an image sensor system capable of reducing a frame rate and polling instructions. Background Art

[0002] In an optical navigation device (e.g., an optical mouse), an image sensor captures frames at a predetermined frame rate and then calculates the movement increments between different frames. The predetermined frame rate can be changed to correspond to different modes, such as active mode or standby mode. In addition, an MCU (Micro Control Unit) polls the image sensor for movement increments (i.e., requests the image sensor to output movement increments). However, the MCU polling rate and the image sensor frame rate are usually different and asynchronous with each other. Therefore, there is often an asynchronous problem between the movement increment output and the MCU polling. Summary of the invention

[0003] An object of the present invention is to disclose an image sensor system control method, which can reduce the problem caused by the asynchronism between the polling rate and the frame rate.

[0004] Another object of the present invention is to disclose an image sensing system which can reduce the problem caused by the asynchronism between the polling rate and the frame rate.

[0005] An embodiment of the present invention discloses an image sensor system control method, which is used in an image sensor system including an image sensor and a control circuit, including: (a) defining a first report frame sensed at a first report frame time and a second report frame sensed at a second report frame time, wherein the image sensor reports a movement calculated according to the first report frame after the first report frame time, and reports a movement calculated according to the second report frame after the second report frame time; and (b) adjusting a first report time interval between the first report frame time and the second report frame time to adjust the second report frame time to a second target time; wherein the second report frame time is after the first report frame time.

[0006] Another embodiment of the present invention discloses an image sensing system, comprising: a control circuit; and an image sensor, for performing the following steps: (a) defining a first report frame sensed at a first report frame time and a second report frame sensed at a second report frame time, wherein the image sensor reports a movement calculated according to the first report frame after the first report frame time, and reports a movement calculated according to the second report frame after the second report frame time; and (b) adjusting a first report time interval between the first report frame time and the second report frame time to adjust the second report frame time to a second target time; wherein the second report frame time is after the first report frame time.

[0007] Another embodiment of the present invention discloses an image sensor system control method, which is used in an image sensor system including an image sensor and a control circuit, and includes: (a) defining a first report frame sensed at a first report frame time by the image sensor, and defining a second report frame sensed at a second report frame time after the first report frame time by the image sensor; (b) obtaining a first movement of the image sensor system in a first delay time interval by the image sensor, the first delay time being between a first polling time and the first report frame time, wherein the image sensor receives a first polling from the control circuit at the first polling time; (c) calculating a second movement of the image sensor system in a first correction time interval by the image sensor, wherein the first correction time interval includes a first frame time interval between the first report frame time and the second report frame time, and includes a first expected delay time interval after the second report frame time; and (d) outputting a first output movement by the image sensor to the control circuit, such that a difference between the first output movement and a physical movement of the image sensor system is less than a predetermined value.

[0008] Another embodiment of the present invention discloses an image sensing system, comprising: a control circuit; and an image sensor, configured to: (a) define a first report frame sensed at a first report frame time, and define a second report frame sensed at a second report frame time after the first report frame time; (b) obtain a first movement of the image sensing system in a first delay time interval, the first delay time being between a first polling time and the first report frame time, wherein the image sensor receives a first polling from the control circuit at the first polling time; (c) calculate a second movement of the image sensing system in a first correction time interval, wherein the first correction time interval includes a first frame time interval between the first report frame time and the second report frame time, and includes a first expected delay time interval after the second report frame time; and (d) output a first output movement to the control circuit, such that a difference between the first output movement and a physical movement of the image sensing system is less than a predetermined value.

[0009] According to the above embodiment, even if the frame rate changes frequently, the image sensor can report the movement corresponding to the polling instruction. Therefore, the influence caused by the asynchronism between the frame rate and the polling instruction can be reduced. Moreover, the movement of the movement delay can also be compensated, making the movement report of the image sensor more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 A block diagram of an image sensing system according to an embodiment of the present invention is depicted;

[0011] Figure 2 A schematic diagram illustrating steps of adjusting at least one reporting time interval according to an embodiment of the present invention is depicted;

[0012] Figure 3 and Figure 4 A schematic diagram of the steps of setting a non-reporting time interval according to an embodiment of the present invention is depicted;

[0013] Figure 5 A schematic diagram of image signals in different domains according to an embodiment of the present invention is illustrated;

[0014] Figure 6 A flowchart of an image sensing system control method according to an embodiment of the present invention is depicted;

[0015] Figure 7 A schematic diagram of an image sensing system control method according to another embodiment of the present invention is depicted;

[0016] Figure 8 According to another embodiment of the present invention, Figure 7 A flowchart of the detailed steps of the image sensing system control method depicted;

[0017] Fig. 9 According to an embodiment of the present invention, Figure 7 and Figure 8 Simplified steps of the embodiment shown.

[0018] The reference numerals are described as follows:

[0019] 100: Image Sensing System

[0020] 101: Control Circuit

[0021] 103. Image Sensor

[0022] 501: Pixel array

[0023] 503: Reading circuit

[0024] 505: Amplifier

[0025] 507: ADC

[0026] 601, 603, 801-811, 901-907: Steps

[0027] AS: Analog image signal

[0028] DS: Digital video signal

[0029] F_r1, F_r1a: First report frame

[0030] F_r2, F_r2a: Second report frame

[0031] F_r3, F_r3a: Third report frame

[0032] F_nr1, F_nr11...F_nr37: First non-return frame

[0033] F_nr12: Second non-return frame

[0034] F_nr21, F_nr22, F_nr23: Third non-return frame

[0035] MP_1, MP_2, MP_3, MP_4, MP_5: Polling instructions

[0036] MP_1a: First poll

[0037] MP_2a: Second poll

[0038] MP_3a: Third poll

[0039] MT_1: First move

[0040] MT_2: Second move

[0041] MT_3: Third move

[0042] MT_4: Fourth move

[0043] MT_5: Fifth move

[0044] T_C1: First calibration time interval

[0045] T_C2: Second calibration time interval

[0046] T_E1: First expected delay time interval

[0047] T_LTE1: First delay time interval

[0048] T_LTE2: Second delay time interval

[0049] T_MTI1: Target reporting delay time interval

[0050] T_MRI1: Report delay time interval

[0051] T_RI1: First reporting time interval

[0052] T_RI2: Second reporting time interval

[0053] T_R1, T_R1a: First report frame time

[0054] T_R2, T_R2a: Second report frame time

[0055] T_R3, T_R3a: Third report frame time

[0056] T_T1: First target time

[0057] T_T2: Second target time

[0058] T_T3, T_T4: target time

[0059] T_nri1: First non-return time interval

[0060] T_nri2: Second non-report time interval

[0061] T_nri3: Third non-return time interval DETAILED DESCRIPTION

[0062] The present invention will be described below with multiple embodiments. Please note that the components in each embodiment can be implemented by hardware (e.g., a device or circuit) or firmware (e.g., at least one program written in a microprocessor). In addition, the "first", "second" and similar descriptions in the following description are only used to define different components, parameters, data, signals or steps. They are not used to limit their order. For example, the first device and the second device can be different devices with the same structure.

[0063] Figure 1 FIG. 1 is a block diagram of an image sensing system 100 according to an embodiment of the present invention. Figure 1 As shown, the image sensing system 100 includes a control circuit 101 and an image sensor 103. The control circuit 101 can be the above-mentioned MCU or any other device capable of achieving the same function, such as a processor. The image sensor 103 is used to sense multiple frames and calculate the motion delta between different frames. The control circuit 101 generates a polling instruction to the image sensor 103, and the image sensor 103 outputs the motion delta in response to the polling instruction. The control circuit 101 and the image sensor 103 can be set in the same device or in different devices.

[0064] Figure 2 A schematic diagram of the steps of adjusting at least one reporting time interval according to an embodiment of the present invention is shown. Figure 2 The embodiment includes three examples: Example 1, Example 2 and Example 3. In each example, a first report frame F_r1 and a second report frame F_r2 are defined. The first report frame F_r1 is sensed at a first report frame time T_R1, and the second report frame F_r2 is sensed at a second report frame time T_R2 after the first report frame time T_R1. The image sensor 103 reports the movement calculated according to the first report frame F_r1 after the first report frame time T_R1, and reports the movement calculated according to the second report frame F_r2 after the second report frame time T_R2. In one embodiment, the image sensor 103 reports the movement in response to the polling instruction that is closest to and after the frame used for movement calculation. For example, in Figure 2 In the embodiment, the image sensor 103 responds to the polling command MP_2 (second polling command) to report the movement calculated according to the first report frame F_r1 , and responds to the polling command MP_3 (third polling command) to report the movement calculated according to the second report frame F_r2 .

[0065] The first report frame F_r1 and the second report frame F_r2 can be defined by various methods. In one embodiment, the order information of the first report frame F_r1 and the second report frame F_r2 are respectively the same as the order information of the frames that have been synchronized with the polling instruction from the control circuit 101. For example, by counting the order of the frames within the predetermined time interval, it can be known that the 15th frame, the 30th frame, and the 45th frame within the predetermined time interval are synchronized with the polling instruction. In this way, in the subsequent sensing action, the 15Kth frame (K is a positive integer) can be set as the report frame. However, the report frame can be defined by other methods and is not limited to these examples.

[0066] After defining the first report frame F_r1 and the second report frame F_r2, the first report time interval T_RI1 between the first report frame time T_R1 and the second report frame time T_R2 is adjusted so that the second report frame time T_R2 becomes the second target time T_T2 after the adjustment. In other words, the sensing time of the second report frame F_r1 is adjusted to the second target time T_T2. In one embodiment, the initial first report time interval T_RI1 is equal to the time interval between two adjacent polling instructions. For example, the initial first report time interval T_RI1 is equal to the time interval between the polling instruction MP_1 (first polling instruction) and the polling instruction MP_2 (second polling instruction).

[0067] In addition, in one embodiment, the first reporting time interval T_RI1 is adjusted after the first reporting frame F_r1 is sensed and before the second reporting frame F_r2 is sensed. In another embodiment, the first reporting time interval T_RI1 is adjusted before the first reporting frame F_r1 and the second reporting frame F_r2 are sensed. Please also note that even if the first reporting frame F_r1 and the second reporting frame F_r2 have not been sensed, the image sensor 103 can still obtain the first reporting frame time T_R1 and the second reporting frame time T_R2 because the first reporting frame time T_R1 and the second reporting frame time T_R2 can still be defined through the above steps.

[0068] In one embodiment, a target report delay time is calculated, which is the time interval between the target time and the polling instruction closest to the target time among the polling instructions before the target time. For example, a target report delay time interval T_MTI1 is calculated between the first target time T_T1 and the polling instruction MP_1 before the first target time T_T1. In addition, a report delay time interval is calculated, which is the time interval between the report frame and the polling instruction closest to the report frame among the polling instructions before the report frame. For example, a report delay time interval T_MRI1 is calculated between the first report frame F_r1 and the polling instruction MP_1.

[0069] Then, the first reporting time interval T_RI1 is adjusted according to the relationship between the target reporting delay time interval T_MTI1 and the reporting delay time interval T_MRI1. In one embodiment, if the reporting delay time interval T_MRI1 is less than the target reporting delay time interval T_MTI1, the first reporting time interval T_RI1 is increased by the offset value. In this way, the second reporting frame F_r2 will be pushed to the first target time T_T1. Figure 2 Example 1 is shown in the following example.

[0070] On the contrary, if the reporting delay time interval T_MRI1 is greater than the target reporting delay time interval T_MTI1, the first reporting time interval T_RI1 is reduced by the offset value. In this way, the second reporting frame F_r2 is pulled to the first target time T_T1. Figure 2 In Example 2, in Example 3, the reporting delay time interval T_MRI1 is equal to the target reporting delay time interval T_MTI1, which means that the second reporting frame F_r2 is synchronized with the second target time T_T2, and therefore the first reporting time interval T_RI1 will not be adjusted.

[0071] In one embodiment, the shift value may be a constant value. In another embodiment, the offset value is a percentage of the first reporting time interval T_RI1, such as 3% or 5% of the first reporting time interval T_RI1. In this case, the first reporting time interval T_RI1 may be gradually and consistently adjusted until the second reporting frame time T_R2 is adjusted to the second target time T_T2. In another embodiment, the second reporting frame time T_R2 may be adjusted in real time to meet the second target time T_T2. In this case, the offset value is equal to the difference between the target reporting delay time interval T_MTI1 and the reporting delay time interval T_MRI1. Such steps may also be used for other target times T_T3, T_T4.

[0072] In one embodiment, the time difference between the first target time T_T1 and the polling instruction (e.g., polling instruction MP_2) closest to the first target time T_T1 among the polling instructions is the necessary time for generating the movement calculated according to the first report frame F_r1. In addition, the time difference between the second target time T_T2 and one of the polling instructions (e.g., polling instruction MP_3) closest to the second target time T_T2 is the necessary time for generating the movement calculated according to the second report frame F_r2. Such necessary time may be the time for buffering the frame for movement calculation and / or the time required for calculating the movement. However, if the speed of the image sensor 103 is fast enough, the necessary time may be regarded as 0. In this case, the first target time T_T1 may be the same as the time when the polling instruction MP_1 is received, and the second target time T_T1 may be the same as the time when the polling instruction MP_2 is received.

[0073] Above Figure 2 The steps can be used in the report frame after the second report frame F_r2. Please refer to Figure 2 .exist Figure 2 In the embodiment of the present invention, a third reporting frame F_r3 is defined. The third reporting frame F_r3 is sensed at a third reporting frame time T_R3 after the second reporting frame time T_R2. The image sensor 103 reports the movement calculated according to the third reporting frame F_r3 after the third reporting frame time T_R3. For example, the image sensor 103 responds to the polling instruction MP_3 and reports the movement calculated according to the third reporting frame F_r3.

[0074] Afterwards, the second reporting time interval T_RI2 between the second reporting frame time T_R2 and the third reporting frame time T_R3 is adjusted to adjust the third reporting frame time T_R3 to the third target time P_3. In one embodiment, the steps of adjusting the first reporting time interval T_RI1 and the second reporting time interval T_RI2 are performed simultaneously. In addition, in one embodiment, the first reporting time interval T_RI1 and the second reporting time interval T_RI2 have the same value.

[0075] Other details of adjusting the second reporting time interval T_RI2 have been described in the above embodiment and will not be repeated here. The step of adjusting the reporting time interval can be performed periodically, for example, adjusting the reporting time interval once every 100 frames. Alternatively, the step of adjusting the reporting time interval can be triggered by an instruction. In addition, the step of adjusting the reporting time interval can be performed by including Figure 1 The step of adjusting the reporting time interval may be triggered by the mode switching of the optical device of the control circuit 101 and the image sensor 103. For example, the step of adjusting the reporting time interval may be triggered when the optical mouse switches from the standby mode to the active mode. Such a step may also be used in the fourth reporting frame F_r4.

[0076] In one embodiment, at least one non-report frame is sensed between the report frames. Figure 3 and Figure 4 A schematic diagram of the steps of setting a non-reporting time interval according to an embodiment of the present invention is depicted. Please also note that for ease of understanding, Figure 3 and Figure 4 The embodiment is not shown Figure 1 and Figure 2 The target time line. Figure 3 There are three examples: Example A, Example B and Example C. Figure 3In an embodiment, a first non-reporting time interval of at least one first non-reporting frame F_nr11 ... F_nr37 is calculated. In a subsequent sensing action, the first non-reporting frame F_nr11 ... F_nr37 is sensed based on the first non-reporting time interval. The first non-reporting time interval may be a time interval between the first report frame time and the first non-reporting frame closest to the first report frame. For example, the first non-reporting time interval may be a time interval T_nri1 between the first report frame time F_r1 and the first non-reporting frame F_nr1. The non-reporting frames F_nr2 and F_nr3 may also follow the same rule.

[0077] In addition, the first non-reporting time interval may be a time interval between two adjacent first non-reporting frames. For example, the first non-reporting time interval T_nri1 may be a time interval between the first non-reporting frames F_nr11 and F_nr12 in Example B. In addition, the first non-reporting time interval may be a time interval between the second reporting frame and the first non-reporting frame closest to the second reporting frame. For example, the first non-reporting time interval T_nri1 is a time interval between the second reporting frame F_r2 and the first non-reporting frame F_nr1 in Example A, or a time interval between the second reporting frame F_r2 and the first non-reporting frame F_r2 in Example B.

[0078] In one embodiment, the value of the first non-reporting time interval is equal to 1 / X of the first reporting time interval T_RI1, where X is a positive integer (1, 2, 3, ...). The first reporting time interval T_RI1 mentioned here can be the reporting time interval adjusted by the above embodiment. In one embodiment, X is 2 N , N is a natural number (0, 1, 2, 3...). Figure 3 In examples A, B, and C, N is 1, 2, and 3 respectively. If N is 0, there is no non-return frame.

[0079] The first non-reporting time interval can be set according to different requirements. In one embodiment, the image sensor 103 dynamically and automatically sets the first non-reporting time interval according to different modes of the optical mouse including the control circuit 101 and the image sensor 103. For example, if the optical mouse is in standby mode or has a low speed, the first non-reporting time interval can be set to be smaller, so that the image sensor 103 senses fewer frames within a predetermined time interval, thereby consuming less power. On the contrary, if the speed of the optical mouse is higher, the first non-reporting time interval can be set to be larger, so that the image sensor 103 senses more frames within a predetermined time interval, making the movement calculation more accurate.

[0080] exist Figure 3In the embodiment of the present invention, the non-reporting time intervals in a single reporting time interval are the same and fixed throughout the sensing operation (e.g., in the example of Example A). However, the scope of the present invention is not limited to a single reporting time interval or the non-reporting time intervals being the same throughout the sensing operation. Figure 4 Three examples are included: Example D, Example E, and Example F. In Example D, non-reporting time intervals of different values ​​exist in a single reporting time interval. For example, in addition to the first non-reporting time interval T_nri1 in Example D, a second non-reporting time interval T_nri2 of the second non-reporting frame F_nr12 in Example D is also calculated. The second non-reporting time interval T_nri2 is different from the first non-reporting time interval T_nri1.

[0081] As mentioned above, the non-reporting time interval may be a first reporting time interval. Therefore, in Figure 4 In Example D of Example D, for the first non-reporting time interval T_nri1, N is 1, and for the second non-reporting time interval T_nri2, N is 2. Similarly, the third non-reporting time interval T_nri3 (N=3) of the third non-reporting frames F_nr21, F_nr22, and F_nr23 in Example D can be set according to the above steps. The definition of the second non-reporting time interval T_nri2 and the third non-reporting time interval T_nri3 is the same as the definition of the first non-reporting time interval T_nri1, so it is not repeated here. Similarly, in Example E, the N of the non-reporting time intervals of the non-reporting frames F_nr11, F_nr12, and F_nr13 is 2, and the N of the non-reporting time intervals of the non-reporting frames F_nr21 and F_nr22 are 1 and 2 respectively. Similarly, the non-reporting time interval N of the non-reporting frame F_nr11 in Example F is 1, the non-reporting time interval N of the non-reporting frames F_nr21 and F_nr22 in Example F is 2, and the non-reporting time interval N of the non-reporting frames F_nr31 ... F_nr37 in Example F is 3. Therefore, the non-reporting time intervals in a single reporting time interval or in the entire sensing action may be different. The non-reporting frame F_nr24 may also follow the same rule.

[0082] The above-described embodiments may be performed in the analog domain or the digital domain. Figure 5 FIG. 2 is a schematic diagram of image signals in different domains according to an embodiment of the present invention. Figure 5As shown, the optical sensor 103 includes a pixel array 501, a read circuit 503, an amplifier 505, and an ADC (analog to digital converter) 507. The pixel circuit in the pixel array 501 is used to generate a charge corresponding to the received light. The read circuit 503 is used to read the charge to generate an analog image signal AS. The amplifier 505, which can be removed in some embodiments, is used to generate an amplified image signal. The ADC 507 is used to convert the analog image signal AS into a digital image signal DS. The above embodiments can be performed based on the analog image signal AS or the digital image signal DS.

[0083] According to the above embodiments, a method for controlling an image sensor system can be obtained. The method for controlling an image sensor system is applied to an image sensor system (eg, Figure 1 100 in the image sensing system). Figure 6 The flowchart of the image sensor system control method according to one embodiment of the present invention is shown, which includes the following steps:

[0084] Step 601: Define a first report frame (eg, Figure 2 ) and a second report frame sensed at a second report frame time (eg, Figure 2 The first report frame F_r2).

[0085] The image sensor reports the movement calculated according to the first report frame after a first report frame time, and reports the movement calculated according to the second report frame after a second report frame time. The second report frame time is after the first report frame time.

[0086] Step 603: Adjust the first reporting time interval between the first reporting frame time and the second reporting frame time (eg, Figure 2 ) to adjust the second reporting frame time to the second target time (eg, Figure 2 The second target time T_T2 in the example.

[0087] In one embodiment, the image sensing system disclosed in the present invention further includes a light source, such as Figure 1 The light source LS shown. In addition, the image sensor 103 senses frames according to the frame rate, and the frames include reporting frames and non-reporting frames. In one embodiment, the light emission frequency of the light source is synchronized with the frame rate. Specifically, when the image sensor senses a picture, the light source emits light, and when the image sensor does not sense a picture, the light source does not emit light.

[0088] The embodiments disclosed in the present invention can be applied to gaming mice, but are not limited to them. Movement latency, i.e., the time required to capture an image for movement calculation, is one of the key factors in evaluating gaming mice. A good gaming mouse sensor needs to have low movement latency to report actual physical movement. In order to solve such a problem, the present application also discloses the following embodiments.

[0089] Figure 7 FIG. 2 is a schematic diagram of an image sensing system control method according to another embodiment of the present invention. Figure 7 As shown, a first report frame F_r1a, a second report frame F_r2a, and a third report frame F_r3a are defined. The image sensor 103 senses the first report frame F_r1a at the first report frame time T_R1a, senses the second report frame F_r2a at the second report frame time T_R2a, and senses the third report frame F_r3a at the third report frame time T_R3a. The first report frame F_r1a, the second report frame F_r2a, and the third report frame F_r3a can be defined by the steps described in the above embodiment, but are not limited thereto.

[0090] In addition, the image sensor 103 calculates the first movement MT_1 of the image sensor system 100 in the first delay time interval T_LTE1 between the first polling time and the first report frame time T_R1a. The image sensor 103 receives the first polling MP_1a from the control circuit 101 at the first polling time. The first delay time interval T_LTE1 is the movement delay mentioned above.

[0091] In addition, the image sensor 103 calculates a second movement MT_2 of the image sensing system within the first calibration time interval T_C1. The first calibration time interval T_C1 includes a first frame time interval between the first report frame time T_R1a and the second report frame time T_R2a, and includes a first expected delay time interval T_E1 after the second report frame time T_R2a. In one embodiment, the first expected delay time interval T_E1 is a delay time interval between the first report frame time T_R1a and the second polling time. The image sensor 103 receives a second polling MP_2a from the control circuit 101 at the first polling time at the second polling time. The first expected delay time interval T_E1 may be the above-mentioned movement waiting time.

[0092] Next, the image sensor 103 subtracts the first movement MT_1 from the second movement MT_2 to generate a first output movement. Then, the image sensor 103 outputs the first output movement to the control circuit 101. That is, the image sensor 103 responds to the second polling MP_2a and reports the first output movement equal to (MT-2-MT_1) to the control circuit 101. The existing image sensor only reports the movement between the first report frame F_r1a and the second report frame F_r2a, without considering the movement in the movement delay. Therefore, Figure 7 The method disclosed in can report more accurate movements.

[0093] According to the same rule, for the third report frame F_r3a, the image sensor 103 obtains the third movement MT_3 of the image sensing system 100 at the second delay time interval T_LTE2 between the third polling time and the third report frame time T_R3a. The image sensor 103 receives the third polling MP_3a from the control circuit 101 at the third polling time. In addition, the image sensor 103 calculates the fourth movement MT_4 of the image sensing system 100 at the second calibration time interval T_C2, wherein the second calibration time interval T_C2 includes the second frame time interval between the third report frame time T_R3a and the first report frame time, and includes the second expected delay time interval after the first report frame time. In one embodiment, the second expected delay time interval is the first delay time interval T_LTE1 mentioned above. Then, the image sensor 103 subtracts the third movement MT_3 from the fourth movement MT_4 to generate a second output movement. Since the third report frame F_r3a is before the first report frame F_r1a and the second report frame F_r2a, the first movement MT_1 can be obtained before starting to calculate the second movement MT_2. For example, before starting to calculate the second movement MT_2, the fifth movement MT_5 of the second expected delay time interval can be regarded as the first movement MT_1.

[0094] Figure 7 The image sensing system control method in can be followed Figure 2 Example 3 of the image sensing system control method shown in FIG. That is, the first report frame F_r1a, the second report frame F_r2a, and the third report frame F_r3a are synchronized with different target times. In this case, Figure 7 The first report frame F_r1a, the second report frame F_r2a and the third report frame F_r3a in the graph may be respectively Figure 2 In this case, because the time interval between two adjacent polls is fixed and the time interval between the target time and the polling is fixed (e.g. Figure 2In the target report delay time interval T_MTI1), the first delay time interval T_LTE1, the first expected delay time interval T_E1, and the second delay time interval T_LTE2 are the same and fixed values. However, the first delay time interval T_LTE1, the first expected delay time interval T_E1, and the second delay time interval T_LTE2 can also be obtained by other methods. For example, the first delay time interval T_LTE1, the first expected delay time interval T_E1, and the second delay time interval T_LTE2 can be predetermined values, or real-time predicted values.

[0095] Figure 8 According to another embodiment of the present invention, Figure 7 The image sensing system control method disclosed in the present invention is not limited to the following steps. Figure 8 Flowchart shown. Figure 8 The image sensing system control method can be Figure 1 The image sensing system 100 and Figure 5 The image sensor 103 shown in the figure is used to implement the method, which includes the following steps:

[0096] Step 801: Wait for a new frame.

[0097] Step 803: Accumulate PIX_C, which represents the accumulated movement of pixels.

[0098] by Figure 7 Taking the first report frame F_r1a as an example, the pixel movement of the frames between the third report frame F_r3a and the first report frame F_r1a is accumulated. Step 803 can be implemented Figure 3 and Figure 4 Example shown.

[0099] Step 805: Determine whether the frame is a report frame, if so, proceed to step 807, if not, return to steps 801 and 803, and continue to accumulate PIX_C.

[0100] Step 807: Calculate the movement of a frame time interval and movement delay, such as Figure 7 The second move MT_2 in .

[0101] In more detail, in one embodiment, the second movement MT_2 can be obtained by the following equation.

[0102] And MT_PI×DPI_C=MT_2

[0103] MRP1 is the first frame time interval between the first report frame time T_R1a and the second report frame time T_R2a. Please note that in one embodiment, because the report frame has been adjusted to the target time, MRP1 is a known parameter, such as Figure 2 T_LTE1 is Figure 7 The first delay time interval shown, DPI is a predetermined DPI (Dot Per Inch, dots per inch) value. In one embodiment, DPI has a value of the first frame time interval. MT_PI is the movement per inch of the image sensing system, MT_2 is Figure 7 The second move in .

[0104] MT_PI can be obtained by the following equation (2):

[0105]

[0106] PIX_C is the accumulated movement of pixels in step 801, and PIX_N is the number of pixels per inch.

[0107] Please note that step 807 can be used for calculating other movements of the correction time interval, and is not limited to the second movement MT_2.

[0108] Step 809: Remove the movement in the last movement delay from the movement in the frame time interval and the movement in the movement delay in step 807. For example, in one embodiment, the first movement MT_1 is removed from the second movement MT_2.

[0109] In one embodiment, the first movement MT_1 is calculated by the following equation:

[0110] And MT_PI×DPI_L=MT_1

[0111] Where MRP1 is the first frame time interval in step 807, T_LTE1 is Figure 7 The first delay time interval is shown, DPI is a predetermined DPI value, and MT_PI is the movement per inch of the image sensing system.

[0112] Step 809 can be applied to other moves in the previous move delay, not limited to the first move MT_1.

[0113] For example, the fifth movement MT_5 is calculated according to the following equation:

[0114] And MT_PI×DPI_L=MT_5

[0115] MRP2 is Figure 2 and Figure 7The second frame time interval in , T_LTE2 is the second expected delay time interval, DPI is the predetermined DPI value, MT_PI is the movement per inch of the image sensing system, and MT_5 is the fifth movement.

[0116] Step 811: Update the last move in the previous move delay. Figure 7 In the process, the fifth movement MT_5 is calculated while the fourth movement MT_4 is calculated, and the fifth movement MT_5 is updated to the movement in the previous movement delay while the second movement MT_2 is calculated.

[0117] According to the aforementioned embodiments, simplified steps of an image sensor system control method can be obtained, which is applied to an image sensor system including an image sensor and a control circuit. Fig. 9 According to an embodiment of the present invention, Figure 7 and Figure 8 Simplified steps of the embodiment shown. Fig. 9 The following steps are involved:

[0118] Step 901: Define a first report frame (eg, Figure 7 F_r1a in the image sensor, and a second report frame (eg, Figure 7 F_r2a in).

[0119] The first report frame and the second report frame can be based on Figure 2 The method described in the description is defined, but not limited to this.

[0120] Step 903: Obtain a first delay time interval (eg Figure 7 The first movement of the image sensing system (e.g. T_LTE1) Figure 7 MT_1), the first delay time is between the first polling time and the first report frame time (e.g. Figure 7 , wherein the image sensor receives a first poll from the control circuit at a first polling time (eg, Figure 7 MP_1a in ).

[0121] Step 905: Calculate a first calibration time interval (eg, Figure 7 The second movement of the image sensing system (eg, Figure 7 The first correction time interval includes the first report frame time and the second report frame time (for example, Figure 7 , and includes a first expected delay time interval after the second report frame time (eg, Figure 7 T_E1 in ).

[0122] if Fig. 9 The image sensing system control method shown follows Figure 2 In the embodiment, since the first report frame time and the second report frame time can be the same fixed value, the first delay time interval and the first expected delay time interval can be the same fixed value. However, the first delay time interval and the first expected delay time interval can also be obtained or set by other methods.

[0123] Step 907: Move the first output of the image sensor to the control circuit so that the difference between the first output movement and the physical movement of the image sensing system is less than a predetermined value.

[0124] In other words, in step 907, Fig. 9 The first output movement outputted by the step is close to the physical movement of the image sensing system. If the image sensing system is applied to an optical mouse, the first output movement is close to the physical trajectory of the optical mouse. As described above, in one embodiment, the first output movement is generated by subtracting the first movement from the second movement. Other steps can be obtained according to the above embodiments and will not be repeated here.

[0125] According to the above embodiment, even if the frame rate changes frequently, the image sensor can report the movement corresponding to the polling instruction. Therefore, the influence caused by the asynchronism between the frame rate and the polling instruction can be reduced. Moreover, the movement of the movement delay can also be compensated, making the movement report of the image sensor more accurate.

[0126] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for controlling an image sensing system, characterized in that: The image sensing system used in the image sensor and the control circuit includes: (a) defining a first report frame sensed at a first report frame time and a second report frame sensed at a second report frame time, wherein the image sensor reports a movement calculated according to the first report frame after the first report frame time, and reports a movement calculated according to the second report frame after the second report frame time; and (b) adjusting a first report time interval between the first report frame time and the second report frame time to adjust the second report frame time to a second target time; The second report frame time is after the first report frame time.

2. The image sensing system control method according to claim 1, characterized in that: Further including: Calculate a first non-return time interval of a first non-return frame; as well as Sensing the first non-report frame according to the first non-report time interval; The first non-return time interval is at least one of the following time intervals: the time interval between the first report frame time and the first non-report frame closest to the first report frame time, the time interval between two adjacent first non-report frames, and the time interval between the second report frame time and the first non-report frame closest to the second report frame time.

3. The image sensing system control method according to claim 2, characterized in that: Further including: Calculating a second non-reporting time interval of a second non-reporting frame, wherein the second non-reporting time interval is different from the first non-reporting time interval; Sensing the second non-report frame according to the second non-report time interval; The second non-return time interval is at least one of the following time intervals: The time interval between the first report frame time and the second non-report frame closest to the first report frame time, the time interval between two adjacent second non-report frames, and the time interval between the second report frame time and the first non-report frame closest to the second report frame time.

4. The image sensing system control method according to claim 2, characterized in that: The first non-reporting time interval is the first reporting frame time. X is a positive integer.

5. The image sensing system control method according to claim 4, characterized in that: X is 2 N , N is a natural number.

6. The image sensing system control method according to claim 1, wherein: Further including: (c) defining a third report frame sensed at a third report frame time after the second report frame time, wherein the image sensor reports a movement calculated according to the third report frame after the third report frame time; (d) adjusting a second report time interval between the second report frame time and the third report frame time to adjust the third report frame time to a third target time; The steps (b) and (d) are performed simultaneously and the first reporting time interval and the second reporting time interval have the same value.

7. The image sensing system control method according to claim 6, characterized in that: Further including: Calculating a third non-return time interval of a third non-return frame; sensing the third non-report frame according to the third non-report time interval; The third non-return time interval is at least one of the following time intervals: the time interval between the second report frame time and the third non-report frame closest to the second report frame time, the time interval between two adjacent third non-report frames, and the time interval between the third report frame time and the third non-report frame closest to the third report frame time.

8. The image sensing system control method according to claim 1, wherein: Further including: receiving a polling instruction from the control circuit with the image sensor; wherein the time difference between the first target time and the polling instruction closest to the first target time is the necessary time for generating the movement calculated according to the first reporting frame; The time difference between the second target time and the polling instruction closest to the second target time is the necessary time for generating the movement calculated according to the second reporting frame.

9. The image sensing system control method according to claim 1, wherein: The sequence information of the first report frame and the second report frame is the same as the sequence information of the frame synchronized with the polling instruction from the control circuit.

10. The image sensing system control method according to claim 9, characterized in that: Further including: counting the order of frames within a predetermined time interval; The order of the first report frame and the second report frame is the same as the order information of the frame synchronized with the polling instruction from the control circuit.

11. An image sensing system, characterized in that: include: Control circuit; as well as An image sensor is used to perform the following steps: (a) defining a first report frame sensed at a first report frame time and a second report frame sensed at a second report frame time, wherein the image sensor reports a movement calculated according to the first report frame after the first report frame time, and reports a movement calculated according to the second report frame after the second report frame time; as well as (b) adjusting a first report time interval between the first report frame time and the second report frame time to adjust the second report frame time to a second target time; The second report frame time is after the first report frame time.

12. The image sensing system according to claim 11, wherein: The image sensor is further configured to perform the following steps: Calculating a first non-report time interval of a first non-report frame; and Sensing the first non-report frame according to the first non-report time interval; The first non-return time interval is at least one of the following time intervals: the time interval between the first report frame time and the first non-report frame closest to the first report frame time, the time interval between two adjacent first non-report frames, and the time interval between the second report frame time and the first non-report frame closest to the second report frame time.

13. The image sensing system according to claim 12, wherein: The image sensor is further configured to perform the following steps: Calculating a second non-reporting time interval of a second non-reporting frame, wherein the second non-reporting time interval is different from the first non-reporting time interval; Sensing the second non-report frame according to the second non-report time interval; The second non-return time interval is at least one of the following time intervals: The time interval between the first report frame time and the second non-report frame closest to the first report frame time, the time interval between two adjacent second non-report frames, and the time interval between the second report frame time and the first non-report frame closest to the second report frame time.

14. The image sensing system according to claim 12, wherein: The first non-reporting time interval is the first reporting frame time. X is a positive integer.

15. The image sensing system according to claim 14, wherein: X is 2 N , N is a natural number.

16. The image sensing system according to claim 11, wherein: The image sensor is further configured to perform the following steps: (c) defining a third report frame sensed at a third report frame time after the second report frame time, wherein the image sensor reports a movement calculated according to the third report frame after the third report frame time; (d) adjusting a second report time interval between the second report frame time and the third report frame time to adjust the third report frame time to a third target time; The steps (b) and (d) are performed simultaneously and the first reporting time interval and the second reporting time interval have the same value.

17. The image sensing system according to claim 16, wherein: The image sensor is further configured to perform the following steps: Calculating a third non-return time interval of a third non-return frame; sensing the third non-report frame according to the third non-report time interval; The third non-return time interval is at least one of the following time intervals: the time interval between the second report frame time and the third non-report frame closest to the second report frame time, the time interval between two adjacent third non-report frames, and the time interval between the third report frame time and the third non-report frame closest to the third report frame time.

18. The image sensing system according to claim 11, wherein: The image sensor is further configured to perform the following steps: receiving a polling instruction from the control circuit with the image sensor; wherein the time difference between the first target time and the polling instruction closest to the first target time is the necessary time for generating the movement calculated according to the first reporting frame; The time difference between the second target time and the polling instruction closest to the second target time is the necessary time for generating the movement calculated according to the second reporting frame.

19. The image sensing system according to claim 11, wherein: The sequence information of the first report frame and the second report frame is the same as the sequence information of the frame synchronized with the polling instruction from the control circuit.

20. The image sensing system according to claim 19, wherein: Further including: counting the order of frames within a predetermined time interval; The order of the first report frame and the second report frame is the same as the order information of the frame synchronized with the polling instruction from the control circuit.

21. An image sensing system control method, characterized in that: The image sensing system used in the image sensor and the control circuit includes: (a) defining a first report frame sensed at a first report frame time by the image sensor, and defining a second report frame sensed at a second report frame time after the first report frame time by the image sensor; (b) obtaining, by the image sensor, a first movement of the image sensor system during a first delay time interval, the first delay time being between a first polling time and the first report frame time, wherein the image sensor receives a first polling from the control circuit during the first polling time; (c) calculating, with the image sensor, a second movement of the image sensing system for a first calibration time interval, wherein the first calibration time interval includes a first frame time interval between the first report frame time and the second report frame time, and includes a first expected delay time interval after the second report frame time; and (d) outputting a first output movement to the control circuit with the image sensor so that a difference between the first output movement and a physical movement of the image sensing system is less than a predetermined value.

22. The image sensing system control method according to claim 21, characterized in that: The step (d) outputs the first output movement in response to a second polling, wherein the second polling is generated by the control circuit and received by the image sensor at a second polling time, and a time difference between the second polling time and the second report frame time is the first expected delay time interval.

23. The image sensing system control method according to claim 21, characterized in that: The second movement is generated according to the following equation: And MT_PI×DPI_C=MT_2 Wherein, MRP1 is the first frame time interval and T_LTE1 is the first expected delay time interval, DPI is a predetermined DPI value, MT_PT is the movement per inch of the image sensor system and MT_2 is the second movement.

24. The image sensing system control method according to claim 23, characterized in that: MT_PI is generated according to the following equation: Where PIX_C is the accumulated movement of pixels and PIX_N is the number of pixels per inch.

25. The image sensing system control method according to claim 21, wherein: The method further comprises the following steps before step (a): defining a third report frame sensed by the image sensor at a third report frame time before the first report frame time; Obtaining a third movement of the image sensor system in a second delay time interval with the image sensor, the second delay time being between a third polling time and the third report frame time, wherein the image sensor receives a third polling from the control circuit during the third polling time; calculating, with the image sensor, a fourth movement of the image sensing system in a second calibration time interval, wherein the second calibration time interval includes a second frame time interval between the third report frame time and the first report frame time, and includes a second expected delay time interval after the first report frame time; subtracting the third movement from the fourth movement with the image sensor to produce a second output movement; and The image sensor calculates a fifth movement of the second expected delay time interval as the first movement.

26. The image sensing system control method according to claim 25, characterized in that: The second movement is generated according to the following equation: And MT_PI×DPI_L=MT_5 Wherein, MRP2 is the second frame time interval and T_LTE2 is the second expected delay time interval, DPI is a predetermined DPI value, MT_PT is the movement per inch of the image sensor system and MT_5 is the fifth movement.

27. The image sensing system control method according to claim 26, wherein: MT_PI is generated according to the following equation: Where PIX_C is the accumulated movement of pixels and PIX_N is the number of pixels per inch.

28. The image sensing system control method according to claim 26, wherein: The image sensing system includes a light source, and the image sensor senses frames at a frame rate, which is synchronized with the light emission frequency of the light source.

29. The image sensing system control method according to claim 21, wherein: The image sensing system is an optical mouse.

30. An image sensing system, characterized in that: include: Control circuit; as well as Image sensors to perform: (a) defining a first report frame sensed at a first report frame time, and defining a second report frame sensed at a second report frame time after the first report frame time; (b) obtaining a first movement of the image sensor system during a first delay time interval, the first delay time being between a first polling time and the first report frame time, wherein the image sensor receives a first polling from the control circuit during the first polling time; (c) calculating a second movement of the image sensor system for a first correction time interval, wherein the first correction time interval includes a first frame time interval between the first report frame time and the second report frame time, and includes a first expected delay time interval after the second report frame time; and (d) outputting a first output movement to the control circuit such that a difference between the first output movement and a physical movement of the image sensing system is less than a predetermined value.

31. The image sensing system of claim 30, wherein: The step (d) outputs the first output movement in response to a second polling, wherein the second polling is generated by the control circuit and received by the image sensor at a second polling time, and a time difference between the second polling time and the second report frame time is the first expected delay time interval.

32. The image sensing system of claim 30, wherein: The second movement is generated according to the following equation: And MT_PI×DPI_C=MT_2 Wherein, MRP1 is the first frame time interval and T_LTE1 is the first expected delay time interval, DPI is a predetermined DPI value, MT_PT is the movement per inch of the image sensor system and MT_2 is the second movement.

33. The image sensing system of claim 32, wherein: MT_PI is generated according to the following equation: Where PIX_C is the accumulated movement of pixels and PIX_N is the number of pixels per inch.

34. The image sensing system of claim 30, wherein: The image sensor further performs the following steps in step (a): defining a third report frame sensed at a third report frame time prior to the first report frame time; Obtaining a third movement of the image sensor system during a second delay time interval, the second delay time being between a third polling time and the third report frame time, wherein the image sensor receives a third polling from the control circuit during the third polling time; Calculating a fourth movement of the image sensor system for a second correction time interval, wherein the second correction time interval includes a second frame time interval between the third report frame time and the first report frame time, and includes a second expected delay time interval after the first report frame time; subtracting the third movement from the fourth movement to produce a second output movement; and A fifth movement of the second expected delay time interval is calculated as the first movement.

35. The image sensing system of claim 34, wherein: The second movement is generated according to the following equation: And MT_PI×DPI_L=MT_5 Wherein, MRP2 is the second frame time interval and T_LTE2 is the second expected delay time interval, DPI is a predetermined DPI value, MT_PT is the movement per inch of the image sensor system and MT_5 is the fifth movement.

36. The image sensing system of claim 35, wherein: MT_PI is generated according to the following equation: Where PIX_C is the accumulated movement of pixels and PIX_N is the number of pixels per inch.

37. The image sensing system of claim 30, wherein: The image sensing system includes a light source, and the image sensor senses frames at a frame rate, which is synchronized with the light emission frequency of the light source.

38. The image sensing system of claim 30, wherein: The image sensing system is an optical mouse.

Images