Frame rate control method

Abstract

The invention discloses a frame rate control method, which belongs to the field of image acquisition, and comprises the step of adjusting the frame rate of a next frame image according to the motion state, speed and pixel change rate of a current frame image relative to a previous frame image. The motion state comprises integral movement and non-integral movement; firstly, whether overall movement occurs is judged; when the movement is overall movement, determining the initial frame rate of the next frame of image according to the speed of the current frame of image, and then determining the final frame rate of the next frame of image according to the pixel change rate of the current frame of image relative to the previous frame of image and the initial frame rate of the next frame of image; and if not, determining the final frame rate of the next frame of image directly according to the pixel change rate of the current frame of image relative to the previous frame of image. According to the method, the frame rate of the next frame of image can be dynamically adjusted only through simple calculation and judgment, good adaptability is achieved for a dynamic scene, and the collection quality of the next frame of image can be ensured so that back-end processing can be facilitated.

Description

A frame rate control method Technical Field

[0001] This invention relates to the field of image technology, and in particular to a frame rate control method. Background Technology

[0002] Currently, most image acquisition devices (or modules) acquire image information at a fixed frame rate, such as 30Hz, 50Hz, or 120Hz, or by using backend interpolation to generate fake image frames to change the frame rate. Setting the frame rate too high will lead to unnecessary power consumption, while setting it too low will affect the performance of subsequent image processing algorithms.

[0003] In addition, existing frame rate control methods, when applied to dynamic scenes (where the captured images are changing or moving), result in poor image acquisition quality. This is especially true for high-speed and constantly changing scenes, where existing frame rate control methods struggle to keep up with the changing speed and content, thus severely limiting the performance of machine vision modules or other application modules. Summary of the Invention

[0004] To address the problem that existing technologies struggle to dynamically set the image acquisition frame rate according to scene changes, the present invention aims to provide a frame rate control method that at least partially solves the aforementioned problem.

[0005] To achieve the above objectives, the technical solution of the present invention is: a frame rate control method, the method comprising the following steps: adjusting the frame rate of the next frame image according to the motion state, speed and pixel change rate of the current frame image relative to the previous frame image.

[0006] In some preferred embodiments, the motion state of the current frame image relative to the previous frame image includes overall movement and non-overall movement; then the step of adjusting the frame rate of the next frame image according to the motion state, speed, and pixel change rate of the current frame image relative to the previous frame image specifically includes: determining whether the motion state of the current frame image relative to the previous frame image is overall movement; when it is overall movement, then first determining the initial frame rate of the next frame image according to the speed of the current frame image, and then determining the final frame rate of the next frame image according to the pixel change rate of the current frame image relative to the previous frame image and the initial frame rate of the next frame image; when it is non-overall movement, then directly determining the final frame rate of the next frame image according to the pixel change rate of the current frame image relative to the previous frame image.

[0007] In some preferred embodiments, the step of determining the initial frame rate of the next frame image based on the speed of the current frame image specifically includes: if the speed of the current frame image is less than a minimum speed threshold, then the initial frame rate F of the next frame image is... 2AThe minimum frame rate is preset; if the speed of the current frame image is greater than the maximum speed threshold, then the initial frame rate F of the next frame image is set. 2A The preset maximum frame rate; in addition to the two cases mentioned above, the initial frame rate F of the next frame image is... 2A The acceleration G1 is the product of the frame rate F1 of the previous frame and the current acceleration G1, where the current acceleration G1 is the ratio between the velocity of the current frame and the velocity of the previous frame.

[0008] In some preferred embodiments, the step of determining the final frame rate of the next frame image based on the pixel change rate of the current frame image relative to the previous frame image and the initial frame rate of the next frame image specifically includes: if the pixel change rate of the current frame image relative to the previous frame image is less than a minimum change rate threshold, then the final frame rate F of the next frame image is... 2B The initial frame rate F of the next frame image 2A The product of (1-k); if the pixel change rate of the current frame image relative to the previous frame image is greater than the highest change rate threshold, then the final frame rate F of the next frame image is... 2B The initial frame rate F of the next frame image 2A The product of (1+k); in the other two cases, the final frame rate F of the next frame image. 2B The initial frame rate F of the next frame image 2A .

[0009] In some preferred embodiments, the step of directly determining the final frame rate of the next frame image based on the pixel change rate of the current frame image relative to the previous frame image specifically includes: if the pixel change rate of the current frame image relative to the previous frame image is less than a minimum change rate threshold, then the final frame rate F of the next frame image is determined. 2B The frame rate F1 of the previous frame is the product of (1-k); if the pixel change rate of the current frame relative to the previous frame is greater than the highest change rate threshold, then the final frame rate F of the next frame is... 2B The frame rate F1 of the previous frame is the product of (1+k); in other cases, the final frame rate F of the next frame is... 2B The frame rate F1 of the previous frame image.

[0010] In some preferred embodiments, the motion state of the current frame image relative to the previous frame image is determined by the following steps: extracting a target image block from the current frame image and recording its position coordinates (x1, y1); determining a matching image block from the previous frame image that is compatible with the target image block and recording its position coordinates (x2, y2); determining whether the position coordinates (x1, y1) of the target image block are the same as the position coordinates (x2, y2) of the matching image block. If they are the same, the motion state of the current frame image relative to the previous frame image is determined to be non-overall motion; otherwise, it is overall motion.

[0011] In some preferred embodiments, the target image patch is not limited to one segment, and the segments are distributed at intervals.

[0012] In some preferred embodiments, when the motion state of the current frame image relative to the previous frame image is an overall movement, the method further includes the following steps: determining the movement direction and distance of the current frame image relative to the previous frame image based on the position coordinates (x1, y1) of the target image block and the position coordinates (x2, y2) of the matching image block; determining the matching region between the current frame image and the previous frame image based on the movement direction and distance, wherein the matching region is the common region of the current frame image and the previous frame image.

[0013] In some preferred embodiments, within the matching region, the step of determining the final frame rate of the next frame image based on the pixel change rate of the current frame image relative to the previous frame image and the initial frame rate of the next frame image is performed.

[0014] In some preferred embodiments, the pixel change rate of the current frame image relative to the previous frame image is obtained through the following steps: First, the change rate of each pixel corresponding to the coordinate position in the current frame image and the previous frame image is calculated one by one using the formula Q=abs((P1(i,j)-P0(i,j)) / P0(i,j), where P1(i,j) is the pixel with coordinate position (i,j) in the current frame image, P0(i,j) is the pixel with coordinate position (i,j) in the previous frame image, and abs is a function used to calculate the absolute value; then, the maximum value of each pixel change rate is taken as the pixel change rate of the current frame image relative to the previous frame image.

[0015] The beneficial effects of the present invention by adopting the above technical solution are as follows: The frame rate control method provided by the present invention only needs to perform motion state judgment on the current frame image and the previous frame image and perform simple calculations on speed and pixel change rate to dynamically adjust the frame rate of the next frame image. It not only has good adaptability to dynamic scenes, but also ensures the acquisition quality of the next frame image to facilitate backend processing. Attached Figure Description

[0016] Figure 1 is a schematic diagram of the method flow of the present invention.

[0017] Figure 2 is a schematic diagram of the setting of the target image block in the current frame image in this invention.

[0018] Figure 3 is a schematic diagram of the pixels contained in a frame of an image in this invention.

[0019] Figure 4 is a schematic diagram of the matching region between the current frame image and the previous frame image in this invention. Detailed Implementation

[0020] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings. It should be noted that these descriptions are for the purpose of aiding understanding the present invention, but do not constitute a limitation thereof. Furthermore, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0021] A frame rate control method is provided for controlling the frame rate when an image acquisition device (or module) acquires image data. The method involves adjusting the frame rate of the next frame image based on the motion state, speed, and pixel change rate of the current frame image relative to the previous frame image.

[0022] The motion state of the current frame image relative to the previous frame image includes overall movement and non-overall movement. Overall movement can be the movement of the image acquisition device (or module), or the overall translation of the scene within the field of view of the image acquisition device (or module) in the same direction.

[0023] As shown in Figure 1, the frame rate of the next frame image is adjusted through the following steps: First, determine whether the motion state of the current frame image relative to the previous frame image is a global movement; if it is a global movement, first determine the initial frame rate of the next frame image based on the speed of the current frame image, and then determine the final frame rate of the next frame image based on the pixel change rate of the current frame image relative to the previous frame image and the initial frame rate of the next frame image; if it is not a global movement, directly determine the final frame rate of the next frame image based on the pixel change rate of the current frame image relative to the previous frame image.

[0024] In the case of overall movement, the initial frame rate of the next frame is determined by the following three cases: 1) If the speed S2 of the current frame is less than the minimum speed threshold STH1, then the initial frame rate F of the next frame is determined by the following three cases: 2A The preset minimum frame rate is FTH1.

[0025] 2) If the velocity S2 of the current frame image is greater than the maximum velocity threshold STH2, then the initial frame rate F of the next frame image is... 2AThe preset maximum frame rate is THH2.

[0026] 3) In addition to the two cases mentioned above, the initial frame rate F of the next frame image 2A It is the product of the frame rate F1 of the previous frame and the current acceleration G1, where the current acceleration G1 is the ratio between the velocity S2 of the current frame and the velocity S1 of the previous frame.

[0027] Among them, the speed S2 of the current frame image refers to the motion speed of the current frame image relative to the previous frame image, the speed S1 of the previous frame image refers to the motion speed of the previous frame image relative to the frame before that, and the minimum speed threshold STH1, the maximum speed threshold STH2, the preset minimum frame rate FTH1, and the preset maximum frame rate THH2 are all preset values.

[0028] After the overall movement and the initial frame rate F of the next frame image was determined. 2A In this case, the final frame rate of the next frame image is determined by the following three situations: 1) If the pixel change rate of the current frame image relative to the previous frame image is less than the minimum change rate threshold, then the final frame rate F of the next frame image is determined by the following three situations: 2B The initial frame rate F of the next frame image 2A The product of (1-k).

[0029] 2) If the pixel change rate of the current frame image relative to the previous frame image is greater than the highest change rate threshold, then the final frame rate F of the next frame image is... 2B The initial frame rate F of the next frame image 2A The product of (1+k).

[0030] 3) In addition to the two cases mentioned above, the final frame rate F of the next frame image is... 2B The initial frame rate F of the next frame image 2A .

[0031] In the case of non-overall movement, the final frame rate of the next frame is determined directly by the following three cases: 1) If the pixel change rate Q2 of the current frame relative to the previous frame is less than the minimum change rate threshold QTH1, then the final frame rate F of the next frame is determined. 2B It is the product of the frame rate F1 of the previous frame and (1-k).

[0032] 2) If the pixel change rate Q2 of the current frame image relative to the previous frame image is greater than the maximum change rate threshold QTH2, then the final frame rate F of the next frame image is... 2B It is the product of the frame rate F1 of the previous frame and (1+k).

[0033] 3) In addition to the two cases mentioned above, the final frame rate F of the next frame image is... 2BF1 is the frame rate of the previous frame.

[0034] Among them, the frame rate F1 of the previous frame is known, the minimum rate of change threshold QTH1 and the maximum rate of change threshold QTH2 are preset values, and k is a pre-set constant, for example, the value range is [1 / 32~1 / 4].

[0035] In this embodiment, the motion state of the current frame image relative to the previous frame image is determined by the following steps: First, the target image block is extracted from the current frame image and its position coordinates (x1, y1) are recorded.

[0036] Then, a matching image patch that fits the target image patch is determined from the previous frame image, and its position coordinates (x2, y2) are recorded.

[0037] Finally, it is determined whether the position coordinates (x1, y1) of the target image block are the same as the position coordinates (x2, y2) of the matching image block. If they are the same, the motion state of the current frame image relative to the previous frame image is determined to be non-overall movement; otherwise, it is overall movement.

[0038] The size of the target image patch can be a preset value, such as 3×3 or 4×4. The position of the target image patch in the current frame can be the center or another position near the center. Usually, because the interval between two adjacent frames is short, there will also be a matching image patch in the previous frame that fits the target image patch. Matching is usually performed using a similarity algorithm.

[0039] It's easy to understand that, to prevent special cases, the target image patch in the current frame is usually cropped in multiple ways, and they are spaced apart. For example, the image is divided into several quadrants with the center point as the origin, and one target image patch is taken from each quadrant. All target image patches are located on the same circle centered at the origin, as shown in Figure 2. For overall movement, the positional coordinate differences between each target image patch and its corresponding matching image patch are the same. When the positional coordinate differences between most (e.g., more than 90%) target image patches and their corresponding matching image patches are the same, the movement state of the current frame image relative to the previous frame image is considered to be overall movement.

[0040] In this embodiment, the pixel change rate Q2 of the current frame image relative to the previous frame image is obtained through the following steps: First, the change rate of each pixel corresponding to the coordinate position in the current frame image and the previous frame image is calculated one by one using the formula Q=abs((P1(i,j)-P0(i,j)) / P0(i,j), where P1(i,j) is the pixel with coordinate position (i,j) in the current frame image, P0(i,j) is the pixel with coordinate position (i,j) in the previous frame image, and abs is a function used to calculate the absolute value; then, the maximum value of each pixel change rate is taken as the pixel change rate of the current frame image relative to the previous frame image.

[0041] Figure 3 shows a frame of an image containing a total of m×n pixels.

[0042] It's easy to understand that the motion state of the current frame relative to the previous frame is an overall movement, which is determined by the pixel change rate Q2 of the current frame relative to the previous frame and the initial frame rate F of the next frame. 2A Determine the final frame rate F of the next frame image. 2B When calculating Q2, only the matching region between the current frame image and the previous frame image is considered. The matching region between the current frame image and the previous frame image is the common area of ​​both images, as shown in Figure 4. This matching region is determined through the following steps: First, based on the position coordinates (x1, y1) of the target image block and the position coordinates (x2, y2) of the matching image block, the movement direction and distance of the current frame image relative to the previous frame image are determined; then, based on this movement direction and distance, the matching region between the current frame image and the previous frame image is determined.

[0043] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and these variations still fall within the protection scope of the present invention.

Claims

1. A frame rate control method, characterized in that: The method includes the following steps: adjusting the frame rate of the next frame image based on the motion state, speed, and pixel change rate of the current frame image relative to the previous frame image.

2. The method according to claim 1, characterized in that: The motion state of the current frame image relative to the previous frame image includes overall movement and non-overall movement; therefore, the step of adjusting the frame rate of the next frame image based on the motion state, speed, and pixel change rate of the current frame image relative to the previous frame image specifically includes: determining whether the motion state of the current frame image relative to the previous frame image is overall movement; when it is overall movement, firstly determine the initial frame rate of the next frame image based on the speed of the current frame image, and then determine the final frame rate of the next frame image based on the pixel change rate of the current frame image relative to the previous frame image and the initial frame rate of the next frame image; when it is non-overall movement, directly determine the final frame rate of the next frame image based on the pixel change rate of the current frame image relative to the previous frame image.

3. The method according to claim 2, characterized in that: The step of determining the initial frame rate of the next frame image based on the speed of the current frame image specifically includes: if the speed of the current frame image is less than a minimum speed threshold, then the initial frame rate F of the next frame image is... 2A The minimum frame rate is preset; if the speed of the current frame image is greater than the maximum speed threshold, then the initial frame rate F of the next frame image is set. 2A The preset maximum frame rate; in addition to the two cases mentioned above, the initial frame rate F of the next frame image is... 2A The acceleration G1 is the product of the frame rate F1 of the previous frame and the current acceleration G1, where the current acceleration G1 is the ratio between the velocity of the current frame and the velocity of the previous frame.

4. The method according to claim 3, characterized in that: The step of determining the final frame rate of the next frame image based on the pixel change rate of the current frame image relative to the previous frame image and the initial frame rate of the next frame image specifically includes: if the pixel change rate of the current frame image relative to the previous frame image is less than a minimum change rate threshold, then the final frame rate F of the next frame image is... 2B The initial frame rate F of the next frame image 2A The product of (1-k); if the pixel change rate of the current frame image relative to the previous frame image is greater than the highest change rate threshold, then the final frame rate F of the next frame image is... 2B The initial frame rate F of the next frame image 2A The product of (1+k); in the other two cases, the final frame rate F of the next frame image. 2B The initial frame rate F of the next frame image 2A .

5. The method according to claim 2, characterized in that: The step of directly determining the final frame rate of the next frame image based on the pixel change rate of the current frame image relative to the previous frame image specifically includes: if the pixel change rate of the current frame image relative to the previous frame image is less than a minimum change rate threshold, then the final frame rate F of the next frame image is... 2B The frame rate F1 of the previous frame is the product of (1-k); if the pixel change rate of the current frame relative to the previous frame is greater than the highest change rate threshold, then the final frame rate F of the next frame is... 2B The frame rate F1 of the previous frame is the product of (1+k); in other cases, the final frame rate F of the next frame is... 2B The frame rate F1 of the previous frame image.

6. The method according to claim 2, characterized in that: The following steps are used to determine whether the motion state of the current frame image relative to the previous frame image is a whole-frame movement: Extract a target image block from the current frame image and record its position coordinates (x1, y1); determine a matching image block from the previous frame image that is compatible with the target image block and record its position coordinates (x2, y2); determine whether the position coordinates (x1, y1) of the target image block are the same as the position coordinates (x2, y2) of the matching image block. If they are the same, the motion state of the current frame image relative to the previous frame image is determined to be a non-whole-frame movement; otherwise, it is a whole-frame movement.

7. The method according to claim 6, characterized in that: The target image patch is not limited to one segment, and the segments are distributed at intervals.

8. The method according to claim 6, characterized in that: When the motion state of the current frame image relative to the previous frame image is an overall movement, the method further includes the following steps: determining the movement direction and distance of the current frame image relative to the previous frame image based on the position coordinates (x1, y1) of the target image block and the position coordinates (x2, y2) of the matching image block; determining the matching region between the current frame image and the previous frame image based on the movement direction and distance, wherein the matching region is the common region of the current frame image and the previous frame image.

9. The method according to claim 8, characterized in that: Within the matching region, the step of determining the final frame rate of the next frame image based on the pixel change rate of the current frame image relative to the previous frame image and the initial frame rate of the next frame image is performed.

10. The method according to claim 1, characterized in that: The pixel change rate of the current frame image relative to the previous frame image is obtained through the following steps: First, the change rate of each pixel corresponding to the coordinate position in the current frame image and the previous frame image is calculated one by one using the formula Q=abs((P1(i,j)-P0(i,j)) / P0(i,j), where P1(i,j) is the pixel with coordinate position (i,j) in the current frame image, P0(i,j) is the pixel with coordinate position (i,j) in the previous frame image, and abs is a function used to calculate the absolute value; then, the maximum value of each pixel change rate is taken as the pixel change rate of the current frame image relative to the previous frame image.

Images