An image distortion correction method for video stabilization

Abstract

The present invention discloses an image distortion correction method for video stabilization, which relates to the technical field of image correction and includes the following steps: Step 1, calculate the mapping relationship tables in the x direction and the y direction respectively. The mapping relationship table is a corresponding relationship table between the distance between each pixel point in the pre-distortion image and the center point and the distance between each pixel point in the corrected image and the center point; Step 2, based on the mapping relationship table, map the pre-distortion image into the corrected image. A camera with a larger FOV corrects the image through the mapping relationship table based on the image distortion correction method, thereby improving the effect of video stabilization.

Description

Technical Field

[0001] The present invention relates to the technical field of image correction, and particularly to an image distortion correction method for video stabilization. Background Art

[0002] Video image stabilization methods are generally divided into two types: 1) directly using an optical lens with an anti-shake function for anti-shake. Such cameras are built-in with gyroscopes and micro-motors, which can dynamically adjust the attitude of the lens through the micro-motors according to the attitude data of the built-in gyroscope to offset random jitters. This method has good effects, but generally has high costs and is easily damaged. 2) Digital anti-shake. Generally, it estimates the change in the camera attitude through gyroscope data or the change in the position of key points between consecutive frames, and then calculates the displacement and rotation angle that should be compensated for each frame, and performs corresponding transformations on the current frame to obtain a stabilized image. This method has lower costs and higher reliability, and is currently more widely used.

[0003] The essence of the digital anti-shake method is to perform translation and rotation transformations on the current frame to offset random jitters. If good effects are to be achieved, there is a potential assumption that although the position of the same target is different in different frames (because the camera shakes), the size of the target should be basically unchanged. This assumption holds for cameras with a small Field of View (FOV). For cameras with a large FOV, generally the target is narrower in the middle of the frame and wider at the edges of the frame, so this assumption does not hold. Therefore, for cameras with a large FOV, if the existing digital anti-shake method is directly applied for anti-shake, the effects will be greatly affected and the video stabilization effect will be poor. Summary of the Invention

[0004] Based on the technical problems existing in the background art, the present invention proposes an image distortion correction method for video stabilization, which can ensure that the size of the same target is basically unchanged when it appears at different positions in the image collected by a camera with a large FOV, and improves the video stabilization effect.

[0005] An image distortion correction method for video stabilization proposed by the present invention includes the following steps:

[0006] Step 1: Calculate the mapping relation tables in the x direction and the y direction respectively. The mapping relation table is a corresponding relation table between the distance between each pixel point in the pre-distortion image and the center point and the distance between each pixel point in the post-correction image and the center point.

[0007] Step 2: Based on the mapping relation tables, map the pre-distortion image into a post-correction image.

[0008] Further, in Step 1, the calculation processes of the mapping relation tables in the x - direction and y - direction are the same. The corresponding formula between the distance before distortion and the distance after distortion in the mapping relation table in the x - direction is as follows:

[0009]

[0010] Among them, is the distance between the pixel point after mapping and the center point, is the distance between the pixel point before mapping and the center point, is the given coefficient calculated, is the given angle, is the approximate radius of the edge of the camera imaging screen.

[0011] Further, the calculation formulas of the coefficient and the approximate radius are as follows:

[0012]

[0013]

[0014] Among them, is the field - of - view angle of the camera, is the focal length of the camera.

[0015] Further, in Step 2, based on the mapping relation table, the remap function of opencv is used to map the image before distortion into the corrected image.

[0016] A computer device includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the image distortion correction method as described above.

[0017] A computer - readable storage medium stores a number of classification programs. The number of classification programs is used to be called by a processor and implement the image distortion correction method as described above.

[0018] Those of ordinary skill in the art can understand that all or part of the steps to implement the above - mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer - readable storage medium. When the program is executed, it performs the steps including the above - mentioned method embodiments; and the aforementioned storage medium includes: various media such as ROM, RAM, magnetic disks, or optical discs that can store program codes.

[0019] The advantages of an image distortion correction method for video stabilization provided by the present invention are as follows: An image distortion correction method for video stabilization provided in the structure of the present invention is applicable to camera anti-shake with a large FOV. For example, video recorders worn on the body generally belong to cameras with a large FOV; such cameras perform mapping correction on images based on a set mapping relationship table, thereby improving the effect of video stabilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a flowchart of the present invention;

[0021] Figure 2 It is a schematic diagram of the geometric relationship of the present invention;

[0022] Figure 3 They are two distant-view images collected by a certain video recorder;

[0023] Figure 4 are Figure 3 the images after correcting the images of by using the algorithm of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0024] Next, the technical solution of the present invention will be described in detail through specific embodiments. Many specific details are set forth in the following description in order to fully understand the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without departing from the connotation of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed below.

[0025] For distant targets, the influence of translation caused by the camera's jitter can be ignored, and it is considered that rotation causes the target to change its position in the picture. In this process, since the actual size of the target remains unchanged and the distance from the camera focus remains unchanged, the angle formed by the target edge and the camera focus is also unchanged. However, the images captured by general cameras cannot achieve this. This embodiment performs mapping correction on the images based on a set mapping relationship table, thereby improving the effect of video stabilization.

[0026] As Figures 1 to 4 shown, an image distortion correction method for video stabilization proposed by the present invention includes the following steps:

[0027] Step 1: Calculate the mapping relationship tables in the x direction and the y direction respectively. The mapping relationship table is a corresponding relationship table between the distance between each pixel point and the center point in the pre-distortion image and the distance between each pixel point and the center point in the corrected image;

[0028] Step 2: Based on the mapping relationship table, map the pre-distortion image into the corrected image;

[0029] Based on the mapping relation table, this embodiment uses the remap function of opencv or other similar mapping functions to map the image before distortion into the corrected image.

[0030] Through steps one to two, this method can correct the images collected by cameras with a relatively large field of view (FOV). It can ensure that for the images collected by such cameras, the size of the same target is basically unchanged when it appears at different positions in the picture, and the effect of video stabilization can be improved.

[0031] Since the distortion occurs during the process of the world point being projected onto the camera coordinate system through the lens, the undistortion operation is performed in the camera coordinate system. Since the world point is often projected onto the normalized camera plane, the image is undistorted on the normalized camera plane. In step one, the calculation processes of the mapping relation tables in the x - direction and y - direction are the same. This embodiment takes the mapping relation table in the x - direction as an example for illustration, and the mapping relation table in the y - direction can be obtained in the same way.

[0032] As Figure 2 , taking the horizontal x - direction as an example for illustration (the algorithm for the vertical y - direction is exactly the same). O is the focus of the camera lens, A is the left - most imaging point, F is the right - most imaging point, E is the mid - point of AF and serves as the center point. The line segment OE is the focal length of the camera, denoted as f; ∠AOF is actually the field of view, denoted as ; ∠DOE is a very small angle near the center, denoted as , Take a very small fixed value, generally less than 0.01 radians; C is an arbitrary pixel point, ∠COE is an arbitrary angle, denoted as ; ∠BOC is a very small angle next to ∠COE, and its size is the same as ∠DOE, which is also . It can be seen that in the case where the included angles are all , the length of the line segment BC is greater than the length of the line segment DE. This is also the reason why generally, when the target is at the edge of the image, its size is larger than when it is in the middle of the image. The ratio of the two line segments is formula (1)

[0033] ; (1)

[0034] Among them, is the length of the line segment BC, is the length of the line segment DE.

[0035] First, a mapping relation needs to be constructed to map the value of each pixel point in the currently collected image to a new position in a new image, and in this mapping relation, the pixel size of the same distant target in the picture remains unchanged.

[0036] The length of the entire line segment BE The ratio of the length to that of line segment DE can be obtained by integrating formula (1). After integration, considering the value range of, formula (2) can be obtained after simplification:

[0037] ; (2)

[0038] Assume that in the corrected image, the approximate radius of the image edge is

[0039] ; (3)

[0040] Then the angles corresponding to the pixel point, the focus O, and the center point E are , where is the distance from the center point E before mapping. Further considering canceling out the non-uniformity in formula (2), the mapping relationship can be obtained:

[0041] ; (4)

[0042] Among them, is the distance between the pixel point after mapping and the center point.

[0043] Considering it is a proportional relationship, multiply the right side of the formula by a coefficient , and formula (5) is obtained:

[0044] ; (5)

[0045] The main function of K is to basically align the coordinate ranges before and after mapping. The empirical formula (6) can be used:

[0046] ; (6)

[0047] Generally, the distortion correction algorithm mainly corrects radial distortion and tangential distortion. Its effect is more to ensure that the straight line does not become bent, but it cannot ensure that the size of the same target is basically unchanged when it appears at different positions in the image.

[0048] Figure 3 The building pointed by the red arrow in. When the resolution is 1280*720, in the left half of the image (i.e., the area where the left arrow is located), the width of the building in the middle is 52 pixels. When the camera lens deflects and causes the building to shift to the edge in the right half of the image (i.e., the area where the right arrow is located), the width is 82 pixels. It can be foreseen that no matter how the offset and rotation parameters are adjusted, such a width difference cannot obtain a good matching effect.

[0049] Figure 4This is the result after applying the algorithm of this embodiment for distortion correction. In the left half of the figure (i.e., the area where the left arrow is located), the width of the building is 70 pixels when it is in the middle. When the camera lens deflects, causing the building to shift to the edge in the right half of the figure (i.e., the area where the right arrow is located), the width is 68 pixels. The target size basically remains unchanged.

[0050] The image distortion correction method of this embodiment is applicable to camera anti-shake with a large FOV. For cameras with a small FOV, this embodiment generally may not be applied. For example, video recorders for personal use generally belong to cameras with a large FOV. If the existing video stabilization algorithm is directly applied to such cameras for anti-shake, the effect will be greatly affected. Practice has proved that applying this method to correct the distortion of the images captured by such cameras and then applying the video stabilization algorithm will significantly improve the final anti-shake effect.

[0051] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution and inventive concept of the present invention, making equivalent substitutions or changes, should be covered by the protection scope of the present invention.

Claims

1. An image distortion correction method for video stabilization, characterized in that It includes the following steps: Step 1: Calculate the mapping relationship tables in the x - direction and y - direction respectively. The mapping relationship table is a corresponding relationship table between the distance between each pixel point and the center point in the pre - distortion image and the distance between each pixel point and the center point in the corrected image; Step 2: Based on the mapping relationship table, map the pre - distortion image into a corrected image; In Step 1, the calculation processes of the mapping relationship tables in the x - direction and y - direction are the same. The corresponding formula between the pre - distortion distance and the post - distortion distance in the mapping relationship table in the x - direction is as follows: Among them, is the distance between the mapped pixel point and the center point, is the distance between the pixel point before mapping and the center point, is the calculated given coefficient, is the given included angle, is the approximate radius of the edge of the camera imaging screen; Coefficient and approximate radius The calculation formula is as follows: Among them, is the field of view angle of the camera, is the focal length of the camera.

2. The image distortion correction method for video stabilization according to claim 1, wherein In Step 2, based on the mapping relationship table, use the remap function of opencv to map the pre - distortion image into a corrected image.

3. A computer device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the image distortion correction method described in any one of claims 1 - 2.

4. A computer-readable storage medium, characterized in that, There are several classification programs stored on the computer - readable storage medium. The several classification programs are used to be called by the processor and execute the image distortion correction method described in any one of claims 1 - 2.

Images