Image distortion correction method for video stabilization

Abstract

The invention discloses an image distortion correction method for video stabilization, which relates to the technical field of image correction and comprises the following steps of: 1, respectively calculating a mapping relation table in an x direction and a mapping relation table in a y direction, the mapping relation table is a corresponding relation table of the distance between each pixel point and the central point in the image before distortion and the distance between each pixel point and the central point in the image after correction; step 2, based on the mapping relation table, mapping the image before distortion into a corrected image; the camera with the large FOV performs mapping correction on the image through the mapping relation table based on the image distortion correction method, so that the effect of video stabilization is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image correction, and in particular to an image distortion correction method for video stabilization. Background Art

[0002] There are generally two methods for stabilizing video images: 1) Directly use an optical lens with an anti-shake function for anti-shake. This type of camera has a built-in gyroscope and micromotor, which can dynamically adjust the lens posture through the micromotor according to the posture data of the built-in gyroscope to offset random jitter. This method works well, but it is generally expensive and easy to damage. 2) Digital anti-shake. Generally, the change in the camera posture is estimated through the gyroscope data or the change in the position of the key points of the previous and next frames, and then the displacement and rotation angle that should be compensated for each frame are calculated, and the current frame is transformed accordingly to obtain a stabilized image. This method is low-cost, high-reliability, and currently has a wider range of applications.

[0003] The essence of the digital anti-shake method is to offset the random jitter by translating and rotating the current frame. If you want a good effect, there is a potential assumption that the same target should be basically unchanged in size even though its position is different in different frames (because the camera is shaking). This assumption is valid for cameras with a small field of view (FOV). For cameras with a large FOV, the target is generally narrower in the middle of the picture and wider at the edge of the picture, so this assumption does not hold. Therefore, for cameras with a large FOV, if the existing digital anti-shake method is directly applied to the anti-shake, the effect will be greatly affected, and the video stabilization effect will not be good. Summary of the invention

[0004] Based on the technical problems existing in the background technology, the present invention proposes an image distortion correction method for video stabilization, which can ensure that in the image captured by a camera with a larger FOV, the size of the same target remains basically unchanged when it appears at different positions in the picture, thereby improving the video stabilization effect.

[0005] The present invention provides an image distortion correction method for video stabilization, comprising the following steps: Step 1: Calculate the mapping relationship table in the x direction and the y direction respectively, wherein the mapping relationship table is a corresponding relationship table of the distance between each pixel point and the center point in the image before distortion and the distance between each pixel point and the center point in the image after correction; Step 2: Based on the mapping relationship table, the image before distortion is mapped to the corrected image.

[0006] Furthermore, in step 1, the calculation process of the mapping relationship table in the x direction and the y direction is the same, wherein the corresponding formula between the distance before distortion and the distance after distortion in the mapping relationship table in the x direction is as follows:

[0007] in, is the distance between the pixel point and the center point after mapping, is the distance between the pixel and the center point before mapping, For the given coefficients of calculation, For a given angle, The approximate radius of the edge of the camera image.

[0008] Furthermore, the coefficient and approximate radius The calculation formula is as follows:

[0009]

[0010] in, is the camera’s field of view, is the focal length of the camera.

[0011] Furthermore, in step 2, based on the mapping relationship table, the image before distortion is mapped to the corrected image using the mapping function remap of OpenCV.

[0012] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned image distortion correction method when executing the computer program.

[0013] A computer-readable storage medium stores a plurality of classification programs, wherein the plurality of classification programs are used to be called by a processor and implement the above-mentioned image distortion correction method.

[0014] A person of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk, etc., various media that can store program codes. The advantages of the image distortion correction method for video stabilization provided by the present invention are that the image distortion correction method for video stabilization provided in the structure of the present invention is applicable to the anti-shake of cameras with a large FOV. For example, a portable video recorder is generally a camera with a large FOV; such cameras perform mapping correction on the image based on the setting of the mapping relationship table, thereby improving the effect of video stabilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a schematic diagram of the process of the present invention; Figure 2 It is a schematic diagram of the geometric relationship of the present invention; Figure 3 Two perspective images collected by a video recorder; Figure 4 for Figure 3 The image is corrected by the algorithm of the present invention. DETAILED DESCRIPTION

[0016] Below, the technical solution of the present invention is described in detail through specific embodiments. Many specific details are set forth in the following description to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific implementation disclosed below.

[0017] For distant targets, the camera shake can ignore the effect of translation, and it is considered that the target changes its position in the picture due to rotation. In this process, because the actual size of the target remains unchanged and the distance from the camera focus remains unchanged, the angle formed by the target edge and the camera focus remains unchanged. However, images captured by general cameras cannot do this. This embodiment is based on setting a mapping relationship table to perform mapping correction on the image, thereby improving the effect of video stabilization.

[0018] like Figures 1 to 4 As shown, the present invention proposes an image distortion correction method for video stabilization, comprising the following steps: Step 1: Calculate the mapping relationship table in the x direction and the y direction respectively, wherein the mapping relationship table is a corresponding relationship table of the distance between each pixel point and the center point in the image before distortion and the distance between each pixel point and the center point in the image after correction; Step 2: Based on the mapping relationship table, the image before distortion is mapped to the corrected image; This embodiment uses the mapping function remap of OpenCV or other similar mapping functions to map the image before distortion to the image after correction based on the mapping relationship table.

[0019] Through steps 1 and 2, this method can correct images captured by cameras with a large field of view (FOV). It can ensure that the size of the same target in images captured by such cameras remains basically unchanged when it appears at different positions in the picture, which can improve the effect of video stabilization.

[0020] Distortion occurs in the process of projecting a world point to a camera coordinate system through a lens, so the dedistortion operation is performed in the camera coordinate system. Since the world point is often projected to a normalized camera plane, the image is dedistorted on the normalized camera plane. In step 1, the calculation process of the mapping relationship table in the x direction and the y direction is the same. This embodiment takes the mapping relationship table in the x direction as an example for explanation, and the mapping relationship table in the y direction can be obtained in the same way.

[0021] like Figure 2 , take the horizontal x direction as an example to explain (the algorithm for the vertical y direction is exactly the same). O is the focus of the camera lens, A is the leftmost imaging point, F is the rightmost imaging point, E is the midpoint of AF, as the center point, the line segment OE is the focal length of the camera, recorded as f; ∠AOF is actually the field of view angle recorded as ; ∠DOE is a very small angle near the center, recorded as , Take a very small fixed value, generally less than 0.01 radians; C is an arbitrary pixel point, ∠COE is an arbitrary angle, recorded as ; ∠BOC is a very small angle next to ∠COE, the same size as ∠DOE, and also It can be seen that the angles are In this case, the length of line segment BC is greater than the length of line segment DE. This is also the reason why the size of the target at the edge of the image is larger than that at the center of the image. The ratio of the two line segments is given by formula (1) ; (1) in, is the length of line segment BC, is the length of line segment DE.

[0022] First, a mapping relationship needs to be constructed to map the value of each pixel point of the current captured image to a new position in a new image. In this mapping relationship, the pixel size of the same target in the distance remains unchanged in the picture.

[0023] Length of the entire line segment BE The ratio of the length of the line segment DE can be obtained by integrating formula (1). After integration, considering After simplifying the range of values, we can get formula (2): ; (2) Assuming that the corrected image has an approximate radius of the edge of the image for ; (3) Then the angle between the pixel point and the focus O and the center point E is ,in is the distance from the center point E before mapping. Further considering the offset of the unevenness in formula (2), the mapping relationship can be obtained: ; (4) in, is the distance between the pixel point and the center point after mapping.

[0024] Considering that it is a proportional relationship, multiply the right side of the formula by a coefficient , we get formula (5): ; (5) The main function of K is to basically align the coordinate ranges before and after mapping. The empirical formula (6) can be used: ; (6) The general distortion correction algorithm is mainly used to correct radial distortion and tangential distortion. Its effect is more to ensure that the straight line does not change its curvature, but it cannot ensure that the size of the same target remains basically unchanged when it appears in different positions of the picture.

[0025] Figure 3 The red arrow in the figure points to the building. When the resolution is 1280*720, in the left half of the image (i.e. the area where the left arrow is located), the width of the building is 52 pixels when it is in the middle. When the camera lens is deflected, the width of the building is 82 pixels when it is offset to the edge in the right half of the image (i.e. the area where the right arrow is located). It can be predicted that with such a difference in width, no matter how the offset and rotation parameters are adjusted, a good matching effect cannot be obtained.

[0026] Figure 4 This is the result after distortion correction using the algorithm of this embodiment. In the left half of the image (i.e., the area where the left arrow is located), the width of the building is 70 pixels when it is in the middle. When the camera lens is deflected, the building is offset to the edge in the right half of the image (i.e., the area where the right arrow is located), and the width is 68 pixels. The target size has basically not changed.

[0027] The image distortion correction method of this embodiment is applicable to the anti-shake of cameras with a large FOV. This embodiment is generally not applicable to cameras with a small FOV. For example, a portable video recorder is generally a camera with a large FOV. If such cameras are directly anti-shake using the existing video stabilization algorithm, the effect will be greatly affected. Practice has shown that the image captured by such cameras can be significantly improved by applying the method for distortion correction and then applying the video stabilization algorithm.

[0028] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. An image distortion correction method for video stabilization, characterized in that: The steps include: Step 1: Calculate the mapping relationship table in the x direction and the y direction respectively, wherein the mapping relationship table is a corresponding relationship table of the distance between each pixel point and the center point in the image before distortion and the distance between each pixel point and the center point in the image after correction; Step 2: Based on the mapping relationship table, the image before distortion is mapped to the corrected image.

2. The image distortion correction method for video stabilization according to claim 1, characterized in that: In step 1, the calculation process of the mapping relationship table in the x direction and the y direction is the same, wherein the corresponding formula between the distance before distortion and the distance after distortion in the mapping relationship table in the x direction is as follows: Among them, x m is the distance between the pixel point and the center point after mapping, x is the distance between the pixel point and the center point before mapping, K is the given coefficient of calculation, Δθ is the given angle, and d is the approximate radius of the edge of the camera imaging screen.

3. The image distortion correction method for video stabilization according to claim 2, characterized in that: The calculation formula of coefficient K and approximate radius d is as follows: in, is the field of view of the camera, and f is the focal length of the camera.

4. The image distortion correction method for video stabilization according to claim 1, characterized in that: In step 2, based on the mapping relationship table, the OpenCV mapping function remap is used to map the image before distortion to the corrected image.

5. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the image distortion correction method as described in any one of claims 1 to 4 is implemented.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of classification programs, and the plurality of classification programs are used to be called by a processor and execute the image distortion correction method as described in any one of claims 1 to 4.

Images