Image anti-shake method, device and equipment based on gyroscope and medium

Abstract

The embodiment of the invention provides an image anti-shake method and device based on a gyroscope, equipment and a medium, and belongs to the technical field of image processing, and the method specifically comprises the following steps: carrying out correction operation on gyroscope data according to an original image; obtaining an original motion track of the image capturing device according to the corrected gyroscope data; processing the original motion track according to noise to obtain a corrected image; and judging whether the corrected image meets an output requirement or not according to a preset rule, and adjusting the noise according to a judgment result until the corrected image meeting the output requirement is obtained. According to the scheme of the invention, the black edge detection is introduced to judge whether the corrected image has the black edge, the black edge constraint is triggered, the performance of the Kalman filter is effectively controlled and the black edge of the corrected image is removed only by adjusting the process noise or the measurement noise, and the black edge of the corrected image does not need to be removed by introducing additional post-processing. And a good anti-shake effect is still achieved in a low-illumination scene or a scene with a large-area moving object in a picture.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of image processing technology, and in particular to a gyroscope-based image stabilization method, device, equipment, and medium. Background Art

[0002] At present, gyro anti-shake technology mainly includes gyro data correction, motion smoothing, motion compensation and other modules. Gyro data correction is to correct the zero bias, temperature drift, crosstalk between axes, coordinate axis and delay of gyro angular velocity data to obtain corrected gyro data; motion smoothing is to smooth and filter the corrected gyro motion information to generate new smooth motion information; motion compensation is to use the original motion information and the smoothed motion information to obtain compensation information and correct the input image.

[0003] Since digital image stabilization relies on feature points in the image to estimate the camera's motion, the low signal-to-noise ratio of the image under low illumination makes it impossible to extract robust feature points, and thus it is impossible to perform accurate motion estimation. In addition, when there are large-scale moving objects in the scene, digital image stabilization can only estimate the motion of the objects in the scene, rather than the actual motion of the camera, resulting in a poor stabilization effect. Since the gyroscope is not affected by the ambient illumination and can directly reflect the movement of the camera, the gyroscope stabilization still has a good stabilization effect in low illumination or when there are large-scale moving objects in the picture.

[0004] Based on the real-time requirements of gyroscope-based image stabilization, motion smoothing usually adopts a zero-delay Kalman filter method. Kalman filtering has two parameters that affect the filter strength, namely process noise Q and measurement noise R. One of the existing technologies is based on a fixed filter strength parameter, which results in the smoothing strength not being able to be adaptively adjusted according to the degree of jitter, limiting the anti-shake effect, and the corrected image may have black edges, requiring additional overhead to be introduced for post-processing to remove the black edges. The other is to adjust the process noise Q and measurement noise R at the same time, but the adjustment of the two parameters makes it difficult to control the filter performance, affecting the anti-shake effect.

[0005] It can be seen that there is an urgent need for an image stabilization method that can remove the black edges of the corrected image while effectively controlling the filter performance, improving the video stabilization effect, and still have a good stabilization effect in low illumination scenes or scenes with large-area moving objects in the picture. Summary of the invention

[0006] In view of this, the embodiments of the present disclosure provide a gyroscope-based image stabilization method, device, equipment and medium, which can remove the black edges of the corrected image while effectively controlling the filter performance, thereby improving the image stabilization effect, and still have a good stabilization effect in low illumination or scenes with large-area moving objects in the picture.

[0007] In a first aspect, an embodiment of the present disclosure provides a gyroscope-based image stabilization method, comprising:

[0008] Performing a correction operation on the gyroscope data according to the original image to obtain corrected gyroscope data;

[0009] Obtaining an original motion trajectory of the image capture device according to the corrected gyroscope data;

[0010] Processing the original motion trajectory according to the noise to obtain a corrected image; wherein the noise is either process noise or measurement noise;

[0011] It is determined whether the corrected image meets the output requirement according to a preset rule, and the noise is adjusted according to the determination result until a corrected image that meets the output requirement is obtained.

[0012] According to a specific implementation of the embodiment of the present disclosure, judging whether the corrected image meets the output requirement according to a preset rule, and adjusting the noise according to the judgment result until a corrected image that meets the output requirement is obtained includes:

[0013] According to the preset rule, it is determined whether there is a black edge between the corrected image and the preset output image, and the noise is adjusted according to the determination result until a corrected image that meets the output requirement is obtained.

[0014] According to a specific implementation of the embodiment of the present disclosure, judging whether there is a black edge between the corrected image and the preset output image according to the preset rule, and adjusting the noise according to the judgment result until a corrected image that meets the output requirement is obtained, includes:

[0015] Determining whether there is a black edge between the corrected image and the preset output image according to the preset rule;

[0016] If it exists, the noise is adjusted to obtain the latest noise, the corrected image is obtained using the latest noise, and the process returns to the step of determining whether there is a black edge between the corrected image and the preset output image according to the preset rules; otherwise, the corrected image is determined to meet the output requirements and is output.

[0017] According to a specific implementation of the embodiment of the present disclosure, judging whether the corrected image meets the output requirement according to a preset rule, and adjusting the noise according to the judgment result until a corrected image that meets the output requirement is obtained includes:

[0018] Determine whether the noise reaches a first threshold, and if so, determine that the corrected image meets the output requirement and output it.

[0019] According to a specific implementation of the embodiment of the present disclosure, judging whether the corrected image meets the output requirement according to a preset rule, and adjusting the noise according to the judgment result until a corrected image that meets the output requirement is obtained includes:

[0020] It is determined whether the number of iterations reaches a second threshold value, and if so, it is determined that the corrected image meets the output requirement and is output.

[0021] According to a specific implementation manner of the embodiment of the present disclosure, the preset rule includes determining whether the coordinates of four image vertices of the corrected image exceed the boundaries of a preset output image.

[0022] According to a specific implementation method of an embodiment of the present disclosure, the correction operation includes performing one or more of zero bias and temperature drift correction operations based on the data after the gyroscope is powered on, performing gyroscope sensitivity and inter-axis crosstalk correction operations, performing coordinate axis synchronization operations of the gyroscope and the image capture device, or performing time synchronization operations of the gyroscope and the image capture device.

[0023] In a second aspect, an embodiment of the present disclosure provides an image stabilization device based on a gyroscope, comprising:

[0024] A data correction module, used for performing correction operation on the gyroscope data according to the original image to obtain corrected gyroscope data;

[0025] A motion estimation module, used for obtaining an original motion trajectory of the image capture device according to the corrected gyroscope data;

[0026] A motion smoothing module processes the original motion trajectory according to noise to obtain a corrected image; wherein the noise is either process noise or measurement noise;

[0027] The image correction module is used to determine whether the corrected image meets the output requirements according to preset rules, and adjust the noise according to the judgment result until a corrected image that meets the output requirements is obtained.

[0028] In a third aspect, an embodiment of the present disclosure further provides an electronic device, the electronic device comprising:

[0029] at least one processor; and,

[0030] a memory communicatively connected to the at least one processor; wherein,

[0031] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the gyroscope-based image stabilization method in the aforementioned first aspect or any implementation of the first aspect.

[0032] In a fourth aspect, an embodiment of the present disclosure further provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable the computer to execute the gyroscope-based image stabilization method in the aforementioned first aspect or any implementation of the first aspect.

[0033] In a fifth aspect, an embodiment of the present disclosure further provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer executes the gyroscope-based image stabilization method in the aforementioned first aspect or any implementation of the first aspect.

[0034] The gyroscope-based image stabilization scheme in the disclosed embodiment includes: performing a correction operation on the gyroscope data according to the original image to obtain corrected gyroscope data; obtaining the original motion trajectory of the image capture device according to the corrected gyroscope data; processing the original motion trajectory according to noise to obtain a corrected image; wherein the noise is either process noise or measurement noise; judging whether the corrected image meets the output requirements according to preset rules, and adjusting the noise according to the judgment result until the corrected image that meets the output requirements is obtained.

[0035] The beneficial effects of the embodiments of the present disclosure are as follows: through the scheme of the present disclosure, black edge detection is introduced to determine whether black edges appear in the corrected image, black edge constraints are triggered, and the Kalman filter performance is effectively controlled only by adjusting process noise or measurement noise, and the black edges of the corrected image are removed at the same time. There is no need to introduce additional post-processing to remove the black edges of the corrected image, and the anti-shake effect is improved at the same time; in addition, considering the real-time requirements of video stabilization, the noise update will be stopped when the number of iterations exceeds a certain upper limit; the gyroscope-based image stabilization method still has a good anti-shake effect in low illumination or scenes with large-area moving objects in the picture. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0037] Figure 1 A schematic diagram of motion smoothing provided by an embodiment of the present disclosure;

[0038] Figure 2 A schematic diagram of an anti-shake image correction provided by an embodiment of the present disclosure;

[0039] Figure 3A schematic diagram of a flow chart of a gyroscope-based image stabilization method provided in an embodiment of the present disclosure;

[0040] Figure 4 A schematic diagram of motion smoothing adaptive adjustment provided by an embodiment of the present disclosure;

[0041] Figure 5 A schematic diagram of a camera and gyroscope coordinate axis provided in an embodiment of the present disclosure;

[0042] Figure 6 A schematic diagram of the alignment of coordinate axes of a camera and a gyroscope provided in an embodiment of the present disclosure;

[0043] Figure 7 A schematic diagram of the inconsistency between the coordinate axes of a camera and a gyroscope provided in an embodiment of the present disclosure;

[0044] Figure 8 A schematic diagram of motion information output by an image estimation and gyroscope provided in an embodiment of the present disclosure;

[0045] Fig. 9 A schematic diagram of a cross-correlation and delay result provided by an embodiment of the present disclosure;

[0046] Fig.10 A schematic diagram of a given output size provided by an embodiment of the present disclosure;

[0047] Fig.11 A schematic diagram of motion trajectory smoothing under different process noises provided by an embodiment of the present disclosure;

[0048] Fig.12 A schematic diagram of the structure of a gyroscope-based image stabilization device provided in an embodiment of the present disclosure;

[0049] Fig.13 A schematic diagram of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0050] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0051] The following describes the embodiments of the present disclosure through specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present disclosure.

[0052] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein may be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present disclosure, it should be understood by those skilled in the art that an aspect described herein may be implemented independently of any other aspect, and two or more of these aspects may be combined in various ways. For example, any number of aspects described herein may be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein may be used to implement this device and / or practice this method.

[0053] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present disclosure. The drawings only show components related to the present disclosure rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0054] Additionally, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects described may be practiced without these specific details.

[0055] Gyro anti-shake technology mainly includes gyro data correction, motion smoothing, motion compensation and other modules. Gyro data correction is to correct the zero bias, temperature drift, crosstalk, coordinate axis and delay of the gyro angular velocity data to obtain the corrected gyro data; motion smoothing is to smooth and filter the corrected gyro motion information to generate a new smooth motion information, such as Figure 1 As shown; motion compensation uses the original motion information and the smoothed motion information to obtain compensation information and correct the input image.

[0056] Since digital image stabilization relies on feature points in the image to estimate the camera's motion, the low signal-to-noise ratio of the image in low illumination makes it impossible to extract robust feature points, and thus it is impossible to perform accurate motion estimation. In addition, when there are large-scale moving objects in the scene, digital image stabilization can only estimate the motion of the objects in the scene, rather than the actual motion of the camera, resulting in a poor stabilization effect. Since the gyroscope is not affected by the ambient illumination and can directly reflect the movement of the camera, the gyroscope stabilization still has a good stabilization effect in low illumination or scenes with large-scale moving objects in the picture.

[0057] Based on the real-time requirements of gyroscope anti-shake, motion smoothing usually adopts zero-delay Kalman filtering method. Kalman filtering has two parameters that affect the filter strength, namely process noise Q and measurement noise R. One of the existing technologies is based on a fixed filter strength parameter, which results in the smoothing strength not being able to be adaptively adjusted according to the degree of jitter, limiting the anti-shake effect, and the corrected image may have black edges, such as Figure 2 As shown in the figure, additional overhead needs to be introduced for post-processing to remove the black edges. Another method is to adjust the process noise Q and the measurement noise R at the same time, but the adjustment of the two parameters is difficult to control the filter performance and affects the anti-shake effect.

[0058] The embodiment of the present disclosure provides a gyroscope-based image stabilization method, which can be applied to a gyroscope stabilization process in an image processing scenario.

[0059] See also Figure 3 , is a flow chart of a gyroscope-based image stabilization method provided by an embodiment of the present disclosure. Figure 3 and Figure 4 As shown, the method mainly comprises the following steps:

[0060] S101, performing a correction operation on the gyroscope data according to the original image to obtain corrected gyroscope data;

[0061] S102, obtaining an original motion trajectory of an image capture device according to the corrected gyroscope data;

[0062] S103, processing the original motion trajectory according to the noise to obtain a corrected image; wherein the noise is either process noise or measurement noise;

[0063] S104, judging whether the corrected image meets the output requirement according to a preset rule, and adjusting the noise according to the judgment result until a corrected image that meets the output requirement is obtained.

[0064] In specific implementation, in the process of using a gyroscope to obtain motion data, the original image and gyroscope data are corrected, noise is processed, and finally a high-quality corrected image is obtained.

[0065] The process steps are as follows:

[0066] Step 1 Gyroscope data correction:

[0067] In practical applications, the image capture device may be a camera or an imaging device such as a video camera, and the original image is captured by the image capture device. The image capture device often relies on gyroscope data to obtain the motion information of the device. First, the gyroscope provides raw angular velocity data on the image capture device, which usually contains some errors or deviations. Therefore, in order to improve the accuracy of the data, the raw gyroscope data needs to be corrected.

[0068] The calibration process is usually carried out by comparing with known reference data or models. The calibration algorithm adjusts the output of the gyroscope sensor based on this data to remove the measurement error. The corrected gyroscope data can more accurately reflect the actual motion trajectory of the capture device.

[0069] Step 2, obtain the original motion trajectory:

[0070] The corrected gyroscope data can be used to calculate the motion trajectory of the image capture device. Specifically, the angular velocity information provided by the gyroscope, combined with the time series, can be used to calculate the motion path and direction change of the device in three-dimensional space. These motion trajectory data provide important reference information for subsequent image correction.

[0071] Step 3: motion trajectory smoothing:

[0072] In the actual motion trajectory data, in addition to the measurement error due to the gyroscope, there may also be noise generated by the external environment or the device itself (such as process noise or measurement noise). These noises may affect the accuracy and quality of image capture. Therefore, after obtaining the original motion trajectory, the noise needs to be processed.

[0073] The method of dealing with noise can be to use a filtering algorithm (such as adaptive Kalman filtering) or other appropriate noise suppression techniques to remove the noise component from the motion trajectory data to obtain a smoother and more accurate motion trajectory. The key to this process is to be able to distinguish between the noise component and the real motion information and effectively suppress the noise.

[0074] Step 4, corrected image generation:

[0075] After obtaining the noise-processed motion trajectory, it can be applied to the image correction process. Through the precise motion trajectory, the system can perform geometric transformation correction on the image, such as translation, rotation, etc., to correct the distortion or offset in the original image caused by the movement of the device.

[0076] A calibrated image is an optimized and corrected image that usually has better visual quality and higher accuracy. This process ensures the quality and accuracy of the image during the movement of the device.

[0077] Step 5: Determine whether the corrected image meets the output requirements:

[0078] The quality of the corrected image needs to meet certain standards, which can usually be set according to different application scenarios. Possible output requirements include image clarity, degree of distortion, and degree of match with the reference image.

[0079] The system evaluates the corrected image according to preset rules to determine whether it meets the output standards. If the image does not meet the requirements, it may be necessary to adjust the noise processing parameters, re-perform noise suppression, or further optimize the correction process of the gyroscope data.

[0080] Step 6, adjust the noise and optimize the image:

[0081] If the corrected image fails to meet the predetermined requirements during the judgment process, the system will automatically adjust the noise processing parameters, such as the parameter settings of the filter, the strength of the noise suppression algorithm, etc., and re-calibrate. Through continuous adjustments, the corrected image will reach the expected output standard.

[0082] For example, if the corrected image exhibits significant blur or unnatural distortion, it may be necessary to further increase the strength of the noise filter or adjust the correction algorithm for the gyroscope data to reduce the error.

[0083] Through the above steps, the system can extract the accurate motion trajectory from the original image and gyroscope data, process the noise, and finally generate a high-quality corrected image. The key to the whole process is the effective processing of noise and the accurate correction of gyroscope data, so as to ensure that the image quality meets the output requirements.

[0084] On the basis of the above embodiment, judging whether the corrected image meets the output requirement according to a preset rule, and adjusting the noise according to the judgment result until a corrected image that meets the output requirement is obtained, includes:

[0085] According to the preset rule, it is determined whether there is a black edge between the corrected image and the preset output image, and the noise is adjusted according to the determination result until a corrected image that meets the output requirement is obtained.

[0086] When implementing it, Figure 5As shown, according to the preset rules, the system will determine whether there is a black border between the corrected image and the given output size of the preset output image. Specifically, the system will analyze whether the image edge meets the predetermined quality standards (for example, whether there is a black border or distortion). If the judgment result is that there is a black border, the system will adjust the noise processing algorithm, calculate the motion compensation information based on the latest motion trajectory obtained by motion smoothing to optimize the motion trajectory correction, and obtain the image correction matrix based on the motion compensation information and correct the image coordinates until a corrected image that meets the preset output requirements is obtained.

[0087] For example, compensation information is obtained according to the original motion trajectory and the smoothed motion trajectory, and each frame image of the original video sequence is corrected accordingly to obtain a stabilized image.

[0088] For example, the process of correcting the anti-shake is:

[0089] 1. Original motion trajectory

[0090] Suppose there is a video whose camera displacement and rotation data for each frame are as follows (unit: pixels and degrees):

[0091] Frame 1: Translation (0,0), Rotation (0°)

[0092] Frame 2: Translation (2,1), Rotation (1°)

[0093] Frame 3: Translation (3,2), Rotation (2°)

[0094] Frame 4: Translation (5,3), Rotation (4°)

[0095] Frame 5: Translation (6,6), Rotation (5°)

[0096] 2. Smoothed motion trajectory

[0097] After Kalman filtering, the smoothed motion trajectory is obtained:

[0098] Frame 1: Translation (0,0), Rotation (0°)

[0099] Frame 2: Translation (1.5, 0.5), Rotation (0.5°)

[0100] Frame 3: Translation (3,2), Rotation (1.5°)

[0101] Frame 4: Translation (4,3), Rotation (3°)

[0102] Frame 5: Translation (5,5), Rotation (4°)

[0103] 3. Calculate compensation information

[0104] Next, calculate the compensation information, which is the difference between the original motion trajectory and the smoothed motion trajectory:

[0105] Frame 1: offset (0,0), rotate (0°)

[0106] Frame 2: offset (0.5, 0.5), rotate (0.5°)

[0107] Frame 3: offset (0,0), rotate (0.5°)

[0108] Frame 4: offset (1,0), rotate (1°)

[0109] Frame 5: Compensate (1,1), rotate (1°)

[0110] 4. Correct the original frame

[0111] According to the compensation information, an image correction matrix is ​​calculated, and the image is corrected according to the image correction matrix.

[0112] Frame 1: original frame + compensation (0,0), rotation (0°) → image remains unchanged

[0113] Frame 2: original frame + compensation (0.5, 0.5), rotation (0.5°) → image slightly translated and rotated

[0114] Frame 3: original frame + compensation (0,0), rotation (0.5°) → image slightly rotated

[0115] Frame 4: original frame + compensation (1,0), rotation (1°) → image is translated and rotated to the right

[0116] Frame 5: original frame + compensation (1,1), rotation (1°) → image is translated and rotated to the lower right

[0117] Differentiate each corresponding data in the original motion trajectory and the smoothed motion trajectory to obtain new motion information, and generate an image correction matrix based on it, where the expression of the correction matrix is:

[0118]

[0119] in, is the camera’s intrinsic parameter matrix, obtained by calibration

[0120] R = cosa + sina * n + 1 - cosa * nn_T is the rotation matrix of the camera

[0121]

[0122]

[0123]

[0124] Among them, ax represents the rotation angle around the x-axis after differentiation, ay represents the rotation angle around the y-axis after differentiation, and az represents the rotation angle around the z-axis after differentiation.

[0125] Then, the coordinates corresponding to the four vertices of the corrected image are calculated according to the correction matrix and the preset correction formula, where the correction formula is:

[0126]

[0127] LT_x_corr=LT_x*M0+LT_y*M1+M2,

[0128] LT_y_corr=LT_x*M3+LT_y*M4+M5

[0129] RT_x_corr=RT_x*M0+RT_y*M1+M2,

[0130] RT_y_corr=RT_x*M3+RT_y*M4+M5

[0131] LB_x_corr=LB_x*M0+LB_y*M1+M2,

[0132] LB_y_corr=LB_x*M3+LB_y*M4+M5

[0133] RB_x_corr=RB_x*M0+RB_y*M1+M2,

[0134] RB_y_corr=RB_x*M3+RB_y*M4+M5.

[0135] 5. Output stable image

[0136] After the above correction steps, the images of all frames are adjusted to obtain a corrected image for each frame.

[0137] Furthermore, judging whether there is a black edge between the corrected image and the preset output image according to the preset rule, and adjusting the noise according to the judgment result until a corrected image that meets the output requirement is obtained, includes:

[0138] Determining whether there is a black edge between the corrected image and the preset output image according to the preset rule;

[0139] If it exists, the noise is adjusted to obtain the latest noise, the corrected image is obtained using the latest noise, and the process returns to the step of determining whether there is a black edge between the corrected image and the preset output image according to the preset rules; otherwise, the corrected image is determined to meet the output requirements and is output.

[0140] In specific implementation, for example, in the actual application of image processing, especially in the process of video stabilization, image stitching or motion compensation, black edges often appear after image correction. Black edges are usually caused by the fact that some pixel areas cannot completely cover the original image after the image is rotated, translated or deformed, causing these areas to appear as black or transparent areas. In order to remove these black edges, a series of judgment and adjustment steps are usually required. The following is a specific embodiment, which illustrates how to judge whether there are black edges between the corrected image and the preset output image according to preset rules, and adjust the noise as needed to optimize the results.

[0141] Example: Black edge adjustment during image correction

[0142] Suppose we have an original image and a corrected image after motion compensation. After a certain rotation and translation, the corrected image may have black edges. Our goal is to determine whether there are black edges and get a new corrected image after adjusting the noise as needed.

[0143] 1. Definition of preset rules

[0144] Before we start adjusting the image, we first need to define some preset rules. Let's assume that these rules are as follows:

[0145] Rule 1: The calibrated image and the preset output image (ideal image) should have the same size and ratio.

[0146] Rule 2: If the edge area of ​​the corrected image displays black (or transparent) pixels exceeding a certain ratio, the image is considered to have black edges.

[0147] Rule 3: If the black border area of ​​the corrected image is large, you may need to adjust the noise to reduce anomalies caused by edge effects.

[0148] 2. Determine the black edge

[0149] First, we compare the rectified image with the preset output image to check whether the rectified image meets rule 1, that is, whether the size and proportion are consistent. If the size matches, then we check whether there is a black border.

[0150] We check the four edge areas of the corrected image and calculate the pixel ratio of the black area. If the area occupied by black pixels exceeds a certain threshold (such as 10%), the corrected image is considered to have black edges.

[0151] If black borders are detected, the system will record the size and position of the black borders to prepare for subsequent adjustments.

[0152] 3. Noise adjustment

[0153] Optionally, the step of adjusting the noise includes:

[0154] The process noise is increased by a first preset value to obtain the latest process noise or the measurement noise is reduced by a second preset value to obtain the latest measurement noise.

[0155] Kalman filtering has two parameters that affect the filtering strength: process noise Q and measurement noise R. Q represents the degree of confidence in the uncertainty of the filtering process. The larger the Q, the closer the filtering result is to the measured value, and vice versa. R represents the degree of confidence in the measurement noise. The larger the R, the closer the filtering result is to the predicted value, and vice versa. In general, the noise can be adjusted by fixing one of the noises and adjusting the other noise processing method, or both noises can be adjusted at the same time. In order to effectively control the performance of the Kalman filter, here only the filtering result is controlled by adjusting Q, such as Fig.11 The figures show the smooth motion trajectories of Q=0.4 and Q=0.004 respectively. Obviously, the smaller the Q, the smoother the motion trajectory and the better the anti-shake effect, but the probability of black edges appearing in the corrected image will also be greater.

[0156] For example, the initial process noise Q0 and measurement noise R0 are set to 0.004 and 0.5 respectively. When the image is corrected by black edge detection after the first motion smoothing, the process noise Q1 is adjusted to Q0+0.001. After the second motion smoothing, the corrected image still has black edges. The process noise Q2 is adjusted to Q1+0.001 until the Nth iteration process noise QN=QN-1+0.001. If there is no black edge in the corrected image, the process noise Q is stopped from being updated. The Nth smoothed motion trajectory is combined with the original motion trajectory to generate compensation information, and the corrected image after anti-shake is output.

[0157] Alternatively, the initial process noise Q0 and R0 are set to 0.004 and 0.5 respectively. When the image is corrected by black edge detection after the first motion smoothing, the process noise R1 is adjusted to R0-0.001. After the second motion smoothing, the corrected image still has black edges. The process noise R2 is adjusted to R1-0.001 until the Nth iteration process noise RN=RN-1-0.001. If the corrected image does not have black edges, the update of the measurement noise R is stopped. The Nth smoothed motion trajectory and the original motion trajectory are used to generate compensation information, and the anti-shake corrected image is output.

[0158] 4. Regenerate the corrected image

[0159] After the noise adjustment, the system will filter the motion trajectory according to the updated noise value, generate compensation information for the Nth smoothed motion trajectory and the original motion trajectory, obtain a new image correction matrix according to the compensation information, correct the image according to the latest image correction matrix, obtain the latest corrected image coordinate values, and then regenerate the latest corrected image. This process will include resampling and interpolation operations on the image to ensure that the image quality is as close as possible to the preset output image requirements.

[0160] The system will again determine whether the adjusted calibrated image still has black edges. If the black edges disappear after adjustment and the image quality meets the output requirements, it means that the current image has met the preset standards.

[0161] If black edges still exist, or the image quality does not meet expectations, the system may continue to perform further noise adjustments and image optimizations until the expected results are achieved.

[0162] 5. Output the final image

[0163] When the corrected image has been noise-adjusted and no black edges exist, the system determines that the image meets the preset output requirements. Finally, the processed and optimized image is returned as the output result.

[0164] Through the above steps, the black edge problem in the image can be effectively determined and adjusted. If the corrected image has black edges, the system will make noise adjustments, optimize the image quality, and continuously check whether it meets the output requirements. In the end, we will get an image without black edges and with the expected quality.

[0165] This processing flow is ideal for video stabilization, image stitching, and other applications that require image correction.

[0166] On the basis of the above embodiment, judging whether the corrected image meets the output requirement according to a preset rule, and adjusting the noise according to the judgment result until a corrected image that meets the output requirement is obtained, includes:

[0167] Determine whether the noise reaches a first threshold, and if so, determine that the corrected image meets the output requirement and output it.

[0168] In a specific implementation, it can be determined whether there is a black edge between the corrected image and the preset output image according to the preset rule;

[0169] If it exists, the noise is adjusted to obtain the latest noise. The process is as described in the above steps. The corrected image is obtained by using the latest noise, and it is determined whether the latest noise is greater than or equal to the first threshold. If so, it is determined that the corrected image meets the output requirement and is output. Otherwise, return to the step of determining whether there is a black edge between the corrected image and the preset output image according to the preset rule;

[0170] If not, it is determined that the corrected image meets the output requirement and is output.

[0171] Meanwhile, in specific implementation, it is also possible to first determine whether the latest noise is greater than or equal to the first threshold and then determine whether there is a black edge between the corrected image and the preset output image:

[0172] Determine whether the latest noise is greater than or equal to a first threshold; if less than the first threshold, determine whether there is a black edge between the corrected image and the preset output image according to the preset rule; if not, determine that the corrected image meets the output requirement and output it; otherwise, return to the step of adjusting the noise to obtain the latest noise, obtain the corrected image using the latest noise, and determine whether the latest noise is greater than or equal to the first threshold;

[0173] If it is greater than or equal to the first threshold, it is determined that the corrected image meets the output requirement and is output.

[0174] For example, in one embodiment, a Kalman filter algorithm can be used to smooth the motion trajectory. The implementation of the Kalman filter is shown in Table 1 below, which includes two parameters that affect the filtering strength: process noise Q and measurement noise R. Q represents the degree of trust in the uncertainty in the filtering process. The larger the Q, the closer the filtering result is to the measured value, and vice versa. R represents the degree of trust in the measurement noise. The larger the R, the closer the filtering result is to the predicted value, and vice versa. In order to effectively control the performance of the Kalman filter, only Q is adjusted to control the filtering result, such as Fig.11 The figures show the smooth motion trajectories of Q=0.4 and Q=0.004 respectively. Obviously, the smaller the Q, the smoother the motion trajectory and the better the anti-shake effect, but the probability of black edges appearing in the corrected image will also be greater.

[0175] Table 1

[0176]

[0177] The initial process noise Q0 and R0 are set to 0.004 and 0.5 respectively. After the first motion smoothing, when the image is corrected by black edge detection, the process noise Q1 is adjusted to Q0+0.001. After the second motion smoothing, the corrected image still has black edges. The process noise Q2 is adjusted to Q1+0.001. The motion trajectory is smoothed according to the adjusted noise, and compensation information is generated by combining the smoothed motion trajectory with the original motion trajectory. A graphic correction matrix is ​​generated according to the compensation information. The image is corrected according to the correction matrix to determine whether the corrected image has black edges. Until the Nth update of the process noise QN=QN-1+0.001, if the corrected image does not have black edges, the process noise Q is stopped from being updated, and the corrected image after anti-shake is output. Considering the real-time anti-shake of the gyroscope, when the process noise Q=4, the filtering result is basically the same as the original input, that is, there is almost no smoothing effect. Therefore, except for the correction image mentioned above, the update of Q is stopped when there is no black edge.

[0178] Alternatively, the initial process noises Q0 and R0 are set to 0.004 and 0.5 respectively. When the image is corrected with black edges after the first motion smoothing using black edge detection, the process noise R1 is adjusted to R0-0.001. After the second motion smoothing, the corrected image still has black edges. The process noise R2 is adjusted to R1-0.001 until the Nth iteration process noise RN is set to RN-1-0.001. If no black edges appear in the corrected image, the updating of the measurement noise R is stopped, and the corrected image after anti-shake is output.

[0179] Furthermore, judging whether the corrected image meets the output requirement according to a preset rule, and adjusting the noise according to the judgment result until a corrected image that meets the output requirement is obtained, includes:

[0180] It is determined whether the number of iterations reaches a second threshold value, and if so, it is determined that the corrected image meets the output requirement and is output.

[0181] In a specific implementation, the initial number of iterations may be set and the preset rule may be used to determine whether there is a black edge between the corrected image and the preset output image;

[0182] If it exists, the noise can be adjusted in the manner described in the above steps to obtain the latest noise (the process noise is increased by a first preset value to obtain the latest process noise or the measurement noise is reduced by a second preset value to obtain the latest measurement noise), the second preset value is added to the initial number of iterations to obtain the latest number of iterations, the corrected image is obtained using the latest noise, and it is determined whether the latest iteration value is greater than or equal to a second threshold value. If so, it is determined that the corrected image meets the output requirements and is output, otherwise, return to the step of determining whether there is a black edge between the corrected image and the preset output image according to the preset rule;

[0183] If not, the corrected image is determined to meet the output requirements and is output; it should be noted that the number of iterations is the number of times a black border is determined between the corrected image and the preset output image according to a preset rule.

[0184] Meanwhile, in specific implementation, it is also possible to first determine whether the latest iteration value is greater than or equal to the second threshold value and then determine whether there is a black edge between the corrected image and the preset output image:

[0185] Determine whether the latest iteration value is greater than or equal to a second threshold value; if less than the second threshold value, determine whether there is a black edge between the corrected image and the preset output image according to the preset rule; if not, determine that the corrected image meets the output requirement and output it; otherwise, return to adjust the noise to obtain the latest noise, add the second preset value to the initial iteration number to obtain the latest iteration number, use the latest noise to obtain the corrected image, and determine whether the latest iteration value is greater than or equal to the second threshold value;

[0186] If it is greater than or equal to the second threshold, it is determined that the corrected image meets the output requirement and is output.

[0187] For example, the number of iterations is the number of times of judging whether there is a black edge between the corrected image and the preset output image according to the preset rules. When the number of iterations exceeds N=4 / 0.001=4000, the noise is also stopped from being updated, and the correction matrix is ​​output for correction of the current frame image.

[0188] Optionally, the preset rule includes determining whether the coordinates of four image vertices of the corrected image exceed a preset boundary of the output image.

[0189] When implementing it, Fig.10 As shown, W_ori and H_ori are the width and height of the original image, W_crop and H_crop are the width and height of the output image, and the corresponding four vertex coordinates are:

[0190] LT_x=(W_ori-W_crop) / 2, LT_y=(H_ori-H_crop) / 2

[0191] RT_x=W_ori-(W_ori-W_crop) / 2, RT_y=(H_ori-H_crop) / 2

[0192] LB_x=(W_ori-W_crop) / 2, LB_y=H_ori-(H_ori-H_crop) / 2

[0193] RB_x=W_ori-(W_ori-W_crop) / 2, RB_y=H_ori-(H_ori-H_crop) / 2

[0194] The motion estimation of digital image stabilization first needs to obtain a 3x3 affine matrix based on the coordinates of the feature points of the previous and next frames and the least squares calculation, and then use gradient descent to iteratively solve the affine matrix, and finally obtain 7 types of motion information, including displacement in the x and y directions, scaling in the x and y directions, shearing in the x and y directions, and rotation angle. The three types of motion information of gyroscope stabilization are directly output by the gyroscope, including the rotation angle traj_ax around the x-axis, the rotation angle traj_ay around the y-axis, and the rotation angle traj_az around the z-axis. Gyroscope stabilization uses the original motion trajectory traj_ax, traj_ay, traj_az and the smooth motion trajectory smooth_traj_ax, smooth_traj_ay, smooth_traj_az to respectively differentiate and obtain new rotation angles ax, ay, az around the three axes, and generates an image correction matrix based on the new three motion information. in

[0195] is the camera’s intrinsic parameter matrix, obtained by calibration

[0196] R = cosa + sina * n + 1 - cosa * nn_T is the rotation matrix of the camera

[0197]

[0198]

[0199]

[0200] After obtaining the image correction matrix, the four corrected image vertices can be calculated using the correction formula The solution is as follows:

[0201] LT_x_corr=LT_x*M0+LT_y*M1+M2,

[0202] LT_y_corr=LT_x*M3+LT_y*M4+M5

[0203] RT_x_corr=RT_x*M0+RT_y*M1+M2,

[0204] RT_y_corr=RT_x*M3+RT_y*M4+M5

[0205] LB_x_corr=LB_x*M0+LB_y*M1+M2,

[0206] LB_y_corr=LB_x*M3+LB_y*M4+M5

[0207] RB_x_corr=RB_x*M0+RB_y*M1+M2,

[0208] RB_y_corr=RB_x*M3+RB_y*M4+M5

[0209] If LT_x_corr is less than 0, LT_y_corr is less than 0, RT_x_corr is greater than W_ori, RT_y_corr is less than 0, LB_x_corr is less than 0, LB_y_corr is greater than H_ori, RB_x_corr is greater than W_ori, and RB_y_corr is greater than H_ori. If any of the above conditions is met, the corrected image is judged to have black edges.

[0210] That is, if the upper left corner horizontal coordinate LT_x_corr of the corrected image is less than 0, or the upper left corner vertical coordinate LT_y_corr of the corrected image is less than 0, or the upper right corner horizontal coordinate RT_x_corr of the corrected image is greater than the set width W_ori of the output image size, or the upper right corner vertical coordinate RT_y_corr of the corrected image is less than 0, or the lower left corner horizontal coordinate LB_x_corr of the corrected image is less than 0, the lower left corner vertical coordinate LB_y_corr of the corrected image is greater than the set height H_ori of the output image size, or the lower right corner horizontal coordinate RB_x_corr of the corrected image is greater than the set width W_ori of the output image size, or the lower right corner vertical coordinate RB_y_corr of the corrected image is greater than the set height H_ori of the output image size, then it is judged that the corrected image has a black border.

[0211] Based on the above embodiments, the correction operation includes one or more of zero bias and temperature drift correction operations based on the data after the gyroscope is powered on, gyroscope sensitivity and inter-axis crosstalk correction operations based on preset calibration operations, coordinate axis synchronization operations of the gyroscope and the image capture device, or time synchronization operations of the gyroscope and the image capture device.

[0212] In specific implementation, when performing correction operations, one or more of the following can be selected, based on actual needs such as accuracy requirements, to perform zero bias and temperature drift correction operations based on the gyroscope power-on data, perform gyroscope sensitivity and inter-axis crosstalk correction operations, perform coordinate axis synchronization operations of the gyroscope and image capture device, or perform time synchronization operations of the gyroscope and image capture device.

[0213] For example, the process of gyroscope data correction can be as follows:

[0214] (1) Bias and temperature drift

[0215] Ideally, the output of the gyroscope is always 0 when it is stationary. However, due to the device characteristics, it may output a non-zero bias in reality. In addition, the zero bias of the gyroscope may be different each time it is powered on. Therefore, 1 to 5 seconds of data are collected each time it is powered on and averaged to obtain the zero bias.

[0216] The zero bias will change with the external temperature, and the relationship between the zero bias and temperature can be obtained by polynomial fitting.

[0217] BiasT=a*T 3 +b*T 2 +c*T+d

[0218] Where T is temperature, a, b, c, d are fitting coefficients

[0219] (2) Sensitivity and crosstalk between axes

[0220] The sensitivity error of the gyroscope itself causes the true value to be inconsistent with the measured value, but there is a certain proportional relationship; in addition, due to the manufacturing process and installation problems, the x / y / z axes of the gyroscope cannot be strictly orthogonal, resulting in crosstalk between axes. The six-face calibration is introduced to obtain the sensitivity coefficient s x ,s y ,s x , and the inter-axis crosstalk coefficient m xy , m xz , m yx , m yz , m zx , m zy .

[0221]

[0222] where w' x , w' y , w' z are the clockwise and counterclockwise angular velocities of the x / y / z axes output by the gyroscope, w x , w y , w z Provides clockwise and counterclockwise angular velocity of x / y / z axis for high precision turntable.

[0223] (3) Coordinate axis synchronization

[0224] The gyroscope and camera have their own coordinate axes such as Figure 5 As shown, when the two are installed, there may be inconsistencies in the coordinate axes. It is necessary to use a 3x3 transformation matrix to transform the coordinate axes of the gyroscope data to make it consistent with the camera coordinate axes.

[0225] For example:

[0226] When the gyro is installed in a position such as Figure 6 As shown in the figure, the coordinate axes of the gyroscope and the camera are consistent. The motion data obtained by the gyroscope is The camera motion data is

[0227] When the gyro is installed in a position such as Figure 7 As shown in the figure, the coordinate axes of the gyroscope and the camera are inconsistent at this time, where the positive direction of the gyroscope x-axis is consistent with the negative direction of the camera y-axis, and the gyroscope y-axis is consistent with the camera x-axis. The motion data obtained by the gyroscope is The camera motion data is

[0228] (4) Motion synchronization

[0229] The motion data output by the gyroscope may have a certain delay with the camera movement. It is necessary to calculate the delay between the two and compensate for it to ensure that the gyroscope and the camera are synchronized in time during movement. Figure 8 The motion information of the image estimation and the motion information of the gyroscope output are used to calculate the relationship between the delay and the cross-correlation result R using the motion information of the two and the cross-correlation formula. xy τ, such as Fig. 9 As shown in Figure 1, the time delay τ0 corresponding to the maximum cross-correlation result is the time delay between the gyroscope and the camera.

[0230] R xy τ=∫x(t)y(t+τ)dt

[0231] τ0=argmax(R xy τ)

[0232] Where x(t) and y(t) are the motion information of the gyroscope and camera respectively.

[0233] The gyroscope-based image stabilization method provided in this embodiment introduces black edge detection to determine whether black edges appear in the corrected image. When the black edge constraint is triggered, the Kalman filter performance is effectively controlled and the black edges of the corrected image are effectively removed by adjusting only the process noise or the measurement noise. There is no need to introduce additional post-processing to remove the black edges of the corrected image, while improving the stabilization effect. In addition, considering the real-time requirements of video stabilization, the noise update will be stopped when the number of iterations exceeds a certain upper limit. The gyroscope-based image stabilization method still has a good stabilization effect in low illumination or scenes with large-area moving objects in the picture.

[0234] Corresponding to the above method embodiment, see Fig.12 The embodiment of the present disclosure further provides a gyroscope-based image stabilization device 120, comprising:

[0235] The data correction module 1201 is used to perform a correction operation on the gyroscope data according to the original image to obtain the corrected gyroscope data;

[0236] A motion estimation module 1202, configured to obtain an original motion trajectory of the image capture device according to the corrected gyroscope data;

[0237] A motion smoothing module 1203 processes the original motion trajectory according to noise to obtain a corrected image; wherein the noise is either process noise or measurement noise;

[0238] The image correction module 1204 is used to determine whether the corrected image meets the output requirements according to a preset rule, and adjust the noise according to the determination result until a corrected image that meets the output requirements is obtained.

[0239] Fig.12 The device shown can correspondingly execute the contents of the above method embodiment. For the parts not described in detail in this embodiment, refer to the contents recorded in the above method embodiment and will not be repeated here.

[0240] See also Fig.13 The embodiment of the present disclosure further provides an electronic device 130, which includes: at least one processor and a memory in communication with the at least one processor. The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the gyroscope-based image stabilization method in the aforementioned method embodiment.

[0241] The embodiment of the present disclosure also provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable the computer to execute the gyroscope-based image stabilization method in the aforementioned method embodiment.

[0242] The embodiments of the present disclosure also provide a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, which, when executed by a computer, enable the computer to execute the gyroscope-based image stabilization method in the aforementioned method embodiment.

[0243] Reference below Fig.13, which shows a schematic diagram of the structure of an electronic device 130 suitable for implementing the embodiment of the present disclosure. The electronic device in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Fig.13 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0244] like Fig.13 As shown, the electronic device 130 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 1301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1302 or a program loaded from a storage device 1308 to a random access memory (RAM) 1303. In the RAM 1303, various programs and data required for the operation of the electronic device 130 are also stored. The processing device 1301, the ROM 1302, and the RAM 1303 are connected to each other via a bus 1304. An input / output (I / O) interface 1305 is also connected to the bus 1304.

[0245] Typically, the following devices may be connected to the I / O interface 1305: input devices 1306 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1308 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1309. The communication device 1309 may allow the electronic device 130 to communicate wirelessly or wired with other devices to exchange data. Although the electronic device 130 with various devices is shown in the figure, it should be understood that it is not required to implement or have all the devices shown. More or fewer devices may be implemented or have alternatively.

[0246] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 1309, or installed from a storage device 1308, or installed from a ROM 1302. When the computer program is executed by the processing device 1301, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0247] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0248] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0249] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device can execute the relevant steps of the method embodiment.

[0250] Alternatively, the computer-readable medium carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device can perform the relevant steps of the method embodiment.

[0251] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0252] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0253] The units involved in the embodiments described in the present disclosure may be implemented by software or by hardware.

[0254] It should be understood that various parts of the present disclosure may be implemented in hardware, software, firmware, or a combination thereof.

[0255] The above is only a specific implementation of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present disclosure should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A gyroscope-based image stabilization method, characterized in that: include: Performing a correction operation on the gyroscope data according to the original image to obtain corrected gyroscope data; Obtaining an original motion trajectory of the image capture device according to the corrected gyroscope data; Processing the original motion trajectory according to the noise to obtain a corrected image; wherein the noise is either process noise or measurement noise; It is determined whether the corrected image meets the output requirement according to a preset rule, and the noise is adjusted according to the determination result until a corrected image that meets the output requirement is obtained.

2. The method according to claim 1, characterized in that The step of judging whether the corrected image meets the output requirement according to a preset rule, and adjusting the noise according to the judgment result until a corrected image that meets the output requirement is obtained, comprises: According to the preset rule, it is determined whether there is a black edge between the corrected image and the preset output image, and the noise is adjusted according to the determination result until a corrected image that meets the output requirement is obtained.

3. The method according to claim 2, characterized in that The determining, according to the preset rule, whether there is a black edge between the corrected image and the preset output image, and adjusting the noise according to the determination result until a corrected image that meets the output requirement is obtained, includes: Determining whether there is a black edge between the corrected image and the preset output image according to the preset rule; If it exists, the noise is adjusted to obtain the latest noise, the corrected image is obtained using the latest noise, and the process returns to the step of determining whether there is a black edge between the corrected image and the preset output image according to the preset rules; otherwise, the corrected image is determined to meet the output requirements and is output.

4. The method according to claim 1, characterized in that: The step of judging whether the corrected image meets the output requirement according to a preset rule, and adjusting the noise according to the judgment result until a corrected image that meets the output requirement is obtained, comprises: Determine whether the noise reaches a first threshold, and if so, determine that the corrected image meets the output requirement and output it.

5. The method according to claim 1, characterized in that The step of judging whether the corrected image meets the output requirement according to a preset rule, and adjusting the noise according to the judgment result until a corrected image that meets the output requirement is obtained, comprises: It is determined whether the number of iterations reaches a second threshold value, and if so, it is determined that the corrected image meets the output requirement and is output.

6. The method according to claim 1, characterized in that The preset rule includes determining whether the coordinates of four image vertices of the corrected image exceed the boundaries of a preset output image.

7. The method according to any one of claims 1 to 6, characterized in that: The correction operation includes one or more of performing zero bias and temperature drift correction operations based on the data after the gyroscope is powered on, performing gyroscope sensitivity and inter-axis crosstalk correction operations, performing coordinate axis synchronization operations of the gyroscope and the image capture device, or performing time synchronization operations of the gyroscope and the image capture device.

8. A gyroscope-based image stabilization device, characterized in that: include: A data correction module, used for performing correction operation on the gyroscope data according to the original image to obtain corrected gyroscope data; A motion estimation module, used for obtaining an original motion trajectory of the image capture device according to the corrected gyroscope data; A motion smoothing module processes the original motion trajectory according to noise to obtain a corrected image; wherein the noise is either process noise or measurement noise; The image correction module is used to determine whether the corrected image meets the output requirements according to preset rules, and adjust the noise according to the judgment result until a corrected image that meets the output requirements is obtained.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the gyroscope-based image stabilization method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the gyroscope-based image stabilization method according to any one of claims 1 to 7.

Images