Anti-shake method, device and electronic equipment

Abstract

The application discloses a method, device and electronic equipment for anti-shake, and belongs to the field of shake detection. The method comprises the following steps: obtaining a first shake amount detected by a gyroscope on a target device during shooting by a first camera on the target device; obtaining a target error value of the gyroscope, the target error value being obtained based on a target image frame obtained by shooting by a second camera on the target device; calibrating the first shake amount based on the target error value to obtain a target shake amount; and performing an anti-shake operation on the first camera based on the target shake amount.

Description

Technical Field

[0001] This application belongs to the field of shake detection, and specifically relates to a shake stabilization method, device and electronic device. Background Technology

[0002] When shooting with video equipment (such as digital cameras and smartphone cameras), especially when shooting handheld, camera shake is a common occurrence, which can lead to blurry images. Therefore, how to prevent camera shake during shooting is an important issue.

[0003] Related technologies typically use gyroscopes installed on camera equipment to detect camera shake during shooting, and then move the lens accordingly to achieve image stabilization. However, gyroscopes have large bias errors and are easily affected by temperature changes, resulting in poor image stabilization performance due to large gyroscope detection errors. Summary of the Invention

[0004] This application provides a stabilization method, device, and electronic device that can solve the problem of poor stabilization effect caused by large gyroscope detection errors in related technologies.

[0005] In a first aspect, embodiments of this application provide a shake stabilization method, including: During the shooting process of the first camera on the target device, the first jitter amount detected by the gyroscope on the target device is obtained; The target error value of the gyroscope is obtained, and the target error value is obtained based on the target image frame captured by the second camera on the target device; The first jitter amount is calibrated based on the target error value to obtain the target jitter amount; Based on the target jitter level, image stabilization is performed on the first camera.

[0006] Secondly, embodiments of this application provide a shake stabilization device, including: The acquisition module is used to acquire a first jitter amount detected by the gyroscope on the target device during the shooting process of the first camera on the target device; and to acquire a target error value of the gyroscope, the target error value being obtained based on the target image frame captured by the second camera on the target device. A calibration module is used to calibrate the first jitter amount based on the target error value to obtain the target jitter amount; The image stabilization module is used to perform image stabilization on the first camera based on the target shaking amount.

[0007] Thirdly, embodiments of this application provide an electronic device including a processor and a memory, wherein the memory stores programs or instructions executable on the processor, and the programs or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0008] Fourthly, embodiments of this application provide a computer-readable storage medium on which a program or instructions are stored, which, when executed, implement the steps of the method described in the first aspect.

[0009] Fifthly, embodiments of this application provide a computer program product comprising a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.

[0010] The at least one technical solution provided in the embodiments of this application can achieve the following technical effects: In this embodiment, the first jitter amount is not only based on the first jitter amount detected by the gyroscope during the shooting process of the first camera on the target device, but also calibrated by combining the target error value obtained by the target image frame captured by the second camera. This ensures that the obtained target jitter amount has high accuracy. The first camera is then stabilized using the target jitter amount with high accuracy, which solves the problem of poor stabilization effect caused by large gyroscope detection error in related technologies. Attached Figure Description

[0011] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a flowchart of a deshake method provided in an embodiment of this application; Figure 2 This is a flowchart of another anti-shake method provided in the embodiments of this application; Figure 3 This is a flowchart of another anti-shake method provided in the embodiments of this application; Figure 4 A schematic diagram of a gyroscope detecting jitter provided in an embodiment of this application; Figure 5 This application provides a schematic diagram for determining a displacement vector; Figure 6 This is a flowchart illustrating a specific anti-shake method provided in an embodiment of this application; Figure 7 This is a structural block diagram of a shake stabilization device provided in an embodiment of this application; Figure 8 This is a structural block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0013] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0014] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0015] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0016] The image stabilization method provided in this application is applied to camera image stabilization. Specifically, during the shooting process of a first camera on a target device, a first jitter amount detected by the gyroscope on the target device is obtained. Based on the target error value of the gyroscope, the first jitter amount is calibrated to obtain a target jitter amount. Based on the target jitter amount, image stabilization is performed on the first camera. The target error value is obtained based on target image frames captured by a second camera on the target device.

[0017] The image stabilization method provided in this application embodiment can be executed by a target device, wherein the target device can be a device with image capture function, such as a digital camera, camcorder, smartphone, and tablet computer.

[0018] The image stabilization method provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0019] Please see Figure 1 , Figure 1 This is a flowchart of a deshake method provided in an embodiment of this application. Figure 1 As shown, the method includes the following steps: Step 110: During the shooting process of the first camera on the target device, obtain the first jitter amount detected by the gyroscope on the target device.

[0020] In this embodiment, the target device can be an image capture device, such as a digital camera, camcorder, or smartphone. The target device has one or more cameras; for example, a multi-camera device typically includes multiple cameras such as ultra-wide-angle cameras, wide-angle cameras, and telephoto cameras, depending on the different focal lengths of the cameras. The first camera is one of the cameras on the target device, which can be selected by the target user according to their actual shooting needs. All cameras on the target device are in a turned-off state by default. By detecting the cameras that are in an active state, the camera used by the target user during the shooting process can be determined, i.e., the first camera on the target device.

[0021] In this embodiment, the target device is further equipped with a gyroscope, which can be used to detect the jitter of the target device. During the shooting process of the first camera, the gyroscope can detect the initial jitter of the target device during the shooting process. In fact, when using a gyroscope for jitter detection, the bias error of the gyroscope's jitter detection is relatively large, and this bias error changes with temperature, resulting in a large error in the jitter detection by the gyroscope and unstable jitter detection.

[0022] It should be noted that since the target device and its camera and gyroscope are all in a relatively stationary state, the jitter of the target device, the jitter of the camera on the target device, and the jitter of the gyroscope on the target device can be the same jitter.

[0023] Step 120: Obtain the target error value of the gyroscope, which is based on the target image frame captured by the second camera on the target device.

[0024] In this embodiment, the target error value may be the bias error of the gyroscope. When using a second camera on the target device to take pictures, the target image frame captured by the second camera can be obtained through the image sensor on the target device. The image sensor may be located on the second camera and is used to acquire the target image frame captured by the second camera. The target image frame may include one or more image frames.

[0025] In one embodiment of this application, the second camera and the first camera may be the same camera. When the first camera and the second camera are the same camera, during the shooting process of the camera, target image frames captured by the camera within a historical time period can be acquired, and the target error value of the gyroscope can be obtained based on the target image frames.

[0026] In one embodiment of this application, the second camera may also be a camera that was not used during the shooting process of the first camera. That is, the first camera and the second camera may be different cameras. In fact, unless otherwise specified below, the first camera and the second camera can be considered to be different cameras.

[0027] In this embodiment, the target device includes multiple cameras. When the first camera is detected to be in an active state, it can be determined that the first camera is in use, and an unused camera can be selected from the multiple cameras as the second camera. For example, the target device includes three cameras: an ultra-wide-angle camera, a wide-angle camera, and a telephoto camera. When the wide-angle camera (the first camera) is detected to be in an active state, one camera can be selected from the ultra-wide-angle camera and the telephoto camera as the second camera.

[0028] In this embodiment of the application, by selecting an unused camera as the second camera, the shake of the first camera can be calibrated while shooting with the first camera, resulting in higher accuracy in detecting the real-time shake of the first camera and higher image quality when shooting with the first camera.

[0029] In one embodiment of this application, the target error value of the gyroscope can be obtained in the following manner: when the first camera is activated, a target image frame captured by a second camera on the target device is acquired; based on the target image frame, a second jitter amount of the target device is determined; during the capture process of the second camera, a third jitter amount detected by the gyroscope on the target device is acquired; based on the second jitter amount and the third jitter amount, the target error value of the gyroscope is obtained.

[0030] In this embodiment, a second jitter amount of the target device can be determined based on the target image frame captured by the second camera. During the process of the second camera capturing the target image frame, a third jitter amount can be detected by the gyroscope on the target device. Then, the difference between the second jitter amount and the third jitter amount can be determined as the target error value of the gyroscope.

[0031] In determining the second jitter amount based on the target image frame, the second jitter amount can be determined based on the image condition of the target image frame during the process of capturing the target image frame using the second camera of the target device. For example, if the target image frame has ghosting or blurriness, the second jitter amount of the target device can be determined based on the degree of blurriness of the target image frame. Alternatively, multiple target image frames can be obtained by capturing the same entity using the second camera, and the second jitter amount of the target device can be determined based on the image differences (displacement vectors between target image frames) between the multiple target image frames.

[0032] The image stabilization method provided in this application is typically used during the shooting process of a target user. Specifically, when the first camera on the target device is detected to be activated, the second camera on the target device is turned on, and the second camera is used to capture a target image frame. During the shooting process of the second camera, the gyroscope on the target device acquires a third jitter amount. Simultaneously, the second jitter amount of the target device can be determined based on the target image frame captured by the second camera. After acquiring the second jitter amount and the third jitter amount, a target error value can be obtained based on the second jitter amount and the third jitter amount.

[0033] For example, when a target user uses the phone's camera function (e.g., the target user taps the phone's camera application and selects the first camera to start shooting), the gyroscope's jitter detection is calibrated. This calibration can be done periodically (e.g., every second) to obtain the real-time target error value for the gyroscope's jitter detection. When the target user taps to shoot (starts preparing for imaging), the gyroscope detects the jitter of the target device during this shooting process (the first jitter value), and calibrates the first jitter value based on the target error value to obtain the target jitter value of the target device. The first camera is then repositioned accordingly based on this target jitter value to achieve image stabilization.

[0034] In this embodiment of the application, since determining the second jitter amount based on the target image frame and obtaining the third jitter amount by the gyroscope are two different jitter detection methods, the target error value of the gyroscope can be determined based on the second jitter amount and the third jitter amount. The target error value can be used to calibrate the jitter amount obtained by the gyroscope, thereby further improving the accuracy of jitter detection.

[0035] In this embodiment of the application, the target device may be equipped with an optical image stabilization system. After obtaining the target error value of the gyroscope, the target error value can be updated to the optical image stabilization system in real time. The optical image stabilization system can calibrate the jitter detected by the gyroscope based on the target error value.

[0036] Step 130: Calibrate the first jitter amount based on the target error value to obtain the target jitter amount.

[0037] In this embodiment of the application, for example, the difference between the first jitter amount and the target error value can be determined as the target jitter amount.

[0038] Step 140: Based on the target jitter amount, perform image stabilization on the first camera.

[0039] In this embodiment, the first camera can be shifted based on the target jitter amount to achieve image stabilization. Since the target jitter amount is obtained by calibrating the first jitter amount based on a target error value, it is closer to the actual jitter amount of the target device than the first jitter amount. Therefore, image stabilization using the target jitter amount is more effective, resulting in more stable and clearer images captured by the first camera.

[0040] In this embodiment, the first camera being used for image stabilization can be performed using a target jitter amount. The target jitter amount is obtained after calibration based on a first jitter amount determined by the target image frame. This results in higher accuracy of the jitter amount and improves the image quality captured by the first camera.

[0041] In one embodiment of this application, the image stabilization operation includes optical image stabilization.

[0042] In the embodiments of this application, optical image stabilization (OIS) technology is a technique that compensates for image shift caused by hand tremors or other unstable factors by physically moving the lens or image sensor. OIS technology typically relies on a gyroscope to detect the amount of shaking of the target device, and then adjusts the position of the lens or sensor to maintain image stability.

[0043] The image stabilization method provided in this application can be applied to OIS technology to calibrate the jitter amount detected by the gyroscope, further improve the accuracy of gyroscope jitter detection, improve the image stabilization effect of optical image stabilization technology, and effectively improve the image stability using optical image stabilization technology.

[0044] It should be noted that the image stabilization method provided in this application embodiment can be applied not only to OIS technology, but also to other image stabilization technologies that use gyroscopes, such as Electronic Image Stabilization (EIS) technology using gyroscopes. EIS technology is a technology that compensates for image jitter through software algorithms.

[0045] Specifically, related technologies typically use a static method to calibrate the bias error of a gyroscope, and then use the calibrated gyroscope data to calibrate with the image sensor that needs to be calibrated to obtain EIS algorithm parameters. The image stabilization method provided in the embodiments of this application can be applied to the calibration process of the gyroscope in the above-mentioned electronic image stabilization technology to improve the accuracy of gyroscope jitter detection and improve the bias error of the gyroscope when using the EIS calibration algorithm in related technologies.

[0046] In this embodiment, during the shooting process of the first camera on the target device, a first jitter amount detected by the gyroscope on the target device is obtained; a target error value of the gyroscope is obtained, which is based on the target image frame captured by the second camera on the target device; the first jitter amount is calibrated based on the target error value to obtain a target jitter amount; and the first camera is stabilized based on the target jitter amount. Thus, it not only relies on the first jitter amount detected by the gyroscope during the shooting process of the first camera on the target device, but also combines the target error value obtained from the target image frame captured by the second camera to calibrate the first jitter amount, ensuring that the obtained target jitter amount has high accuracy. Using this high-accuracy target jitter amount to perform the stabilization operation on the first camera solves the problem of poor stabilization effect caused by large gyroscope detection errors in related technologies.

[0047] Please see Figure 2 , Figure 2 This is a flowchart of another anti-shake method provided in an embodiment of this application. For example... Figure 2 As shown, the method includes the following steps: Step 210: During the shooting process of the first camera on the target device, obtain the first jitter amount detected by the gyroscope on the target device.

[0048] Step 220: When the first camera is activated, acquire a target image frame captured by the second camera on the target device, wherein the target image frame includes multiple image frames.

[0049] In this embodiment, the target image frame includes multiple image frames obtained by continuously capturing the same object. If the shaking of the target device during the capture of the target image frame is ignored, the image content of the multiple image frames is the same. That is, the difference between the multiple image frames is caused by the shaking of the target device during the capture process.

[0050] Step 230: Based on multiple image frames, determine the displacement vector of the target object, wherein the target object is an object that exists in all of the multiple image frames.

[0051] In this embodiment, the target object is an object present in all of the multiple image frames. Since the differences between the multiple image frames are caused by camera shake during the shooting process, the displacement vector of the target object in the multiple image frames can be used to indirectly represent the amount of camera shake during the capture of the multiple image frames. That is, the amount of camera shake during the capture of the multiple image frames can be determined by comparing the position of the target object in the multiple image frames.

[0052] Furthermore, to avoid interference from the jitter detection process due to the movement of the target object itself, the target object can be a stationary object, such as a table, house, or tree in an image frame. Since the target object is stationary, its displacement vector across multiple image frames is caused by the jitter of the target device. By detecting the displacement vector of the target object, the jitter of the target device can be determined.

[0053] For example, in one embodiment of this application, the plurality of image frames includes adjacent first image frames and second image frames. Step 230, determining the displacement vector of the target object based on the plurality of image frames, includes: performing feature point detection on the first image frame and the second image frame to obtain feature points of the first image frame and the second image frame; determining the target object by performing feature point matching on the feature points on the first image frame and the feature points on the second image frame, wherein the target object includes the feature points that match between the first image frame and the second image frame; and determining the displacement vector of the target object based on the first position of the target object in the first image frame and the second position of the target object in the second image frame.

[0054] In this embodiment of the application, for any two adjacent image frames among the plurality of image frames, the target object can be determined and its displacement vector within the two adjacent image frames can be determined in the following manner. It should be noted that "adjacent image frames" here means that these two image frames are obtained sequentially in time, and there are no other image frames between them.

[0055] In determining the target object, feature points in two adjacent image frames can be detected first. These feature points can be points with significant features, such as corner points or edge points. Furthermore, there are no restrictions on the feature point detection algorithm used; for example, the Harris corner detection algorithm or the Scale-Invariant Feature Transform (SIFT) algorithm can be employed.

[0056] After determining the feature points of two adjacent image frames, feature point matching can be performed on these two frames. This matching is used to match feature points at the same location of the same object. It's important to note that this feature point matching is performed on the feature points of two image frames, not on the feature points within a single image frame. Specifically, descriptors of the feature points from the two image frames can be extracted first, and then the similarity between the descriptors can be calculated to match the feature points of the two image frames. The SIFT algorithm or the Oriented Fast and Rotated Brief (ORB) algorithm can be used for descriptor extraction.

[0057] After feature point matching is performed on the feature points of two adjacent image frames, matching feature point pairs of the two adjacent image frames can be obtained. Based on the matching feature point pairs, motion estimation is performed on the two adjacent image frames. Specifically, for any pair of matching feature points in the matching feature point pair, the displacement vector of the matching feature point pair can be determined based on the first position of the matching feature point pair in the first image frame and the second position of the matching feature point pair in the second image frame. For example, if image frame 1 and image frame 2 are adjacent image frames, and feature point A in image frame 1 matches feature point B in image frame 2, the displacement vector of the feature point pair (feature point A and feature point B) can be determined based on the position of feature point A in image frame 1 and the position of feature point B in image frame 2.

[0058] Among them, optical flow or direct motion estimation methods can be used to determine the displacement vector of feature point pairs between two image frames. For example, the Lucas-Kanade optical flow method can be used.

[0059] In one embodiment of this application, the above-mentioned method of determining a target object by matching feature points on the first image frame and feature points on the second image frame includes: obtaining matching feature point pairs by matching feature points on the first image frame and feature points on the second image frame; obtaining target feature point pairs that satisfy target conditions from the matching feature point pairs; and determining the target object based on the target feature point pairs.

[0060] In this embodiment, the target condition may include an object that is stationary during the second camera's capture. Since the movement vector of the feature points is used to detect jitter caused by the target device, and the movement of the feature points themselves can interfere with the detection of jitter caused by the target device, to avoid this interference, feature point pairs of stationary objects can be selected from matching feature point pairs as target feature point pairs, and these target feature point pairs are determined as the target objects. The displacement vector of the target objects is then further determined. This reduces the interference of the movement of the feature points themselves on the jitter detection of the target device, improving the jitter detection accuracy of the target device.

[0061] Step 240: Determine the second jitter amount of the target device based on the displacement vector of the target object.

[0062] In this embodiment of the application, for any two adjacent image frames, the matching feature point pairs between the two adjacent image frames may include multiple feature point pairs, wherein each feature point pair may correspond to a displacement vector. That is, multiple feature point pairs may correspond to multiple displacement vectors. Since the multiple displacement vectors are all generated by the jitter of the target device, theoretically the multiple displacement vectors are the same, and this displacement vector can be determined as the displacement vector between the two adjacent image frames.

[0063] However, in practical applications, it cannot be guaranteed that all selected feature point pairs correspond to objects that are absolutely stationary, and the displacement vectors corresponding to multiple feature point pairs may have individual differences. Therefore, the displacement vector with the most identical displacement vectors among the multiple displacement vectors can be selected as the displacement vector between two adjacent image frames.

[0064] In this embodiment of the application, for any two adjacent image frames, there may be a displacement vector between the two adjacent image frames. For example, the target image frames include image frame 1, image frame 2 and image frame 3. Image frame 1 and image frame 2 are adjacent, image frame 2 and image frame 3 are adjacent, and there may be a displacement vector between image frame 1 and image frame 2, and there may be a displacement vector between image frame 2 and image frame 3.

[0065] Furthermore, the jitter of two adjacent image frames can be determined based on the displacement vector between them. For example, the displacement vector of two adjacent image frames can be directly determined as the jitter of the two adjacent image frames.

[0066] Step 250: During the second camera's shooting process, acquire the third jitter amount detected by the gyroscope on the target device.

[0067] Step 260: Based on the second jitter amount and the third jitter amount, obtain the target error value of the gyroscope.

[0068] Step 270: Calibrate the first jitter amount based on the target error value to obtain the target jitter amount.

[0069] Step 280: Based on the target jitter amount, perform image stabilization operation on the first camera.

[0070] In this embodiment, a displacement vector of a target object is determined based on multiple image frames, wherein the target object is an object present in all of the multiple image frames; based on the displacement vector of the target object, a first jitter amount of the target device is determined. Thus, by determining the displacement vector of a target object present in multiple image frames, the jitter amount of the target device can be indirectly determined, providing a method for analyzing image frames to determine the jitter amount of the target device.

[0071] This application describes in detail a method for determining the jitter of a target device based on the displacement vector of the target object. Please refer to [link to relevant documentation]. Figure 3 , Figure 3 This is a flowchart of another anti-shake method provided in an embodiment of this application. For example... Figure 3 As shown, the method includes the following steps: Step 310: During the shooting process of the first camera on the target device, obtain the first jitter amount detected by the gyroscope on the target device.

[0072] Step 320: When the first camera is activated, acquire a target image frame captured by the second camera on the target device, wherein the target image frame includes multiple image frames.

[0073] Step 330: Based on multiple image frames, determine the displacement vector of the target object, wherein the target object is an object that exists in all of the multiple image frames, and the displacement vector includes at least one of a first displacement vector and a second displacement vector.

[0074] The first displacement vector is used to indicate the displacement of the target object in the horizontal direction, and the second displacement vector is used to indicate the displacement of the target object in the vertical direction.

[0075] For reference Figure 4 , Figure 4 This is a schematic diagram illustrating a gyroscope for detecting jitter, provided as an embodiment of this application. Figure 4 As shown, a gyroscope can be used to detect the jitter of a target device at three angles: pitch, yaw, and roll. In other words, when calibrating the jitter detected by the gyroscope, calibration can be performed on the pitch, yaw, and roll angles. Since the jitter at the roll angle does not cause image blurring in the camera image, calibration can be performed only on the jitter detected by the gyroscope at the pitch and yaw angles.

[0076] Simultaneously, in order to detect the jitter of the target equipment in the pitch and yaw angles, the displacement vectors of the target object in the horizontal and vertical directions can be obtained. Specifically, refer to the description of the following steps. Steps 340 and 350 are not in any particular order during actual implementation; step 340 can be executed first, followed by step 350, or vice versa.

[0077] Step 340: If the displacement vector includes the first displacement vector, determine the first jitter component of the target device at the yaw angle based on the first displacement vector and the image distance of the second camera.

[0078] In the embodiments of this application, reference can be made to Figure 5 , Figure 5 This application provides a schematic diagram of determining a displacement vector. The displacement vector can be represented as a two-dimensional vector (dx, dy), and may include the first displacement vector of the target object in the horizontal direction (e.g., Figure 5 dx) and the second displacement vector in the vertical direction (e.g. Figure 5 (dy in the figure). Based on the first displacement vector of the target object in the horizontal direction, the first jitter component of the target device at the yaw angle can be determined.

[0079] Specifically, the image distance of the second camera can be obtained first. The image distance of the second camera is a fixed parameter of the second camera, and different cameras can correspond to different image distances. For example... Figure 5 As shown, the first displacement vector, the image distance of the second camera, and the first jitter component of the target device at the yaw angle can have the following geometric relationship: ; Where dx is the first displacement vector, and v is the image distance of the second camera. This represents the first jitter component of the target device at the yaw angle.

[0080] Therefore, the arctangent function ATAN() can be used to determine the first jitter component of the target device at the yaw angle based on the first displacement vector and the image distance of the second camera, using the following formula: ; Where dx is the first displacement vector, and v is the image distance of the second camera. This represents the first jitter component of the target equipment at the yaw angle. Used to represent the arctangent function.

[0081] Correspondingly, the gyroscope on the target device can detect the jitter of the target device at three angles: pitch, yaw, and roll. That is, the first jitter includes a third jitter component and a fourth jitter component. The third jitter component is used to indicate the jitter of the target device detected by the gyroscope at the yaw angle, and the fourth jitter component is used to indicate the jitter of the target device detected by the gyroscope at the pitch angle.

[0082] Step 350: If the displacement vector includes a second displacement vector, determine the second jitter component of the target device in the pitch angle based on the second displacement vector and the image distance of the second camera.

[0083] In this embodiment of the application, similar to the method for determining the first jitter component, such as... Figure 5 As shown, the second displacement vector, the image distance of the second camera, and the second jitter component of the target device at the pitch angle can have the following geometric relationship: ; Where dy is the second displacement vector, and v is the image distance of the second camera. This is the second jitter component of the target equipment at the pitch angle.

[0084] Therefore, the arctangent function ATAN() can be used to determine the second jitter component of the target device in the pitch angle based on the second displacement vector and the image distance of the second camera, using the following formula: ; Where dy is the second displacement vector, and v is the image distance of the second camera. This is the second jitter component of the target equipment at the pitch angle.

[0085] Step 360: During the second camera's shooting process, acquire the third jitter amount detected by the gyroscope on the target device.

[0086] In this embodiment of the application, the gyroscope on the target device can detect the jitter of the target device at three angles: pitch angle, yaw angle, and roll angle. That is, the third jitter includes a fifth jitter component and a sixth jitter component. The fifth jitter component is used to indicate the jitter of the target device at the yaw angle detected by the gyroscope, and the sixth jitter component is used to indicate the jitter of the target device at the pitch angle detected by the gyroscope.

[0087] Step 370: Based on the second jitter amount and the third jitter amount, obtain the target error value of the gyroscope.

[0088] In this embodiment of the application, the target error value includes a first error value of the jitter detected by the gyroscope at the pitch angle and a second error value of the jitter detected at the yaw angle. The first error value is the error value between the first jitter component and the fifth jitter component, and the second error value is the error value between the second jitter component and the sixth jitter component.

[0089] A first error value can be obtained based on the first jitter component and the fifth jitter component detected by the gyroscope, and a second error value can be obtained based on the second jitter component and the sixth jitter component detected by the gyroscope.

[0090] In one embodiment of this application, the number of the second jitter amount and the number of the third jitter amounts are both M, where M is a positive integer greater than 1. Step 370, which involves obtaining the target error value of the gyroscope based on the second jitter amount and the third jitter amount, includes: determining M differences between the second jitter amount and the third jitter amount based on the M second jitter amounts and the M third jitter amounts; performing derivative processing on the M differences to obtain M angular velocity values ​​of the gyroscope; and determining the average of the M angular velocity values ​​as the target error value.

[0091] In this embodiment, the second jitter amount is M second jitter amounts obtained based on multiple image frames, wherein the M second jitter amounts include M first jitter components and M second jitter components. The third jitter amount is M third jitter amounts detected by the gyroscope when the second camera captures multiple image frames, wherein the M third jitter amounts include M fifth jitter components and M sixth jitter components.

[0092] When determining the first error value of the gyroscope based on M first jitter components and M fifth jitter components, the following method can be used. First, determine the M differences between the M first jitter components and the M fifth jitter components. Here, the difference is the difference between the first jitter component and the fifth jitter component generated by the second camera during the shooting process. For example, during the process of using the second camera to capture image frame 2, the first jitter component can be determined based on the displacement vector between image frame 2 and the previous image frame. The fifth jitter component detected by the gyroscope when the second camera captures image frame 2 can be obtained, and a difference can be determined based on the first jitter component and the fifth jitter component. That is to say, M differences can be determined based on the M first jitter components and the M fifth jitter components, as shown in the following formula.

[0093] ; in, It is a data sequence consisting of M first jitter components determined based on multiple image frames. This is a data sequence consisting of M fifth jitter components detected by the gyroscope when multiple image frames are captured by the second camera. These data sequences can be arranged in chronological order according to the multiple image frames captured by the second camera. It is a data sequence consisting of M differences determined based on M first jitter components and M fifth jitter components.

[0094] After determining the data sequence consisting of M differences, for each of the M differences, the time parameter can be differentiated to obtain the M angular velocity values ​​corresponding to the M differences. Here, the angular velocity values ​​are the angular velocities of the target equipment at the yaw angle. Specifically, refer to the following formula: ; in, It is a data sequence composed of M differences determined by M first jitter components and M fifth jitter components. This is a data sequence obtained based on M angular velocity values ​​determined by M differences.

[0095] After determining the M angular velocity values ​​of the target device at the yaw angle, the average of these M angular velocity values ​​can be determined as the bias error of the gyroscope at the yaw angle, i.e., the first error value, to filter out high-frequency components. Specifically, refer to the following formula: ; in, This represents the bias error of the gyroscope at the yaw angle. This is a data sequence obtained based on M angular velocity values ​​determined by M differences.

[0096] Accordingly, the method for determining the second error value of the gyroscope based on the M second jitter components and the M sixth jitter components can be the same as described above. First, the M differences between the M second jitter components and the M sixth jitter components can be determined by referring to the following formula.

[0097] ; in, It is a data sequence consisting of M second jitter components determined based on multiple image frames. This refers to the data sequence composed of M sixth jitter components detected by the gyroscope when multiple image frames are captured by the second camera. This data sequence can be arranged according to the chronological order of the multiple image frames captured by the second camera. It is a data sequence consisting of M differences determined based on M second jitter components and M sixth jitter components.

[0098] After determining the data sequence consisting of M differences, for each of the M differences, the time parameter can be differentiated to obtain the M angular velocity values ​​corresponding to the M differences. Here, the angular velocity values ​​are the angular velocities of the target device at the pitch angle. Specifically, refer to the following formula: ; in, It is a data sequence composed of M differences determined based on M second jitter components and M sixth jitter components. This is a data sequence obtained based on M angular velocity values ​​determined by M differences.

[0099] After determining the M angular velocity values ​​of the target device at the yaw and pitch angles, the average of these M angular velocity values ​​can be determined as the bias error of the gyroscope at the pitch angle, i.e., the second error value, to filter out high-frequency components. Specifically, refer to the following formula: ; in, This represents the bias error of the gyroscope at the pitch angle. This is a data sequence obtained based on M angular velocity values ​​determined by M differences.

[0100] After determining the target error values ​​(first error value and second error value), the jitter detected by the gyroscope can be calibrated based on the target error values. Specifically, when the target user uses the first camera to capture an image, the gyroscope can detect the first jitter of the first camera when capturing the image. The first jitter includes the third jitter component of the target device at the yaw angle and the fourth jitter component of the target device at the pitch angle, both detected by the gyroscope when the first camera captures the image.

[0101] Step 380: Based on the target error value, calibrate the first jitter amount to obtain the target jitter amount.

[0102] In this embodiment, the first jitter amount can be calibrated using the target error value to obtain the target jitter amount. The target jitter amount includes a seventh jitter component of the first camera at the yaw angle when capturing an image, and an eighth jitter component of the first camera at the pitch angle when capturing an image. The difference between the third jitter component and the bias error (first error value) of the gyroscope at the yaw angle can be determined as the seventh jitter component of the first camera at the yaw angle when capturing an image; the difference between the fourth jitter component and the bias error (second error value) of the gyroscope at the pitch angle can be determined as the eighth jitter component of the first camera at the pitch angle when capturing an image.

[0103] Step 390: Based on the target jitter amount, perform image stabilization on the first camera.

[0104] In this embodiment of the application, the first camera can be stabilized at the yaw angle and the pitch angle based on the seventh jitter component of the first camera at the yaw angle and the eighth jitter component of the first camera at the pitch angle, respectively.

[0105] In this embodiment, the jitter of the target device is determined based on the displacement vector from two angles: pitch angle and yaw angle. By accurately determining the first jitter, the bias error detection accuracy of the gyroscope is improved, thereby making the calibration effect of the jitter of the gyroscope better.

[0106] Please see Figure 6 , Figure 6 This is a flowchart illustrating a specific anti-shake method provided in an embodiment of this application. For example... Figure 6 As shown, the method includes the following steps: Step 610: During the shooting process of the first camera on the target device, obtain the first jitter amount detected by the gyroscope on the target device.

[0107] In this embodiment of the application, the first jitter amount includes a third jitter component and a fourth jitter component. The third jitter component is used to indicate the jitter amount of the target device detected by the gyroscope in the yaw angle, and the fourth jitter component is used to indicate the jitter amount of the target device detected by the gyroscope in the pitch angle.

[0108] Step 615: When the first camera is activated, acquire a target image frame captured by the second camera on the target device, wherein the target image frame includes an adjacent first image frame and a second image frame.

[0109] Step 620: Perform feature point detection on the first image frame and the second image frame to obtain the feature points of the first image frame and the feature points of the second image frame.

[0110] Step 625: Determine the target object by performing feature point matching on the feature points on the first image frame and the feature points on the second image frame. The target object includes the feature points that are matched between the first image frame and the second image frame.

[0111] In this embodiment of the application, the target object can be determined in the following way: by performing feature point matching on the feature points on the first image frame and the feature points on the second image frame, a matching feature point pair is obtained; from the matching feature point pairs, a target feature point pair that meets the target conditions is obtained; and based on the target feature point pair, the target object is determined.

[0112] Step 630: Determine the displacement vector of the target object based on the first position of the target object in the first image frame and the second position of the target object in the second image frame.

[0113] In this embodiment of the application, the displacement vector includes a first displacement vector and a second displacement vector. The first displacement vector is used to indicate the displacement of the target object in the horizontal direction, and the second displacement vector is used to indicate the displacement of the target object in the vertical direction.

[0114] Step 635: Determine the second jitter amount of the target device based on the displacement vector of the target object.

[0115] In this embodiment of the application, the second jitter amount includes at least one of a first jitter component and a second jitter component. The first jitter component is used to indicate the jitter amount at the yaw angle, and the second jitter component is used to indicate the jitter amount at the pitch angle. The second jitter amount of the target device can be determined by referring to the following method.

[0116] When the displacement vector includes a first displacement vector, a first jitter component of the target device at the yaw angle is determined based on the first displacement vector and the image distance of the second camera; when the displacement vector includes a second displacement vector, a second jitter component of the target device at the pitch angle is determined based on the second displacement vector and the image distance of the second camera.

[0117] Step 640: During the second camera's shooting process, acquire the third jitter amount detected by the gyroscope on the target device.

[0118] In this embodiment of the application, the third jitter amount includes a fifth jitter component and a sixth jitter component. The fifth jitter component is the jitter amount of the target device detected by the gyroscope in the yaw angle, and the sixth jitter component is the jitter amount of the target device detected by the gyroscope in the pitch angle.

[0119] Step 645: Based on the second jitter amount and the third jitter amount, obtain the target error value of the gyroscope.

[0120] In this embodiment, the number of the second jitter and the third jitter are both M, where M is a positive integer greater than 1. The target error value of the gyroscope can be determined as follows: based on the M second jitter and the M third jitter, determine the M differences between the second jitter and the third jitter; perform differentiation on the M differences to obtain the M angular velocity values ​​of the gyroscope; and determine the average of the M angular velocity values ​​as the target error value of the gyroscope.

[0121] The target error value includes a first error value for the jitter detected by the gyroscope at the pitch angle and a second error value for the jitter detected at the yaw angle. The first error value is the error between the first jitter component and the fifth jitter component, and the second error value is the error between the second jitter component and the sixth jitter component.

[0122] Step 650: Calibrate the first jitter amount based on the target error value to obtain the target jitter amount.

[0123] In this embodiment, all cameras on the target device are off by default. When a camera on the target device is detected to be active (this camera may be in a ready-to-use state, such as when the target user opens the camera application but has not yet clicked to take a picture), this camera can be identified as the first camera. Simultaneously, an unused camera on the target device can be selected as the second camera and used to capture multiple target image frames. The gyroscope on the target device can simultaneously detect the third jitter of the target device when each target image frame is captured by the second camera. The second jitter of the target device is determined by the displacement vector between multiple target image frames. The difference between the second jitter and the third jitter is determined as the target error value of the gyroscope, which is used to calibrate the jitter detected by the gyroscope subsequently.

[0124] When the first camera takes a picture (e.g., when the user clicks the capture button), the gyroscope can detect a first amount of jitter in the target device during the first camera's capture. Then, the first amount of jitter can be calibrated using the target error value; that is, the difference between the first amount of jitter and the target error value is determined as the target amount of jitter in the target device.

[0125] Specifically, the target jitter includes a seventh jitter component at the yaw angle and an eighth jitter component at the pitch angle when the first camera captures an image. The difference between the third jitter component and the bias error (first error value) of the gyroscope at the yaw angle can be determined as the seventh jitter component at the yaw angle when the first camera captures an image; the difference between the fourth jitter component and the bias error (second error value) of the gyroscope at the pitch angle can be determined as the eighth jitter component at the pitch angle when the first camera captures an image.

[0126] Step 655: Based on the target jitter amount, perform image stabilization on the first camera.

[0127] In this embodiment, during the shooting process of the first camera on the target device, a first jitter amount detected by the gyroscope on the target device is obtained; a target error value of the gyroscope is obtained, which is based on the target image frame captured by the second camera on the target device; the first jitter amount is calibrated based on the target error value to obtain a target jitter amount; and the first camera is stabilized based on the target jitter amount. Thus, it not only relies on the first jitter amount detected by the gyroscope during the shooting process of the first camera on the target device, but also combines the target error value obtained from the target image frame captured by the second camera to calibrate the first jitter amount, ensuring that the obtained target jitter amount has high accuracy. Using this high-accuracy target jitter amount to perform the stabilization operation on the first camera solves the problem of poor stabilization effect caused by large gyroscope detection errors in related technologies.

[0128] It is important to understand that Figures 1 to 6 The explanations of the same or corresponding steps can be cross-referenced. For example, Figure 1 The explanations of steps 130 and 140 are applicable to Figure 2 Steps 270 and 280 in the process.

[0129] Meanwhile, it should be understood that the image stabilization method provided in this application embodiment has the following beneficial effects: First, by utilizing a multimodal sensor data fusion method, and combining the target device displacement vector detected by the camera and the target device jitter detected by the gyroscope, the bias error of the gyroscope is calibrated in real time, which can effectively improve the stability, accuracy, and reliability of the images captured by the target device. Second, compared with related technologies that rely on additional equipment (fixing the target device with a bracket or table) and asynchronous, non-real-time (not calibrating the bias error of the gyroscope while shooting) testing and calibration methods, the image stabilization method provided in this application has the advantages of high efficiency, low power consumption, and low cost. Third, the image stabilization method provided in this application is not only suitable for static shooting but also applicable to achieving efficient optical image stabilization in dynamic scenes. Fourth, the image stabilization method provided in this application can be combined with the EIS control calibration algorithm using the gyroscope in related technologies to achieve better electronic image stabilization in the EIS control calibration algorithm.

[0130] Please see Figure 7 , Figure 7 This is a structural block diagram of a shake stabilization device provided in an embodiment of this application. For example... Figure 7 As shown in the figure, this application embodiment provides a stabilization device 700, which includes: an acquisition module 710, a calibration module 720, and a stabilization operation module 730.

[0131] The acquisition module 710 is used to acquire a first jitter amount detected by the gyroscope on the target device during the shooting process of the first camera on the target device; and to acquire a target error value of the gyroscope, wherein the target error value is obtained based on the target image frame captured by the second camera on the target device. The calibration module 720 is used to calibrate the first jitter amount based on the target error value to obtain the target jitter amount; The image stabilization module 730 is used to perform image stabilization on the first camera based on the target shaking amount.

[0132] In this embodiment, during the shooting process of the first camera on the target device, a first jitter amount detected by the gyroscope on the target device is obtained; a target error value of the gyroscope is obtained, which is based on the target image frame captured by the second camera on the target device; the first jitter amount is calibrated based on the target error value to obtain a target jitter amount; and the first camera is stabilized based on the target jitter amount. Thus, it not only relies on the first jitter amount detected by the gyroscope during the shooting process of the first camera on the target device, but also combines the target error value obtained from the target image frame captured by the second camera to calibrate the first jitter amount, ensuring that the obtained target jitter amount has high accuracy. Using this high-accuracy target jitter amount to perform the stabilization operation on the first camera solves the problem of poor stabilization effect caused by large gyroscope detection errors in related technologies.

[0133] The image stabilization device provided in this application embodiment can realize all the processes implemented in the above method embodiments, and will not be described again here to avoid repetition.

[0134] like Figure 8 As shown in the illustration, this application also provides an electronic device 800. The electronic device 800 includes a processor 810 and a memory 820. The memory 820 stores programs or instructions, which, when executed by the processor 810, implement the steps of any of the methods described above. For example, when the program is executed by the processor 810, it implements the following process: during the shooting process of a first camera on a target device, a first jitter amount detected by a gyroscope on the target device is obtained; a target error value of the gyroscope is obtained, the target error value being based on a target image frame captured by a second camera on the target device; the first jitter amount is calibrated based on the target error value to obtain a target jitter amount; and image stabilization is performed on the first camera based on the target jitter amount. In this way, it not only relies on the first jitter detected by the gyroscope during the shooting process of the first camera on the target device, but also combines the target error value obtained by the target image frame captured by the second camera to calibrate the first jitter, ensuring that the obtained target jitter has high accuracy. The first camera is then stabilized with the target jitter with high accuracy, which solves the problem of poor stabilization effect caused by large gyroscope detection error in related technologies.

[0135] This application also provides a readable storage medium storing a program or instructions that, when executed by a processor, implement the steps of various embodiments of the debouncing method and achieve the same technical effect. To avoid repetition, these steps will not be repeated here.

[0136] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0137] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above method embodiments and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0138] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above method embodiments and achieve the same technical effects. To avoid repetition, it will not be described again here.

[0139] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0140] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0141] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A method for stabilizing image quality, characterized in that, include: During the shooting process of the first camera on the target device, the first jitter amount detected by the gyroscope on the target device is obtained; The target error value of the gyroscope is obtained, and the target error value is obtained based on the target image frame captured by the second camera on the target device; The first camera and the second camera are different cameras; The first jitter amount is calibrated based on the target error value to obtain the target jitter amount; Based on the target jitter level, the first camera is subjected to image stabilization. The step of obtaining the target error value of the gyroscope includes: When the first camera is activated, a target image frame captured by the second camera on the target device is acquired; Based on the target image frame, determine the second jitter amount of the target device; During the second camera's shooting process, a third jitter amount is acquired by the gyroscope detected by the target device; The target error value of the gyroscope is obtained based on the second jitter amount and the third jitter amount.

2. The method according to claim 1, characterized in that, The second camera is the camera that was not used during the recording process of the first camera.

3. The method according to claim 1, characterized in that, The image stabilization operation includes optical image stabilization.

4. The method according to claim 1, characterized in that, The target image frame includes multiple image frames; determining the second jitter amount of the target device based on the target image frame includes: Based on multiple image frames, determine the displacement vector of the target object, wherein the target object is an object that exists in all of the multiple image frames; Based on the displacement vector of the target object, the second jitter amount of the target device is determined.

5. The method according to claim 4, characterized in that, The plurality of image frames includes adjacent first image frames and second image frames; determining the displacement vector of the target object based on the plurality of image frames includes: Feature point detection is performed on the first image frame and the second image frame to obtain the feature points of the first image frame and the feature points of the second image frame; The target object is determined by matching the feature points on the first image frame and the feature points on the second image frame. The target object includes the feature points that are matched between the first image frame and the second image frame. The displacement vector of the target object is determined based on the first position of the target object in the first image frame and the second position of the target object in the second image frame.

6. The method according to claim 5, characterized in that, The step of determining the target object by matching feature points on the first image frame and feature points on the second image frame includes: By performing feature point matching on the feature points of the first image frame and the feature points of the second image frame, matching feature point pairs are obtained. Obtain the target feature point pairs that satisfy the target conditions from the matching feature point pairs; The target object is determined based on the target feature point pair.

7. The method according to claim 4, characterized in that, The displacement vector includes at least one of a first displacement vector and a second displacement vector, wherein the first displacement vector is used to indicate the displacement of the target object in the horizontal direction and the second displacement vector is used to indicate the displacement of the target object in the vertical direction; the second jitter includes at least one of a first jitter component and a second jitter component, wherein the first jitter component is used to indicate the jitter at the yaw angle and the second jitter component is used to indicate the jitter at the pitch angle. Determining the second jitter amount of the target device based on the displacement vector of the target object includes: When the displacement vector includes a first displacement vector, the first jitter component of the target device at the yaw angle is determined based on the first displacement vector and the image distance of the second camera; When the displacement vector includes a second displacement vector, the second jitter component of the target device at the pitch angle is determined based on the second displacement vector and the image distance of the second camera.

8. The method according to any one of claims 4-7, characterized in that, The number of the second jitter and the third jitter are both M, where M is a positive integer greater than 1; obtaining the target error value of the gyroscope based on the second jitter and the third jitter includes: Based on M second jitter values ​​and M third jitter values, determine M differences between the second jitter values ​​and the third jitter values; The M differences are processed to obtain the M angular velocity values ​​of the gyroscope; The average of the M angular velocity values ​​is determined as the target error value of the gyroscope.

9. A shake-stabilizing device, characterized in that, include: The acquisition module is used to acquire the first jitter amount detected by the gyroscope on the target device during the shooting process of the first camera on the target device; The target error value of the gyroscope is obtained, and the target error value is obtained based on the target image frame captured by the second camera on the target device; The first camera and the second camera are different cameras; A calibration module is used to calibrate the first jitter amount based on the target error value to obtain the target jitter amount; The image stabilization module is used to perform image stabilization on the first camera based on the target shake amount; Specifically, in the process of obtaining the target error value of the gyroscope, the acquisition module is used to: when the first camera is activated, acquire a target image frame captured by the second camera on the target device; and determine the second jitter amount of the target device based on the target image frame. During the second camera's capture process, a third jitter value is acquired by the gyroscope on the target device; based on the second jitter value and the third jitter value, the target error value of the gyroscope is obtained.

10. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a program or instructions that run on the processor, the program or instructions which, when executed by the processor, implement the steps of the method as described in any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The medium stores a program or instructions that, when executed, implement the steps of the method as described in any one of claims 1-8.

12. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method described in any one of claims 1-8.

Images