Image processing method and device, electronic equipment and storage medium

Abstract

The invention relates to an image processing method and device, electronic equipment and a storage medium. The method comprises the steps of obtaining a current posture when the electronic equipment collects an image and a historical posture before the electronic equipment collects the image; determining a motion state of the electronic equipment according to the current posture and the historical posture; predicting a target posture of the electronic equipment according to the current posture, the historical posture and the motion state; and performing anti-shake processing on the image by using the target attitude to obtain an image after anti-shake processing. Through the method, the image anti-shake processing effect can be improved.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to an image processing method and device, electronic equipment, and storage medium. Background Art

[0002] When electronic devices including mobile phones, cameras, wearable devices, etc. are capturing images through image acquisition components (such as cameras, etc.), if the position or posture of the electronic device changes, such as when the user holds the electronic device to capture images and there is hand shaking, or the electronic device actively moves, the captured image will cause shaking, affecting the image clarity and visual effect.

[0003] The current mainstream anti-shake technology is applicable to focal lengths between 0.5X and 5X (12mm~120mm). However, with the development of electronic equipment, lens configurations are getting higher and higher. With electronic cropping, the focal length can reach 100X (equivalent focal length 2400mm) or even 120X (equivalent focal length 2880mm). In this case, the movements of electronic devices are linearly magnified exponentially. If traditional electronic anti-shake algorithms are used, the stability and tracking of the processed images will be obviously insufficient. Summary of the invention

[0004] The present disclosure provides an image processing method and device, an electronic device, and a storage medium.

[0005] According to a first aspect of an embodiment of the present disclosure, there is provided an image processing method, including:

[0006] Acquire a current posture of the electronic device when capturing an image, and a historical posture of the electronic device before capturing the image;

[0007] Determining a motion state of the electronic device according to the current posture and the historical posture;

[0008] Predicting a target posture of the electronic device according to the current posture, the historical posture, and the motion state;

[0009] The image is subjected to anti-shake processing using the target posture to obtain an anti-shake processed image.

[0010] In some embodiments, predicting the target posture of the electronic device according to the current posture, the historical posture and the motion state includes:

[0011] Performing filtering processes in different ways on the current posture and the historical posture to obtain different filtering postures;

[0012] Based on the motion state of the electronic device, selecting a filtering posture to be interpolated from the different filtering postures;

[0013] Interpolation processing is performed based on the filtered posture to be interpolated to obtain the target posture of the electronic device.

[0014] In some embodiments, the interpolation processing is performed based on the filtered posture to be interpolated to obtain the target posture of the electronic device, including:

[0015] Determining an interpolation coefficient corresponding to the filtering posture to be interpolated based on the duration of the motion state and the filtering posture to be interpolated;

[0016] An interpolation process is performed based on the filter posture to be interpolated and the interpolation coefficient corresponding to the filter posture to be interpolated to obtain the target posture of the electronic device.

[0017] In some embodiments, the filtering posture to be interpolated includes two filtering postures, the filtering postures are represented by quaternions, and the interpolation process includes spherical linear interpolation of quaternions;

[0018] The determining, based on the duration of the motion state and the filter posture to be interpolated, an interpolation coefficient corresponding to the filter posture to be interpolated includes:

[0019] Determining weights corresponding to two filter postures to be interpolated in the spherical linear interpolation of the quaternion based on the duration of the motion state;

[0020] Determining a cosine angle between the two filtered postures to be interpolated;

[0021] Based on the cosine angle and the weights corresponding to the two filtering postures, interpolation coefficients corresponding to the filtering postures to be interpolated are determined.

[0022] In some embodiments, the method further comprises:

[0023] Get the duration threshold;

[0024] The step of determining weights corresponding to two filter postures to be interpolated in the spherical linear interpolation of the quaternion based on the duration of the motion state includes:

[0025] In response to the duration of the motion state being greater than a duration threshold, determining weights corresponding to two filter postures to be interpolated in the spherical linear interpolation of the quaternion; wherein the weight of a preset initial filter posture in the two filter postures to be interpolated is positively correlated with the duration of the motion state, and the sum of the weights corresponding to the two filter postures to be interpolated is a first preset value;

[0026] In response to the duration of the motion state being less than or equal to the duration threshold, the weight of the preset initial filtering posture is determined to be the first preset value, and the weight of the filtering posture to be interpolated other than the preset initial filtering posture is 0.

[0027] In some embodiments, the method further comprises:

[0028] Acquire the focal length when the electronic device captures the image;

[0029] The acquisition time threshold includes:

[0030] In the case where the focal length belongs to a preset focal length, determining a duration threshold corresponding to the focal length according to a preset mapping relationship; wherein the preset mapping relationship includes a mapping between a focal length and a duration threshold;

[0031] or,

[0032] When the focal length does not belong to the preset focal length, the duration threshold corresponding to the focal length is determined according to the duration thresholds respectively corresponding to a first preset focal length that is greater than the focal length and a second preset focal length that is less than the focal length; wherein the first preset focal length and the second preset focal length both belong to the preset focal length.

[0033] In some embodiments, the first preset focal length and the second preset focal length are both preset focal lengths adjacent to the focal length; and determining the duration threshold corresponding to the focal length according to the duration thresholds respectively corresponding to the first preset focal length that is greater than the focal length and the second preset focal length that is less than the focal length, comprises:

[0034] Determine the first focal length weight of the first preset focal length by the ratio of the difference between the focal length and the second preset focal length to the difference between the first preset focal length and the second preset focal length;

[0035] Determine a ratio of a difference between the first preset focal length and the focal length and a difference between the first preset focal length and the second preset focal length as a second focal length weight of the second preset focal length;

[0036] The sum of the result of weighting the preset duration threshold corresponding to the first preset focal length by the first focal length weight and the result of weighting the preset duration threshold corresponding to the second preset focal length by the second focal length weight is determined as the duration threshold corresponding to the focal length.

[0037] In some embodiments, the filtering of the current posture and the historical posture in different ways to obtain different filtered postures includes:

[0038] Performing weighted processing on the current posture and the historical posture respectively, and obtaining a first filtered posture based on the weighted postures;

[0039] Performing Kalman filtering on the current posture and the historical posture to obtain a second filtered posture;

[0040] The current posture is weighted, and a third filtering posture is obtained based on the weighted current posture and the historical posture.

[0041] In some embodiments, the weighting of the current posture and the historical posture is performed respectively, and the first filtered posture is obtained based on the weighted postures, including:

[0042] The sum of the result of weighting the current posture with the first preset weight and the result of weighting the historical posture with the second preset weight is determined as the first filtering posture; wherein the first preset weight is positively correlated with the difference between the current posture and the historical posture; the sum of the first preset weight and the second preset weight is the second preset value.

[0043] In some embodiments, performing weighted processing on the current posture and obtaining a third filtered posture based on the weighted current posture and the historical posture includes:

[0044] The sum of a result of weighting the current posture with a third preset weight and the historical posture is determined as the third filtering posture; wherein the third preset weight is less than a preset weight threshold.

[0045] In some embodiments, the motion state includes shaking, moving, and stillness; and selecting a filtering posture to be interpolated from the different filtering postures based on the motion state of the electronic device includes:

[0046] In a case where the motion state includes shaking, determining the third filtering posture and the first filtering posture as the filtering postures to be interpolated;

[0047] In a case where the motion state includes moving or being stationary, the first filtering posture and the second filtering posture are determined as the filtering postures to be interpolated.

[0048] In some embodiments, the current posture includes a current angular velocity, and the historical posture includes a historical angular velocity; and determining the motion state of the electronic device according to the current posture and the historical posture includes:

[0049] respectively determining whether the current angular velocity and the historical angular velocity are between a first preset angular velocity threshold and a second preset angular velocity threshold;

[0050] When the current angular velocity and the historical angular velocity are both between the first preset angular velocity threshold and the second preset angular velocity threshold, determining that the motion state of the electronic device is shaking;

[0051] or,

[0052] When the current angular velocity or the historical angular velocity is not between the first preset angular velocity threshold and the second preset angular velocity threshold, the motion state of the electronic device is determined according to the current angular velocity and the historical angular velocity.

[0053] In some embodiments, when the current angular velocity or the historical angular velocity is greater than or equal to the first preset angular velocity threshold, or less than or equal to the second preset angular velocity threshold, determining the motion state of the electronic device according to the current angular velocity and the historical angular velocity includes:

[0054] In a case where the current angular velocity or the historical angular velocity is greater than or equal to the first preset angular velocity threshold, or less than or equal to the second preset angular velocity threshold, according to the current angular velocity and the historical angular velocity, in response to satisfying one of the following: the shaking frequency of the electronic device is greater than a preset frequency threshold, the direction of the angular velocity changes, and the duration of the change in the direction of the angular velocity is greater than a first preset duration threshold, determining that the motion state of the electronic device is shaking;

[0055] In response to the current angular velocity and the historical angular velocity being both less than the first preset angular velocity threshold, determining that the motion state of the electronic device is stationary;

[0056] In response to satisfying one of the following conditions: the shaking frequency is less than or equal to the preset frequency threshold, the direction of the angular velocity has not changed, and the duration of the change in the direction of the angular velocity is less than or equal to the first preset duration threshold, the motion state of the electronic device is determined to be moving.

[0057] In some embodiments, performing anti-shake processing on the image using the target posture to obtain the anti-shake processed image includes:

[0058] Transforming the coordinates of the pixel points in the image using the target posture to obtain an image after coordinate transformation;

[0059] The image after the coordinate transformation is cropped to obtain the image after the anti-shake processing.

[0060] According to a second aspect of an embodiment of the present disclosure, there is provided an image processing apparatus, including:

[0061] A first acquisition module, used to acquire a current posture of the electronic device when capturing an image, and a historical posture of the electronic device before capturing the image;

[0062] A determination module, used to determine the motion state of the electronic device according to the current posture and the historical posture;

[0063] A prediction module, configured to predict a target posture of the electronic device according to the current posture, the historical posture and the motion state;

[0064] The anti-shake module is used to perform anti-shake processing on the image using the target posture to obtain an image after anti-shake processing.

[0065] In some embodiments, the prediction module is used to filter the current posture and the historical posture in different ways to obtain different filtered postures; based on the motion state of the electronic device, select the filtered posture to be interpolated from the different filtered postures; and perform interpolation processing based on the filtered posture to be interpolated to obtain the target posture of the electronic device.

[0066] In some embodiments, the prediction module is used to determine the interpolation coefficient corresponding to the filtering posture to be interpolated based on the duration of the motion state and the filtering posture to be interpolated; and perform interpolation processing based on the filtering posture to be interpolated and the interpolation coefficient corresponding to the filtering posture to be interpolated to obtain the target posture of the electronic device.

[0067] In some embodiments, the filtering posture to be interpolated includes two filtering postures, the filtering postures are represented by quaternions, and the interpolation process includes spherical linear interpolation of quaternions;

[0068] The prediction module is used to determine the weights corresponding to two filter postures to be interpolated in the spherical linear interpolation of the quaternion based on the duration of the motion state; determine the cosine angle between the two filter postures to be interpolated; and determine the interpolation coefficients corresponding to each filter posture to be interpolated based on the cosine angle and the weights corresponding to the two filter postures.

[0069] In some embodiments, the apparatus further comprises:

[0070] The second acquisition module is used to obtain a duration threshold;

[0071] The prediction module is used to determine the weights corresponding to two filter postures to be interpolated in the spherical linear interpolation of the quaternion in response to the duration of the motion state being greater than a duration threshold; wherein the weight of a preset initial filter posture in the two filter postures to be interpolated is positively correlated with the duration of the motion state, and the sum of the weights corresponding to the two filter postures to be interpolated is a first preset value; in response to the duration of the motion state being less than or equal to the duration threshold, the weight of the preset initial filter posture is determined to be the first preset value, and the weight of the filter posture to be interpolated other than the preset initial filter posture is 0.

[0072] In some embodiments, the apparatus further comprises:

[0073] A third acquisition module, used to acquire the focal length when the electronic device captures the image;

[0074] The second acquisition module is used to determine the duration threshold corresponding to the focal length according to a preset mapping relationship when the focal length belongs to a preset focal length; wherein the preset mapping relationship includes a mapping between a focal length and a duration threshold; or, when the focal length does not belong to the preset focal length, determine the duration threshold corresponding to the focal length according to the duration thresholds respectively corresponding to a first preset focal length greater than the focal length and a second preset focal length less than the focal length; wherein the first preset focal length and the second preset focal length both belong to the preset focal length.

[0075] In some embodiments, the second acquisition module is used to determine the first focal length weight of the first preset focal length by the ratio of the difference between the focal length and the second preset focal length and the difference between the first preset focal length and the second preset focal length; determine the second focal length weight of the second preset focal length by the ratio of the difference between the first preset focal length and the focal length and the difference between the first preset focal length and the second preset focal length; and determine the sum of the result of weighting the preset duration threshold corresponding to the first preset focal length by the first focal length weight and the result of weighting the preset duration threshold corresponding to the second preset focal length by the second focal length weight as the duration threshold corresponding to the focal length.

[0076] In some embodiments, the prediction module is used to perform weighted processing on the current posture and the historical posture respectively, and obtain a first filtered posture based on each weighted posture; perform Kalman filtering on the current posture and the historical posture to obtain a second filtered posture; perform weighted processing on the current posture, and obtain a third filtered posture based on the weighted current posture and the historical posture.

[0077] In some embodiments, the prediction module is used to determine the first filtering posture as the sum of the result of weighting the current posture with a first preset weight and the result of weighting the historical posture with a second preset weight; wherein the first preset weight is positively correlated with the difference between the current posture and the historical posture; and the sum of the first preset weight and the second preset weight is a second preset value.

[0078] In some embodiments, the prediction module is used to determine the sum of the result of weighting the current posture with a third preset weight and the historical posture as the third filtered posture; wherein the third preset weight is less than a preset weight threshold.

[0079] In some embodiments, the motion state includes jitter, movement, and stillness; the prediction module is used to determine the third filtering posture and the first filtering posture as the filtering posture to be interpolated when the motion state includes jitter; and to determine the first filtering posture and the second filtering posture as the filtering posture to be interpolated when the motion state includes movement or stillness.

[0080] In some embodiments, the current posture includes the current angular velocity, and the historical posture includes the historical angular velocity; the determination module is used to respectively determine whether the current angular velocity and the historical angular velocity are between a first preset angular velocity threshold and a second preset angular velocity threshold; when the current angular velocity and the historical angular velocity are both between the first preset angular velocity threshold and the second preset angular velocity threshold, determine that the motion state of the electronic device is shaking; or, when the current angular velocity or the historical angular velocity is not between the first preset angular velocity threshold and the second preset angular velocity threshold, determine the motion state of the electronic device based on the current angular velocity and the historical angular velocity.

[0081] In some embodiments, the determination module is used to determine that the motion state of the electronic device is shaking based on the current angular velocity and the historical angular velocity, in response to satisfying one of the following conditions: the jitter frequency of the electronic device is greater than a preset frequency threshold, the direction of the angular velocity changes, and the duration of the change in the direction of the angular velocity is greater than a first preset duration threshold; determine that the motion state of the electronic device is stationary in response to the current angular velocity and the historical angular velocity are both less than the first preset angular velocity threshold; and determine that the motion state of the electronic device is moving in response to satisfying one of the following conditions: the jitter frequency is less than or equal to the preset frequency threshold, the direction of the angular velocity does not change, and the duration of the change in the direction of the angular velocity is less than or equal to the first preset duration threshold.

[0082] In some embodiments, the anti-shake module is used to transform the coordinates of the pixel points in the image using the target posture to obtain the image after the coordinate transformation; and to crop the image after the coordinate transformation to obtain the image after the anti-shake processing.

[0083] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0084] A processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the method as described in the first aspect above.

[0085] According to a fourth aspect of an embodiment of the present disclosure, there is provided a storage medium, including:

[0086] When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method as described in the first aspect above.

[0087] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0088] Due to the different motion states of the electronic device, the credibility of the current posture will be different. For example, if the motion state of the electronic device is shaking, since the shaking has a greater impact on the current posture, the credibility of the current posture is low, the credibility of the historical posture is high, and the target posture after reducing the shaking effect is closer to the historical posture; if the motion state of the electronic device is moving (such as the user holding the device to pan and shoot), the current posture after moving is more consistent with the posture of the current captured image, so the credibility of the current posture is high, the credibility of the historical posture is low, and the target posture after reducing the shaking effect is closer to the current posture. Therefore, the embodiment of the present disclosure determines the motion state of the electronic device according to the current posture and the historical posture, and predicts the target posture of the electronic device according to the current posture, the historical posture and the motion state, so as to use the target posture after reducing the shaking effect to perform anti-shake processing on the image, on the one hand, the image after anti-shake processing can be made to match the motion state of the electronic device more, and the adaptive anti-shake of the electronic device in different motion scenes can be achieved, and the anti-shake effect is better; on the other hand, since the future posture of the electronic device is not introduced in the embodiment of the present disclosure, real-time anti-shake can be achieved. This scheme has great advantages in scenes with high requirements for anti-shake real-time performance, such as image preview, automatic driving path planning, etc.

[0089] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0090] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0091] Figure 1 It is an example flowchart of an image processing method shown in an embodiment of the present disclosure.

[0092] Figure 2 This is an example diagram of weights corresponding to two filter postures to be interpolated in the spherical linear interpolation of quaternions determined in an embodiment of the present disclosure.

[0093] Figure 3 3 is an example diagram of image comparison before and after anti-shake processing shown in an embodiment of the present disclosure.

[0094] Figure 4 is a diagram of an image processing device shown in an embodiment of the present disclosure.

[0095] Figure 5 It is a block diagram of an electronic device shown in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0096] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0097] Among the related technologies, anti-shake technologies include electronic image stabilization (EIS) technology and optical image stabilization (OIS) technology. The principle of electronic image stabilization technology is to use the acceleration sensor and gyroscope in the electronic device to detect the amplitude of the electronic device's shaking, so as to dynamically adjust the camera's sensitivity, shutter and imaging algorithm to correct the shaking blur in the image. Optical image stabilization technology converts the real-time shaking information detected by the gyroscope and acceleration sensor into an electrical signal by installing a micro motor that can move in multiple directions inside the lens, thereby predicting the image offset caused by the tilt, and then feeding the image offset back to the motor, allowing it to push the sensor to move with an offset of the same size but opposite direction as the predicted image offset, thereby offsetting the image offset caused by hand shaking and achieving image stabilization.

[0098] However, this type of anti-shake technology has obvious deficiencies in the stability and tracking of images captured at different zoom ratios, especially when the focal length is as high as 100X (2400mm) or even 120X (2880mm).

[0099] In this regard, an embodiment of the present disclosure provides an image processing method, and its execution subject may be an image processing device. For example, the image processing method may be executed by an electronic device or a server or other electronic device including an image acquisition component, wherein the electronic device may be a mobile phone, a camera, a tablet computer, a vehicle-mounted device, a wearable device, etc. If the execution subject is a server or other electronic device, after acquiring the image and posture captured by the electronic device, it is sent to the server or other electronic device, so that the server or other electronic device performs anti-shake processing according to the image and posture, and can also feed back the anti-shake processed image to the electronic device for preview or subsequent image processing. In some possible implementations, the image processing method can be implemented by a processor calling computer-readable instructions stored in a memory. The following is an example of an electronic device being the execution subject image processing device.

[0100] Figure 1 is an example flow chart of an image processing method shown in an embodiment of the present disclosure, comprising Figure 1 It can be seen that the following steps are included:

[0101] S101, acquiring a current posture of an electronic device when capturing an image, and a historical posture of the electronic device before capturing the image;

[0102] S102, determining a motion state of the electronic device according to the current posture and the historical posture;

[0103] S103, predicting a target posture of the electronic device according to the current posture, the historical posture and the motion state;

[0104] S104: performing anti-shake processing on the image using the target posture to obtain an anti-shake processed image.

[0105] In the embodiments of the present disclosure, the electronic device may be the aforementioned terminal device including the image acquisition component, such as a mobile phone, a camera, a tablet computer, a vehicle-mounted device, a wearable device, etc.

[0106] In step S101, the current posture of the electronic device when capturing an image and the historical posture of the electronic device before capturing the image can be obtained based on a posture sensor; wherein the posture sensor is such as a three-axis gyroscope, an angular velocity sensor, a displacement sensor, etc.; correspondingly, the current posture and the historical posture may include a rotation matrix, Euler angles, quaternions, angular velocity, vector displacement, etc.

[0107] In step S102, the motion state of the electronic device is determined based on the current posture and the historical posture. In some embodiments, if the current posture is consistent with the historical posture, such as the current angular velocity is consistent with the historical angular velocity (including the magnitude of the current angular velocity is consistent with the magnitude of the historical angular velocity, and the direction of the current angular velocity is consistent with the direction of the historical angular velocity), it is determined that the electronic device is in a moving (such as translation) motion state. If the current posture is inconsistent with the historical posture, such as the direction of the current angular velocity is inconsistent with the direction of the historical angular velocity, it is determined that the electronic device is in a shaking (such as reciprocating shaking) motion state.

[0108] In some embodiments, based on the current posture and multiple historical postures, the shaking frequency of the electronic device is determined; if the shaking frequency of the electronic device is greater than a preset frequency threshold, it is determined that the electronic device is in a shaking motion state; if the shaking frequency of the electronic device is less than or equal to the preset frequency threshold, it is determined that the electronic device is in a moving motion state. The shaking frequency of the electronic device can be characterized by the number of angular velocities whose difference in angular velocity at adjacent moments is greater than a preset angular velocity difference threshold; it can also be characterized by the number of angular velocities whose directions change at adjacent moments.

[0109] In some embodiments, the current posture and the historical posture are processed based on a preset state model to obtain the motion state of the electronic device. The preset state model can be trained based on a deep learning network, for example, based on posture sample data and label values, a network such as a convolutional neural network (CNN) or a deep learning network (DNN) is trained and adjusted to obtain the state model, wherein the label value is the preset motion state.

[0110] It can be understood that if the motion state of the electronic device is shaking, the shaking has a greater impact on the current posture, so that the credibility of the current posture is lower, and the credibility of the historical posture is higher; if the motion state of the electronic device is moving (such as the user holding the device and panning to shoot), the current posture after the movement is more consistent with the posture of the current captured image, so that the credibility of the current posture is higher, and the credibility of the historical posture is lower.

[0111] In step S103, the electronic device predicts the target posture of the electronic device according to the current posture, historical posture and motion state. In some embodiments, it can be seen from the above that if the motion state of the electronic device is shaking, the credibility of the current posture is low and the credibility of the historical posture is high, so the historical posture can be used as the target posture; or the sum of the weighted result of the current posture and the weighted result of the historical posture is used as the target posture; wherein the weight of the current posture is less than the weight of the historical posture.

[0112] In some embodiments, as can be seen from the foregoing, if the motion state of the electronic device is moving, the credibility of the current posture is higher and the credibility of the historical posture is lower, so that the current posture can be used as the target posture; or the sum of the weighted result of the current posture and the weighted result of the historical posture is used as the target posture; wherein the weight of the current posture is greater than the weight of the historical posture.

[0113] In some embodiments, the current posture and the historical posture can also be smoothed and filtered to obtain the current posture after smoothing and the historical posture after smoothing; thus, based on a method similar to the above-mentioned embodiment, the target posture of the electronic device can be predicted according to the current posture after smoothing, the historical posture after smoothing and the motion state.

[0114] In some embodiments, based on a preset posture model, the current posture, historical posture and motion state are processed to obtain a target posture of the electronic device. The preset posture model can be trained based on a deep learning network, for example, based on posture sample data associated with the motion state and label values, a network such as a convolutional neural network, a deep learning network, etc. is trained and adjusted to obtain the posture model, wherein the label value is the preset target posture.

[0115] It can be understood that the target posture of the electronic device predicted by the embodiment of the present disclosure is a posture that matches the motion state of the electronic device and is determined based on the current posture and historical postures. Therefore, the target posture can represent the posture of the electronic device after reducing the influence of jitter.

[0116] After the target posture of the electronic device is determined, step S104 is executed to perform anti-shake processing on the image currently collected by the electronic device using the target posture to obtain an image after anti-shake processing.

[0117] In some embodiments, the electronic device determines the posture difference between the target posture and the current posture according to the target posture and the current posture, and calculates the transformation matrix corresponding to the image to be processed according to the posture difference; wherein the transformation matrix may include at least one of a rotation matrix and a translation matrix. Thus, the coordinates of each pixel point in the image are transformed according to the transformation matrix corresponding to the image to be processed, and the transformed coordinates of each pixel point are determined, thereby obtaining the target image.

[0118] In some embodiments, based on the aforementioned transformation matrix, the coordinates of the pixel points outside the preset center range in the image are transformed to determine the transformed coordinates of each pixel point outside the preset center range, thereby obtaining the target image based on the transformed coordinates of the pixel points outside the preset center range and the coordinates of the pixel points within the preset center range; wherein, if the transformed coordinates of the pixel points outside the preset center range overlap with the coordinates of the pixel points within the center range, the average of the pixel values ​​of the overlapping pixel points is taken as the pixel value of the pixel point at the coordinates.

[0119] It can be understood that the different motion states of the electronic device will lead to different credibility of the current posture. For example, if the motion state of the electronic device is shaking, since the shaking has a greater impact on the current posture, the credibility of the current posture is low, the credibility of the historical posture is high, and the target posture after reducing the impact of the shaking is closer to the historical posture; if the motion state of the electronic device is moving (such as the user holding the device to pan and shoot), the current posture after moving is more consistent with the posture of the current captured image, so the credibility of the current posture is high, the credibility of the historical posture is low, and the target posture after reducing the impact of the shaking is closer to the current posture. Therefore, the embodiment of the present disclosure determines the motion state of the electronic device according to the current posture and the historical posture, and predicts the target posture of the electronic device according to the current posture, the historical posture and the motion state, so as to use the target posture after reducing the impact of the shaking to perform anti-shake processing on the image. On the one hand, the image after the anti-shake processing can be made to match the motion state of the electronic device more closely, and the adaptive anti-shake of the electronic device in different motion scenes can be achieved, and the anti-shake effect is better; on the other hand, since the future posture of the electronic device is not introduced in the embodiment of the present disclosure, real-time anti-shake can be achieved. This solution has great advantages in scenes with high requirements for anti-shake real-time performance, such as image preview and automatic driving path planning.

[0120] In some embodiments, predicting the target posture of the electronic device according to the current posture, the historical posture and the motion state includes:

[0121] Performing filtering processes in different ways on the current posture and the historical posture to obtain different filtering postures;

[0122] Based on the motion state of the electronic device, selecting a filtering posture to be interpolated from the different filtering postures;

[0123] Interpolation processing is performed based on the filtered posture to be interpolated to obtain the target posture of the electronic device.

[0124] In the embodiments of the present disclosure, the electronic device performs filtering processing on the current posture and the historical posture in different ways to obtain different filtered postures; for example, the current posture and the historical posture are low-pass filtered to obtain the posture after low-pass filtering; the current posture and the historical posture are stabilization filtered to obtain the stabilized posture (which is slightly different from the historical posture); the current posture and the historical posture are median filtered to obtain the posture after median filtering; the current posture and the historical posture are Kalman filtered to obtain the posture after Kalman filtering, etc. The present disclosure does not limit the specific filtering method.

[0125] It can be understood that the posture after the low-pass filter and the posture after the median filter mentioned above belong to the filtered posture after reducing the noise; the posture after stabilization is closer to the historical posture; the posture after the Kalman filter belongs to the predicted optimal estimated posture.

[0126] After obtaining different filtering postures, the electronic device selects the filtering posture to be interpolated from the different filtering postures based on the motion state; for example, if the motion state is jitter, the credibility of the current posture is low, and the credibility of the historical posture is high, so the posture after low-pass filtering and the posture after stabilization can be used as the posture to be interpolated, reducing the impact of the current posture with low credibility on the target posture.

[0127] For example, if the motion state is moving, the credibility of the current posture is higher and the credibility of the historical posture is lower, so the posture after median filtering and the posture after Kalman filtering can be used as the posture to be interpolated to reduce the impact of the historical posture with lower credibility on the target posture.

[0128] After determining the posture to be interpolated, the electronic device performs interpolation processing on the filtered posture to be interpolated based on the nearest neighbor interpolation algorithm, linear interpolation algorithm, bilinear interpolation algorithm, high-order interpolation algorithm and other algorithms to obtain the target posture of the electronic device.

[0129] It can be understood that, on the one hand, the embodiments of the present disclosure filter the current posture and the historical posture in different ways to obtain different filtering postures, which can reduce noise interference; on the other hand, based on the motion state of the electronic device, the filtering posture to be interpolated is selected from different filtering postures, and then interpolation processing is performed based on the filtering posture to be interpolated to obtain the target posture of the electronic device, which can reduce the impact of postures with lower credibility in the motion state on the target posture and improve the accuracy of the target posture.

[0130] In some embodiments, the interpolation processing is performed based on the filtered posture to be interpolated to obtain the target posture of the electronic device, including:

[0131] Determining an interpolation coefficient corresponding to the filtering posture to be interpolated based on the duration of the motion state and the filtering posture to be interpolated;

[0132] An interpolation process is performed based on the filter posture to be interpolated and the interpolation coefficient corresponding to the filter posture to be interpolated to obtain the target posture of the electronic device.

[0133] In the disclosed embodiment, the duration of the motion state represents the duration of maintaining the same motion state. The electronic device determines the interpolation coefficient corresponding to the filtering posture to be interpolated based on the duration of the motion state and the filtering posture to be interpolated. In some embodiments, based on a preset coefficient mapping relationship, the interpolation coefficient of the filtering posture to be interpolated corresponding to the duration of the motion state is determined; wherein the preset coefficient mapping relationship is a mapping relationship obtained through multiple experiments under different motion states, including the mapping between the motion state, the duration, and the coefficients of the filtering posture to be interpolated.

[0134] In some embodiments, based on a preset function corresponding to the motion state, the duration of the motion state is processed to obtain an interpolation coefficient corresponding to the filtering posture to be interpolated; wherein the preset function is a function equation obtained by fitting multiple tests under different motion states.

[0135] After determining the interpolation coefficient corresponding to the filter posture to be interpolated, the electronic device of the embodiment of the present disclosure performs interpolation processing based on the filter posture to be interpolated and the interpolation coefficient corresponding to the filter posture to be interpolated to obtain the target posture of the electronic device. In some embodiments, for each filter posture to be interpolated, the filter posture is weighted using the interpolation coefficient corresponding to the filter posture to obtain a weighted result; the sum of the weighted results corresponding to each filter posture to be interpolated is used as the target posture after linear difference.

[0136] In some embodiments, each interpolation coefficient is processed based on a nonlinear function, and the difference coefficients after each processing are used to weight the filter posture to be interpolated, and the sum of the weighted results corresponding to each filter posture to be interpolated is used as the target posture after nonlinear difference. If the filter posture includes displacement, the nonlinear function can be a quadratic function after multiple test fitting; if the filter posture includes Euler angles, the nonlinear function can be a function including sine components and / or cosine components after multiple test fitting, etc.

[0137] Taking the motion state of jitter as an example, the posture to be interpolated includes the posture after low-pass filtering and the posture after stabilization; if the target posture is the sum of the weighted result of the posture after low-pass filtering and the weighted result of the posture after stabilization, the longer the duration of the jitter state, the lower the credibility of the corresponding current posture, and the higher the credibility of the historical posture, so that the weight corresponding to the posture after stabilization can be increased, and the weight of the posture after low-pass filtering can be reduced, thereby improving the matching degree between the target posture and the duration of the jitter state.

[0138] Taking the motion state of moving as an example, the posture to be interpolated includes the posture after median filtering and the posture after Kalman filtering; if the target posture is the sum of the weighted result of the posture to be interpolated and the weighted result of the posture after Kalman filtering, the longer the duration of the moving state, the higher the credibility of the corresponding current posture, and the lower the credibility of the historical posture, so that the weight corresponding to the posture after Kalman filtering can be increased, and the weight of the posture after median filtering can be reduced, thereby improving the matching degree between the target posture and the duration of the moving state.

[0139] It is understandable that due to the different durations of the motion state, the current posture and the historical posture have different effects on the target posture. In this regard, the embodiment of the present disclosure dynamically determines the interpolation coefficient corresponding to the filter posture to be interpolated based on the duration of the motion state and the filter posture to be interpolated, and then performs interpolation processing based on the filter posture to be interpolated and the interpolation coefficient corresponding to the filter posture to be interpolated to obtain the target posture of the electronic device; compared with the method of fixing the interpolation coefficient, the embodiment of the present disclosure can improve the accuracy of the interpolation coefficient of the filter posture to be interpolated dynamically allocated based on the duration of the motion state and the degree of matching with the motion scene, thereby helping to improve the accuracy of determining the target posture.

[0140] In some embodiments, the filtering posture to be interpolated includes two filtering postures, the filtering postures are represented by quaternions, and the interpolation process includes spherical linear interpolation of quaternions;

[0141] The determining, based on the duration of the motion state and the filter posture to be interpolated, an interpolation coefficient corresponding to the filter posture to be interpolated includes:

[0142] Determining weights corresponding to two filter postures to be interpolated in the spherical linear interpolation of the quaternion based on the duration of the motion state;

[0143] Determining a cosine angle between the two filtered postures to be interpolated;

[0144] Based on the cosine angle and the weights corresponding to the two filtering postures, interpolation coefficients corresponding to the filtering postures to be interpolated are determined.

[0145] In the embodiment of the present disclosure, the filtering posture to be interpolated includes two filtering postures, the filtering postures are represented by quaternions, and the interpolation processing includes spherical linear interpolation of the quaternions; wherein the spherical linear interpolation equation of the quaternion can be expressed by the following formula (1):

[0146]

[0147] Among them, SLERP represents spherical linear interpolation, q represents the target attitude after interpolation, q1 and q2 represent two quaternions to be interpolated, θ represents the cosine angle between the two quaternions to be interpolated; ratio represents the weight corresponding to the filter attitude in the spherical linear interpolation of the quaternion. If the motion state of the electronic device is jitter, q1 can represent the filter attitude after stabilization, and q2 can represent the attitude after low-pass filtering; if the operating state of the electronic device is moving, q1 can represent the filter attitude after Kalman filtering, and q2 can represent the attitude after low-pass filtering.

[0148] The electronic device of the disclosed embodiment determines the weights corresponding to two filtering postures to be interpolated in the spherical linear interpolation of the quaternion based on the duration of the motion state; for example, if the motion state is shaking, the two filtering postures to be interpolated are the posture after low-pass filtering and the posture after stabilization, and based on the preset weight mapping relationship corresponding to the shaking state, the weight of the posture after low-pass filtering corresponding to the duration of the shaking state and the weight of the posture after stabilization are determined; wherein the preset weight mapping relationship corresponding to the shaking state includes the mapping between the duration, the weight of the posture after low-pass filtering, and the weight of the posture after stabilization.

[0149] For example, the motion state is moving, and the two filtered postures to be interpolated are the posture after low-pass filtering and the posture after Kalman filtering. Based on the weight mapping relationship corresponding to the preset moving state, the weight of the posture after low-pass filtering corresponding to the duration of the moving state and the weight of the posture after Kalman filtering are determined; wherein, the weight mapping relationship corresponding to the preset moving state includes the mapping between the duration, the weight of the posture after low-pass filtering, and the weight of the posture after Kalman filtering.

[0150] In addition, the electronic device of the embodiment of the present disclosure further determines the cosine angle between the two filtering postures to be interpolated; wherein the cosine angle θ can be determined by the following formula (2):

[0151] θ=cos -1 (q1.q2); (2)

[0152] After determining the weights corresponding to the two filter postures to be interpolated and the cosine angle between the two filter postures to be interpolated in the spherical linear interpolation of the quaternion, the electronic device determines the interpolation coefficients corresponding to the filter postures to be interpolated based on the cosine angle and the weights corresponding to the two filter postures, as in the above formula (1), is the difference coefficient corresponding to q1, is the difference coefficient corresponding to q2. Alternatively, based on a preset difference coefficient formula, the cosine angle and the weights corresponding to the two filter postures are processed to obtain the interpolation coefficients corresponding to each filter posture to be interpolated; wherein the preset difference coefficient formula may be obtained by fitting multiple tests under different durations of motion states, and the present disclosure does not limit this.

[0153] It can be understood that the embodiment of the present disclosure utilizes a spherical linear interpolation algorithm including quaternions to perform interpolation processing based on two filter postures to be interpolated, which can improve the rationality and accuracy of interpolation of filter postures represented by quaternions; in addition, the embodiment of the present disclosure dynamically determines the weights corresponding to the two filter postures to be interpolated in the spherical linear interpolation of quaternions based on the duration of the motion state, which can improve the degree of matching between the weights and the motion scene of the electronic device, and further improve the accuracy of determining the interpolation coefficients corresponding to each filter posture to be interpolated based on the weights and cosine angles.

[0154] In some embodiments, the filtering of the current posture and the historical posture in different ways to obtain different filtered postures includes:

[0155] Performing weighted processing on the current posture and the historical posture respectively, and obtaining a first filtered posture based on the weighted postures;

[0156] Performing Kalman filtering on the current posture and the historical posture to obtain a second filtered posture;

[0157] The current posture is weighted, and a third filtering posture is obtained based on the weighted current posture and the historical posture.

[0158] In the disclosed embodiment, the electronic device performs weighted processing on the current posture and the historical posture respectively, and obtains a first filtered posture based on each weighted posture, such as the posture after the low-pass filter mentioned above. Exemplarily, the first filtered posture candQtLPF can be expressed by the following formula (3):

[0159] candQtLPF=(1-weight LPF )*Q′ -1 +weight LPF *Q0; (3)

[0160] Among them, Q0 represents the current posture, Q′ -1 Indicates historical posture, weight LPF Indicates the weight of the current posture in the first filtering posture, (1-weight LPF ) represents the weight of the historical posture in the first filtering posture.

[0161] The electronic device of the disclosed embodiment performs weighted processing on the current posture, and obtains a third filtered posture based on the weighted current posture and the historical posture, such as the posture after stabilization. Exemplarily, the third filtered posture candQtSuperStable can be expressed by the following formula (4):

[0162] candQtSuperStable=Q′ -1 +weight ss *Q0; (4)

[0163] Among them, weight SS Represents the weight of the current posture in the third filtering posture.

[0164] The embodiment of the present disclosure performs Kalman filtering on the current posture and the historical posture to obtain a second filtered posture, such as the posture after Kalman filtering described above. The second filtered posture is obtained based on the following formulas (5) to (8):

[0165]

[0166] in, represents the prior estimated posture, F is the preset state transfer matrix, represents the historical posture, B is the preset control matrix, u t-1 It is the preset input control amount.

[0167]

[0168] in, represents the prior estimated covariance, P b-1 is the preset covariance matrix, and G is the preset process noise covariance matrix.

[0169]

[0170] Among them, K t represents the Kalman gain, H is the preset measurement matrix, and R is the preset observation noise covariance matrix.

[0171]

[0172] in, represents the a posteriori estimated posture, corresponding to the aforementioned second filtering posture; z t Represents the actual observation value, corresponding to the current posture mentioned above.

[0173] In addition, the Kalman filter formula also includes the posterior estimated covariance, which can be expressed as follows:

[0174]

[0175] It should be noted that if the current posture includes the current quaternion and the historical posture includes the historical quaternion, the posture in the above formula can be represented by a 7-dimensional state quantity (w, X, Y, Z, x, y, z) including the quaternion, where (w, X, Y, Z) represents the quaternion and (x, y, z) represents the three-axis components of a posture sensor (such as a gyroscope).

[0176] Then, the second posture candQtKalman of the embodiment of the present disclosure can be expressed based on the following formula (10):

[0177]

[0178] in, Indicates taking The first 4 bits of are used to obtain the second filtered attitude represented by the quaternion.

[0179] It can be understood that the embodiment of the present disclosure performs weighted processing on the current posture and the historical posture respectively, and obtains the first filtered posture based on each weighted posture, which can improve the flexibility of adjusting the first filtered posture as needed by adjusting the weight of the current posture and the weight of the historical posture. In addition, the Kalman filter algorithm is a relatively mature prediction algorithm. The embodiment of the present disclosure performs Kalman filtering on the current posture and the historical posture, which can improve the accuracy of the second filtered posture. The embodiment of the present disclosure performs weighted processing on the current posture, and obtains the third filtered posture based on the current posture and the historical posture after weighted processing, which can improve the convenience of adjusting the influence of the current posture on the third filtered posture by adjusting the weight of the current posture.

[0180] In some embodiments, the weighting of the current posture and the historical posture is performed respectively, and the first filtered posture is obtained based on the weighted postures, including:

[0181] The sum of the result of weighting the current posture with the first preset weight and the result of weighting the historical posture with the second preset weight is determined as the first filtering posture; wherein the first preset weight is positively correlated with the difference between the current posture and the historical posture; the sum of the first preset weight and the second preset weight is the second preset value.

[0182] In the embodiment of the present disclosure, the first preset weight corresponds to the aforementioned weight LPF ; Taking the second preset value as 1 as an example, the sum of the first preset weight and the second preset weight is 1, that is, the second preset weight is 1-weight LPF .

[0183] Since the first preset weight represents the influence of the current posture on the first filtering posture, and if the difference Diff(Q′) between the current posture and the historical posture -1 , Q0) is large, indicating that the possibility of being in a moving state is high, so the credibility of the current posture is large, and the credibility of the historical posture is small, so the first preset weight of the embodiment of the present disclosure is positively correlated with the difference between the current posture and the historical posture, and the second preset weight is negatively correlated with the difference between the current posture and the historical posture. Through this scheme, the accuracy of determining the sum of the result after the first preset weight is weighted on the current posture and the result after the second preset weight is weighted on the historical posture as the first filtering posture can be improved.

[0184] In some embodiments, performing weighted processing on the current posture and obtaining a third filtered posture based on the weighted current posture and the historical posture includes:

[0185] The sum of a result of weighting the current posture with a third preset weight and the historical posture is determined as the third filtering posture; wherein the third preset weight is less than a preset weight threshold.

[0186] In the embodiment of the present disclosure, the third preset weight corresponds to the aforementioned weight SS ; The electronic device of the disclosed embodiment determines the sum of the result of weighting the current posture by the third preset weight and the historical posture as the third filtered posture; wherein the third preset weight is less than the preset weight threshold, and illustratively, the third preset weight is 0.00001. Through this solution, the third filtered posture is closer to the historical posture, and the accuracy of determining the target posture based on the posture to be interpolated including the third filtered posture is improved when the running state includes the jitter state.

[0187] In some embodiments, the motion state includes shaking, moving, and stillness; and selecting a filtering posture to be interpolated from the different filtering postures based on the motion state of the electronic device includes:

[0188] In a case where the motion state includes shaking, determining the third filtering posture and the first filtering posture as the filtering postures to be interpolated;

[0189] In a case where the motion state includes moving or being stationary, the first filtering posture and the second filtering posture are determined as the filtering postures to be interpolated.

[0190] In the embodiment of the present disclosure, the motion state includes shaking, moving, and stillness. When the motion state of the electronic device is determined to be shaking according to the current posture and the historical posture, the third filtering posture and the first filtering posture are determined as the filtering postures to be interpolated. Since the difference between the current posture and the historical posture in the shaking state is small, the credibility of the historical posture is higher; it can be seen from the positive correlation between the aforementioned first preset weight and the difference between the current posture and the historical posture that the historical posture has a greater impact on the first filtering posture, and since the third filtering posture is closer to the historical posture, therefore, in the case where the motion state includes shaking, the embodiment of the present disclosure determines the third filtering posture and the first filtering posture as the filtering postures to be interpolated, which can improve the accuracy of determining the target posture in the shaking state based on the filtering posture to be interpolated.

[0191] When the motion state of the electronic device is determined to be moving or stationary based on the current posture and the historical posture, the first filtering posture and the second filtering posture are determined as the filtering postures to be interpolated. Since the difference between the current posture and the historical posture in the moving state is large, the credibility of the current posture is high; and since the third filtering posture is closer to the historical posture, therefore, when the motion state includes moving, the embodiment of the present disclosure determines the second filtering posture and the first filtering posture as the filtering postures to be interpolated, which can improve the accuracy of determining the target posture in the shaking state based on the filtering posture to be interpolated. In addition, since the credibility of the current posture in the stationary state is very high, therefore, when the motion state includes stationary, the embodiment of the present disclosure determines the second filtering posture and the first filtering posture as the filtering postures to be interpolated, which can also improve the accuracy of determining the target posture in the stationary state based on the filtering posture to be interpolated.

[0192] In some embodiments, the method further comprises:

[0193] Get the duration threshold;

[0194] The step of determining weights corresponding to two filter postures to be interpolated in the spherical linear interpolation of the quaternion based on the duration of the motion state includes:

[0195] In response to the duration of the motion state being greater than a duration threshold, determining weights corresponding to two filter postures to be interpolated in the spherical linear interpolation of the quaternion; wherein the weight of a preset initial filter posture in the two filter postures to be interpolated is positively correlated with the duration of the motion state, and the sum of the weights corresponding to the two filter postures to be interpolated is a first preset value;

[0196] In response to the duration of the motion state being less than or equal to the duration threshold, the weight of the preset initial filtering posture is determined to be the first preset value, and the weight of the filtering posture to be interpolated other than the preset initial filtering posture is 0.

[0197] In the embodiment of the present disclosure, the electronic device obtains the duration threshold by obtaining a stored preset duration threshold, or obtains the duration threshold corresponding to the motion state based on the correspondence between the preset motion state and the duration threshold.

[0198] The preset initial filtering posture in the embodiment of the present disclosure is any one of the aforementioned filtering postures. Exemplarily, the preset initial filtering posture is the first filtering posture.

[0199] In response to the duration of the motion state being greater than the duration threshold, the disclosed embodiment determines the weights corresponding to two filter postures to be interpolated in the spherical linear interpolation of the quaternion; wherein the weight of the preset initial filter posture in the two filter postures to be interpolated is positively correlated with the duration of the motion state, and the sum of the weights corresponding to the two filter postures to be interpolated is the first preset value, that is, the weight corresponding to the other filter posture to be interpolated is negatively correlated with the duration of the motion state. Exemplarily, the first preset value is 1.

[0200] Figure 2 An example diagram of weights corresponding to two filter postures to be interpolated in the spherical linear interpolation of quaternions according to the embodiment of the present disclosure. Figure 2 , taking the preset initial filtering posture as the first filtering posture as an example, if the motion state is jitter, the two filtering postures to be interpolated include the first filtering posture and the third filtering posture; if the jitter duration is less than or equal to the duration threshold, the weight of the first filtering posture is located at the initial filtering posture and does not move. At this time, the corresponding weight of the first filtering posture is the first preset value, and the weight of the third filtering posture is 0. If the jitter duration is greater than the duration threshold, the weight of the first filtering posture moves from the initial filtering posture to the third filtering posture (moves in the direction of weight reduction), and the moving distance is positively correlated with the duration of the jitter. As the duration of the jitter state increases, the corresponding weight of the first filtering posture decreases, and the weight of the third filtering posture increases.

[0201] It should be noted that in the process of the weight of the first filtering posture moving from the initial filtering posture to the third filtering posture, if the motion state of the electronic device changes to moving, the weight of the first filtering posture moves in the opposite direction (moves in the direction of increasing weight). As the duration of the moving state increases, the corresponding weight of the first filtering posture increases, and the weight of the third filtering posture decreases.

[0202] Still taking the preset initial filtering posture as the first filtering posture as an example, if the motion state is moving, the two filtering postures to be interpolated include the first filtering posture and the second filtering posture; if the moving duration is less than or equal to the duration threshold, the weight of the first filtering posture is located at the initial filtering posture and does not move. At this time, the corresponding weight of the first filtering posture is the first preset value, and the weight of the second filtering posture is 0. If the moving duration is greater than the duration threshold, the weight of the first filtering posture moves from the initial filtering posture to the second filtering posture (moves in the direction of weight reduction), and the moving distance is positively correlated with the duration of the jitter. As the duration of the moving state increases, the corresponding weight of the first filtering posture decreases, and the weight of the second filtering posture increases.

[0203] It should be noted that in the process of the weight of the first filtering posture moving from the initial filtering posture to the second filtering posture, if the motion state of the electronic device changes to shaking, the weight of the first filtering posture moves in the opposite direction (moves in the direction of increasing weight). As the duration of the shaking state increases, the corresponding weight of the first filtering posture increases, and the weight of the second filtering posture decreases.

[0204] It is understandable that due to the influence of noise on the current posture and / or historical posture, the filtering posture to be interpolated is also affected by noise. In this regard, in the embodiment of the present disclosure, when the duration of the motion state is less than or equal to the duration threshold, the weight of the preset initial filtering posture is determined to be the first preset value, and the weight of the filtering posture to be interpolated other than the preset initial filtering posture is 0, which can reduce the influence of noise on the fluctuation of the weight. In addition, in the embodiment of the present disclosure, when the duration of the motion state is greater than the duration threshold, that is, the motion state is relatively stable, the weight of the preset initial filtering posture is determined to be positively correlated with the duration of the motion state, and the sum of the weights corresponding to the two filtering postures to be interpolated is the first preset value, which can improve the stability and accuracy of determining the weight corresponding to the filtering posture to be interpolated.

[0205] In some embodiments, the method further comprises:

[0206] Acquire the focal length when the electronic device captures the image;

[0207] The acquisition time threshold includes:

[0208] In the case where the focal length belongs to a preset focal length, determining a duration threshold corresponding to the focal length according to a preset mapping relationship; wherein the preset mapping relationship includes a mapping between a focal length and a duration threshold;

[0209] or,

[0210] When the focal length does not belong to the preset focal length, the duration threshold corresponding to the focal length is determined according to the duration thresholds respectively corresponding to a first preset focal length that is greater than the focal length and a second preset focal length that is less than the focal length; wherein the first preset focal length and the second preset focal length both belong to the preset focal length.

[0211] In an embodiment of the present disclosure, the focal length of the electronic device when capturing the image is obtained; wherein the focal length can be represented by a magnification, such as 20X, 30X, or by an equivalent focal length, such as 12mm, 120mm.

[0212] In the embodiment of the present disclosure, when the focal length belongs to a preset focal length, the duration threshold corresponding to the focal length is determined according to the mapping relationship between the preset focal length and the duration threshold. Through this solution, the convenience of determining the duration threshold corresponding to the commonly used focal length can be improved.

[0213] In the case where the focal length does not belong to the preset focal length, the embodiment of the present disclosure determines the duration threshold corresponding to the focal length according to the duration thresholds corresponding to the first preset focal length greater than the focal length and the second preset focal length less than the focal length. For example, if the preset focal lengths are 30X, 100X, and 120X, then the duration threshold corresponding to the focal length of 40X can be the average of the duration threshold corresponding to 30X and the duration threshold corresponding to 120X; or, based on different duration weights, the weighted sum of the duration threshold corresponding to 30X and the duration threshold corresponding to 120X, which is not limited in the present disclosure.

[0214] It can be understood that the embodiment of the present disclosure determines the duration threshold corresponding to the focal length when the electronic device captures the image, which can improve the matching degree between the anti-shake processing of the image based on the target posture determined in combination with the duration threshold and the focal length of the image, thereby improving the anti-shake processing effect; in addition, the embodiment of the present disclosure supports determining the duration threshold of any focal length, is easy to expand, and can dynamically adjust the duration threshold parameters according to different focal lengths and scenes, thereby making up for the insufficient performance of existing anti-shake algorithms under ultra-telephoto (such as 120X).

[0215] It should be noted that the embodiments of the present disclosure can also determine parameters corresponding to other focal lengths, such as sensitivity, shutter speed, etc., based on the above method. For example, when the focal length belongs to a preset focal length, the shutter speed corresponding to the focal length is determined according to a preset mapping relationship; wherein the preset mapping relationship includes a mapping between the focal length and the shutter speed;

[0216] Or, when the focal length does not belong to the preset focal length, the shutter speed corresponding to the focal length is determined according to the shutter speeds respectively corresponding to a first preset focal length greater than the focal length and a second preset focal length less than the focal length.

[0217] In some embodiments, the first preset focal length and the second preset focal length are both preset focal lengths adjacent to the focal length; and determining the duration threshold corresponding to the focal length according to the duration thresholds respectively corresponding to the first preset focal length that is greater than the focal length and the second preset focal length that is less than the focal length, comprises:

[0218] Determine the first focal length weight of the first preset focal length by the ratio of the difference between the focal length and the second preset focal length to the difference between the first preset focal length and the second preset focal length;

[0219] Determine a ratio of a difference between the first preset focal length and the focal length and a difference between the first preset focal length and the second preset focal length as a second focal length weight of the second preset focal length;

[0220] The sum of the result of weighting the preset duration threshold corresponding to the first preset focal length by the first focal length weight and the result of weighting the preset duration threshold corresponding to the second preset focal length by the second focal length weight is determined as the duration threshold corresponding to the focal length.

[0221] In the embodiment of the present disclosure, the first preset focal length and the second preset focal length are both preset focal lengths adjacent to the focal length; for example, the preset focal lengths are 20X, 30X, 70X, and 100X, and the first preset focal length corresponding to the focal length of 40X is 30X, and the second preset focal length is 70X.

[0222] The embodiment of the present disclosure determines the time length threshold parm corresponding to the focal length (such as 40X) when the electronic device captures the image based on the time length threshold corresponding to the first preset focal length and the second preset focal length. 40X It can be expressed by the following formula (11):

[0223]

[0224] Among them, parm 30X Indicates the duration threshold corresponding to the first preset focal length, parm 70X represents the duration threshold corresponding to the second preset focal length, 40 represents the focal length when the electronic device captures the image, 30 represents the first preset focal length, and 70 represents the second preset focal length. Through this solution, the convenience of determining the duration threshold corresponding to the non-preset focal length can be improved.

[0225] In some embodiments, the current posture includes a current angular velocity, and the historical posture includes a historical angular velocity; and determining the motion state of the electronic device according to the current posture and the historical posture includes:

[0226] respectively determining whether the current angular velocity and the historical angular velocity are between a first preset angular velocity threshold and a second preset angular velocity threshold;

[0227] When the current angular velocity and the historical angular velocity are both between the first preset angular velocity threshold and the second preset angular velocity threshold, determining that the motion state of the electronic device is shaking;

[0228] or,

[0229] When the current angular velocity or the historical angular velocity is not between the first preset angular velocity threshold and the second preset angular velocity threshold, the motion state of the electronic device is determined according to the current angular velocity and the historical angular velocity.

[0230] In the embodiment of the present disclosure, the current posture includes the current angular velocity, and the historical posture includes the historical angular velocity. Compared with the moving state, the angular velocity of the electronic device is smaller in the shaking state; compared with the stationary state, the angular velocity of the electronic device is larger in the shaking state. In this regard, the embodiment of the present disclosure respectively determines whether the current angular velocity and the historical angular velocity are between the first preset angular velocity threshold and the second preset angular velocity threshold, and determines that the motion state of the electronic device is shaking when the current angular velocity and the historical angular velocity are both between the first preset angular velocity threshold and the second preset angular velocity threshold. Among them, the first preset angular velocity threshold is a threshold for distinguishing between shaking and stationary states, and the second preset angular velocity threshold is a threshold for distinguishing between shaking and moving states.

[0231] If the current angular velocity or the historical angular velocity is not between the first preset angular velocity threshold and the second preset angular velocity threshold, the motion state of the electronic device is determined according to the current angular velocity and the historical angular velocity. This solution can improve the convenience and immediacy of determining the shaking state, and eliminate the necessity of further determining the motion state of the electronic device when the current angular velocity or the historical angular velocity is not between the first preset angular velocity threshold and the second preset angular velocity threshold.

[0232] In some embodiments, when the current angular velocity or the historical angular velocity is greater than or equal to the first preset angular velocity threshold, or less than or equal to the second preset angular velocity threshold, determining the motion state of the electronic device according to the current angular velocity and the historical angular velocity includes:

[0233] In a case where the current angular velocity or the historical angular velocity is greater than or equal to the first preset angular velocity threshold, or less than or equal to the second preset angular velocity threshold, according to the current angular velocity and the historical angular velocity, in response to satisfying one of the following: the shaking frequency of the electronic device is greater than a preset frequency threshold, the direction of the angular velocity changes, and the duration of the change in the direction of the angular velocity is greater than a first preset duration threshold, determining that the motion state of the electronic device is shaking;

[0234] In response to the current angular velocity and the historical angular velocity being both less than the first preset angular velocity threshold, determining that the motion state of the electronic device is stationary;

[0235] In response to satisfying one of the following conditions: the shaking frequency is less than or equal to the preset frequency threshold, the direction of the angular velocity has not changed, and the duration of the change in the direction of the angular velocity is less than or equal to the first preset duration threshold, the motion state of the electronic device is determined to be moving.

[0236] In the disclosed embodiment, if the current angular velocity and the historical angular velocity are both less than the first preset angular velocity threshold, that is, the current angular velocity and the historical angular velocity are both very small, it is likely that the electronic device is stable (such as the electronic device is mounted on a tripod or a gimbal), so the motion state of the electronic device is determined to be stationary, or a character representing stationary, such as "1", is output. Since the motion state of the electronic device is stationary, the credibility of the current posture of the electronic device is very high, and the hand tracking performance after anti-shake based on the determined target posture is good in this scenario.

[0237] The historical angular velocity of the disclosed embodiment may also include angular velocities at multiple different moments. If there is a current angular velocity or a historical angular velocity that is not between the first preset angular velocity threshold and the second preset angular velocity threshold, based on the current angular velocity and the historical angular velocity, in response to satisfying one of the following: the jitter frequency of the electronic device is greater than the preset frequency threshold, the direction of the angular velocity changes, and the duration of the change in the direction of the angular velocity is greater than the first preset duration threshold, the motion state of the electronic device is determined to be jitter, or a character representing the jitter, such as "0", is output. Among them, the jitter frequency of the electronic device can be characterized by the number of angular velocities whose difference in angular velocity at adjacent moments is greater than the preset angular velocity difference threshold; it can also be characterized by the number of angular velocities whose direction of angular velocity changes at adjacent moments.

[0238] It can be understood that when the electronic device is in a shaking state, there is a high possibility that the shaking frequency is large, the direction of the angular velocity changes, and the duration of the change in the direction of the angular velocity is long. Therefore, the embodiment of the present disclosure responds to one of the following: the shaking frequency of the electronic device is greater than a preset frequency threshold, the direction of the angular velocity changes, and the duration of the change in the direction of the angular velocity is greater than a first preset duration threshold, and the accuracy of determining that the motion state of the electronic device is shaking is high.

[0239] In response to one of the following being met: the shaking frequency is less than or equal to the preset frequency threshold, the direction of the angular velocity has not changed, and the duration of the change in the direction of the angular velocity is less than or equal to the first preset duration threshold, the motion state of the electronic device is determined to be moving, or a character representing movement, such as "1", is output. Among them, since the credibility of the current posture is high in both the moving state and the static state, the character representing movement and the character representing static can be set to be the same, thereby simplifying the logic of determining the target posture based on the subsequent motion state.

[0240] It can be understood that when the electronic device is in a moving state, there is a high possibility that the jitter frequency is small, the direction of the angular velocity has not changed, and the duration of the change in the direction of the angular velocity is short (such as continuing to pan and shoot the scene after the hand hits the wall during the panning and shooting of a handheld electronic device). Therefore, the embodiment of the present disclosure responds to one of the following: the jitter frequency is less than or equal to the preset frequency threshold, the direction of the angular velocity has not changed, and the duration of the change in the direction of the angular velocity is less than or equal to the first preset duration threshold, and the accuracy of determining that the motion state of the electronic device is moving is higher.

[0241] It should be noted that the determination of whether one of the above conditions is met based on the current angular velocity and the historical angular velocity can be based on a state machine to determine whether all the above conditions are met, or can be based on different state machines to determine whether one condition is met, and the present disclosure does not limit this. The state machines designed for different scenarios make up for the shortcomings of common anti-shake algorithms in a single scenario.

[0242] In some embodiments, performing anti-shake processing on the image using the target posture to obtain the anti-shake processed image includes:

[0243] Transforming the coordinates of the pixel points in the image using the target posture to obtain an image after coordinate transformation;

[0244] The image after the coordinate transformation is cropped to obtain the image after the anti-shake processing.

[0245] In the disclosed embodiment, after determining the target posture, the electronic device transforms the coordinates of the pixel points in the image using the target posture to obtain the image after the coordinate transformation; in some embodiments, the electronic device determines the posture difference between the target posture and the current posture according to the target posture and the current posture, and calculates the transformation matrix corresponding to the image to be processed according to the posture difference; wherein the transformation matrix may include at least one of a rotation matrix and a translation matrix. Thus, the coordinates of each pixel point in the image are transformed according to the transformation matrix corresponding to the image to be processed, and the transformed coordinates of each pixel point are determined, thereby obtaining the target image.

[0246] In some embodiments, after determining the target posture, the electronic device generates a homography matrix using the target posture and the current posture, and then transforms the coordinates of the pixel points in the image using the homography matrix to obtain an image after the coordinate transformation.

[0247] The electronic device of the embodiment of the present disclosure crops the image after the coordinate transformation, such as performing center cropping, random cropping, etc., to obtain an image after anti-shake processing. Figure 3 is an example diagram of image comparison before and after anti-shake processing shown in an embodiment of the present disclosure, Figure 3 In the figure, L1 marks the image before anti-shake processing, and L2 marks the image after anti-shake processing; among them, in the image marked by L2, the part marked by the dotted frame M1 is the maximum field of view that does not include the black border, and the part marked by the dotted frame M2 is the image after anti-shake processing and center cropping.

[0248] Figure 4 is a diagram of an image processing device shown in an embodiment of the present disclosure, comprising Figure 4 It is known that, including:

[0249] A first acquisition module 401 is used to acquire a current posture of the electronic device when capturing an image, and a historical posture of the electronic device before capturing the image;

[0250] A determination module 402, configured to determine a motion state of the electronic device according to the current posture and the historical posture;

[0251] A prediction module 403, configured to predict a target posture of the electronic device according to the current posture, the historical posture and the motion state;

[0252] The anti-shake module 404 is used to perform anti-shake processing on the image using the target posture to obtain an image after anti-shake processing.

[0253] In some embodiments, the prediction module 403 is used to filter the current posture and the historical posture in different ways to obtain different filtering postures; based on the motion state of the electronic device, select the filtering posture to be interpolated from the different filtering postures; and perform interpolation processing based on the filtering posture to be interpolated to obtain the target posture of the electronic device.

[0254] In some embodiments, the prediction module 403 is used to determine the interpolation coefficient corresponding to the filtering posture to be interpolated based on the duration of the motion state and the filtering posture to be interpolated; and perform interpolation processing based on the filtering posture to be interpolated and the interpolation coefficient corresponding to the filtering posture to be interpolated to obtain the target posture of the electronic device.

[0255] In some embodiments, the filtering posture to be interpolated includes two filtering postures, the filtering postures are represented by quaternions, and the interpolation process includes spherical linear interpolation of quaternions;

[0256] The prediction module 403 is used to determine the weights corresponding to the two filter postures to be interpolated in the spherical linear interpolation of the quaternion based on the duration of the motion state; determine the cosine angle between the two filter postures to be interpolated; and determine the interpolation coefficients corresponding to each filter posture to be interpolated based on the cosine angle and the weights corresponding to the two filter postures.

[0257] In some embodiments, the apparatus further comprises:

[0258] The second acquisition module 405 is used to acquire a duration threshold;

[0259] The prediction module 403 is used to determine the weights corresponding to two filtering postures to be interpolated in the spherical linear interpolation of the quaternion in response to the duration of the motion state being greater than a duration threshold; wherein the weight of a preset initial filtering posture in the two filtering postures to be interpolated is positively correlated with the duration of the motion state, and the sum of the weights corresponding to the two filtering postures to be interpolated is a first preset value; in response to the duration of the motion state being less than or equal to the duration threshold, the weight of the preset initial filtering posture is determined to be the first preset value, and the weight of the filtering posture to be interpolated other than the preset initial filtering posture is 0.

[0260] In some embodiments, the apparatus further comprises:

[0261] A third acquisition module 406 is used to acquire the focal length when the electronic device captures the image;

[0262] The second acquisition module 405 is used to determine the duration threshold corresponding to the focal segment according to a preset mapping relationship when the focal segment belongs to the preset focal segment; wherein the preset mapping relationship includes a mapping between the focal segment and the duration threshold; or, when the focal segment does not belong to the preset focal segment, determine the duration threshold corresponding to the focal segment according to the duration thresholds respectively corresponding to a first preset focal segment greater than the focal segment and a second preset focal segment less than the focal segment; wherein the first preset focal segment and the second preset focal segment both belong to the preset focal segment.

[0263] In some embodiments, the second acquisition module 405 is used to determine the first focal length weight of the first preset focal length by the ratio of the difference between the focal length and the second preset focal length and the difference between the first preset focal length and the second preset focal length; determine the second focal length weight of the second preset focal length by the ratio of the difference between the first preset focal length and the focal length and the difference between the first preset focal length and the second preset focal length; and determine the sum of the result of weighting the preset duration threshold corresponding to the first preset focal length by the first focal length weight and the result of weighting the preset duration threshold corresponding to the second preset focal length by the second focal length weight as the duration threshold corresponding to the focal length.

[0264] In some embodiments, the prediction module 403 is used to perform weighted processing on the current posture and the historical posture respectively, and obtain a first filtered posture based on each weighted posture; perform Kalman filtering on the current posture and the historical posture to obtain a second filtered posture; perform weighted processing on the current posture, and obtain a third filtered posture based on the weighted current posture and the historical posture.

[0265] In some embodiments, the prediction module 403 is used to determine the sum of the result of weighting the current posture with the first preset weight and the result of weighting the historical posture with the second preset weight as the first filtering posture; wherein the first preset weight is positively correlated with the difference between the current posture and the historical posture; the sum of the first preset weight and the second preset weight is the second preset value.

[0266] In some embodiments, the prediction module 403 is used to determine the sum of the result of weighting the current posture with a third preset weight and the historical posture as the third filtered posture; wherein the third preset weight is less than a preset weight threshold.

[0267] In some embodiments, the motion state includes shaking, moving, and stillness; the prediction module 403 is used to determine the third filtering posture and the first filtering posture as the filtering posture to be interpolated when the motion state includes shaking; and to determine the first filtering posture and the second filtering posture as the filtering posture to be interpolated when the motion state includes moving or stillness.

[0268] In some embodiments, the current posture includes the current angular velocity, and the historical posture includes the historical angular velocity; the determination module 402 is used to respectively determine whether the current angular velocity and the historical angular velocity are between a first preset angular velocity threshold and a second preset angular velocity threshold; when the current angular velocity and the historical angular velocity are both between the first preset angular velocity threshold and the second preset angular velocity threshold, determine that the motion state of the electronic device is shaking; or, when the current angular velocity or the historical angular velocity is not between the first preset angular velocity threshold and the second preset angular velocity threshold, determine the motion state of the electronic device based on the current angular velocity and the historical angular velocity.

[0269] In some embodiments, the determination module 402 is used to determine that the motion state of the electronic device is shaking based on the current angular velocity and the historical angular velocity, in response to satisfying one of the following conditions: the jitter frequency of the electronic device is greater than the preset frequency threshold, the direction of the angular velocity changes, and the duration of the change in the direction of the angular velocity is greater than the first preset duration threshold; determine that the motion state of the electronic device is stationary in response to the current angular velocity and the historical angular velocity are both less than the first preset angular velocity threshold; and determine that the motion state of the electronic device is moving in response to satisfying one of the following conditions: the jitter frequency is less than or equal to the preset frequency threshold, the direction of the angular velocity does not change, and the duration of the change in the direction of the angular velocity is less than or equal to the first preset duration threshold.

[0270] In some embodiments, the anti-shake module 404 is used to transform the coordinates of the pixel points in the image using the target posture to obtain the image after the coordinate transformation; and to crop the image after the coordinate transformation to obtain the image after the anti-shake processing.

[0271] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0272] Figure 5800 is a block diagram of an electronic device shown in an embodiment of the present disclosure. For example, the electronic device 800 may be a mobile phone, a mobile computer, etc.

[0273] Reference Figure 5 , the electronic device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0274] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0275] The memory 804 is configured to store various types of data to support operations on the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0276] The power supply component 806 provides power to the various components of the electronic device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 800.

[0277] The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0278] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0279] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0280] The sensor assembly 814 includes one or more sensors, such as the aforementioned posture sensor, for providing various aspects of status assessment for the electronic device 800. For example, the sensor assembly 814 can detect the open / closed state of the electronic device 800, the relative positioning of the components, such as the display and keypad of the electronic device 800, and the sensor assembly 814 can also detect the position change of the electronic device 800 or a component of the electronic device 800, the presence or absence of contact between the user and the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and the temperature change of the electronic device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0281] The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as Wi-Fi, 4G or 5G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0282] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0283] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, and the instructions can be executed by a processor 820 of an electronic device 800 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0284] A non-transitory computer-readable storage medium, when instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to perform the aforementioned image processing method.

[0285] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims above.

[0286] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire a current posture of the electronic device when capturing an image, and a historical posture of the electronic device before capturing the image; Determining a motion state of the electronic device according to the current posture and the historical posture; Predicting a target posture of the electronic device according to the current posture, the historical posture, and the motion state; The image is subjected to anti-shake processing using the target posture to obtain an anti-shake processed image.

2. The method according to claim 1, characterized in that The step of predicting a target posture of the electronic device according to the current posture, the historical posture and the motion state includes: Performing filtering processes in different ways on the current posture and the historical posture to obtain different filtering postures; Based on the motion state of the electronic device, selecting a filtering posture to be interpolated from the different filtering postures; Interpolation processing is performed based on the filtered posture to be interpolated to obtain the target posture of the electronic device.

3. The method according to claim 2, characterized in that The interpolation processing is performed based on the filtered posture to be interpolated to obtain the target posture of the electronic device, including: Determining an interpolation coefficient corresponding to the filtering posture to be interpolated based on the duration of the motion state and the filtering posture to be interpolated; An interpolation process is performed based on the filter posture to be interpolated and the interpolation coefficient corresponding to the filter posture to be interpolated to obtain the target posture of the electronic device.

4. The method according to claim 3, characterized in that The filtering posture to be interpolated includes two filtering postures, the filtering postures are represented by quaternions, and the interpolation process includes spherical linear interpolation of quaternions; The determining, based on the duration of the motion state and the filter posture to be interpolated, an interpolation coefficient corresponding to the filter posture to be interpolated includes: Determining weights corresponding to two filter postures to be interpolated in the spherical linear interpolation of the quaternion based on the duration of the motion state; Determining a cosine angle between the two filtered postures to be interpolated; Based on the cosine angle and the weights corresponding to the two filtering postures, interpolation coefficients corresponding to the filtering postures to be interpolated are determined.

5. The method according to claim 4, characterized in that The method further comprises: Get the duration threshold; The step of determining weights corresponding to two filter postures to be interpolated in the spherical linear interpolation of the quaternion based on the duration of the motion state includes: In response to the duration of the motion state being greater than a duration threshold, determining weights corresponding to two filter postures to be interpolated in the spherical linear interpolation of the quaternion; wherein the weight of a preset initial filter posture in the two filter postures to be interpolated is positively correlated with the duration of the motion state, and the sum of the weights corresponding to the two filter postures to be interpolated is a first preset value; In response to the duration of the motion state being less than or equal to the duration threshold, the weight of the preset initial filtering posture is determined to be the first preset value, and the weight of the filtering posture to be interpolated other than the preset initial filtering posture is 0.

6. The method according to claim 5, characterized in that The method further comprises: Acquire the focal length when the electronic device captures the image; The acquisition time threshold includes: In the case where the focal length belongs to a preset focal length, determining a duration threshold corresponding to the focal length according to a preset mapping relationship; wherein the preset mapping relationship includes a mapping between a focal length and a duration threshold; or, When the focal length does not belong to the preset focal length, the duration threshold corresponding to the focal length is determined according to the duration thresholds respectively corresponding to a first preset focal length that is greater than the focal length and a second preset focal length that is less than the focal length; wherein the first preset focal length and the second preset focal length both belong to the preset focal length.

7. The method according to claim 6, characterized in that The first preset focal length and the second preset focal length are both preset focal lengths adjacent to the focal length; and determining the duration threshold corresponding to the focal length according to the duration thresholds respectively corresponding to the first preset focal length that is greater than the focal length and the second preset focal length that is less than the focal length, comprises: Determine the first focal length weight of the first preset focal length by the ratio of the difference between the focal length and the second preset focal length to the difference between the first preset focal length and the second preset focal length; Determine a ratio of a difference between the first preset focal length and the focal length to a difference between the first preset focal length and the second preset focal length as a second focal length weight of the second preset focal length; The sum of the result of weighting the preset duration threshold corresponding to the first preset focal length by the first focal length weight and the result of weighting the preset duration threshold corresponding to the second preset focal length by the second focal length weight is determined as the duration threshold corresponding to the focal length.

8. The method according to claim 2, characterized in that: The filtering process of the current posture and the historical posture in different ways to obtain different filtering postures includes: Performing weighted processing on the current posture and the historical posture respectively, and obtaining a first filtered posture based on the weighted postures; Performing Kalman filtering on the current posture and the historical posture to obtain a second filtered posture; The current posture is weighted, and a third filtering posture is obtained based on the weighted current posture and the historical posture.

9. The method according to claim 8, characterized in that The weighting process is performed on the current posture and the historical posture respectively, and a first filtering posture is obtained based on each weighted posture, including: The sum of the result of weighting the current posture with the first preset weight and the result of weighting the historical posture with the second preset weight is determined as the first filtering posture; wherein the first preset weight is positively correlated with the difference between the current posture and the historical posture; the sum of the first preset weight and the second preset weight is the second preset value.

10. The method according to claim 8, characterized in that The weighted processing is performed on the current posture, and a third filtering posture is obtained based on the weighted current posture and the historical posture, including: The sum of a result of weighting the current posture with a third preset weight and the historical posture is determined as the third filtering posture; wherein the third preset weight is less than a preset weight threshold.

11. The method according to claim 8, characterized in that The motion state includes shaking, moving, and stillness; and selecting a filtering posture to be interpolated from the different filtering postures based on the motion state of the electronic device includes: In a case where the motion state includes shaking, determining the third filtering posture and the first filtering posture as the filtering postures to be interpolated; In a case where the motion state includes moving or being stationary, the first filtering posture and the second filtering posture are determined as the filtering postures to be interpolated.

12. The method according to claim 1, characterized in that The current posture includes a current angular velocity, and the historical posture includes a historical angular velocity; and determining the motion state of the electronic device according to the current posture and the historical posture includes: respectively determining whether the current angular velocity and the historical angular velocity are between a first preset angular velocity threshold and a second preset angular velocity threshold; When the current angular velocity and the historical angular velocity are both between the first preset angular velocity threshold and the second preset angular velocity threshold, determining that the motion state of the electronic device is shaking; or, When the current angular velocity or the historical angular velocity is not between the first preset angular velocity threshold and the second preset angular velocity threshold, the motion state of the electronic device is determined according to the current angular velocity and the historical angular velocity.

13. The method according to claim 12, characterized in that In the case where the current angular velocity or the historical angular velocity is greater than or equal to the first preset angular velocity threshold, or less than or equal to the second preset angular velocity threshold, determining the motion state of the electronic device according to the current angular velocity and the historical angular velocity includes: In a case where the current angular velocity or the historical angular velocity is greater than or equal to the first preset angular velocity threshold, or less than or equal to the second preset angular velocity threshold, according to the current angular velocity and the historical angular velocity, in response to satisfying one of the following: the shaking frequency of the electronic device is greater than a preset frequency threshold, the direction of the angular velocity changes, and the duration of the change in the direction of the angular velocity is greater than a first preset duration threshold, determining that the motion state of the electronic device is shaking; In response to the current angular velocity and the historical angular velocity being both less than the first preset angular velocity threshold, determining that the motion state of the electronic device is stationary; In response to satisfying one of the following conditions: the shaking frequency is less than or equal to the preset frequency threshold, the direction of the angular velocity has not changed, and the duration of the change in the direction of the angular velocity is less than or equal to the first preset duration threshold, the motion state of the electronic device is determined to be moving.

14. The method according to claim 1, characterized in that The step of performing anti-shake processing on the image by using the target posture to obtain an image after the anti-shake processing includes: Transforming the coordinates of the pixel points in the image using the target posture to obtain an image after coordinate transformation; The image after the coordinate transformation is cropped to obtain the image after the anti-shake processing.

15. An image processing device, characterized in that: The device comprises: A first acquisition module, used to acquire a current posture of the electronic device when capturing an image, and a historical posture of the electronic device before capturing the image; A determination module, used to determine the motion state of the electronic device according to the current posture and the historical posture; A prediction module, configured to predict a target posture of the electronic device according to the current posture, the historical posture and the motion state; The anti-shake module is used to perform anti-shake processing on the image using the target posture to obtain an image after anti-shake processing.

16. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to execute the method as claimed in any one of claims 1 to 14.

17. A computer-readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method as claimed in any one of claims 1 to 14.

Images