Image processing method and device, electronic equipment, storage medium and program product

Abstract

The invention discloses an image processing method and device, electronic equipment, a storage medium and a program product, and belongs to the field of electronic equipment. The method comprises the following steps: determining an anti-shake compensation degree of a first dynamic photo; based on the anti-shake compensation degree and the anti-shake compensation data, performing shake reduction processing on the first dynamic photo to obtain a second dynamic photo; wherein the anti-shake compensation data is used for correcting deviation generated by shake when the first dynamic picture is shot.

Description

Technical Field

[0001] The present application belongs to the field of electronic equipment, and specifically relates to an image processing method, device, electronic equipment, storage medium and program product. Background Art

[0002] As people's living standards continue to improve, static photos can no longer meet people's photography needs. Therefore, a new shooting mode has emerged, namely the dynamic shooting mode, which can also be called the live shooting mode. When a user uses the live shooting mode of an electronic device to take a photo, after the user clicks the camera shutter button in the electronic device, the electronic device obtains the video of a certain period of time before and after the user presses the shutter button, and outputs this video as the image content of the dynamic photo. Dynamic photos give photos more content by recording the content before and after the moment of taking the photo, allowing users to play each photo like a video when reviewing, hear the sound at the time, and see the changing movements. It has a better immersion than static photos and is easy to restore the atmosphere of the scene at the time.

[0003] In the related technology, when electronic devices are taking dynamic photos, they will use OIS (Optical Image Stabilization) technology to perform anti-shake compensation on the dynamic photos. However, this makes the dynamic photos unable to reflect the shaking during shooting, and lacks information and story related to the shooting environment, resulting in "distortion" of the dynamic photos, making it impossible for the dynamic photos to restore the real shooting environment. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide an image processing method, device, electronic device, storage medium and program product, so that when taking dynamic photos, the shaking during shooting can be reflected, thereby more accurately showing the on-site atmosphere during shooting.

[0005] In a first aspect, an embodiment of the present application provides an image processing method, comprising:

[0006] Determining an anti-shake compensation degree of the first dynamic photo;

[0007] Based on the anti-shake compensation degree and the anti-shake compensation data, performing shake restoration processing on the first dynamic photo to obtain a second dynamic photo;

[0008] The anti-shake compensation data is data used to correct the deviation caused by shaking when taking the first dynamic photo.

[0009] In a second aspect, an embodiment of the present application provides an image processing device, including:

[0010] A degree determination module, used to determine the degree of anti-shake compensation of the first dynamic photo;

[0011] A restoration module, configured to perform a shake restoration process on the first dynamic photo based on the anti-shake compensation degree and the anti-shake compensation data to obtain a second dynamic photo;

[0012] The anti-shake compensation data is data used to correct the deviation caused by shaking when taking the first dynamic photo.

[0013] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.

[0014] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0015] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.

[0016] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method described in the first aspect.

[0017] In the embodiment of the present application, the degree of anti-shake compensation of the first dynamic photo is determined, and based on the degree of anti-shake compensation and anti-shake compensation data, the first dynamic photo is subjected to shake restoration processing to obtain a second dynamic photo; wherein the anti-shake compensation data is data used to correct the deviation caused by shaking in the first dynamic photo when the first dynamic photo is taken. According to this embodiment, after obtaining the dynamic photo, the anti-shake compensation of the dynamic photo can be restored based on the anti-shake compensation data of the dynamic photo, so that the restored dynamic photo can reflect the shaking situation during shooting, and thus more accurately show the atmosphere at the time of shooting. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a flowchart of an image processing method provided by some embodiments of the present application;

[0019] Figure 2 is a schematic diagram of a preview interface provided in some embodiments of the present application;

[0020] Figure 3 is a schematic diagram of an image display interface provided by some embodiments of the present application;

[0021] Figure 4 is a schematic diagram of image cropping provided in some embodiments of the present application;

[0022] Figure 5 is a schematic diagram of an image processing device provided by some embodiments of the present application;

[0023] Figure 6 is a block diagram of an electronic device provided by some embodiments of the present application;

[0024] Figure 7 It is a schematic diagram of the structure of an electronic device provided in some embodiments of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0026] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0027] The image processing method, device, electronic device, storage medium and program product provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings through specific embodiments and their application scenarios.

[0028] The image processing method provided in the embodiment of the present application can be applied to dynamic photo shooting scenes. Figure 1-Figure 4 The image processing method provided in the embodiment of the present application is described in detail. It should be noted that the image processing method provided in the embodiment of the present application can be executed by an electronic device such as a smart phone. In the embodiment of the present application, the image processing method provided in the embodiment of the present application is described by taking an electronic device executing the image processing method as an example.

[0029] See also Figure 1 , is a flow chart of the image processing method provided in the embodiment of the present application, such as Figure 1 As shown, the image processing method includes steps 110 to 12, which are described in detail below.

[0030] Step 110: Determine the degree of anti-shake compensation of the first dynamic photo.

[0031] In some embodiments of the present application, the first dynamic photo may be any dynamic photo after anti-shake compensation displayed in an image storage application of the electronic device, wherein the image storage application may be, for example, an album App (Application), a gallery App, a camera App, etc.

[0032] In some embodiments of the present application, the first dynamic photo may be a dynamic photo taken by a camera app in an electronic device. Based on this, before the above step 110, the user may first take the first dynamic photo by the electronic device. When the user takes the first dynamic photo by the electronic device, the camera app in the electronic device is opened, and when the dynamic photo shooting function of the camera app is turned on, the user enters the following step: Figure 2 In the preview interface 200 shown, the electronic device receives a first input from the user, and in response to the first input, obtains a video within a certain period of time before and after receiving the first input, and generates a first dynamic photo based on the obtained video. Among them, the first input is an input for instructing the electronic device to take a dynamic photo. Exemplarily, the above-mentioned first input includes but is not limited to: a user clicks the shutter button in the preview interface through a touch device such as a finger or a stylus, or a voice command input by the user, or a specific gesture input by the user, or other feasible inputs, which can be determined according to actual use requirements, and the embodiment of the present invention is not limited. The specific gesture in the embodiment of the present application can be any one of a single-click gesture, a sliding gesture, a drag gesture, a pressure recognition gesture, a long press gesture, an area change gesture, a double-press gesture, and a double-click gesture; the click input in the embodiment of the present application can be a single-click input, a double-click input, or any number of click inputs, etc., and can also be a long press input or a short press input. For example, the above-mentioned first input can be: a user clicks the shutter button 210 in the preview interface 200 through a finger. The above-mentioned first input can also be: the user displays a Figure 2 The electronic device of the preview interface 200 shown in the figure issues a voice command indicating starting to take a photo.

[0033] In some embodiments of the present application, as described above, the electronic device generates a first dynamic photo based on the video within a certain period of time before and after receiving the first input. In order to ensure that the electronic device can obtain the video before receiving the first input, when the dynamic photo shooting function of the camera App is turned on, before receiving the first input, the electronic device can continue to cache the video of the first preset time length, so that when the first input is received, the video before the first input can be obtained from the cached video.

[0034] In some embodiments of the present application, the first dynamic photo is a dynamic photo after anti-shake processing. In the process of shooting the first dynamic photo, the electronic device obtains jitter data, generates anti-shake compensation data of the first dynamic photo based on the jitter data, and performs anti-shake compensation on the first dynamic photo based on the anti-shake compensation data, so as to eliminate the image instability caused by jitter in the first dynamic photo, and finally outputs the first dynamic photo after anti-shake compensation. Among them, the anti-shake compensation data is data used to correct the deviation caused by jitter in the first dynamic photo. As mentioned above, the first dynamic photo is generated based on a video within a certain period of time. The essence of the video is an image frame sequence composed of at least two image frames, so the first dynamic photo can be regarded as an image frame sequence composed of at least two image frames. Based on this, the electronic device can obtain the jitter data of each image frame respectively in the process of generating the first dynamic photo in response to the first input, generate the anti-shake compensation data of each image frame based on the jitter data of each image frame, and then perform anti-shake compensation on each frame image based on the anti-shake compensation data of each frame image, so as to obtain the first dynamic photo.

[0035] In some embodiments of the present application, the electronic device stores the anti-shake compensation data of the first dynamic image in the Exif (Exchangeable image file format) file of the first dynamic photo for subsequent use. The Exif file of the first dynamic photo is used to record the attribute information and shooting data of the first dynamic photo. The anti-shake compensation data of the first dynamic image includes the anti-shake compensation data of each image frame in the first dynamic photo. In this way, after obtaining the first dynamic image, the first dynamic photo can be restored to shake based on the anti-shake compensation data of the first dynamic photo.

[0036] In some embodiments of the present application, after obtaining the first dynamic photo, the electronic device determines the degree of anti-shake compensation for the first dynamic photo when it determines that the jitter situation of the first dynamic photo is "distorted" from the actual situation, and performs jitter restoration processing on the first dynamic photo based on the anti-shake compensation degree and the anti-shake compensation data, so as to at least partially restore the jitter when taking the first dynamic photo, thereby solving the distortion problem of the first dynamic photo.

[0037] In some embodiments of the present application, the electronic device may automatically determine the degree of anti-shake compensation of the first dynamic photo based on a preset algorithm, wherein the preset algorithm includes but is not limited to an algorithm based on deep learning, etc., which is not specifically limited in this embodiment.

[0038] In some embodiments of the present application, the electronic device may also determine the degree of anti-shake compensation of the first dynamic photo based on the user's second input. When browsing the first dynamic photo, the user can freely adjust the degree of anti-shake compensation of the first dynamic photo through the second input according to actual needs. The electronic device responds to the second input and determines the adjusted degree of anti-shake compensation as the degree of anti-shake compensation of the first dynamic photo. For example, the default degree of anti-shake compensation of the first dynamic photo is 100%. When the user determines that the first dynamic photo is distorted, the degree of anti-shake compensation of the first dynamic photo is adjusted from the original 100% to 20% through the second input. The electronic device responds to the second input and determines 20% as the degree of anti-shake compensation of the first dynamic photo for subsequent shake restoration processing. Among them, the value range of the degree of anti-shake compensation is 0-100%. The above second input is an input for adjusting the degree of anti-shake compensation of the first dynamic photo. Exemplarily, the above-mentioned second input includes but is not limited to: the user's sliding input of the anti-shake adjustment control in the image display interface through a touch device such as a finger or a stylus, or a voice command input by the user, or a specific gesture input by the user, or other feasible input, which can be determined according to actual usage requirements and is not limited by the embodiment of the present invention. The specific gesture in the embodiment of the present application can be any one of a single-click gesture, a sliding gesture, a drag gesture, a pressure recognition gesture, a long press gesture, an area change gesture, a double-press gesture, and a double-click gesture; the click input in the embodiment of the present application can be a single-click input, a double-click input, or any number of click inputs, etc., and can also be a long press input or a short press input. For example, the above-mentioned second input can be: the user uses his finger to Figure 3 The second input may also be: the user slides the anti-shake adjustment control 310 in the image display interface 300 as shown. Figure 3 In the case of the first dynamic photo of the image frame 320 shown, a voice instruction is sent to the electronic device to instruct the anti-shake compensation degree of the first dynamic photo to be adjusted to a certain value.

[0039] Step 120. Based on the anti-shake compensation degree and the anti-shake compensation data, perform shake restoration processing on the first dynamic photo to obtain a second dynamic photo.

[0040] In some embodiments of the present application, the anti-shake compensation data refers to the anti-shake compensation data of the first dynamic photo. The electronic device can obtain the anti-shake compensation data of the first dynamic photo from the Exif file of the first dynamic photo, thereby performing a shake restoration process on the first dynamic photo based on the anti-shake compensation degree and the anti-shake compensation data, and using the first dynamic photo after the shake restoration process as the second dynamic photo.

[0041] In some embodiments of the present application, when shooting the first dynamic photo, the default anti-shake compensation degree of the first dynamic photo is 100%. Therefore, before the above step 110, the original anti-shake compensation degree of the first dynamic photo is 100%. The anti-shake compensation degree determined in step 110 is a new anti-shake compensation degree that is re-determined and is less than the original anti-shake compensation degree of the first dynamic photo. Therefore, when the first dynamic photo is subjected to shake restoration processing based on the anti-shake compensation degree and the anti-shake compensation data in step 120, it is actually reverse compensation for the first dynamic photo, so as to weaken the anti-shake compensation degree of the first dynamic photo. In this way, the shake restoration processing can at least restore part of the first dynamic photo caused by shaking, such as image blur and image deformation, so that the first dynamic photo can reflect the shaking during shooting to a certain extent, so that the first dynamic photo can more accurately restore the environmental atmosphere during shooting.

[0042] The image processing method provided in the embodiment of the present application determines the degree of anti-shake compensation of the first dynamic photo, and based on the degree of anti-shake compensation and anti-shake compensation data, performs shake restoration processing on the first dynamic photo to obtain a second dynamic photo; wherein the anti-shake compensation data is data used to correct the deviation caused by shaking in the first dynamic photo when the first dynamic photo is taken. According to this embodiment, after obtaining the dynamic photo, the anti-shake compensation of the dynamic photo can be restored based on the anti-shake compensation data of the dynamic photo, so that the restored dynamic photo can reflect the shaking situation during shooting, and thus more accurately show the atmosphere at the time of shooting.

[0043] In some embodiments, the first dynamic photo includes at least two image frames, and the anti-shake compensation data of the first dynamic photo includes the lens compensation angle of each of the at least two image frames. The lens compensation angle is an angle parameter used to correct the deviation of the first lens due to shaking during the process of shooting the first dynamic photo, and the first lens is a lens used to shoot the first dynamic photo. Based on this, the above step 120 can be implemented through the following steps 1210-1230.

[0044] Step 1210: Based on the anti-shake compensation degree and the anti-shake compensation data, determine the lens reverse compensation angle of each image frame, where the direction of the lens reverse compensation angle is opposite to the direction of the lens compensation angle.

[0045] In some embodiments of the present application, the first dynamic photo includes at least two image frames, each image frame has its own corresponding anti-shake compensation data, and the anti-shake compensation data of each image frame includes a corresponding lens compensation angle anglex i and angley i , lens compensation angle anglex for different image frames i and angleyi , different. Among them, anglex i represents the compensation angle of the i-th image frame on the X-axis of the lens coordinate system, angley i Indicates the compensation angle of the i-th image frame on the Y axis of the lens coordinate system. Where i = 1, 2..., n, represents a time sequence, and the value of n can be determined according to the duration of the first dynamic photo and the shooting frequency of the electronic device. For example, if the shooting frequency of the electronic device is 1KHz and the duration of the first dynamic photo is 3 seconds, then the value of n is 3000, that is, the first dynamic photo is an image frame sequence consisting of 3000 image frames.

[0046] In some embodiments of the present application, the anti-shake compensation degree determined in step 110 is the anti-shake compensation degree that needs to be retained in the first dynamic photo, and the default original anti-shake compensation degree of the first dynamic photo is 100%. Therefore, in the above step 1210, the anti-shake compensation degree determined in step 110 is subtracted from 100% to obtain the anti-shake compensation degree that needs to be weakened when performing the shake restoration process. Because the shake restoration process is to weaken the anti-shake compensation degree, when the anti-shake compensation data is the lens compensation angle, the shake restoration process is actually to adjust the lens compensation angle of the image frame in the opposite direction. Based on this, this embodiment defines a lens reverse compensation angle, and the lens reverse compensation angle is used to represent the angle that the lens needs to be adjusted in the opposite direction, wherein the direction of the lens reverse compensation angle is opposite to the direction of the lens compensation angle. Specifically, the lens reverse compensation angle corresponding to each image frame can be calculated according to the following formulas (1) and (2):

[0047] anglex′ i =-(100%-deg)*anglex i (1)

[0048] angley′ i =-(100%-deg)*angley i (2)

[0049] In the above formulas (1) and (2), anglex′ i represents the lens reverse compensation angle of the i-th image frame on the X-axis of the lens coordinate system, angley′ i represents the lens reverse compensation angle of the i-th image frame on the Y axis of the lens coordinate system, and deg represents the degree of anti-shake compensation determined in the above step 110. For example, taking deg as 20%, anglex′ i =-0.8*anglex i ,angley′ i =-0.8*angley i .

[0050] Step 1220. Based on the lens reverse compensation angle of each image frame, perform jitter restoration processing on each image frame to obtain a restored image of each image frame.

[0051] In some embodiments of the present application, each image frame in the first dynamic image has its own corresponding lens reverse compensation angle. Since the lens compensation angles of different image frames are different, the lens reverse compensation angles of different image frames are also different. Based on this, when performing jitter restoration processing on the first dynamic image, jitter restoration processing is performed on each image frame based on the lens reverse compensation angle of each image frame, so as to obtain a restored image of each image frame.

[0052] In some embodiments of the present application, for each image frame, jitter restoration processing may be performed through steps 210 to 240 as shown in the figure.

[0053] Step 210: Determine the distance between the subject in the image frame and the first lens according to the focus information of the first lens, where the first lens is a lens for taking the first dynamic photo.

[0054] In some embodiments of the present application, the focus information of the first lens refers to various data and parameters related to the adjustment of the focus position of the first lens to obtain a clear image during the process of shooting the first dynamic photo through the first lens, including but not limited to information such as focus distance, focus mode, and focus position. The electronic device can record the focus information of the first lens during the process of shooting the first dynamic photo with the first lens, so as to obtain the focus information corresponding to each image frame in the first dynamic photo. The focus distance refers to the distance between the frontmost lens of the lens and the subject. Therefore, after obtaining the focus information of each image frame, the distance between the subject and the first lens in the image frame can be determined according to the focus distance. Among them, the subject of the image frame refers to the person, object, scene or a combination thereof that the first lens is aimed at and recorded in the image frame during the process of shooting the image frame, which is the core subject and expression focus of the image frame.

[0055] Step 220: Determine the horizontal displacement of the image frame based on the distance and the lens back compensation angle of the image frame.

[0056] In some embodiments of the present application, after obtaining the distance between the subject in the image frame and the first lens, the horizontal displacement of the image frame can be determined based on the obtained distance and the lens reverse compensation angle of the image frame using a trigonometric function relationship.

[0057] In some embodiments of the present application, the horizontal displacement of the image frame includes a displacement x on the X-axis of the lens coordinate system. i and the displacement y on the Y axis of the lens coordinate system i, assuming that the distance between the first lens and the subject in the i-th image frame is d i , the lens reverse compensation angle of the i-th image frame includes anglex′ i and anglex′ i , in this case, when calculating the displacement x i When the distance between the first lens and the subject is d i As the hypotenuse of the right triangle, the displacement x i As a right-angled side of a right triangle, anglex′ i The corresponding hypotenuse and displacement x i The angle formed by the corresponding right-angled sides is based on the sine function relationship in trigonometric functions. The displacement x can be derived as shown in the following equation (3): i The calculation formula is:

[0058] x i =d i *sin(anglex′ i ) (3)

[0059] Similarly, in the same way, the displacement y can be derived as shown in equation (4): i The calculation formula is:

[0060] y i =d i *sin(angley′ i ) (4)

[0061] Based on this, in d i 、anglex′ i and angley′ i If the horizontal displacement of the image frame is known, the horizontal displacement of the image frame can be calculated based on the above formulas (3) and (4).

[0062] Step 230: Determine the number of pixels of the horizontal displacement of the image frame based on the horizontal displacement of the image frame and the resolution of the first dynamic photo.

[0063] Here, the resolution of the first dynamic photo is used to indicate how many pixels there are in each inch of the image of the first dynamic photo. The resolution of each image frame of the first dynamic photo is consistent, and the resolution can be obtained from the Exif of the first dynamic photo.

[0064] In some embodiments of the present application, the number of horizontally displaced pixels of an image frame is used to indicate the position of the image frame in a horizontal position.

[0065] In some embodiments of the present application, the number of horizontally displaced pixels of the image frame includes the first number of pixels that need to be moved on the X-axis of the lens coordinate system and the second number of pixels that need to be moved on the Y-axis of the lens coordinate system. The first number of pixels and the second number of pixels can be calculated based on the following formulas (5) and (6), respectively:

[0066] The first pixel number = x i * Resolution (5)

[0067] Second pixel number = y i * Resolution (6)

[0068] For example, with a resolution of 300 dpi, x i 3 inches, y i Taking 2 inches as an example, the number of first pixel points is 3*300=900, and the number of second pixel points is 2*300=600.

[0069] Step 240: Based on the number of horizontally displaced pixels of the image frame, the image frame is cropped to obtain a restored image of the image frame.

[0070] In some embodiments of the present application, after obtaining the number of horizontally displaced pixels of the image frame, the image frame is cropped based on the horizontal displacement direction of the image frame and the direction of the lens reverse compensation angle. When cropping the image frame, starting from the boundary of the image frame, a specified number of pixels in the image frame are cropped along the direction of the lens reverse compensation angle, and the cropped image is determined as the restored image of the image frame. The specified number is the number of horizontally displaced pixels.

[0071] In some embodiments of the present application, the image frame may be cropped using a cropping frame. Figure 4 Taking the cropping of the image frame 400 as an example, before the image frame 400 is cropped, the boundary of the cropping box 410 coincides with the boundary of the image frame 400. Assuming that the anglex′ of the image frame 400 is i The direction is the negative direction of the X axis of the lens coordinate system. The number of the first pixel points is 900. i The direction is the positive direction of the Y axis of the lens coordinate system, the second number of pixels is 600, and when the image frame 400 is cropped by the cropping box 410, the right boundary of the cropping box 410 is moved 900 pixels to the right starting from the right boundary of the image frame 400, and the lower boundary of the cropping box 410 is moved 600 pixels upward starting from the lower boundary of the image frame 400, and the image area outside the moved cropping box 410 is cropped, and only the image area inside the cropping box 410 is retained, and the image obtained after cropping is determined as the restored image of the image frame 400.

[0072] Furthermore, if the size of the restored image obtained by cropping does not match the display area, the size of the restored image can be adjusted by scaling to match the display area.

[0073] Step 1230. Determine an image sequence consisting of restored images of all image frames in the first dynamic photo as the second dynamic photo.

[0074] Through the above method, reverse compensation of the first dynamic photo can be achieved, so as to at least partially restore the shaking of the first dynamic photo and obtain a second dynamic photo that can better reflect the real scene environment.

[0075] In some embodiments, the electronic device may determine the anti-shake compensation data of the first dynamic photo through the following steps 310 to 320.

[0076] Step 310: Obtain device jitter data when taking the first dynamic photo.

[0077] In some embodiments of the present application, when the first dynamic image is captured by the electronic device, the electronic device may shake. For example, when the first dynamic photo is captured by a user holding the electronic device, the shaking of the user's hand will be transmitted to the electronic device, thereby causing the electronic device to shake. Alternatively, when the electronic device is placed in a certain position to capture the first dynamic image, the electronic device may also shake due to reasons such as an uneven placement position and an unstable device. The shaking of the device will cause the first dynamic image to be blurred and other problems. Therefore, when capturing the first dynamic photo, the device shaking data of the electronic device is obtained as a basis for anti-shake compensation.

[0078] In some embodiments of the present application, an inertial measurement unit IMU is provided in the electronic device, and the IMU includes a gyroscope and an accelerometer, wherein the gyroscope can measure the angular velocity of the first lens, and the accelerometer can measure the acceleration of the first lens. Based on this, the electronic device can measure the angular velocity and acceleration of the first lens when shooting each image frame through the IMU during the process of shooting the first dynamic photo through the first lens, and determine the measured angular velocity and / or acceleration of the first lens as the device jitter data. In this way, the obtained device jitter data contains the device jitter data corresponding to each image frame in the first dynamic photo.

[0079] Step 320. Determine anti-shake compensation data for the first dynamic photo based on the device jitter data.

[0080] In some embodiments of the present application, for each image frame in the first dynamic photo, the electronic device may use an OIS anti-shake algorithm to calculate the anti-shake compensation data of the image frame based on the device jitter data of the image frame, and use the anti-shake compensation data corresponding to all image frames in the first dynamic photo as the anti-shake compensation data of the first dynamic photo. The OIS anti-shake algorithm includes but is not limited to a calculation method based on angular velocity and a calculation method based on acceleration.

[0081] In some embodiments of the present application, the principle of the angular velocity-based calculation method is to determine the rotation angle of the first lens relative to the initial position at each moment based on the acquired angular velocity. According to these angle data, combined with the optical parameters of the first lens and the size of the image sensor and other information, the compensation angle required for the first lens can be calculated to offset the jitter of the first lens.

[0082] In some embodiments of the present application, the principle of the acceleration-based calculation method is to determine the acceleration data of the first lens in different directions based on the acquired acceleration, and to obtain the velocity data of the first lens by integrating the acceleration data, and to obtain the displacement data of the first lens by integrating again. Based on the displacement data and the optical system parameters of the first lens, the compensation angle of the first lens required to compensate for the shake of the first lens can be calculated.

[0083] In the above manner, anti-shake compensation data corresponding to the device shake data can be obtained, and anti-shake compensation is performed on the first dynamic image based on the anti-shake compensation data, thereby eliminating problems such as unclearness caused by device shake in the first dynamic image.

[0084] Among the current related technologies, both OIS technology and EIS (Electronic Image Stabilization) technology are essentially to prevent hand shaking, but cannot prevent the shaking of the subject. The subject of dynamic photos is usually moving, and the movement of the subject will also cause the instability of dynamic photos. Due to this factor, dynamic photos after traditional anti-shake processing are still not stable enough.

[0085] In view of this, in some embodiments, the electronic device may determine the anti-shake compensation data for performing anti-shake compensation on the first dynamic photo through the following steps 410 to 420.

[0086] Step 410. Obtain device jitter data when taking the first dynamic photo and motion data of the subject in the first dynamic photo.

[0087] In some embodiments of the present application, the method of obtaining the device jitter data can refer to the above description of step 310, and will not be described again here to avoid repetition.

[0088] In some embodiments of the present application, the motion data of the subject is data that can describe the motion state and characteristics of the subject. In order to facilitate the use of OIS technology to stabilize the first dynamic photo, the motion data of the subject may include the angular velocity, acceleration, etc. of the subject.

[0089] In some embodiments of the present application, when the motion data of the subject includes the angular velocity and acceleration of the subject, the electronic device may determine the motion data of the subject through the following steps 4101-4102.

[0090] Step 4101. Determine the motion trajectory of the subject in the first dynamic photo.

[0091] In some embodiments of the present application, the optical flow method can be used to determine the motion trajectory of the subject in the first dynamic photo. The principle of the optical flow method to determine the motion trajectory is mainly to calculate the motion vector of the pixel based on the brightness change of the pixel in the image frame sequence, thereby determining the motion trajectory of the subject. When using the optical flow method to determine the motion trajectory of the subject, the image frame in the first dynamic photo can be converted into a grayscale image, which can reduce the amount of calculation and also highlight the brightness information in the image frame, which is convenient for subsequent analysis of the pixel points. The brightness conservation equation is established. It is assumed that within a short time interval, the brightness of the object in the image frame does not change with time. That is, for each pixel point (x, y) in the image frame, at time t and time t+Δt, its brightness I(x, y, t) and I(x+Δx, y+Δy, t+Δt) satisfy I(x, y, t)=I(x+Δx, y+Δy, t+Δt), where Δt represents the time interval between two adjacent image frames, Δx represents the X-axis displacement change between two adjacent image frames, and Δy represents the Y-axis displacement change between two adjacent image frames. Taylor expansion of I(x+Δx, y+Δy, t+Δt) at (x, y, t) is performed, and high-order infinitesimal terms are ignored to obtain I x u+I y v+I t =0, this is the basic equation for optical flow calculation, where They are the speed of the pixel in the X-axis direction and the Y-axis direction, that is, the two components of the optical flow vector, I x , I y and I tare the partial derivatives of image frame I with respect to x, y and t respectively. In order to solve the optical flow vector, it is necessary to establish constraints based on the above brightness conservation equation. Since one equation cannot directly solve two unknowns u and v, some additional assumptions and constraints are usually introduced, such as the method based on the local smoothness assumption, assuming that the optical flow vectors of adjacent pixels do not change much, and solving the optical flow by minimizing the smoothness of the optical flow vector. Common algorithms include the Lucas-Kanade algorithm and the Horn-Schunck algorithm. The Lucas-Kanade algorithm assumes that the optical flow is constant in a small window, and obtains the optical flow vector by solving a linear equation system composed of brightness conservation equations of multiple pixels; the Horn-Schunck algorithm considers the smoothness of the optical flow in a global scope, and obtains the optical flow field by solving a variational problem. After calculating the optical flow field, it is necessary to select some representative feature points to track the motion trajectory of the subject. These feature points are usually points with obvious features in the image, such as corner points, edge points, etc. They have high recognition and stability in the image, which is convenient for tracking in subsequent image frames. Harris corner detection and other algorithms can be used to extract feature points in an image. Based on the calculated optical flow vector, the feature points in the previous image frame are mapped to the next image frame. By continuously tracking the position changes of these feature points between adjacent image frames, the motion trajectory of the subject can be obtained. The optical flow method can quickly capture the motion of the subject and can also track fast-moving subjects well.

[0092] In some embodiments of the present application, in addition to the optical flow method, other methods can also be used to determine the motion trajectory of the subject. For example, a tracking method based on feature points, a deep learning-based method, etc. can be used, which are not listed here one by one.

[0093] Step 4102. Based on the motion trajectory, determine the angular velocity and acceleration of the subject.

[0094] In some embodiments of the present application, the position coordinates of the subject in each image frame are assumed to be (x, y). Based on this, the speed of the subject can be calculated by taking a difference between the position data of the subject in two adjacent image frames. Where Δt is the time interval between two image frames. After obtaining the speed of the subject, the acceleration of the subject can be calculated by differentiating the speed. The direction of acceleration is the same as the direction of velocity.

[0095] In some embodiments of the present application, by rotating the object in adjacent image frames by an angle θ i By taking the difference, the angular velocity of the object can be calculated as

[0096] Step 420. Determine anti-shake compensation data for the first dynamic photo based on the device shake data and motion data.

[0097] In some embodiments of the present application, based on the device jitter data and the motion data, the comprehensive jitter data of the first dynamic photo is determined, and then based on the comprehensive jitter data, the anti-shake compensation data of the first dynamic photo is determined.

[0098] In some embodiments of the present application, the device jitter data of the first dynamic photo includes the device jitter data of each image frame in the first dynamic photo. Similarly, the motion data of the subject in the first dynamic photo includes the motion data of the subject in each image frame in the first dynamic photo. Different image frames correspond to different timestamps. Therefore, the user manual data and motion data of the first dynamic photo are both time series data. In view of this, in order to improve data processing efficiency, when determining the comprehensive jitter data of the first dynamic photo based on the device jitter data and the motion data, the device jitter data and the motion data can be fast Fourier transformed respectively to obtain first frequency domain data of the device jitter data and second frequency domain data of the motion data; the first frequency domain data and the second frequency domain data are added to obtain comprehensive frequency domain data; and the comprehensive frequency domain data are inverse Fourier transformed to obtain comprehensive jitter data of the dynamic photo.

[0099] Exemplarily, the user manual data of the first dynamic photo includes a first acceleration sequence a i and the first angular velocity sequence ω i , the first acceleration sequence a i The first dynamic photo includes the acceleration of the first lens corresponding to each image frame, the first angular velocity sequence ω i The first dynamic photo includes the angular velocity of the first lens corresponding to each image frame in the first dynamic photo, and the motion data of the subject in the first dynamic photo includes the second acceleration sequence a j and the second angular velocity sequence ω j , the second acceleration sequence a j The second angular velocity sequence ω includes the acceleration of the subject in each image frame. j The first frequency domain data is obtained by performing Fourier transform on the user manual data of the first dynamic photo, and the first frequency shift data includes the first acceleration sequence a i The frequency domain data a i , FD and the first angular velocity sequence ω i Frequency domain data ω i，FD , by performing Fourier transform on the motion data of the subject in the first dynamic photo, second frequency domain data is obtained, and the second frequency domain data includes a second acceleration sequence aj The frequency domain data a j，FD and the second angular velocity sequence ω j Frequency domain data ω j，FD , add the first frequency domain data and the second frequency domain data to obtain comprehensive frequency domain data, which includes comprehensive acceleration a s and the integrated angular velocity ω s , where a s =a i，FD +a j，FD ,ω s =ω i，FD +ω j，FD Finally, by calculating the comprehensive acceleration a s and the integrated angular velocity ω s By performing an inverse Fourier transform, the comprehensive jitter data can be obtained.

[0100] The above method of determining the comprehensive jitter data is more efficient than directly processing the user manual data and motion data in the time domain.

[0101] In some embodiments of the present application, in order to improve the accuracy of the anti-shake compensation data finally obtained, before determining the comprehensive shake data of the first dynamic photo based on the device shake data and the motion data, the device shake data and the motion data may be preprocessed to filter out the noise data therein. Since the frequency of the user's hand shake is relatively high, the device shake data of the first dynamic photo is high-frequency data. Therefore, the device shake data of the first dynamic photo may be filtered through a high-pass filter to filter out the noise data in the device shake data. Since the motion frequency of the subject is relatively low, the motion data of the first dynamic photo is low-frequency data. Therefore, the motion data may be filtered through a low-pass filter to filter out the noise data in the motion data.

[0102] In some embodiments of the present application, after the comprehensive jitter data is obtained, the comprehensive jitter data can be transmitted to the OIS algorithm to calculate the anti-shake compensation information of the first dynamic photo. Here, the same OIS algorithm as in step 320 can be used, and to avoid repetition, it will not be described in detail here.

[0103] In the above manner, when the first dynamic image is anti-shake processed, in addition to considering the image instability caused by the user's hand shaking, the image instability caused by the movement of the subject is also considered, so that the subject in the first dynamic image is clearer.

[0104] The image processing method provided in the embodiment of the present application can be executed by a dynamic photo device. In the embodiment of the present application, a dynamic photo device executing the dynamic photo method is taken as an example to illustrate the dynamic photo device provided in the embodiment of the present application.

[0105] See also Figure 5 , is a schematic diagram of an image processing device provided in an embodiment of the present application, such as Figure 5 As shown, the image processing device 500 includes the following modules:

[0106] A degree determination module 501, used to determine the degree of anti-shake compensation of the first dynamic photo;

[0107] A restoration module 502, configured to perform a shake restoration process on the first dynamic photo based on the anti-shake compensation degree and the anti-shake compensation data to obtain a second dynamic photo;

[0108] The anti-shake compensation data is data used to correct the deviation caused by shaking when taking the first dynamic photo.

[0109] The image processing device provided in the embodiment of the present application determines the degree of anti-shake compensation of the first dynamic photo, and performs a shake restoration process on the first dynamic photo based on the degree of anti-shake compensation and the anti-shake compensation data to obtain a second dynamic photo; wherein the anti-shake compensation data is data used to correct the deviation caused by shaking in the first dynamic photo when the first dynamic photo is taken. According to this embodiment, after obtaining the dynamic photo, the anti-shake compensation of the dynamic photo can be restored based on the anti-shake compensation data of the dynamic photo, so that the restored dynamic photo can reflect the shaking situation during shooting, and thus more accurately show the atmosphere at the time of shooting.

[0110] In some embodiments, the first dynamic photo includes at least two image frames, and the anti-shake compensation data includes a lens compensation angle of each of the at least two image frames;

[0111] The restoration module 502 is specifically used for:

[0112] Determine a lens reverse compensation angle for each image frame based on the anti-shake compensation degree and the anti-shake compensation data, wherein the direction of the lens reverse compensation angle is opposite to the direction of the lens compensation angle;

[0113] Based on the lens reverse compensation angle of each image frame, each image frame is subjected to jitter restoration processing to obtain a restored image of each image frame;

[0114] An image sequence consisting of restored images of all image frames in the first dynamic photo is determined as the second dynamic photo.

[0115] In some embodiments, the restoration module 502 is specifically configured to:

[0116] For each image frame, determining the distance between the subject in the image frame and the first lens according to the focus information of the first lens, where the first lens is a lens for taking the first dynamic photo;

[0117] Determine the horizontal displacement of the image frame based on the distance and the lens back compensation angle of the image frame;

[0118] Determining the number of pixels of the horizontal displacement of the image frame based on the horizontal displacement of the image frame and the resolution of the first dynamic photo;

[0119] Based on the number of horizontal displacement pixels of the image frame and the direction of the lens reverse compensation angle, the image frame is cropped to obtain a restored image of the image frame.

[0120] In some embodiments, the apparatus 500 further includes an anti-shake module, configured to:

[0121] Before performing shake restoration processing on the first dynamic photo based on the anti-shake compensation degree and the anti-shake compensation data to obtain the second dynamic photo, obtaining device shake data when the first dynamic photo was taken;

[0122] Based on the device jitter data, anti-shake compensation data of the first dynamic photo is determined.

[0123] In some embodiments, the apparatus 500 further includes an anti-shake module, configured to:

[0124] Before performing shake restoration processing on the first dynamic photo based on the anti-shake compensation degree and the anti-shake compensation data to obtain the second dynamic photo, obtaining device shake data when the first dynamic photo was taken and motion data of the subject in the first dynamic photo;

[0125] Based on the device shake data and the motion data, anti-shake compensation data of the first dynamic photo is determined.

[0126] In some embodiments, the motion data includes angular velocity and acceleration, and the anti-shake module is specifically used to:

[0127] Determine a motion trajectory of a subject in a first dynamic photo;

[0128] Based on the motion trajectory, the angular velocity and acceleration of the subject are determined.

[0129] In some embodiments, the anti-shake module is specifically used to:

[0130] Determine comprehensive jitter data of the first dynamic photo based on the device jitter data and the motion data;

[0131] Based on the comprehensive jitter data, anti-shake compensation data of the first dynamic photo is determined.

[0132] In some embodiments, the anti-shake module is specifically used to:

[0133] Performing fast Fourier transform on the device jitter data and the motion data respectively to obtain first frequency domain data and second frequency domain data;

[0134] Adding the first frequency domain data and the second frequency domain data to obtain comprehensive frequency domain data;

[0135] The comprehensive frequency domain data is subjected to inverse Fourier transform to obtain the comprehensive jitter data of the dynamic photo.

[0136] The image processing device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0137] The image processing device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0138] The image processing device provided in the embodiment of the present application can achieve Figures 1 to 4 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0139] Alternatively, if Figure 6 As shown, an embodiment of the present application also provides an electronic device 600, including a processor 601 and a memory 602, wherein the memory 602 stores programs or instructions that can be executed on the processor 601, and when the program or instructions are executed by the processor 601, the various steps of the above-mentioned image processing method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, they are not described here.

[0140] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0141] Figure 7 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0142] The electronic device 700 includes but is not limited to: a radio frequency unit 701, a network module 702, an audio output unit 703, an input unit 704, a sensor 705, a display unit 706, a user input unit 707, an interface unit 708, a memory 709, and a processor 710.

[0143] Those skilled in the art will appreciate that the electronic device 700 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 710 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Figure 7 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0144] The processor 710 is used to determine the anti-shake compensation degree of the first dynamic photo;

[0145] The processor 710 is further configured to perform shake restoration processing on the first dynamic photo based on the anti-shake compensation degree and the anti-shake compensation data to obtain a second dynamic photo;

[0146] The anti-shake compensation data is data used to correct the deviation caused by shaking when taking the first dynamic photo.

[0147] The electronic device provided in the embodiment of the present application determines the degree of anti-shake compensation of the first dynamic photo, and based on the degree of anti-shake compensation and anti-shake compensation data, performs a shake restoration process on the first dynamic photo to obtain a second dynamic photo; wherein the anti-shake compensation data is data used to correct the deviation caused by shaking in the first dynamic photo when the first dynamic photo is taken. According to this embodiment, after obtaining the dynamic photo, the anti-shake compensation of the dynamic photo can be restored based on the anti-shake compensation data of the dynamic photo, so that the restored dynamic photo can reflect the shaking situation during shooting, and thus more accurately show the atmosphere at the time of shooting.

[0148] In some embodiments, the first dynamic photo includes at least two image frames, and the anti-shake compensation data includes a lens compensation angle of each of the at least two image frames;

[0149] The processor 710 is specifically configured to:

[0150] Determine a lens reverse compensation angle for each image frame based on the anti-shake compensation degree and the anti-shake compensation data, wherein the direction of the lens reverse compensation angle is opposite to the direction of the lens compensation angle;

[0151] Based on the lens reverse compensation angle of each image frame, each image frame is subjected to jitter restoration processing to obtain a restored image of each image frame;

[0152] An image sequence consisting of restored images of all image frames in the first dynamic photo is determined as the second dynamic photo.

[0153] In some embodiments, the processor 710 is specifically configured to:

[0154] For each image frame, determining the distance between the subject in the image frame and the first lens according to the focus information of the first lens, where the first lens is a lens for taking the first dynamic photo;

[0155] Determine the horizontal displacement of the image frame based on the distance and the lens back compensation angle of the image frame;

[0156] Determining the number of pixels of the horizontal displacement of the image frame based on the horizontal displacement of the image frame and the resolution of the first dynamic photo;

[0157] Based on the number of horizontal displacement pixels of the image frame and the direction of the lens reverse compensation angle, the image frame is cropped to obtain a restored image of the image frame.

[0158] In some embodiments, the processor 710 is further configured to:

[0159] Before performing shake restoration processing on the first dynamic photo based on the anti-shake compensation degree and the anti-shake compensation data to obtain the second dynamic photo, obtaining device shake data when the first dynamic photo was taken;

[0160] Based on the device jitter data, anti-shake compensation data of the first dynamic photo is determined.

[0161] In some embodiments, the processor 710 is further configured to:

[0162] Before performing shake restoration processing on the first dynamic photo based on the anti-shake compensation degree and the anti-shake compensation data to obtain the second dynamic photo, obtaining device shake data when the first dynamic photo was taken and motion data of the subject in the first dynamic photo;

[0163] Based on the device shake data and the motion data, anti-shake compensation data of the first dynamic photo is determined.

[0164] In some embodiments, the motion data includes angular velocity and acceleration, and the processor 710 is specifically configured to:

[0165] Determine a motion trajectory of a subject in a first dynamic photo;

[0166] Based on the motion trajectory, the angular velocity and acceleration of the subject are determined.

[0167] In some embodiments, the processor 710 is specifically configured to:

[0168] Determine comprehensive jitter data of the first dynamic photo based on the device jitter data and the motion data;

[0169] Based on the comprehensive jitter data, anti-shake compensation data of the first dynamic photo is determined.

[0170] In some embodiments, the processor 710 is specifically configured to:

[0171] Performing fast Fourier transform on the device jitter data and the motion data respectively to obtain first frequency domain data and second frequency domain data;

[0172] Adding the first frequency domain data and the second frequency domain data to obtain comprehensive frequency domain data;

[0173] The comprehensive frequency domain data is subjected to an inverse Fourier transform to obtain the comprehensive jitter data of the dynamic photo. It should be understood that in the embodiment of the present application, the input unit 704 may include a graphics processing unit (GPU) 7041 and a microphone 7042, and the graphics processor 7041 processes the image data of the static picture or video obtained by the image capture device (such as a camera) in the video capture mode or the image capture mode. The display unit 706 may include a display panel 7061, and the display panel 7061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 707 includes a touch panel 7071 and at least one of other input devices 7072. The touch panel 7071 is also called a touch screen. The touch panel 7071 may include two parts: a touch detection device and a touch controller. Other input devices 7072 may include, but are not limited to, a physical keyboard, function keys (such as a volume control button, a switch button, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0174] The memory 709 can be used to store software programs and various data. The memory 709 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 709 may include a volatile memory or a non-volatile memory, or the memory x09 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 709 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0175] The processor 710 may include one or more processing units; optionally, the processor 710 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to the operating system, user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 710.

[0176] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned image processing method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0177] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0178] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned image processing method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0179] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0180] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned image processing method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0181] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0182] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0183] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. An image processing method, characterized in that: include: Determining a degree of anti-shake compensation for the first dynamic photo; Based on the anti-shake compensation degree and the anti-shake compensation data, performing shake restoration processing on the first dynamic photo to obtain a second dynamic photo; The anti-shake compensation data is data used to correct the deviation caused by shaking when taking the first dynamic photo.

2. The method according to claim 1, characterized in that The first dynamic photo includes at least two image frames, and the anti-shake compensation data includes a lens compensation angle of each of the at least two image frames; The performing shake restoration processing on the first dynamic photo based on the anti-shake compensation degree and the anti-shake compensation data to obtain a second dynamic photo includes: Determining a lens reverse compensation angle of each of the image frames based on the anti-shake compensation degree and the anti-shake compensation data, wherein a direction of the lens reverse compensation angle is opposite to a direction of the lens compensation angle; Based on the lens reverse compensation angle of each image frame, each image frame is subjected to jitter restoration processing to obtain a restored image of each image frame; An image sequence consisting of restored images of all the image frames in the first dynamic photo is determined as the second dynamic photo.

3. The method according to claim 2, characterized in that The step of performing jitter restoration processing on each image frame based on the lens reverse compensation angle of each image frame to obtain a restored image of each image frame includes: For each of the image frames, determining a distance between a subject in the image frame and the first lens according to focus information of the first lens, where the first lens is a lens for taking the first dynamic photo; Determining a horizontal displacement of the image frame based on the distance and a lens reverse compensation angle of the image frame; Determining the number of pixels of the horizontal displacement of the image frame based on the horizontal displacement of the image frame and the resolution of the first dynamic photo; The image frame is cropped based on the number of horizontally displaced pixels of the image frame and the direction of the lens reverse compensation angle to obtain a restored image of the image frame.

4. The method according to any one of claims 1 to 3, characterized in that: Before performing shake restoration processing on the first dynamic photo based on the anti-shake compensation degree and the anti-shake compensation data to obtain the second dynamic photo, the method further includes: Obtaining device jitter data when taking the first dynamic photo; Based on the device jitter data, anti-shake compensation data of the first dynamic photo is determined.

5. The method according to any one of claims 1 to 3, characterized in that: Before performing shake restoration processing on the first dynamic photo based on the anti-shake compensation degree and the anti-shake compensation data to obtain the second dynamic photo, the method further includes: Acquire device jitter data when taking the first dynamic photo and motion data of the subject in the first dynamic photo; Based on the device shake data and the motion data, anti-shake compensation data of the first dynamic photo is determined.

6. The method according to claim 5, characterized in that The motion data includes angular velocity and acceleration, and obtaining the motion data of the subject in the first dynamic photo includes: Determining a motion trajectory of the subject in the first dynamic photo; Based on the motion trajectory, an angular velocity and an acceleration of the object are determined.

7. The method according to claim 5 or 6, characterized in that: The determining, based on the device jitter data and the motion data, anti-shake compensation data of the first dynamic photo includes: Determining comprehensive jitter data of the first dynamic photo based on the device jitter data and the motion data; Based on the comprehensive jitter data, anti-shake compensation data of the first dynamic photo is determined.

8. The method according to claim 7, characterized in that The determining, based on the device jitter data and the motion data, comprehensive jitter data of the dynamic photo includes: Performing fast Fourier transform on the device jitter data and the motion data respectively to obtain first frequency domain data and second frequency domain data; Adding the first frequency domain data and the second frequency domain data to obtain comprehensive frequency domain data; Perform an inverse Fourier transform on the comprehensive frequency domain data to obtain comprehensive jitter data of the dynamic photo.

9. An image processing device, characterized in that: include: A degree determination module, used to determine the degree of anti-shake compensation of the first dynamic photo; a restoration module, configured to perform a shake restoration process on the first dynamic photo based on the anti-shake compensation degree and the anti-shake compensation data, so as to obtain a second dynamic photo; The anti-shake compensation data is data used to correct the deviation caused by shaking when taking the first dynamic photo.

10. The device according to claim 9, characterized in that The device further comprises an anti-shake module, which is used for: Before performing shake restoration processing on the first dynamic photo based on the anti-shake compensation degree and the anti-shake compensation data to obtain the second dynamic photo, obtaining device shake data when taking the first dynamic photo and motion data of the subject in the first dynamic photo; Based on the device shake data and the motion data, anti-shake compensation data of the first dynamic photo is determined.

11. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the image processing method according to any one of claims 1 to 8 are implemented.

12. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the image processing method according to any one of claims 1 to 8 are implemented.

13. A computer program product, characterized in that The program product is stored in a storage medium, and the program product is executed by at least one processor to implement the steps of the image processing method according to any one of claims 1 to 8.

Images