Image anti-shake method, device, electronic device and computer-readable storage medium

Abstract

The embodiments of the present application disclose an image stabilization method, device, electronic device, and computer-readable storage medium. The method may include: obtaining the original posture data corresponding to the image to be processed at the current moment, the original posture data at least including the first posture data of the image acquisition device when acquiring the image to be processed; processing the original posture data through a posture processing model to obtain the target posture data corresponding to the image to be processed, the target posture data is used to characterize the posture of the image acquisition device after eliminating the shaking; according to the target posture data and the first posture data, the image to be processed is transformed to obtain the target image. The above-mentioned image stabilization method, device, electronic device, and computer-readable storage medium can effectively eliminate the blurring phenomenon caused by shaking of the image, achieve the effect of image stabilization, and improve the visual effect of the image.

Description

Technical Field

[0001] The present application relates to the field of imaging technology, and in particular to an image stabilization method, device, electronic device, and computer-readable storage medium. Background Art

[0002] When an electronic device captures an image through an image acquisition device (such as a camera), if the position or posture of the electronic device changes (for example, the user does not hold the electronic device steady when capturing images, or the electronic device is in motion, etc.), the captured image will shake, reducing the image clarity and affecting the visual effect of the image. Therefore, how to effectively eliminate image jitter to improve the visual effect of the image has become one of the hot research topics in the field of imaging technology. Summary of the invention

[0003] The embodiments of the present application disclose an image stabilization method, device, electronic device and computer-readable storage medium, which can effectively eliminate the blurring phenomenon caused by image shaking, achieve the effect of image stabilization, and improve the visual effect of the image.

[0004] The present application discloses an image stabilization method, including:

[0005] Acquire original posture data corresponding to the image to be processed at the current moment, wherein the original posture data at least includes first posture data when the image acquisition device acquires the image to be processed;

[0006] The original posture data is processed by a posture processing model to obtain target posture data corresponding to the image to be processed, and the target posture data is used to characterize the posture of the image acquisition device after the jitter is eliminated; wherein the posture processing model is trained based on a plurality of original sample posture data and a smooth posture curve corresponding to the plurality of original sample posture data, and the smooth posture curve is obtained by smoothing the plurality of original sample posture data;

[0007] According to the target posture data and the first posture data, the image to be processed is transformed to obtain a target image.

[0008] The present application discloses an image stabilization device, comprising:

[0009] A data acquisition module, used to acquire original posture data corresponding to the image to be processed at the current moment, wherein the original posture data at least includes first posture data when the image acquisition device acquires the image to be processed;

[0010] A posture processing module, used for processing the original posture data through a posture processing model to obtain target posture data corresponding to the image to be processed, wherein the target posture data is used to characterize the posture of the image acquisition device after the jitter is eliminated; wherein the posture processing model is obtained by training based on a plurality of original sample posture data and a smooth posture curve corresponding to the plurality of original sample posture data, wherein the smooth posture curve is obtained by smoothing the plurality of original sample posture data;

[0011] The transformation module is used to perform image transformation on the image to be processed according to the target posture data and the first posture data to obtain a target image.

[0012] An embodiment of the present application discloses an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor implements the method described above.

[0013] An embodiment of the present application discloses a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method described above is implemented.

[0014] The embodiment of the present application discloses an image anti-shake method, device, electronic device and computer-readable storage medium, which obtains the original posture data corresponding to the image to be processed at the current moment, and the original posture data at least includes the first posture data of the image acquisition device when acquiring the image to be processed, and processes the original posture data through a posture processing model to obtain the target posture data corresponding to the image to be processed, and the target posture data is used to characterize the posture of the image acquisition device after the shaking is eliminated, and then the image to be processed is transformed according to the target posture data and the first posture data to obtain the target image. In the embodiment of the present application, since the posture processing model is obtained by training based on a plurality of original sample posture data and a smooth posture curve corresponding to the plurality of original sample posture data, the posture processing model can accurately obtain the target posture data after the shaking is eliminated corresponding to the image to be processed at the current moment, so as to accurately compensate the shaking of the image acquisition device by using the target posture data, and obtain the target image with the shaking eliminated, which can effectively eliminate the blurring phenomenon of the image caused by the shaking, achieve the effect of image anti-shake, and improve the visual effect of the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 is a block diagram of an image processing circuit in one embodiment;

[0017] Figure 2 is a flow chart of an image anti-shake method in one embodiment;

[0018] Figure 3 is a schematic diagram of a gesture processing model in one embodiment;

[0019] Figure 4 A flowchart of an embodiment of the present invention is to process the original posture data by the posture processing model to obtain the target posture data corresponding to the image to be processed;

[0020] Figure 5A is a schematic diagram of a gesture processing model in another embodiment;

[0021] Figure 5B To follow the time axis Figure 5A Schematic diagram of the posture processing model in the expansion;

[0022] Figure 6 is a flow chart of an image stabilization method in another embodiment;

[0023] Figure 7 is a schematic diagram of a smooth posture curve in one embodiment;

[0024] Figure 8 is a block diagram of an image anti-shake device in one embodiment;

[0025] Fig. 9 FIG. 4 is a structural block diagram of an electronic device in an embodiment. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0027] It should be noted that the terms "including" and "having" and any variations thereof in the embodiments of the present application and the accompanying drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products or devices.

[0028] It is to be understood that the terms "first", "second" etc. used in the present application can be used to describe various elements in this article, but these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, without departing from the scope of the present application, the first posture data can be referred to as the second posture data, and similarly, the second posture data can be referred to as the first posture data. Both the first posture data and the second posture data belong to posture data, but they are not the same posture data. In addition, it should be noted that the terms "multiple" etc. used in the embodiments of the present application refer to two or more.

[0029] For image stabilization, the targets mainly include optical stabilization and digital stabilization. Among them, optical stabilization is to avoid the phenomenon of instrument shaking during image capture by setting optical components (such as lens settings, etc.). Digital stabilization is to analyze the image collected by the image sensor and perform stabilization compensation on the image through software. Common digital stabilization methods include image stabilization using image feature matching; motion estimation using the collected image, and motion compensation of the image based on the motion estimation results to achieve image stabilization. However, the current digital stabilization methods still have problems such as relatively single application scenarios or complex calculation processes and poor stabilization effects. Therefore, there is still a need to improve image stabilization technology.

[0030] In an embodiment of the present application, an image stabilization method, device, electronic device and computer-readable storage medium are provided. Through a posture processing model, the target posture data after the jitter is eliminated corresponding to the image to be processed at the current moment can be accurately obtained, and then the image acquisition device is subjected to jitter compensation to obtain the target image with the jitter eliminated. The blurring phenomenon of the image caused by the jitter can be effectively eliminated, the image stabilization effect is achieved, and the visual effect of the image is improved.

[0031] The embodiment of the present application provides an electronic device, which may include but is not limited to a mobile phone, a smart wearable device, a tablet computer, a PC (Personal Computer), a vehicle terminal, a digital camera, etc., and the embodiment of the present application does not limit this. The electronic device includes an image processing circuit, which can be implemented using hardware and / or software components, and may include various processing units that define an ISP (Image Signal Processing) pipeline. Figure 1 FIG. 1 is a block diagram of an image processing circuit in one embodiment. For ease of explanation, Figure 1 Only various aspects of the image processing technology related to the embodiments of the present application are shown.

[0032] like Figure 1As shown, the image processing circuit includes an ISP processor 140 and a control logic 150. The image data captured by the imaging device 110 is first processed by the ISP processor 140, and the ISP processor 140 analyzes the image data to capture image statistics that can be used to determine one or more control parameters of the imaging device 110. The imaging device 110 may include one or more lenses 112 and an image sensor 114. The image sensor 114 may include a color filter array (such as a Bayer filter), and the image sensor 114 may obtain light intensity and wavelength information captured by each imaging pixel, and provide a set of raw image data that can be processed by the ISP processor 140. The posture sensor 120 (such as a three-axis gyroscope, a Hall sensor, an accelerometer, etc.) can provide the collected image processing parameters (such as anti-shake parameters) to the ISP processor 140 based on the interface type of the posture sensor 120. The posture sensor 120 interface can use a SMIA (Standard Mobile Imaging Architecture) interface, other serial or parallel camera interfaces, or a combination of the above interfaces.

[0033] It should be noted that although Figure 1 Only one imaging device 110 is shown in the figure, but in the embodiment of the present application, at least two imaging devices 110 may be included, each imaging device 110 may correspond to one image sensor 114, or multiple imaging devices 110 may correspond to one image sensor 114, which is not limited here. The working process of each imaging device 110 may refer to the above description.

[0034] In addition, the image sensor 114 may also send raw image data to the gesture sensor 120 , and the gesture sensor 120 may provide the raw image data to the ISP processor 140 based on the interface type of the gesture sensor 120 , or the gesture sensor 120 may store the raw image data in the image memory 130 .

[0035] The ISP processor 140 processes the raw image data pixel by pixel in a variety of formats. For example, each image pixel may have a bit depth of 8, 10, 12, or 14 bits, and the ISP processor 140 may perform one or more image processing operations on the raw image data and collect statistical information about the image data. The image processing operations may be performed at the same or different bit depth precisions.

[0036] The ISP processor 140 may also receive image data from the image memory 130. For example, the gesture sensor 120 interface sends raw image data to the image memory 130, and the raw image data in the image memory 130 is then provided to the ISP processor 140 for processing. The image memory 130 may be a part of a memory device, a storage device, or an independent dedicated memory in an electronic device, and may include a DMA (Direct Memory Access) feature.

[0037] When receiving raw image data from the image sensor 114 interface or from the posture sensor 120 interface or from the image memory 130, the ISP processor 140 may perform one or more image processing operations, such as time domain filtering. The processed image data may be sent to the image memory 130 for further processing before being displayed. The ISP processor 140 receives the processed data from the image memory 130 and performs image data processing in the original domain and in the RGB and YCbCr color spaces on the processed data. The image data processed by the ISP processor 140 may be output to the display 160 for viewing by the user and / or further processed by a graphics engine or GPU (Graphics Processing Unit). In addition, the output of the ISP processor 140 may also be sent to the image memory 130, and the display 160 may read the image data from the image memory 130. In one embodiment, the image memory 130 may be configured to implement one or more frame buffers.

[0038] The statistical data determined by the ISP processor 140 may be sent to the control logic 150. For example, the statistical data may include image sensor 114 statistical information such as vibration frequency of a gyroscope, automatic exposure, automatic white balance, automatic focus, flicker detection, black level compensation, lens 112 shading correction, etc. The control logic 150 may include a processor and / or microcontroller that executes one or more routines (such as firmware), and the one or more routines may determine control parameters of the imaging device 110 and control parameters of the ISP processor 140 based on the received statistical data. For example, the control parameters of the imaging device 110 may include attitude sensor 120 control parameters (such as gain, integration time for exposure control, anti-shake parameters, etc.), camera flash control parameters, camera anti-shake shift parameters, lens 112 control parameters (such as focal length for focus or zoom), or a combination of these parameters. The ISP control parameters may include gain levels and color correction matrices for automatic white balance and color adjustment (for example, during RGB processing), and lens 112 shading correction parameters.

[0039] For example, in combination Figure 1The image processing circuit of the present application is used to illustrate the image stabilization method, device, electronic device and computer-readable storage medium provided in the embodiments of the present application. While the imaging device 110 collects each frame of image data, the posture sensor 120 can obtain the posture data of the imaging device 110 in real time. When the ISP processor 140 obtains the image to be processed at the current moment, it can obtain the original posture data corresponding to the image to be processed at the current moment collected by the posture sensor 120. The original posture data may at least include the first posture data of the imaging device 110 when collecting the image to be processed. The ISP processor 140 can process the original posture data through the posture processing model to obtain the target posture data corresponding to the image to be processed. The target posture data is used to characterize the posture of the imaging device 110 after the shaking is eliminated. Then, according to the target posture data and the first posture data, the image to be processed is transformed to obtain the target image with the shaking eliminated.

[0040] like Figure 2 As shown, in one embodiment, an image stabilization method is provided, which can be applied to the above electronic device. The method may include the following steps:

[0041] Step 210, obtaining original posture data corresponding to the image to be processed at the current moment, wherein the original posture data at least includes first posture data when the image acquisition device acquires the image to be processed.

[0042] The electronic device can obtain the image to be processed at the current moment. The image to be processed at the current moment may refer to the image that the electronic device needs to perform image de-shaking processing at the current moment. The image to be processed may be an image captured in real time by an image acquisition device (i.e., the imaging device 110 described above, such as a camera), or an image stored in a memory. It should be noted that the image to be processed at the current moment does not refer to the image captured by the image acquisition device at the current moment. There is usually a certain time delay from the image acquisition device capturing the image to the processor performing image de-shaking processing on the image. For example, the image acquisition device captures the 10th frame of image at the current moment, while the image to be processed at the current moment is the 3rd frame of image. It can be understood that if the performance of the processor is very high, there may be no time delay from the image acquisition device capturing the image to the processor performing image de-shaking processing. In this case, the image to be processed at the current moment is the image captured by the image acquisition device at the current moment.

[0043] In the process of the image acquisition device acquiring images, the attitude sensor in the electronic device can acquire the attitude data of the electronic device in real time, and the attitude data of the electronic device acquired by the attitude sensor can be used as the attitude data of the image acquisition device. The attitude sensor may include one or more of a gyroscope sensor, an acceleration sensor, a gravity sensor, etc., but is not limited thereto. Optionally, the attitude data may include Euler angle attitude, and the Euler angle angular velocity acquired by the attitude sensor at each acquisition time may be integrated to obtain the attitude data corresponding to each acquisition time.

[0044] Specifically, each frame of image collected by the image acquisition device may carry a first timestamp (used to represent the time when the image was collected), and each frame of posture data collected by the posture sensor may carry a second timestamp (used to represent the time when the posture data was collected). The first timestamp corresponding to each frame of image and the second timestamp corresponding to each frame of posture data may be matched to achieve synchronization between image collection and posture data, and the posture data with the same second timestamp as the first timestamp of the image may be used as the posture data when the image acquisition device collected the image.

[0045] The electronic device can obtain the original posture data corresponding to the image to be processed at the current moment, and the original posture data at least includes the first posture data of the image acquisition device when acquiring the image to be processed, wherein the second timestamp of the first posture data can be equal to the first timestamp of the image to be processed.

[0046] In some embodiments, the original posture data corresponding to the image to be processed at the current moment, in addition to the first posture data when the image acquisition device acquires the image to be processed, may also include the second posture data corresponding to the M frames of historical images acquired by the image acquisition device before the image to be processed, and / or the third posture data corresponding to the N frames of future images acquired by the image acquisition device after the image to be processed, wherein M and N are both positive integers.

[0047] As an implementation, M may be greater than N, for example, M may be 100 or 120, and N may be 10 or 15; or M may be 12 or 13, and N may be 2 or 5, etc. The present application embodiment does not limit the specific values ​​of M and N. Optionally, according to the first timestamp corresponding to the image to be processed at the current moment, the second posture data corresponding to the M frames of historical images collected by the image acquisition device before the first timestamp may be obtained, and / or, according to the first timestamp corresponding to the image to be processed at the current moment, the third posture data corresponding to the N frames of future images collected by the image acquisition device after the first timestamp may be obtained.

[0048] For example, if the image to be processed at the current moment is the 20th frame image, the 10 frames of images collected before the 20th frame image (i.e., the 10th to 19th frames of images) can be used as historical images, and the posture data corresponding to the 10th to 19th frames of images can be obtained. For another example, if the image to be processed at the current moment is the 20th frame image, the 5 frames of images collected after the 20th frame image (i.e., the 21st to 25th frames of images) can be used as future images, and the posture data corresponding to the 21st to 25th frames of images can be obtained.

[0049] In some embodiments, before obtaining the original posture data corresponding to the image to be processed at the current moment, it also includes: caching the posture data corresponding to the first number of images acquired by the image acquisition device, and the first number of images includes at least M frames of historical images and / or N frames of future images.

[0050] After the posture sensor collects the posture data, the electronic device may cache the multiple frames of posture data collected by the posture sensor. Optionally, only the posture data corresponding to the first number of images collected by the image acquisition device may be cached. For example, if the first number is 50, only 50 frames of posture data whose second timestamps are the same as the first timestamps of the 50 frames of images may be cached.

[0051] Optionally, the posture data collected by the posture sensor within the target acquisition time period corresponding to the first number of images collected by the image acquisition device can also be cached. For example, the first number is 50, and the target acquisition time period for the image acquisition device to collect the 50 frames of images is a to b, then the posture data collected by the posture sensor within the target acquisition time period a to b can be directly cached.

[0052] By caching the posture data corresponding to the first number of images collected by the image acquisition device, it can be ensured that when each frame of the image to be processed is de-jittered, M second posture data and N third posture data can be obtained to meet the processing requirements of the subsequent posture processing model.

[0053] Step 220, the original posture data is processed by the posture processing model to obtain target posture data corresponding to the image to be processed, and the target posture data is used to characterize the posture of the image acquisition device after the jitter is eliminated.

[0054] After obtaining the original posture data corresponding to the image to be processed at the current moment, the obtained original posture data can be input into the posture processing model, and the posture processing model can process the original posture data to obtain the target posture data corresponding to the image to be processed. The posture processing model can be trained based on multiple original sample posture data and smooth posture curves corresponding to the multiple original sample posture data, wherein the smooth posture curve is obtained by smoothing the multiple original sample posture data. By training the posture processing model with multiple original sample posture data and smooth posture curves corresponding to the multiple original sample posture data, the posture processing model can be equipped with the ability to accurately predict and obtain the target posture data to eliminate the jitter based on the original posture data. The target posture data corresponding to the image to be processed obtained by the posture processing model can smoothly transition with the posture data before and after the image to be processed. Therefore, the image blurring caused by jitter can be eliminated by correcting the image to be processed using the target posture data.

[0055] The posture processing model may include but is not limited to network models such as CNN (Convolutional Neural Networks) and RNN (Recurrent Neural Network). The specific architecture of the posture processing model is not limited in the embodiments of the present application.

[0056] As an implementation, the original posture data may include the first posture data of the image acquisition device when acquiring the image to be processed, as well as the second posture data and / or the third posture data mentioned above. The posture processing model may analyze a plurality of posture data that are continuous at the acquisition time, extract the characteristics of the posture data, and obtain the target posture data corresponding to the image to be processed based on the characteristics. The posture processing model may obtain the target posture data corresponding to the image to be processed by analyzing more posture data, and may improve the accuracy of the target posture data.

[0057] Step 230 , performing image transformation on the image to be processed according to the target posture data and the first posture data to obtain a target image.

[0058] The electronic device can use the target posture data to perform jitter compensation on the image to be processed, and perform image transformation on the image to be processed according to the target posture data and the first posture data, so as to convert the image to be processed captured by the image acquisition device under the first posture data into the target image when the image acquisition device captures the image with the target posture data.

[0059] As a specific implementation, the posture difference data between the target posture data and the first posture data can be determined according to the target posture data and the first posture data, and the transformation matrix corresponding to the image to be processed can be calculated according to the posture difference data. Wherein, the transformation matrix may include at least one of a rotation matrix and a translation matrix. The image to be processed can be transformed according to the transformation matrix corresponding to the image to be processed, and the pixel coordinates of each pixel point in the image to be processed can be transformed according to the transformation matrix to determine the pixel coordinates of each pixel point after transformation, thereby obtaining the target image.

[0060] The electronic device uses the posture processing model to obtain the target posture data corresponding to the image to be processed. The target posture data can smoothly transition with the posture data before and after the image to be processed. Therefore, the target image obtained after the jitter compensation of the target posture data is more stable, and the image blur caused by the jitter can be eliminated. Compared with the method of filtering the original posture data using a filtering algorithm in the related art, the use of a neural network can obtain more accurate target posture data, and there is no need to cache a lot of posture data, which reduces memory overhead and improves the overall performance of the electronic device.

[0061] In the embodiment of the present application, since the posture processing model is trained based on multiple original sample posture data and smooth posture curves corresponding to the multiple original sample posture data, the posture processing model can accurately obtain the target posture data after de-jittering corresponding to the image to be processed at the current moment, so as to use the target posture data to accurately perform jitter compensation on the image acquisition device to obtain the target image with de-jittering, which can effectively eliminate the blurring of the image caused by jitter, achieve the effect of image stabilization, and improve the visual effect of the image.

[0062] In some embodiments, the posture processing model may be a CNN. The step of processing the original posture data through the posture processing model to obtain target posture data corresponding to the image to be processed may include: performing a dilated convolution process on the original posture data through one or more dilated convolution layers in the posture processing model, and obtaining the target posture data corresponding to the image to be processed according to the result of the dilated convolution process.

[0063] The posture processing model may include one or more dilated convolution layers, and the original posture data may be input into the posture processing model, and the features of the original posture data may be extracted through one or more dilated convolution layers in the posture processing model. Starting from the second dilated convolution layer, each dilated convolution layer may perform a dilated convolution operation on the features output by the previous dilated convolution layer, and the scale of the features output by each dilated convolution layer may be the same. The dilated convolution layer may increase the receptive field, but will not reduce the scale of the features, and will not increase the amount of calculation, which can reduce the resources occupied by the posture processing model and improve the operational efficiency of the posture processing model.

[0064] As an implementation method, the original posture data may include the first posture data of the image acquisition device when acquiring the image to be processed, as well as the second posture data and / or the third posture data mentioned above. Each posture data contained in the original posture data may be simultaneously input into the posture processing model. The posture processing model may generate features (such as feature vectors or feature matrices, etc.) corresponding to the image to be processed according to each input posture data, and input the generated features into the first atrous convolution layer. The first atrous convolution layer may perform atrous convolution operations on the input features, and input the features obtained by the operations into the second atrous convolution layer, ..., and so on, until the last atrous convolution layer, and then determine the target posture data corresponding to the image to be processed according to the features output by the last atrous convolution layer.

[0065] Figure 3 FIG. 1 is a schematic diagram of a gesture processing model in one embodiment. Figure 3 As shown, the posture processing model 310 may include n hole convolution layers, and M second posture data, first posture data and N third posture data may be input into the posture processing model 310. The posture processing model 310 may process the input posture data through n hole convolution layers to obtain target posture data.

[0066] It should be noted that, in addition to one or more dilated convolutional layers, the posture processing model may also include other network modules, such as a pooling layer, a fully connected layer, etc., and is not limited to the above-mentioned dilated convolutional layer.

[0067] In some embodiments, the gesture processing model may be an RNN. Figure 4 As shown, the step processes the original posture data through the posture processing model to obtain the target posture data corresponding to the image to be processed, which may include the following steps:

[0068] Step 402, in the order of collection time from earliest to latest, according to the collection time corresponding to each piece of posture data contained in the original posture data, each piece of posture data is sequentially input into the posture processing model.

[0069] In the embodiment of the present application, the original posture data corresponding to the image to be processed at the current moment may include the first posture data when the image acquisition device acquires the image to be processed, and the second posture data corresponding to the M frames of historical images acquired by the image acquisition device before the image to be processed, and / or the third posture data corresponding to the N frames of future images acquired by the image acquisition device after the image to be processed. The various posture data contained in the original posture data can be input into the posture processing model as a sequence information in the order of acquisition time from the earliest to the latest.

[0070] For example, the image to be processed at the current moment is the 51st frame image captured by the image acquisition device, and the original posture data corresponding to the image to be processed at the current moment may include the posture data corresponding to the 1st frame image to the 60th frame image captured by the image acquisition device. Then, the posture data corresponding to the 1st frame image to the 60th frame image can be taken as a sequence information and input into the posture processing model in sequence according to the second timestamps (representing the collection time of the posture data) corresponding to each frame posture data from first to last.

[0071] Step 404, using the posture processing model, obtains the predicted posture data output this time according to the posture data input this time and the predicted posture data output last time.

[0072] Step 406, determine whether the posture data input this time is the last posture data in the original posture data at the time of collection. If so, execute step 408. If not, use the predicted posture data output this time as the new predicted posture data output last time, and continue to execute step 404.

[0073] Step 408: determine the predicted posture data output this time as the target posture data corresponding to the image to be processed.

[0074] The posture processing model can continuously perform cyclic iterative calculations based on the time axis. The posture processing model can obtain the posture data input this time, and process the posture data input this time and the predicted posture data output last time to obtain the predicted posture data output this time. It can be determined whether the posture data input this time is the last posture data at the time of collection in the original posture data. If the posture data input this time is the last posture data at the time of collection in the original posture data (that is, the last posture data input in the original posture data), the cyclic iteration can be stopped, and the predicted posture data output this time is determined as the target posture data corresponding to the image to be processed.

[0075] If the posture data input this time is not the last posture data at the time of collection in the original posture data, the predicted posture data output this time can be used as the new predicted posture data output last time, and continue to obtain the newly input posture data, and continue to execute the steps of obtaining the predicted posture data output this time based on the posture data input this time and the predicted posture data output last time through the posture processing model.

[0076] like Figure 5A and Figure 5B As shown, Figure 5A is a schematic diagram of a gesture processing model in another embodiment, Figure 5B To follow the time axis Figure 5ASchematic diagram of the posture processing model in the figure. The posture processing model 510 may include an input layer (not shown), a hidden layer C, and an output layer (not shown). According to the order of the acquisition time corresponding to each posture data, the posture data input this time and the predicted posture data output by the hidden layer C last time can be calculated in sequence through the hidden layer C to obtain the predicted posture data output by the hidden layer C this time. For example, for the posture data input at time t, the hidden layer C can calculate the posture data input at time t and the predicted posture data output by the hidden layer C at time t-1 to obtain the predicted posture data output at time t. The predicted posture data output at time t can be used together with the posture data input at time t+1 to determine the predicted posture data output at time t+1.

[0077] As an implementation, the hidden layer may include a neural network, and the neural network may include LSTM (Long Short-Term Memory) neurons, etc., but is not limited thereto.

[0078] It should be noted that the posture processing model can be other neural network models besides the above-mentioned CNN and RNN.

[0079] In the embodiment of the present application, the posture processing model can accurately obtain the target posture data after de-jittering corresponding to the image to be processed at the current moment, which can effectively improve the effect of image de-jittering and improve the visual effect of the image.

[0080] like Figure 6 As shown, in another embodiment, an image stabilization method is provided, which can be applied to the above electronic device. The method may include the following steps:

[0081] Step 602, obtaining original sample posture data corresponding to a plurality of acquisition moments, and performing smoothing filtering on the original sample posture data to obtain a smooth posture curve corresponding to the original sample posture data.

[0082] Before training the posture processing model, a training data set may be generated, and the training data set may include a large amount of original sample posture data. Further, the training data set may include one or more groups of sample data, each group of sample data may include original sample posture data corresponding to multiple acquisition moments, and a smoothing filter process may be performed on the multiple original sample posture data included in each group of sample data to obtain a smooth posture curve corresponding to each group of sample data.

[0083] In some embodiments, the original sample gesture data may be smoothed and filtered by a smoothing filter, and the smoothing filter may include but is not limited to any one of a mean filter, a median filter, a weighted filter, etc. Optionally, a plurality of original sample gesture data may be arranged in order from the earliest to the latest acquisition time, and the smoothing filter may slide in sequence in the arranged original sample gesture data according to the target window, and the original sample gesture data in the target window may be smoothed and filtered to obtain the filtered sample gesture data. The filtered sample gesture data corresponding to each original sample gesture data may form a smooth gesture curve.

[0084] Optionally, the filter parameters of the smoothing filter can be pre-set fixed parameters, which can be calculated based on multiple experimental data. The filter parameters may include but are not limited to the window size of the above-mentioned target window, etc. The filter parameters can be used to determine the filtering strength of the smoothing filter. For example, the larger the target window, the greater the filtering strength, and the smaller the target window, the smaller the filtering strength, etc.

[0085] Optionally, the filter parameters of the smoothing filter can also be determined according to the original sample posture data. The filter parameters of the smoothing filter can be determined according to the original sample posture data, and the original sample posture data is smoothed and filtered by the smoothing filter using the filter parameters to obtain a smooth posture curve corresponding to the original sample posture data.

[0086] As an implementation method, the data fluctuation degree between the original sample posture data corresponding to multiple acquisition moments can be determined, and the filter parameters of the smoothing filter can be determined according to the data fluctuation degree. Optionally, the difference between the original sample posture data of each two adjacent acquisition moments can be calculated, and the data fluctuation degree can be determined based on the calculated difference. For example, the larger the difference, the greater the data fluctuation degree, and the smaller the difference, the smaller the data fluctuation degree.

[0087] In some embodiments, the filtering effect corresponding to the filter parameters of the determined smoothing filter may be positively correlated with the degree of data fluctuation between multiple original sample posture data, that is, the greater the data fluctuation degree, the stronger the filtering effect corresponding to the determined filter parameters may be, which can ensure that the obtained smooth posture curve is smoother; the smaller the data fluctuation degree, the weaker the filtering effect corresponding to the determined filter parameters may be, which can make the smooth posture curve more consistent with the actual original sample posture data, ensure the stability and smoothness of the smooth posture curve, and improve the accuracy of the posture processing model.

[0088] Furthermore, the original sample posture data corresponding to the multiple acquisition moments can be divided into multiple groups, and the acquisition moments corresponding to the original sample posture data contained in each group are continuous. For example, 50 original sample posture data can be included, and the 50 original sample posture data can be divided into 3 groups, and the acquisition moments corresponding to the original sample posture data in each group are continuous.

[0089] Optionally, a plurality of original sample posture data may be directly divided into a plurality of groups on average. For example, 50 original sample posture data may be divided into 5 groups on average, and each group includes 10 original sample posture data.

[0090] Optionally, the difference between the original sample posture data of each two adjacent acquisition moments can also be calculated, and the original sample posture data can be divided into multiple groups based on the calculated difference. For example, it can be determined in order from the first to the last acquisition moment whether the difference between the current original sample posture data and the previous original sample posture data is greater than the difference threshold; if it is not greater than the difference threshold, the current original sample posture data can be divided into the same group as the previous original sample posture data; if it is greater than the difference threshold, the current original sample posture data can be divided into a new group. This ensures that the fluctuation degree of the original sample posture data in each group is small, and the generated smooth sub-curve is more accurate.

[0091] After the original sample posture data are divided into multiple groups, the data fluctuation degree corresponding to each group can be determined according to the original sample posture data contained in each group, and the filter parameters can be determined according to the data fluctuation degree. The original sample posture data contained in each group can be smoothed and filtered by the smoothing filter corresponding to each group to obtain the smooth sub-curve corresponding to each group, and the smooth posture curve can be determined according to the smooth sub-curve.

[0092] The method of determining the data fluctuation degree corresponding to each group from the original sample posture data contained in each group can be similar to the method of determining the data fluctuation degree between the original sample posture data corresponding to multiple acquisition moments described above, and will not be repeated here. For each group of original sample posture data, the filter parameters of the smoothing filter corresponding to each group can be set according to the data fluctuation degree of the original sample posture data in each group, which can make the smoothing filter more targeted and improve the accuracy of the smoothed posture curve.

[0093] For example, Figure 7 FIG. 1 is a schematic diagram of a smooth posture curve in one embodiment. Figure 7As shown, curve 720 is an original posture curve composed of original sample posture data corresponding to multiple acquisition moments, and curve 710 is a smooth posture curve obtained by smoothing and filtering the original sample posture data corresponding to multiple acquisition moments. Compared with the original posture curve, the smooth posture curve is smoother and the transition is more natural. By using the posture data on the smooth posture curve to perform jitter compensation on the image collected by the image acquisition device, image blur caused by jitter can be effectively eliminated.

[0094] Step 604, inputting the original sample posture data and the smoothed posture curve corresponding to the sample collection time into the posture processing model to be trained, and processing the original sample posture data corresponding to the sample collection time by the posture processing model to be trained to obtain predicted posture data.

[0095] Step 606 , determining the output error through the posture processing model to be trained according to the predicted posture data and the smooth posture data corresponding to the smooth posture curve at the time of sample collection.

[0096] Step 608: Adjust the parameters in the posture processing model to be trained according to the output error.

[0097] The electronic device can train the posture processing model to be trained according to the training data set. The original sample posture data and the smooth posture curve corresponding to the sample collection time can be input into the posture processing model to be trained, and the original sample posture data corresponding to the sample collection time can be processed by the posture processing model to be trained to obtain the predicted posture data corresponding to the sample collection time. The posture processing model to be trained can compare the predicted posture data corresponding to the sample collection time with the smooth posture data corresponding to the sample collection time in the smooth posture curve, and calculate the output error between the predicted posture data corresponding to the sample collection time and the smooth posture data through a preset loss function. The parameters in the posture processing model to be trained can be adjusted according to the output error, and the next training can be performed until the convergence condition is reached, that is, the posture processing model that has been trained is obtained.

[0098] Optionally, the convergence condition may include an output error less than an error threshold, the number of parameter iterations (ie, the number of training times reaches a threshold, etc.), but is not limited thereto.

[0099] In some embodiments, in order to improve the accuracy of the posture processing model, more posture data can be used to train the posture processing model each time. The original sample posture data corresponding to the sample collection time, the original sample posture data corresponding to the M collection times between the sample collection times, and the original sample posture data corresponding to the N collection times after the sample collection time can be input into the posture processing model to be trained, and the posture processing model to be trained is used to process each of the input original sample posture data to obtain the predicted posture data corresponding to the sample collection time. Then, the output error is calculated based on the predicted posture data corresponding to the sample collection time and the smoothed posture data. Each time the posture processing model is trained with more posture data, the accuracy of the posture processing model can be improved, and the accuracy of the output result of the posture processing model can be improved, thereby further improving the effect of image de-shaking.

[0100] Step 610: Acquire original posture data corresponding to the image to be processed at the current moment, where the original posture data at least includes first posture data when the image acquisition device acquires the image to be processed.

[0101] Step 612, the original posture data is processed by the trained posture processing model to obtain target posture data corresponding to the image to be processed, and the target posture data is used to characterize the posture of the image acquisition device after the jitter is eliminated.

[0102] Step 614: Perform image transformation on the image to be processed according to the target posture data and the first posture data to obtain a target image.

[0103] The description of steps 610 to 614 may refer to the relevant description in the above embodiments, and will not be repeated here.

[0104] In the embodiment of the present application, the original sample posture data corresponding to multiple acquisition moments and the corresponding smooth posture curves can be used to train the posture processing model, so that the trained posture processing model can have the ability to output smooth target posture data, so as to use the target posture data to accurately compensate for jitter of the image acquisition device and achieve the effect of image stabilization. Compared with the method of directly using filters to obtain target posture data in related technologies, the target posture data obtained by the posture processing model is more accurate, and there is no need to cache too much data during the image stabilization process, which reduces memory overhead and improves the system performance of electronic equipment. In addition, the power consumption during the image stabilization process can be reduced.

[0105] like Figure 8 As shown, in one embodiment, an image stabilization device 800 is provided, which can be applied to the above electronic device. The image stabilization device 800 can include a data acquisition module 810, a posture processing module 820 and a transformation module 830.

[0106] The data acquisition module 810 is used to acquire the original posture data corresponding to the image to be processed at the current moment, and the original posture data at least includes the first posture data when the image acquisition device acquires the image to be processed.

[0107] In one embodiment, the original posture data also includes: second posture data corresponding to M frames of historical images collected by the image acquisition device before the image to be processed, and / or third posture data corresponding to N frames of future images collected by the image acquisition device after the image to be processed, wherein M and N are both positive integers.

[0108] In one embodiment, the image stabilization device 800 further includes a buffer module.

[0109] The cache module is used to cache the posture data corresponding to the first number of images collected by the image acquisition device, and the first number of images at least includes M frames of historical images and / or N frames of future images.

[0110] The posture processing module 820 is used to process the original posture data through the posture processing model to obtain the target posture data corresponding to the image to be processed, and the target posture data is used to characterize the posture of the image acquisition device after eliminating the jitter; wherein, the posture processing model is trained based on multiple original sample posture data and the smooth posture curves corresponding to the multiple original sample posture data, and the smooth posture curve is obtained by smoothing the multiple original sample posture data.

[0111] The transformation module 830 is used to perform image transformation on the image to be processed according to the target posture data and the first posture data to obtain the target image.

[0112] In one embodiment, the posture processing module 820 is further used to perform dilated convolution processing on the original posture data through one or more dilated convolution layers in the posture processing model, and obtain target posture data corresponding to the image to be processed according to the result of the dilated convolution processing.

[0113] In one embodiment, the posture processing module 820 is also used to input each posture data into the posture processing model in sequence according to the collection time corresponding to each posture data contained in the original posture data in the order from the earliest collection time; obtain the predicted posture data output this time according to the posture data input this time and the predicted posture data output last time through the posture processing model, and use the predicted posture data output this time as the new predicted posture data output last time; and continue to execute the step of obtaining the predicted posture data output this time according to the posture data input this time and the predicted posture data output last time through the posture processing model, until the posture data input this time is the last posture data at the collection time in the original posture data, and the predicted posture data output this time is determined as the target posture data corresponding to the image to be processed.

[0114] In one embodiment, the image stabilization device 800 further includes a training module.

[0115] The training module is used to train the posture processing model according to multiple original sample posture data and smooth posture curves corresponding to the multiple original sample posture data.

[0116] The training module includes a filtering unit, a prediction unit, an error determination unit and an adjustment unit.

[0117] The filtering unit is used to obtain the original sample posture data corresponding to multiple acquisition moments, and perform smoothing filtering on the original sample posture data to obtain a smooth posture curve corresponding to the original sample posture data.

[0118] In one embodiment, the filtering unit is further used to determine filter parameters of a smoothing filter according to the original sample posture data, and to perform smoothing filtering on the original sample posture data by using the smoothing filter with the filter parameters to obtain a smooth posture curve corresponding to the original sample posture data.

[0119] In one embodiment, the filtering unit is also used to divide the original sample posture data into multiple groups, and the collection time corresponding to the original sample posture data contained in each group is continuous; determine the data fluctuation degree corresponding to each group according to the original sample posture data contained in each group, and determine the filter parameters according to the data fluctuation degree, and the filter parameters are used to set the smoothing filter corresponding to each group; and through the smoothing filter corresponding to each group, respectively perform smoothing filtering processing on the original sample posture data contained in each group to obtain the smoothing sub-curve corresponding to each group, and determine the smooth posture curve according to the smoothing sub-curve.

[0120] The prediction unit is used to input the original sample posture data and the smooth posture curve corresponding to the sample collection time into the posture processing model to be trained, and the original sample posture data corresponding to the sample collection time is processed by the posture processing model to be trained to obtain the predicted posture data.

[0121] In one embodiment, the prediction unit is also used to input the original sample posture data corresponding to the sample collection moment, the original sample posture data corresponding to M collection moments between the sample collection moments, and the original sample posture data corresponding to N collection moments after the sample collection moment into the posture processing model to be trained; the input original sample posture data is processed by the posture processing model to be trained to obtain the predicted posture data corresponding to the sample collection moment.

[0122] The error determination unit is used to determine the output error according to the predicted posture data and the smooth posture data corresponding to the smooth posture curve at the time of sample collection through the posture processing model to be trained.

[0123] The adjustment unit is used to adjust the parameters in the posture processing model to be trained according to the output error.

[0124] In the embodiment of the present application, the original sample posture data corresponding to multiple acquisition moments and the corresponding smooth posture curve can be used to train the posture processing model, so that the trained posture processing model can have the ability to output smooth target posture data, so as to use the target posture data to accurately compensate for the jitter of the image acquisition device and achieve the effect of image anti-shake. Compared with the method of directly using a filter to obtain the target posture data in the related art, the target posture data obtained by the posture processing model is more accurate, and there is no need to cache too much data during the image anti-shake process, which reduces memory overhead and improves the system performance of the electronic device.

[0125] Fig. 9 FIG. 1 is a structural block diagram of an electronic device in an embodiment. Fig. 9 As shown, the electronic device 900 may include one or more of the following components: a processor 910, and a memory 920 coupled to the processor 910, wherein the memory 920 may store one or more computer programs, and the one or more computer programs may be configured to implement the methods described in the above embodiments when executed by one or more processors 910.

[0126] The processor 910 may include one or more processing cores. The processor 910 uses various interfaces and lines to connect various parts of the entire electronic device 900, and executes various functions and processes data of the electronic device 900 by running or executing instructions, programs, code sets or instruction sets stored in the memory 920, and calling data stored in the memory 920. Optionally, the processor 910 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 910 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 910, but may be implemented separately through a communication chip.

[0127] The memory 920 may include a random access memory (RAM) or a read-only memory (ROM). The memory 920 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 920 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc. The data storage area may also store data created by the electronic device 900 during use, etc.

[0128] It is understandable that the electronic device 900 may include more or fewer structural elements than those in the above structural block diagram, for example, a power module, physical buttons, a WiFi (Wireless Fidelity) module, a speaker, a Bluetooth module, a sensor, etc., and is not limited here.

[0129] An embodiment of the present application discloses a computer-readable storage medium storing a computer program, wherein the computer program implements the method described in the above embodiment when executed by a processor.

[0130] An embodiment of the present application discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program can be executed by a processor to implement the methods described in the above embodiments.

[0131] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a ROM, etc.

[0132] As used herein, any reference to memory, storage, database, or other medium may include nonvolatile and / or volatile memory. Suitable nonvolatile memory may include ROM, programmable ROM (Programmable ROM, PROM), erasable PROM (Erasable PROM, EPROM), electrically erasable PROM (Electrically Erasable PROM, EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which is used as an external cache memory. By way of illustration and not limitation, RAM may be in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), memory bus direct RAM (RDRAM) and direct memory bus dynamic RAM (DRDRAM).

[0133] It should be understood that the "one embodiment" or "an embodiment" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. Those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required for the present application.

[0134] In the various embodiments of the present application, it should be understood that the size of the serial numbers of the above-mentioned processes does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0135] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0136] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0137] The above is a detailed introduction to an image stabilization method, device, electronic device, and computer-readable storage medium disclosed in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those of ordinary skill in the art, according to the ideas of the present application, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present application.

Claims

1. An image stabilization method, characterized in that: include: Acquire original posture data corresponding to the image to be processed at the current moment, wherein the original posture data at least includes first posture data when the image acquisition device acquires the image to be processed; The original posture data is processed by a posture processing model to obtain target posture data corresponding to the image to be processed, and the target posture data is used to characterize the posture of the image acquisition device after the jitter is eliminated; wherein the posture processing model is trained based on a plurality of original sample posture data and a smooth posture curve corresponding to the plurality of original sample posture data, and the smooth posture curve is obtained by smoothing the plurality of original sample posture data; According to the target posture data and the first posture data, performing image transformation on the image to be processed to obtain a target image; The processing of the original posture data by the posture processing model to obtain target posture data corresponding to the image to be processed includes: In the order of collection time from earliest to latest, according to the collection time corresponding to each piece of the posture data included in the original posture data, each piece of the posture data is sequentially input into the posture processing model; Obtaining predicted posture data output this time according to the posture data input this time and predicted posture data output last time through the posture processing model; The predicted posture data output this time is used as the new predicted posture data output last time, and the step of obtaining the predicted posture data output this time through the posture processing model based on the posture data input this time and the predicted posture data output last time is re-executed, until the posture data input this time is the last posture data at the time of collection in the original posture data, and the predicted posture data output this time is determined as the target posture data corresponding to the image to be processed.

2. The method according to claim 1, characterized in that The original posture data also includes: second posture data corresponding to M frames of historical images collected by the image acquisition device before the image to be processed, and / or third posture data corresponding to N frames of future images collected by the image acquisition device after the image to be processed, wherein M and N are both positive integers.

3. The method according to claim 2, characterized in that Before obtaining the original posture data corresponding to the image to be processed at the current moment, the method further includes: The posture data respectively corresponding to a first number of images acquired by the image acquisition device are cached, wherein the first number of images at least includes the M frames of historical images and / or the N frames of future images.

4. The method according to claim 1, characterized in that: Before obtaining the original posture data corresponding to the image to be processed at the current moment, the method further includes: Acquire original sample posture data corresponding to a plurality of acquisition moments, and perform smoothing filtering on the original sample posture data to obtain a smooth posture curve corresponding to the original sample posture data; Inputting the original sample posture data corresponding to the sample collection moment and the smooth posture curve into the posture processing model to be trained, and processing the original sample posture data corresponding to the sample collection moment by the posture processing model to be trained to obtain predicted posture data; Determine the output error by the posture processing model to be trained according to the predicted posture data and the smooth posture data corresponding to the sample collection time in the smooth posture curve; Parameters in the posture processing model to be trained are adjusted according to the output error.

5. The method according to claim 4, characterized in that The performing smoothing filtering on the original sample posture data to obtain a smooth posture curve corresponding to the original sample posture data includes: The filter parameters of a smoothing filter are determined according to the original sample posture data, and the original sample posture data is smoothed and filtered by the smoothing filter using the filter parameters to obtain a smoothed posture curve corresponding to the original sample posture data.

6. The method according to claim 5, characterized in that The method of determining filter parameters of a smoothing filter according to the original sample posture data, and performing smoothing filtering on the original sample posture data by using a smoothing filter with the filter parameters to obtain a smoothed posture curve corresponding to the original sample posture data includes: Dividing the original sample posture data into a plurality of groups, wherein the collection time corresponding to the original sample posture data contained in each group is continuous; Determining the data fluctuation degree corresponding to each group according to the original sample posture data contained in each group, and determining the filter parameters according to the data fluctuation degree, wherein the filter parameters are used to set the smoothing filter corresponding to each group; The original sample posture data contained in each group is respectively smoothed and filtered by the smoothing filter corresponding to each group to obtain the smoothing sub-curve corresponding to each group, and the smooth posture curve is determined according to the smoothing sub-curve.

7. The method according to any one of claims 4 to 6, characterized in that: The method of inputting the original sample posture data corresponding to the sample collection moment and the smooth posture curve into the posture processing model to be trained, and processing the original sample posture data corresponding to the sample collection moment by the posture processing model to be trained to obtain predicted posture data includes: Inputting the original sample posture data corresponding to the sample collection time, the original sample posture data corresponding to M collection times between the sample collection time, and the original sample posture data corresponding to N collection times after the sample collection time into the posture processing model to be trained; The input original sample posture data is processed by the posture processing model to be trained to obtain the predicted posture data corresponding to the sample collection moment.

8. An image stabilization device, characterized in that: include: A data acquisition module, used to acquire original posture data corresponding to the image to be processed at the current moment, wherein the original posture data at least includes first posture data when the image acquisition device acquires the image to be processed; A posture processing module, used for processing the original posture data through a posture processing model to obtain target posture data corresponding to the image to be processed, wherein the target posture data is used to characterize the posture of the image acquisition device after the jitter is eliminated; wherein the posture processing model is obtained by training based on a plurality of original sample posture data and a smooth posture curve corresponding to the plurality of original sample posture data, and the smooth posture curve is obtained by smoothing the plurality of original sample posture data; A transformation module, used for performing image transformation on the image to be processed according to the target posture data and the first posture data to obtain a target image; The posture processing module is also used to input each posture data into the posture processing model in sequence according to the collection time corresponding to each posture data contained in the original posture data in the order from the earliest collection time; obtain the predicted posture data output this time according to the posture data input this time and the predicted posture data output last time through the posture processing model; use the predicted posture data output this time as the new predicted posture data output last time, and re-execute the step of obtaining the predicted posture data output this time according to the posture data input this time and the predicted posture data output last time through the posture processing model, until the posture data input this time is the last posture data at the collection time in the original posture data, and the predicted posture data output this time is determined as the target posture data corresponding to the image to be processed.

9. An electronic device, characterized in that: The method comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor implements the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images