Image processing method and device, storage medium and electronic equipment

Abstract

The invention provides an image processing method and device, electronic equipment and a computer readable storage medium, and relates to the technical field of image processing. The method comprises the following steps: in a target area, determining a pixel point characteristic value of a corresponding line based on a gray value of each line of pixel points in each video frame; obtaining a first angular velocity and a first timestamp, and determining a first position corresponding to the target moment; determining a first inter-frame pixel displacement according to the first position, determining a second inter-frame pixel displacement based on video matching, and determining a compensation pixel displacement; acquiring position data in a single direction and a second timestamp, and determining a second position corresponding to the target moment; determining inter-frame motion displacement according to the second position, and obtaining a first stable parameter based on the inter-frame motion displacement and the compensation pixel displacement; and determining a second stability parameter based on the first stability parameter, the inter-frame motion displacement and the compensation pixel displacement, so as to process the image according to the first and second stability parameters. According to the invention, the optical anti-vibration effect is improved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of image processing, and in particular to an image processing method, an image processing device, an electronic device, and a computer-readable storage medium. Background Art

[0002] OIS (Optical Image Stabilization) uses special optical elements or components in the lens to counteract the shake of the camera or camera to keep the image stable.

[0003] In the prior art, optical image stabilization calibration uses a 2D method and optical flow to detect feature points. However, in areas with unclear textures or in scenes with large motion between previous and next frames, the feature point matching accuracy is reduced and the iterative solution process is time-consuming.

[0004] Therefore, in order to solve the above problems, the embodiments of the present disclosure provide an image processing method, an image processing device, an electronic device, and a computer-readable storage medium. Summary of the invention

[0005] The purpose of the embodiments of the present disclosure is to provide an image processing method, an image processing device, an electronic device and a computer-readable storage medium, thereby solving to a certain extent the problems existing in the related technologies, such as reduced feature point matching accuracy and a long iterative solution process in areas with unclear textures or in scenes with large motion amplitudes between previous and subsequent frames.

[0006] According to a first aspect of the present disclosure, an image processing method is provided, comprising: in a target area, determining a pixel feature value of a corresponding row based on the grayscale value of each row of a video frame; acquiring a first angular velocity and a first timestamp through a first sensor, and determining a first position acquired through the first sensor at a target time according to the first angular velocity and the first timestamp, wherein the target time is a time selected from a rolling shutter exposure time corresponding to the target area; determining a first inter-frame pixel displacement between adjacent video frames according to the first position, determining a second inter-frame pixel displacement between the adjacent video frames according to a video matching method based on the pixel feature value, and determining a compensation pixel displacement according to the first inter-frame pixel displacement and the second inter-frame pixel displacement; acquiring position data and a second timestamp in a single direction through a second sensor, and determining a second position acquired through the second sensor at the target time according to the position data and the second timestamp; determining an inter-frame motion displacement of each of the adjacent video frames according to the second position, and obtaining a first stabilization parameter based on the inter-frame motion displacement and the compensation pixel displacement; determining a second stabilization parameter based on the first stabilization parameter, the inter-frame motion displacement, and the compensation pixel displacement, so as to perform anti-shake processing on the image according to the first stabilization parameter and the second stabilization parameter.

[0007] In an exemplary embodiment of the present disclosure, the method further includes: constructing a loss function using the first stability parameter and the second stability parameter; and based on the loss function, iteratively obtaining a target first stability parameter and a target second stability parameter using a global optimization algorithm.

[0008] In an exemplary embodiment of the present disclosure, the use of the first stability parameter and the second stability parameter to construct a loss function includes: pre-constructing a pixel array and determining the movement angle of each pixel in the pixel array, wherein the pixel feature value of each pixel in the pixel data is increasing; calculating the rolling shutter exposure time of each row of pixels in the pixel array to obtain the exposure moment of each row of pixels in the current video frame; obtaining a third timestamp and a displacement angle through the first sensor, constructing a first interpolation function based on the third timestamp and the displacement angle, and determining a first displacement angle corresponding to the exposure moment of each row of pixels using the first interpolation function; The second sensor obtains a fourth timestamp, constructs a second interpolation function based on the fourth timestamp, the second stability parameter and the inter-frame motion displacement, and determines the second displacement angle corresponding to the exposure moment of each row of pixels based on the second interpolation function; constructs a third interpolation function based on the first displacement angle, the second displacement angle and the pixel feature values ​​of each row of pixels, and obtains the pixel feature values ​​of each row of pixels of the next video frame based on the third interpolation function; determines the pixel feature values ​​of the next video frame mapped to the current video frame, and constructs the loss function based on the pixel feature values ​​of the next video frame and the pixel feature values ​​of the next video frame mapped to the current video frame.

[0009] In an exemplary embodiment of the present disclosure, in the target area, the characteristic value of the pixel point of the corresponding row is determined based on the grayscale value of the pixel point of each row of the video frame, including: taking the central area of ​​each of the video frames as the target area, and obtaining the grayscale value of the pixel point of each row in the target area; determining the average value of the grayscale value of each row of the pixel point, and taking the average value of the grayscale value as the characteristic value of the pixel point of the corresponding row.

[0010] In an exemplary embodiment of the present disclosure, the first sensor is a gyroscope, and the target moment is the middle moment of the rolling shutter exposure time corresponding to the central area; the first angular velocity and the first timestamp are obtained through the first sensor, and the first position obtained through the first sensor at the target moment is determined according to the first angular velocity and the first timestamp, including: obtaining the angular velocity and timestamp of the gyroscope, taking the angular velocity of the gyroscope as the first angular velocity, and the timestamp of the gyroscope as the first timestamp, and constructing a fourth interpolation function according to the first angular velocity and the first timestamp; obtaining the gyroscope position corresponding to the target moment based on the fourth interpolation function to obtain the first position.

[0011] In an exemplary embodiment of the present disclosure, the second sensor is a Hall sensor; the position data and the second timestamp in a single direction are obtained through the second sensor, and the second position obtained by the second sensor at the target moment is determined based on the position data and the second timestamp, including: obtaining the timestamp of the Hall sensor and its sensor coordinate data in the direction of a single coordinate axis, using the sensor coordinate axis data as the position data, and the timestamp of the Hall sensor as the second timestamp; constructing a fifth difference function based on the position data and the second timestamp, and determining the sensor coordinate data corresponding to the target moment based on the fifth difference function as the second position.

[0012] In an exemplary embodiment of the present disclosure, determining the compensating pixel displacement according to the first inter-frame pixel displacement and the second inter-frame pixel displacement includes: calculating the difference between the first inter-frame pixel displacement and the second inter-frame pixel displacement to obtain the compensating pixel displacement.

[0013] According to a second aspect of the present disclosure, an image processing device is provided, including: a feature value determination module, for determining, in a target area, a feature value of a pixel point in a corresponding row based on the grayscale value of each row of a video frame; a first position determination module, for obtaining a first angular velocity and a first timestamp through a first sensor, and determining a first position obtained by the first sensor at a target moment according to the first angular velocity and the first timestamp, wherein the target moment is a moment selected from a rolling shutter exposure time corresponding to the target area; a compensated pixel displacement determination module, for determining a first inter-frame pixel displacement between adjacent video frames according to the first position, determining a second inter-frame pixel displacement between the adjacent video frames according to a method based on video matching according to the pixel point feature value, and determining a second inter-frame pixel displacement between the adjacent video frames according to the method based on the video matching. The compensation pixel displacement is determined according to the first inter-frame pixel displacement and the second inter-frame pixel displacement; a second position determination module is used to obtain position data and a second timestamp in a single direction through a second sensor, and determine the second position obtained by the second sensor at the target moment according to the position data and the second timestamp; a first stability parameter determination module is used to determine the inter-frame motion displacement of each of the adjacent video frames according to the second position, and obtain a first stability parameter based on the inter-frame motion displacement and the compensation pixel displacement; an image processing module is used to determine a second stability parameter based on the first stability parameter, the inter-frame motion displacement and the compensation pixel displacement, so as to perform anti-shake processing on the image according to the first stability parameter and the second stability parameter.

[0014] In an exemplary embodiment of the present disclosure, the graphic stabilization device also includes a parameter optimization unit, and the parameter optimization unit is used to: construct a loss function using the first stabilization parameter and the second stabilization parameter; based on the loss function, iteratively obtain the target first stabilization parameter and the target second stabilization parameter using a global optimization algorithm.

[0015] According to a third aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute any one of the above-mentioned methods by executing the executable instructions.

[0016] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, any one of the methods described above is implemented.

[0017] The exemplary embodiments of the present disclosure may have some or all of the following beneficial effects:

[0018] In the image processing method provided in the exemplary embodiment of the present disclosure, in the target area, the characteristic value of the pixel point of the corresponding row is determined based on the gray value of the pixel point of each row of the video frame; the first angular velocity and the first timestamp are obtained by the first sensor, and the first position obtained by the first sensor at the target time is determined according to the first angular velocity and the first timestamp, and the target time is a time selected from the rolling shutter exposure time corresponding to the target area; the first inter-frame pixel displacement between adjacent video frames is determined according to the first position, the second inter-frame pixel displacement between adjacent video frames is determined according to the method based on the video matching according to the pixel point characteristic value, and the compensation pixel displacement is determined according to the first inter-frame pixel displacement and the second inter-frame pixel displacement; the position data and the second timestamp in a single direction are obtained by the second sensor, and the second position obtained by the second sensor at the target time is determined according to the position data and the second timestamp; the inter-frame motion displacement of each adjacent video frame is determined according to the second position, and the first stabilization parameter is obtained based on the inter-frame motion displacement and the compensation pixel displacement; the second stabilization parameter is determined based on the first stabilization parameter, the inter-frame motion displacement and the compensation pixel displacement, so as to perform anti-shake processing on the image according to the first stabilization parameter and the second stabilization parameter. The disclosed embodiment simplifies the solution model, reduces interference factors, and improves the motion detection accuracy between the front and back video frames by detecting the motion of the front and back video frames along a single direction, thereby improving the calibration accuracy. At the same time, under the same processor, calibration along a single axis greatly reduces the calibration time.

[0019] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0021] Figure 1 A schematic diagram schematically shows a flow chart of an image processing method according to an embodiment of the present disclosure;

[0022] Figure 2 A schematic diagram of an OIS lens module according to an embodiment of the present disclosure is schematically shown;

[0023] Figure 3 A schematic diagram of a video frame according to an embodiment of the present disclosure is schematically shown;

[0024] Figure 4 Schematically shows an error distribution diagram of 1D OIS hall calibration parameter optimization according to an embodiment of the present disclosure;

[0025] Figure 5 The diagram schematically shows a 1D OIS motion hall code change curve according to an embodiment of the present disclosure;

[0026] Figure 6 A block diagram schematically shows an image processing device according to an embodiment of the present disclosure;

[0027] Figure 7 A schematic diagram of an electronic device according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0028] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as being limited to the examples set forth herein; on the contrary, these embodiments are provided so that the present disclosure will be more comprehensive and complete, and the concepts of the example embodiments are fully conveyed to those skilled in the art. The described features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known technical solutions are not shown or described in detail to avoid obscuring various aspects of the present disclosure.

[0029] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0030] OIS (Optical Image Stabilization) uses special optical elements or components in the lens to counteract the shake of the camera or camera to keep the image stable.

[0031] In the related art, OIS calibration is mostly 2D calibration. Exemplarily, the method can be implemented as follows: use optical flow to match the feature points of the previous and next frames, use RANSAC (Random Sample Consensus, parameter estimation iteration method) to remove the wrong feature points, and obtain the final feature point matching result, so that the feature point P2 of the next frame can be calculated from the feature point P1 of the previous frame; use the gyroscope and OIS hall (Hall sensor) information to calculate the mapping relationship of the feature points between the two frames, and the feature point P3 of the next frame can be calculated from the feature point P1 of the previous frame; through the optimization algorithm, ensure that loss = P2-P3 constantly tends to 0, determine the stability parameters, so as to stabilize the image through the stability parameters and achieve the optical image stabilization effect.

[0032] However, in areas with unclear texture or in scenes with large motion between previous and next frames, the above method has reduced feature point matching accuracy and a long iterative solution process.

[0033] In order to solve the problems existing in the above methods, this exemplary embodiment proposes an image processing method, an image processing device, an electronic device, and a computer-readable storage medium. The technical solution of the embodiment of the present disclosure is described in detail below:

[0034] This exemplary embodiment first provides an image processing method. Figure 1 As shown, the image processing method specifically includes the following steps:

[0035] Step S110: in the target area, determining the characteristic value of the pixel point of the corresponding row based on the gray value of the pixel point of each row of the video frame;

[0036] Step S120: acquiring a first angular velocity and a first timestamp through a first sensor, and determining a first position acquired through the first sensor at a target time according to the first angular velocity and the first timestamp, where the target time is a time selected from the rolling exposure time corresponding to the target area;

[0037] Step S130: determining a first inter-frame pixel displacement between adjacent video frames according to the first position, determining a second inter-frame pixel displacement between adjacent video frames according to a method based on video matching based on pixel feature values, and determining a compensation pixel displacement according to the first inter-frame pixel displacement and the second inter-frame pixel displacement;

[0038] Step S140: acquiring position data and a second timestamp in a single direction through a second sensor, and determining a second position acquired through the second sensor at a target time according to the position data and the second timestamp;

[0039] Step S150: determining the inter-frame motion displacement of each adjacent video frame according to the second position, and obtaining a first stability parameter based on the inter-frame motion displacement and the compensation pixel displacement;

[0040] Step S160: Determine a second stabilization parameter based on the first stabilization parameter, the inter-frame motion displacement, and the compensated pixel displacement, so as to perform anti-shake processing on the image according to the first stabilization parameter and the second stabilization parameter.

[0041] In the image processing method provided in the exemplary embodiment of the present disclosure, by detecting the motion of the front and rear video frames along a single direction, the solution model is simplified, interference factors are reduced, and the motion detection accuracy between the front and rear video frames is improved, thereby improving the calibration accuracy. At the same time, under the same processor, calibration along a single axis greatly reduces the calibration time.

[0042] Next, in another embodiment, the above steps are described in more detail.

[0043] In step S110 , in the target area, the characteristic value of the pixel point of the corresponding row is determined based on the grayscale value of the pixel point of each row of the video frame.

[0044] The disclosed embodiment is used to achieve optical image stabilization, which mainly compensates for the camera motion at the current moment by translating the lens, thereby achieving an anti-shake effect. Limited by the hardware conditions of the lens module, the maximum compensation range of each coordinate axis is usually about ±1°. The schematic diagram of the OIS lens module is shown in FIG. Figure 2 In this lens module, the Hall sensor detects the position of the OIS module and reports the data, and combines the tilt angle information of the gyroscope to drive the OIS to move in the X and Y axes to stabilize the imaging.

[0045] In the embodiment of the present disclosure, the video frame is a still image in the video, which is the smallest unit in the video sequence. The video frame is composed of pixels, each pixel represents a point in the image, and contains information about color and brightness. In order to achieve a better anti-shake effect, the target area can be selected from the central area of ​​each video frame, such as Figure 3 As shown, Figure 3 is a certain video frame, 301 is the central area of ​​the video frame, and the central area is the selected target area.

[0046] The above-mentioned grayscale value is a numerical value used to represent the brightness or grayscale intensity of the image. In the above-mentioned target area, determining the characteristic value of the pixel point of the corresponding row based on the grayscale value of the pixel point of each row of the video frame can be implemented as follows: taking the central area of ​​each video frame as the target area, and obtaining the grayscale value of each row of pixel points in the target area; determining the average value of the grayscale value of each row of pixel points, and taking the average value of the grayscale value as the pixel point characteristic value of the corresponding row of pixel points. It should be noted that the above-mentioned scenario is only an exemplary description, and the protection scope of the embodiments of the present disclosure is not limited thereto. For example, the above-mentioned pixel point characteristic value may also be a numerical value after other processing of the grayscale value.

[0047] In step S120, a first angular velocity and a first timestamp are obtained by the first sensor, and a first position obtained by the first sensor at a target time is determined according to the first angular velocity and the first timestamp, where the target time is a time selected from the rolling shutter exposure time corresponding to the target area.

[0048] In the embodiment of the present disclosure, the first sensor is used to obtain the angular velocity and the corresponding timestamp of the terminal device with shooting function. Exemplarily, the first sensor can be a gyroscope, and the first angular velocity and the first timestamp are the angular velocity and the timestamp of the terminal device during the shooting process obtained by the gyroscope. Among them, the gyroscope is a sensor for measuring and sensing rotational motion. It is usually composed of a rotating gyro wheel or other rotating mechanism, which can measure the rotational angular velocity of an object around its three axes (X axis, Y axis and Z axis). The first angular velocity is a physical quantity that describes the speed of the rotation of an object, usually expressed as the angle of rotation around the axis per unit time. The gyroscope measures the first angular velocity by sensing the rotational motion and converting it into an electrical signal. The first timestamp is used to mark the time point of each first angular velocity measurement. The first timestamp can represent the system time at the beginning or end of the measurement. In general, the timestamp can be a counter value, in milliseconds, microseconds or nanoseconds, indicating the time elapsed since a reference time point. Through the first timestamp information provided by the gyroscope, the change of the first angular velocity can be temporally located and analyzed.

[0049] When the target moment is determined based on the first angular velocity and the first timestamp, the first position obtained by the first sensor can be implemented as follows: construct a fourth interpolation function based on the first angular velocity and the first timestamp obtained by the gyroscope; obtain the gyroscope position corresponding to the target moment based on the fourth interpolation function, and the gyroscope position is the above-mentioned first position. Among them, each video frame in the embodiment of the present disclosure is obtained by rolling exposure, and the above-mentioned target moment is the moment selected from the rolling exposure time corresponding to the target area. Exemplarily, the middle moment of the rolling exposure time of the video frame can be selected as the above-mentioned target moment. Exemplarily, the above process can be specifically implemented as follows: determine the angular displacement A1 of the gyroscope based on the first angular velocity and the first timestamp obtained by the gyroscope. Construct an interpolation function based on A1 and the gyroscope timestamp Time1, and find the gyroscope position (first position) A2 corresponding to the middle moment Time2 of each frame.

[0050] In step S130, a first inter-frame pixel displacement between adjacent video frames is determined according to the first position, a second inter-frame pixel displacement between adjacent video frames is determined according to a method based on video matching based on pixel feature values, and a compensation pixel displacement is determined according to the first inter-frame pixel displacement and the second inter-frame pixel displacement.

[0051] In the disclosed embodiment, the above-mentioned inter-frame pixel displacement refers to the position change of a certain pixel between two adjacent frames in a video sequence. By comparing the pixel values ​​of two consecutive frames, the displacement of the pixel in the horizontal and vertical directions can be calculated. Specifically, the above-mentioned first inter-frame pixel displacement is further determined based on the first position obtained by the gyroscope information in step S120. In the case of a very small rotation angle, the above-mentioned first inter-frame pixel displacement pixel_1 can be approximated based on the displacement of the gyroscope between each frame, and the specific formula is as follows:

[0052] pixel_1=tan(ΔA2)*FocalLength

[0053] Among them, the above pixel_1 is the pixel displacement between the first frames, A2 is the gyroscope position corresponding to the middle moment Time2 of each frame, and FocalLength is the focal length of the camera.

[0054] The above-mentioned second inter-frame pixel displacement is the inter-frame pixel displacement pixel_2 determined based on video matching. Among them, the pixel displacement of each frame based on video matching refers to the calculation of the displacement of each pixel point in time by comparing the pixel difference between two adjacent frames in video processing. The specific implementation can be as follows: extract pixel data of adjacent frames; compare the difference of each pixel between the two frames, and use pixel-level difference or more advanced feature matching methods; calculate the displacement of each pixel point in time based on the difference; the displacement can be further processed as needed, such as smoothing, interpolation, or application to image stabilization and other tasks. In the embodiment of the present disclosure, the above-mentioned second inter-frame pixel displacement can be obtained by calculating the difference between the above-mentioned pixel feature values ​​of adjacent video frames.

[0055] In the embodiment of the present disclosure, the above-mentioned determination of the compensation pixel displacement based on the first inter-frame pixel displacement and the second inter-frame pixel displacement can be implemented as follows: the difference between the first inter-frame pixel displacement and the second inter-frame pixel displacement is calculated to obtain the compensation pixel displacement. Exemplarily, the above-mentioned process can be specifically implemented as follows: the difference between the second inter-frame pixel displacement pixel_2 based on video matching and the first inter-frame pixel displacement pixel_1 calculated based on gyroscope information is calculated to obtain the compensation pixel displacement pixel_3=pixel_2-pixel_1 that requires OIS movement.

[0056] In step S140, position data and a second time stamp in a single direction are acquired by a second sensor, and a second position is acquired by the second sensor at a target time according to the position data and the second time stamp.

[0057] In order to solve the problems of inaccurate feature point matching, long time consumption, poor robustness, etc. during 2D OIS hall calibration, the disclosed embodiment constructs a solution model based on OIS moving along one axis to reduce interference factors. Specifically, the second sensor can be a Hall sensor, and the disclosed embodiment realizes 1D calibration by acquiring data information on a single direction coordinate axis of the Hall sensor.

[0058] Among them, the above-mentioned Hall sensor is a sensor based on the working principle of the Hall effect. The Hall effect refers to the fact that when an electric current passes through a specific material in the presence of a magnetic field, a certain voltage will be generated in its transverse direction. The Hall sensor uses this effect to detect and measure the presence and strength of the magnetic field. The Hall sensor is usually composed of a Hall element, an amplifier and an output stage. The Hall element is a thin sheet or chip that contains conductive materials and charge carriers. Under the influence of an external magnetic field, the charge carriers are deflected to generate a Hall voltage. The amplifier amplifies this tiny Hall voltage signal and converts it into a usable voltage or digital signal output through the output stage. These parameters can be converted into coordinate data, that is, the above-mentioned sensor coordinate axis data, according to the requirements of the specific application. Exemplarily, the timestamp of the above-mentioned sensor can be obtained in the following way: use a microcontroller or a single-chip microcomputer to read the output of the Hall sensor, record the current timestamp information at each sampling, and associate the timestamp with the output of the sensor.

[0059] In the disclosed embodiment, the process of obtaining the position data and the second timestamp in a single direction through the second sensor, and determining the second position obtained by the second sensor at the target time according to the position data and the second timestamp can be implemented as follows: obtaining the timestamp of the Hall sensor and its sensor coordinate data in the direction of a single coordinate axis, using the sensor coordinate axis data as the above position data, and using the timestamp of the Hall sensor as the above second timestamp; constructing a fifth difference function based on the above position data and the second timestamp, and determining the sensor coordinate data corresponding to the above target time based on the fifth difference function as the second position. Among them, the target time can be selected from the middle moment of the rolling shutter exposure time of the above video frame. Specifically, the process can be implemented as follows: constructing an interpolation function based on the hall Hall sensor Y-axis data hall_1 and the timestamp hallTime of the OIS, obtaining the hall data hall_2 corresponding to the middle moment Time_2 of each frame, and obtaining the second position corresponding to each of the above video frames at the target time, wherein the above Hall sensor Y-axis data is the position data in the above single direction.

[0060] In step S150 , the inter-frame motion displacement of each adjacent video frame is determined according to the second position, and a first stabilization parameter is obtained based on the inter-frame motion displacement and the compensated pixel displacement.

[0061] In the disclosed embodiment, the inter-frame motion displacement of each adjacent video frame determined based on the second position can be obtained by the difference of the second positions of adjacent frames. Specifically, after obtaining the hall data hall_2 corresponding to the middle time Time_2 of each frame, the position hall_3 of the inter-frame OIS motion can be further obtained.

[0062] The first stabilization parameter is a parameter used to stabilize the image, and can be obtained by compensating for the ratio of pixel displacement to inter-frame motion displacement. Specifically, the first stabilization parameter can be an initial value of pixel / hall: PixelPerHall=pixel_3 / hall_3.

[0063] In step S160 , a second stabilization parameter is determined based on the first stabilization parameter, the inter-frame motion displacement, and the compensated pixel displacement, so as to perform anti-shake processing on the image according to the first stabilization parameter and the second stabilization parameter.

[0064] The second stabilization parameter is a parameter used to stabilize the image. Exemplarily, the process of determining the second stabilization parameter based on the first stabilization parameter, inter-frame motion displacement, and compensated pixel displacement can be implemented as follows: by searching at different time steps, comparing the displacement pixel points of hall_3*PixelPerHall with pixel_3, the initial value HallDelay of the time delay between the OIS hall value and the video can be predicted, and the initial value HallDelay is the second stabilization parameter.

[0065] After obtaining the first stability parameter and the second stability parameter, the embodiment of the present disclosure can write the first stability parameter and the second stability parameter into the above-mentioned device with shooting function, so that the device can process the captured image based on the first stability parameter and the second stability parameter to achieve the optical image stabilization effect.

[0066] In another embodiment of the present disclosure, the embodiment of the present disclosure can also optimize the above-mentioned first stable parameter and second stable parameter. Exemplarily, the optimization process can be implemented as follows: construct a loss function using the first stable parameter and the second stable parameter; based on the loss function, iteratively obtain the target first stable parameter and the target second stable parameter using a global optimization algorithm.

[0067] The above process of constructing the loss function can be implemented as follows: construct an increasing pixel array and determine the movement angle of each pixel in the pixel array; construct a pixel array in advance and determine the movement angle of each pixel in the pixel array, wherein the pixel feature value of each pixel in the pixel data is increasing; calculate the rolling shutter exposure time of each row of pixels in the pixel array to obtain the exposure time of each row of pixels in the current video frame; obtain the third timestamp and displacement angle through the first sensor, construct the first interpolation function based on the third timestamp and displacement angle, and use the first interpolation function to determine the first exposure time corresponding to each row of pixels. displacement angle; obtaining a fourth timestamp through a second sensor, constructing a second interpolation function according to the fourth timestamp, the second stability parameter and the inter-frame motion displacement, and determining the second displacement angle corresponding to the exposure moment of each row of pixels based on the second interpolation function; constructing a third interpolation function according to the first displacement angle, the second displacement angle and the pixel point feature values ​​of each row of pixels, and obtaining the pixel point feature values ​​of each row of pixels of the next video frame based on the third interpolation function; determining the pixel point feature values ​​of the next video frame mapped to the current video frame, and constructing a loss function based on the pixel point feature values ​​of the next video frame and the pixel point feature values ​​of the next video frame mapped to the current video frame.

[0068] In a specific embodiment, the construction of the above loss function may include the following steps:

[0069] S1: Construct an increasing pixel array, P1 represents the position of each row of pixels, FocalLength represents the focal length, and the corresponding motion angle of each pixel can be approximately calculated:

[0070] A3=arctan(P1 / FocalLength)

[0071] S2: Calculate the rolling shutter exposure time ROT corresponding to each row of pixels, and obtain the time of each row of pixels in this frame:

[0072] Time3=ROT+Time2[i]

[0073] S3: Use the gyroscope timestamp Time1 and the gyroscope displacement P1 between each frame to construct the interpolation function gyroPosIntp, and interpolate to obtain the gyroscope displacement angle corresponding to the time of each row of pixels in this frame:

[0074] A4=gyroPosIntp(Time3)

[0075] S4: Use the OIS timestamp hallTime–HallDelay and the OIS inter-frame displacement to construct an interpolation function interpHall, calculate the pixel displacement position of each row of this frame, and then calculate the displacement angle of each row:

[0076] P2=P1+interpHall(Time3)*PixelPerHall

[0077] A5=arctan(P2 / FocalLength)

[0078] S5: Use A4+A5 and FrameValue (pixel feature value) to construct the interpolation function frame1DIntp to obtain the pixel value of each row of the new frame:

[0079] newFrame=frame1DIntp(A3)

[0080] S6: Define the feature value of the previous frame as oldFrame, and the feature value of the next frame as newFrame. Then the feature value of the next frame mapped to the previous frame is:

[0081] NormFrame=(oldFrame-avg(oldFrame))*(std(newFrame) / std(oldFrame))+

[0082] avg(newFrame)

[0083] S7: Construct loss function: loss = abs(NormFrame-newFrame)

[0084] like Figure 4 As shown, after obtaining the above loss function, the PixelPerHallLSB and HallTSDelay times can be predicted through iteration based on the above loss function and using the Basin Hopping global optimization algorithm.

[0085] The present disclosure proposes a method for calibrating 1D-OIS by detecting the feature points of the motion of the previous and next frames along a single axis. The 1D OIS motion hall code change curve is as follows: Figure 5 As shown, this method simplifies the solution model, reduces interference factors, and improves the motion detection accuracy between previous and next frames, thereby improving the OIS hall calibration accuracy.

[0086] It should be noted that although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0087] Furthermore, in this exemplary embodiment, an image processing device is also provided, referring to Figure 6 As shown, the image processing device 600 may include a feature value determination module 610, a first position determination module 620, a compensation pixel displacement determination module 630, a second position determination module 640, a first stability parameter determination module 650 and an image processing module 660. Among them:

[0088] The feature value determination module 610 may be used to determine the feature value of the pixel points in the corresponding row in the target area based on the grayscale value of the pixel points in each row of the video frame;

[0089] A first position determination module 620 may be configured to obtain a first angular velocity and a first timestamp through a first sensor, and determine a first position obtained through the first sensor at a target time according to the first angular velocity and the first timestamp, wherein the target time is a time selected from a rolling shutter exposure time corresponding to a target area;

[0090] The compensation pixel displacement determination module 630 may be used to determine a first inter-frame pixel displacement between adjacent video frames according to the first position, determine a second inter-frame pixel displacement between adjacent video frames according to a method based on video matching based on pixel feature values, and determine a compensation pixel displacement according to the first inter-frame pixel displacement and the second inter-frame pixel displacement;

[0091] The second position determination module 640 may be used to obtain position data and a second timestamp in a single direction through a second sensor, and determine a second position obtained by the second sensor at a target time according to the position data and the second timestamp;

[0092] A first stability parameter determination module 650 may be used to determine the inter-frame motion displacement of each adjacent video frame according to the second position, and obtain a first stability parameter based on the inter-frame motion displacement and the compensated pixel displacement;

[0093] The image processing module 660 may be used to determine a second stabilization parameter based on the first stabilization parameter, the inter-frame motion displacement, and the compensated pixel displacement, so as to perform anti-shake processing on the image according to the first stabilization parameter and the second stabilization parameter.

[0094] In the embodiment of the present disclosure, the above-mentioned image processing device also includes a parameter optimization unit, which is used to: construct a loss function using the first stable parameter and the second stable parameter; based on the loss function, iteratively obtain the target first stable parameter and the target second stable parameter using a global optimization algorithm.

[0095] The parameter optimization unit specifically constructs the loss function by the following method: constructing a pixel array in advance, and determining the movement angle of each pixel in the pixel array, wherein the pixel feature value of each pixel in the pixel data is increasing; calculating the rolling shutter exposure time of each row of pixels in the pixel array, and obtaining the exposure time of each row of pixels in the current video frame; obtaining the third timestamp and displacement angle through the first sensor, constructing the first interpolation function based on the third timestamp and the displacement angle, and determining the first displacement angle corresponding to the exposure time of each row of pixels by using the first interpolation function; and obtaining the third timestamp and displacement angle through the second sensor. The device obtains a fourth timestamp, constructs a second interpolation function according to the fourth timestamp, the second stabilization parameter and the inter-frame motion displacement, and determines the second displacement angle corresponding to the exposure moment of each row of pixels based on the second interpolation function; constructs a third interpolation function according to the first displacement angle, the second displacement angle and the pixel point feature values ​​of each row of pixels, and obtains the pixel point feature values ​​of each row of pixels of the next video frame based on the third interpolation function; determines the pixel point feature values ​​of the next video frame mapped to the current video frame, and constructs a loss function based on the pixel point feature values ​​of the next video frame and the pixel point feature values ​​of the next video frame mapped to the current video frame.

[0096] The specific implementation details of the above-mentioned image processing device have been described in detail in the corresponding position of the above-mentioned image processing method, so they will not be repeated here.

[0097] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0098] Figure 7 Schematic diagram of the structure of an electronic device in the embodiment of the present disclosure. Figure 7 , which shows a structural schematic diagram of an electronic device 700 suitable for implementing the embodiments of the present disclosure. Figure 7 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0099] like Figure 7As shown, the electronic device 700 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 to a random access memory (RAM) 703 to implement the image processing method of the embodiment described in the present disclosure. In the RAM 703, various programs and data required for the operation of the electronic device 700 are also stored. The processing device 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0100] Typically, the following devices may be connected to the I / O interface 705: an input device 706 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 707 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 708 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 709. The communication device 709 may allow the electronic device 700 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 7 The electronic device 700 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0101] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains a program code for executing the method shown in the flowchart, thereby implementing the image processing method as described above. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 709, or installed from a storage device 708, or installed from a ROM 702. When the computer program is executed by the processing device 701, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0102] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0103] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0104] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0105] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device:

[0106] In the target area, the characteristic value of the pixel point of the corresponding row is determined based on the gray value of the pixel point of each row of the video frame;

[0107] Acquire a first angular velocity and a first timestamp through a first sensor, and determine a first position acquired through the first sensor at a target time according to the first angular velocity and the first timestamp, where the target time is a time selected from a rolling shutter exposure time corresponding to a target area;

[0108] Determine a first inter-frame pixel displacement between adjacent video frames according to the first position, determine a second inter-frame pixel displacement between adjacent video frames according to a method based on video matching based on pixel feature values, and determine a compensating pixel displacement according to the first inter-frame pixel displacement and the second inter-frame pixel displacement;

[0109] Acquire position data and a second timestamp in a single direction through a second sensor, and determine a second position acquired through the second sensor at a target time according to the position data and the second timestamp;

[0110] Determining the inter-frame motion displacement of each adjacent video frame according to the second position, and obtaining a first stabilization parameter based on the inter-frame motion displacement and the compensation pixel displacement;

[0111] A second stabilization parameter is determined based on the first stabilization parameter, the inter-frame motion displacement, and the compensated pixel displacement, so as to perform anti-shake processing on the image according to the first stabilization parameter and the second stabilization parameter.

[0112] Optionally, when the above one or more programs are executed by the electronic device, the electronic device may also execute other steps described in the above embodiments.

[0113] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0114] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0115] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit does not, in some cases, limit the unit itself.

[0116] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0117] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0118] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0119] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0120] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. An image processing method, characterized in that: include: In the target area, the characteristic value of the pixel point of the corresponding row is determined based on the gray value of the pixel point of each row of the video frame; Acquire a first angular velocity and a first timestamp through a first sensor, and determine a first position acquired through the first sensor at a target time according to the first angular velocity and the first timestamp, wherein the target time is a time selected from the rolling exposure time corresponding to the target area; Determine a first inter-frame pixel displacement between adjacent video frames according to the first position, determine a second inter-frame pixel displacement between the adjacent video frames according to the pixel point feature value based on a video matching method, and determine a compensating pixel displacement according to the first inter-frame pixel displacement and the second inter-frame pixel displacement; Acquire position data and a second timestamp in a single direction through a second sensor, and determine a second position acquired through the second sensor at the target time according to the position data and the second timestamp; Determining an inter-frame motion displacement of each of the adjacent video frames according to the second position, and obtaining a first stability parameter based on the inter-frame motion displacement and the compensated pixel displacement; A second stabilization parameter is determined based on the first stabilization parameter, the inter-frame motion displacement, and the compensated pixel displacement, so as to perform anti-shake processing on the image according to the first stabilization parameter and the second stabilization parameter.

2. The image processing method according to claim 1, characterized in that: The method further comprises: constructing a loss function using the first stabilization parameter and the second stabilization parameter; Based on the loss function, a global optimization algorithm is used to iteratively obtain a target first stability parameter and a target second stability parameter.

3. The image processing method according to claim 2, characterized in that: The using the first stable parameter and the second stable parameter to construct a loss function includes: Pre-constructing a pixel array, and determining the movement angle of each pixel in the pixel array, wherein the pixel feature value of each pixel in the pixel data is increasing; Calculating the rolling shutter exposure time of each row of pixels in the pixel array to obtain the exposure time of each row of pixels in the current video frame; Acquire a third timestamp and a displacement angle through the first sensor, construct a first interpolation function based on the third timestamp and the displacement angle, and use the first interpolation function to determine a first displacement angle corresponding to the exposure moment of each row of pixels; acquiring a fourth timestamp through the second sensor, constructing a second interpolation function according to the fourth timestamp, the second stability parameter and the inter-frame motion displacement, and determining a second displacement angle corresponding to the exposure moment of each row of pixels based on the second interpolation function; Constructing a third interpolation function according to the first displacement angle, the second displacement angle, and the pixel point feature values ​​of each row of pixels, and obtaining the pixel point feature values ​​of each row of pixels of the next video frame based on the third interpolation function; Determine the pixel feature values ​​of the next video frame mapped to the current video frame, and construct the loss function based on the pixel feature values ​​of the next video frame and the pixel feature values ​​of the next video frame mapped to the current video frame.

4. The image processing method according to claim 1, characterized in that: The step of determining the characteristic value of the pixel points in the corresponding row in the target area based on the grayscale value of the pixel points in each row of the video frame includes: Taking the central area of ​​each of the video frames as the target area, and obtaining the grayscale value of each row of pixels in the target area; An average value of the grayscale values ​​of the pixels in each row is determined, and the average value of the grayscale values ​​is used as the characteristic value of the pixels in the corresponding row.

5. The image processing method according to claim 4, characterized in that: The first sensor is a gyroscope, and the target time is the middle time of the rolling shutter exposure time corresponding to the central area; The step of acquiring a first angular velocity and a first timestamp through a first sensor, and determining a first position acquired through the first sensor at a target time according to the first angular velocity and the first timestamp, includes: Acquire the angular velocity and timestamp of the gyroscope, use the angular velocity of the gyroscope as the first angular velocity, use the timestamp of the gyroscope as the first timestamp, and construct a fourth interpolation function according to the first angular velocity and the first timestamp; The gyroscope position corresponding to the target time is acquired based on the fourth interpolation function to obtain the first position.

6. The image processing method according to claim 5, characterized in that: The second sensor is a Hall sensor; Acquiring position data and a second timestamp in a single direction through a second sensor, and determining a second position acquired through the second sensor at the target time according to the position data and the second timestamp, including: Acquire the timestamp of the Hall sensor and its sensor coordinate data in the direction of a single coordinate axis, use the sensor coordinate axis data as the position data, and use the timestamp of the Hall sensor as the second timestamp; A fifth difference function is constructed based on the position data and the second timestamp, and sensor coordinate data corresponding to the target time is determined based on the fifth difference function as the second position.

7. The image processing method according to claim 1, characterized in that: The determining the compensation pixel displacement according to the first inter-frame pixel displacement and the second inter-frame pixel displacement comprises: The difference between the first inter-frame pixel displacement and the second inter-frame pixel displacement is calculated to obtain the compensated pixel displacement.

8. An image processing device, characterized in that: include: A feature value determination module is used to determine the feature value of the pixel points in the corresponding row in the target area based on the grayscale value of the pixel points in each row of the video frame; a first position determination module, configured to obtain a first angular velocity and a first timestamp through a first sensor, and determine a first position obtained through the first sensor at a target time according to the first angular velocity and the first timestamp, wherein the target time is a time selected from a rolling shutter exposure time corresponding to the target area; a compensation pixel displacement determination module, configured to determine a first inter-frame pixel displacement between adjacent video frames according to the first position, determine a second inter-frame pixel displacement between the adjacent video frames according to the pixel point feature value based on a video matching method, and determine a compensation pixel displacement according to the first inter-frame pixel displacement and the second inter-frame pixel displacement; A second position determination module, configured to obtain position data and a second timestamp in a single direction through a second sensor, and determine a second position obtained by the second sensor at the target time according to the position data and the second timestamp; a first stabilization parameter determination module, configured to determine an inter-frame motion displacement of each of the adjacent video frames according to the second position, and obtain a first stabilization parameter based on the inter-frame motion displacement and the compensated pixel displacement; The image processing module is used to determine a second stabilization parameter based on the first stabilization parameter, the inter-frame motion displacement and the compensated pixel displacement, so as to perform anti-shake processing on the image according to the first stabilization parameter and the second stabilization parameter.

9. The image processing device according to claim 8, characterized in that: The image stabilization device further includes a parameter optimization unit, wherein the parameter optimization unit is used to: constructing a loss function using the first stabilization parameter and the second stabilization parameter; Based on the loss function, a global optimization algorithm is used to iteratively obtain a target first stability parameter and a target second stability parameter.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

11. An electronic device, characterized in that: include: processor; A memory, configured to store executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 7 by executing the executable instructions.

Images