Electronic image stabilization methods, equipment and storage media

Abstract

This application discloses an electronic image stabilization method, device, and storage medium, belonging to the field of terminal technology. The method includes: during video recording, detecting the current first motion state of the electronic device based on motion state data; determining a first image stabilization parameter corresponding to the first motion state from a target image stabilization parameter set, the target image stabilization parameter set including image stabilization parameters corresponding to various motion states, the image stabilization parameter including a range of motion, the range of motion indicating the difference in field of view of the image before and after image stabilization processing; performing image stabilization processing on a first video frame captured by the camera device according to the first image stabilization parameter to obtain a second video frame; and displaying or storing the second video frame. In this way, the range of motion of the image stabilization and the field of view of the output image can be adaptively adjusted according to the real-time motion state, meeting the image stabilization needs of diverse shooting scenarios, improving recording quality and user experience.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to an electronic image stabilization method, device and storage medium. Background Technology

[0002] Image shake in electronic devices can cause blurry or distorted images captured by cameras, affecting image quality. To reduce the impact of shake on image quality, image stabilization has been increasingly adopted in electronic devices. Electronic image stabilization (EIS) is one of the main image stabilization functions. EIS is an image stabilization technology that uses software algorithms to compensate for image shake.

[0003] Electronic image stabilization (EIS) algorithms achieve image stabilization by cropping the edges of the captured image to provide a margin. The margin indicates the difference in the field of view before and after stabilization, representing the stabilization capability. The margin is inversely proportional to the field of view; a larger margin indicates stronger stabilization and a smaller field of view in the output image after stabilization. Currently, when electronic devices record video using cameras, their EIS functions typically stabilize the video frames recorded by the camera according to a fixed margin.

[0004] This method of image stabilization based on a fixed margin has fixed stabilization capability and low flexibility, which may not meet the stabilization needs of diverse shooting scenarios. Summary of the Invention

[0005] This application provides an electronic image stabilization method, device, and storage medium, which can improve the flexibility of image stabilization and meet the image stabilization needs of diverse shooting scenarios.

[0006] In a first aspect, an electronic image stabilization method is provided, applied in an electronic device, the electronic device including a camera device, the method comprising:

[0007] Multiple image stabilization parameters corresponding to different motion states are pre-configured. During recording, based on the motion state data of the electronic device, the current first motion state of the electronic device is detected; based on the first motion state, the first image stabilization parameter corresponding to the first motion state is determined from the target image stabilization parameter group, which includes image stabilization parameters corresponding to multiple motion states; based on the first image stabilization parameter, the first video frame captured by the camera device is image stabilized to obtain the second video frame; the second video frame is then displayed or stored.

[0008] In this context, image stabilization parameters refer to the image stabilization processing parameters used to control the intensity of image stabilization. Image stabilization parameters can include the range of motion, which indicates the difference in the field of view of the image before and after image stabilization. Different motion states correspond to different ranges of motion. For example, the more intense the motion (such as greater amplitude or greater shaking), the larger the corresponding range of motion.

[0009] In this way, the motion state of the electronic device can be detected in real time during recording, and the recorded video frames can be stabilized according to the stabilization parameters such as the range of motion adapted to the real-time motion state. This achieves the effect of adaptively adjusting the range of motion of the stabilization and the field of view of the output image according to the real-time motion state, which can meet the stabilization needs of diverse shooting scenarios, thereby improving the flexibility and accuracy of electronic image stabilization, and thus improving the recording effect and user experience.

[0010] In some embodiments, the multiple motion states may include at least two of the following: tripod state, handheld stationary state, stationary rotation state, walking state, and running state. The tripod state refers to the state in which the electronic device is mounted on a tripod, and the handheld stationary state refers to the state in which the user holds the electronic device and remains stationary.

[0011] In some embodiments, the stabilization parameters may also include other stabilization parameters such as smoothing parameters. Smoothing parameters are used to smooth the shake path, which indicates the trajectory of image shake during shooting. For example, the more intense the motion, the stronger the smoothing capability indicated by the corresponding smoothing parameter. For instance, the smoothing parameter can be a filter coefficient (such as the filter order); the more intense the motion, the greater the filtering strength indicated by the corresponding filter coefficient.

[0012] In some embodiments, the first video frame can be an original video frame captured by a camera device, or a video frame obtained by processing the original video frame captured by the camera device. For example, the first video frame can be a video frame obtained by processing the original video frame captured by the camera device using image processing techniques such as digital zoom. Exemplarily, the magnification of the original video frame captured by the camera device is generally 1x, and the first video frame can be a video frame with other magnifications obtained by processing the original video frame using digital zoom technology, such as a 1.1x video frame or a 1.2x video frame.

[0013] In some embodiments, motion state data refers to data used to determine the motion state of an electronic device, such as inertial sensor data. For example, motion state data may include acceleration data and angular velocity data, such as acceleration data and angular velocity data in three-dimensional space. In addition, motion state data may also include other motion state data such as orientation data, which are not limited in this embodiment.

[0014] As an example, motion state data can be IMU data measured by an IMU. An IMU includes multiple inertial sensors, such as accelerometers and gyroscopes, and may also include magnetometers, etc. Accordingly, IMU data includes acceleration data measured by the accelerometer, angular velocity data measured by the gyroscope, and may also include orientation data measured by the magnetometer, etc.

[0015] In some embodiments, the target stabilization parameter group can be a fixed stabilization parameter group or a stabilization parameter group matched from multiple stabilization parameter groups. For example, the stabilization parameter group corresponding to the target effect tendency specified by the user through the effect tendency adjustment control is not limited in this respect.

[0016] In one embodiment, before recording, a preview interface of the recording mode can be displayed, the preview interface including an effect tendency adjustment control; in response to an adjustment operation on the effect tendency adjustment control, a target effect tendency specified by the adjustment operation is determined; and a target image stabilization parameter group corresponding to the target effect tendency is determined from the image stabilization parameter groups corresponding to different effect tendencies.

[0017] The effect tendency adjustment control is used to adjust the effect tendency, which indicates the tendency of the image stabilization effect between stability and field of view. Different effect tendencies correspond to different image stabilization parameter groups, and each image stabilization parameter group includes image stabilization parameters corresponding to various motion states.

[0018] By setting up effect preference adjustment controls in the video preview interface, users can easily choose the effect preference according to their needs, that is, whether the image stabilization effect leans more towards high stability or a wider field of view. The range of motion of the image stabilization can be adjusted according to the user's selected effect preference, thereby changing the field of view of the output image. This improves the flexibility of electronic image stabilization, meets the user's self-adjustment needs, and enhances recording quality and user experience.

[0019] In some embodiments, the effect tendency adjustment control can be steplessly adjusted within a specific effect tendency range, or it can be adjusted between multiple effect tendency levels. This application embodiment does not limit this.

[0020] In some embodiments, the effect preference adjustment control can be in the form of a slider control, progress bar control, scroll bar control, adjustment knob control, or adjustment arrow control, etc., and this application embodiment does not limit this.

[0021] Once the first stabilization parameter corresponding to the first motion state is determined, the current input image can be stabilized based on the first stabilization parameter.

[0022] In one possible implementation, the second stabilization parameter can be adjusted according to the first stabilization parameter and a preset adjustment step size to obtain the third stabilization parameter. Then, the input image is stabilized according to the third stabilization parameter to obtain the output image.

[0023] By gradually adjusting the current image stabilization parameters according to a preset adjustment step size, the image stabilization parameters can be gradually adjusted from the current parameters to the primary image stabilization parameters. This prevents large changes in the image stabilization parameters from causing excessive changes in the field of view (FOV) of the output images between two consecutive frames, which would negatively impact the user experience.

[0024] The second stabilization parameter can be either the initial stabilization parameter or the stabilization parameter corresponding to the previous input frame, i.e., the stabilization parameter used when stabilizing the previous input frame. For example, if the current input image is the first video frame captured during recording, the second stabilization parameter is the initial stabilization parameter; if the current input image is not the first video frame captured during recording, the second stabilization parameter is the stabilization parameter used when stabilizing the previous input image.

[0025] As an example, the adjustment direction can be determined based on the first and second stabilization parameters; the second stabilization parameter can be adjusted according to the adjustment direction and the preset adjustment step size to obtain the third stabilization parameter.

[0026] In some embodiments, the image transformation parameters of the first video frame can be determined based on the third stabilization parameter; and the first video frame can be transformed based on the image transformation parameters to obtain the second video frame.

[0027] In some embodiments, the range of motion and field of view (FOV) of the first video frame can be adjusted according to the adjusted stabilization parameters to achieve adaptive adjustment of the range of motion and the FOV of the output image based on the real-time motion state.

[0028] In other embodiments, the range of motion can be adaptively adjusted based on the adjusted stabilization parameters while maintaining a constant field of view (FOV). For example, the FOV of the output image can be kept constant by changing the FOV of the input image based on the adjusted stabilization parameters.

[0029] By adaptively adjusting the range of motion while maintaining a constant FOV, the impact of changes in the FOV of the output video frames on the user experience can be avoided, further improving the recording effect and user experience.

[0030] In some embodiments, the implementation of ensuring that the FOV of the output image remains unchanged by changing the FOV of the input image according to the adjusted stabilization parameters may include: the second stabilization parameter includes a second active range, and the third stabilization parameter includes a third active range; based on the third stabilization parameter; determining a second image size based on the second active range, the third active range, and the first image size of the previous video frame; wherein, the first field of view of the output image after stabilizing the input image of the second image size based on the third active range is the same as the second field of view, and the second field of view refers to the field of view of the output image after stabilizing the input image of the first image size based on the second active range; obtaining a first video frame based on the second image size and the first image size, wherein the image size of the first video frame is the first image size, and the ratio of the field of view of the first video frame to the field of view of the previous video frame is equal to the ratio of the second image size to the first image size.

[0031] In some embodiments, obtaining a first video frame based on a second image size and a first image size includes: obtaining an original video frame captured by a camera device; and performing image processing on the original video frame based on the second image size and the first image size to obtain the first video frame. For example, the original video frame captured by the camera device can be processed by an image processor driver to obtain the first video frame.

[0032] In this way, the FOV of the original video frames captured by the camera can be changed by processing the original video frames captured by the camera, thereby obtaining a first video frame that meets the FOV requirements.

[0033] In other embodiments, the camera device can be controlled to capture and output a video frame with an image size equal to the first image size and a field of view that is equal to the ratio of the field of view of the previous video frame to the ratio of the second image size to the first image size, based on the second image size and the first image size; and the first video frame captured and output by the camera device can be acquired.

[0034] In this way, the camera equipment can be directly controlled to capture and output the first video frame that meets the FOV requirement.

[0035] In one embodiment, the second camera intrinsics can be determined based on the second image size, the first image size, and the current first camera intrinsics of the camera device. Then, the first video frame is subjected to image stabilization processing based on the third stabilization parameter and the second camera intrinsics to obtain the second video frame.

[0036] In one embodiment, when the electronic device is detected to have switched from a first motion state to a second motion state based on motion state data, a fourth stabilization parameter corresponding to the second motion state can be determined from the target stabilization parameter set. Then, based on the fourth stabilization parameter, stabilization processing is applied to a third video frame captured by the camera device to obtain a fourth video frame, which is the next video frame after the first video frame. The fourth video frame is then displayed or stored.

[0037] In one embodiment, stabilizing a third video frame captured by a camera device according to a fourth stabilization parameter includes: adjusting a first stabilization parameter according to the fourth stabilization parameter to obtain a fifth stabilization parameter; and stabilizing the third video frame according to the fifth stabilization parameter to obtain a fourth video frame.

[0038] In one embodiment, adjusting the first stabilization parameter according to the fourth stabilization parameter to obtain the fifth stabilization parameter includes: determining the adjustment direction based on the first stabilization parameter and the fourth stabilization parameter; and adjusting the first stabilization parameter according to the adjustment direction and a preset adjustment step size to obtain the fifth stabilization parameter.

[0039] Secondly, an electronic image stabilization method is provided, applied in an electronic device, including a camera device. The method includes: displaying a preview interface of a recording mode, the preview interface including an effect tendency adjustment control, the effect tendency adjustment control being used to adjust the effect tendency, the effect tendency being used to indicate the tendency of the image stabilization effect between stability and field of view, different effect tendencies corresponding to different image stabilization parameters, the image stabilization parameters including a range of motion, the range of motion being used to indicate the difference in field of view of the image before and after image stabilization processing; in response to an adjustment operation on the effect tendency adjustment control, determining a target effect tendency specified by the adjustment operation; in response to a recording start operation, performing image stabilization processing on a first video frame captured by the camera device according to the target image stabilization parameters corresponding to the target effect tendency, obtaining a second video frame; and displaying the second video frame.

[0040] By setting up effect preference adjustment controls in the video preview interface, users can easily choose the effect preference according to their needs, that is, whether the image stabilization effect leans more towards high stability or a wider field of view. The range of motion of the image stabilization can be adjusted according to the user's selected effect preference, thereby changing the field of view of the output image. This improves the flexibility of electronic image stabilization, meets the user's self-adjustment needs, and enhances recording quality and user experience.

[0041] In this context, image stabilization parameters refer to the image stabilization processing parameters used to control the intensity of image stabilization. Image stabilization parameters can include the range of motion, which indicates the difference in the field of view of the image before and after image stabilization. Different motion states correspond to different ranges of motion. For example, the more intense the motion (such as greater amplitude or greater shaking), the larger the corresponding range of motion.

[0042] In some embodiments, the stabilization parameters may also include other stabilization parameters such as smoothing parameters. Smoothing parameters are used to smooth the shake path, which indicates the trajectory of image shake during shooting. For example, the more intense the motion, the stronger the smoothing capability indicated by the corresponding smoothing parameter. For instance, the smoothing parameter can be a filter coefficient (such as the filter order); the more intense the motion, the greater the filtering strength indicated by the corresponding filter coefficient.

[0043] In some embodiments, the effect tendency adjustment control can be steplessly adjusted within a specific effect tendency range, or it can be adjusted between multiple effect tendency levels. This application embodiment does not limit this.

[0044] In some embodiments, the effect preference adjustment control can be in the form of a slider control, progress bar control, scroll bar control, adjustment knob control, or adjustment arrow control, etc., and this application embodiment does not limit this.

[0045] In some embodiments, the stabilization parameters corresponding to the target effect tendency can be stabilization parameters corresponding to various motion states. Before stabilizing the first video frame captured by the camera device according to the target stabilization parameters corresponding to the target effect tendency, the current first motion state of the electronic device can be determined based on the motion state data of the electronic device. Then, the first stabilization parameter corresponding to the first motion state is determined from the stabilization parameters corresponding to various motion states; the first video frame is then stabilized according to the first stabilization parameter to obtain the second video frame.

[0046] In this way, the motion state of the electronic device can be detected in real time during recording, and the recorded video frames can be stabilized according to the stabilization parameters such as the range of motion adapted to the real-time motion state. This achieves the effect of adaptively adjusting the range of motion of the stabilization and the field of view of the output image according to the real-time motion state, which can meet the stabilization needs of diverse shooting scenarios, thereby improving the flexibility and accuracy of electronic image stabilization, and thus improving the recording effect and user experience.

[0047] Thirdly, an electronic image stabilization device is provided, which has the function of implementing the electronic image stabilization method described in the first aspect. The electronic image stabilization device includes at least one module, which is used to implement the electronic image stabilization method provided in the first aspect.

[0048] Fourthly, an electronic image stabilization device is provided, comprising a processor and a memory. The memory stores a program that supports the electronic image stabilization device in executing the electronic image stabilization method provided in the first aspect, and stores data related to implementing the electronic image stabilization method described in the first aspect. The processor is configured to execute the program stored in the memory. The electronic image stabilization device may further include a communication bus for establishing a connection between the processor and the memory.

[0049] Fifthly, a computer-readable storage medium is provided, wherein instructions are stored therein, which, when executed on a computer, cause the computer to perform the electronic image stabilization method described in the first aspect.

[0050] In a sixth aspect, a computer program product containing instructions is provided, which, when run on a computer, causes the computer to perform the electronic stabilization method described in the first aspect.

[0051] The technical effects achieved by the third, fourth, fifth and sixth aspects mentioned above are similar to the technical effects achieved by the corresponding technical means in the first or second aspects mentioned above, and will not be repeated here. Attached Figure Description

[0052] Figure 1 This is a comparative diagram illustrating the electronic image stabilization effect under different activity ranges provided in this application example;

[0053] Figure 2 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0054] Figure 3 This is a block diagram of a software system for an electronic device provided in an embodiment of this application;

[0055] Figure 4 This is a schematic diagram of the video preview interface in portrait and landscape modes provided in the embodiments of this application;

[0056] Figure 5 This is an operational schematic diagram illustrating a user's preference for adjusting effects, provided in an embodiment of this application.

[0057] Figure 6 This is a schematic diagram of an adaptive adjustment process for the active range and FOV of image stabilization provided in an embodiment of this application;

[0058] Figure 7 This is a schematic diagram of another adaptive adjustment process for the active range and FOV of image stabilization provided in an embodiment of this application;

[0059] Figure 8 This is a schematic diagram of a process for adaptively adjusting the active range of image stabilization while maintaining a constant FOV size, provided in an embodiment of this application.

[0060] Figure 9 This is a schematic diagram of another processing procedure provided in this application embodiment for adaptively adjusting the active range of image stabilization while maintaining a constant FOV size;

[0061] Figure 10 This is a flowchart illustrating an electronic image stabilization method provided in this application.

[0062] Figure 11 This is a flowchart illustrating another electronic image stabilization method provided in this application example;

[0063] Figure 12 This is a flowchart illustrating another electronic image stabilization method provided in this application example. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0065] It should be understood that "multiple" as mentioned in this application refers to two or more. In the description of this application, unless otherwise stated, " / " indicates "or," for example, A / B can mean A or B; "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist, for example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, to facilitate a clear description of the technical solutions of this application, the terms "first," "second," etc., are used to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first," "second," etc., do not limit the quantity or execution order, and that "first," "second," etc., do not necessarily imply differences.

[0066] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0067] To facilitate understanding, before providing a detailed description of the electronic image stabilization method provided in the embodiments of this application, the terms involved in the embodiments of this application will be explained first.

[0068] Electronic image stabilization (EIS) is a technology that uses software algorithms to compensate for image shake. For example, it can achieve image stabilization by cropping the edges of the captured image.

[0069] Margin: Margin indicates the range of shake that the electronic image stabilization algorithm can handle or compensate for. The larger the margin, the stronger the stabilization capability and the more stable the stabilization effect.

[0070] Electronic image stabilization algorithms may crop each frame to remove edge regions caused by shaking, ensuring the remaining image is stable. This cropping range can be considered part of an active range. For example, the active range indicates the difference in field of view (FOV) between the image before and after stabilization. Exemplarily, the active range is the ratio between the difference in FOV between the image before and after stabilization and the FOV of the image before stabilization.

[0071] Field of view (FOV): Also known as viewing angle or field of view, the field of view of an image in a shooting scene refers to the angle corresponding to the range of the actual scene in front of the lens that can be captured by the lens and imaged on the image sensor. In other words, the field of view of an image indicates the extent of the scene in the image. The larger the field of view of an image, the larger the range of the scene in the image, and the more scene content included in the image.

[0072] It should be noted that the range of motion is inversely proportional to the field of view. The larger the range of motion, the smaller the field of view of the image stabilized, that is, the smaller the field of view of the output image after image stabilization.

[0073] Please refer to Figure 1 , Figure 1 This is a comparative diagram illustrating the electronic image stabilization effect under different activity ranges provided in this application example. Figure 1 Figure (a) shows the output image obtained by stabilizing the input image based on a relatively small range of motion. Figure 1 Figure (b) shows the output image obtained by stabilizing the input image based on a relatively large range of motion. Here, the range of motion refers to the ratio between the FOV difference between the input and output images and the FOV of the input image; the spatial range between the input and output images can be considered the range of motion. Figure 1 As shown in Figure (a), stabilizing the input image based on a relatively small range of motion results in an output image with a relatively large field of view (FOV). That is, the larger the FOV of the output image, the smaller the range of motion, the weaker the stabilization capability, and the weaker the stability. Figure 1As shown in Figure (b), stabilizing the input image based on a relatively large range of motion results in an output image with a relatively small field of view (FOV). In other words, the smaller the FOV of the output image, the larger the range of motion, the stronger the stabilization capability, and the greater the stability.

[0074] As shown above, in electronic image stabilization technology, the stability of the stabilization effect is inversely proportional to the field of view. That is, the greater the stability, the smaller the field of view; the smaller the stability, the larger the field of view. Therefore, image stabilization effect usually cannot guarantee both high stability and a large field of view at the same time, and an appropriate trade-off must be made between the two as needed.

[0075] Effect bias: Effect bias indicates the tendency of the image stabilization effect between stability and field of view, that is, whether the image stabilization effect is more biased towards high stability or more biased towards a large field of view.

[0076] Inertial Measurement Unit (IMU): An IMU is a device used to measure specific physical quantities of an object, such as acceleration and rotational rate (sometimes also called angular velocity). For example, an IMU may include three accelerometers and three gyroscopes (also called angular velocity sensors or gyroscope sensors), and may also include three magnetometers. The IMU can use these sensors to measure physical quantities such as the acceleration and angular velocity of an object in three-dimensional space. Using these physical quantities, information such as the object's attitude, velocity, and position can be determined.

[0077] Next, the application scenarios and electronic devices involved in the electronic image stabilization method provided in the embodiments of this application will be described.

[0078] The electronic image stabilization method provided in this application is applied to electronic devices with shooting functions to perform image stabilization processing on images captured by the electronic device, thereby reducing the impact of device shake on image quality and improving shooting results.

[0079] In some embodiments, the electronic image stabilization method provided in this application is applied to video recording scenarios to perform image stabilization processing on video frames recorded by electronic devices, thereby improving the stability and quality of the recorded video. For example, it can be applied to video recording scenarios where the motion state changes.

[0080] In some embodiments, the electronic device may also be referred to as a terminal, terminal equipment, user equipment (UE), mobile station (MS), mobile terminal (MT), etc. The electronic device can be a mobile phone, smart TV, wearable device, tablet computer, computer with wireless transceiver capabilities, virtual reality (VR) electronic device, augmented reality (AR) electronic device, wireless terminal in industrial control, wireless terminal in self-driving, wireless terminal in remote medical surgery, wireless terminal in smart grid, wireless terminal in transportation safety, wireless terminal in smart city, wireless terminal in smart home, etc. The embodiments of this application do not limit the specific technology or specific device form used in the electronic device.

[0081] Figure 2 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. See also... Figure 2 The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0082] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0083] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0084] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to the instruction opcode and timing signals to complete the control of fetching and executing instructions.

[0085] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from this memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0086] In some embodiments, the processor 110 may include one or more interfaces, such as an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0087] The charging management module 140 receives charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 receives charging input from the wired charger via a USB interface 130. In some wireless charging embodiments, the charging management module 140 receives wireless charging input via the wireless charging coil of the electronic device 100. While charging the battery 142, the charging management module 140 can also supply power to the electronic device 100 via the power management module 141.

[0088] The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, internal memory 121, external memory, display screen 194, camera 193, and wireless communication module 160, etc. The power management module 141 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 141 may also be located within the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may be located in the same device.

[0089] The wireless communication function of electronic device 100 can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0090] Antennas 1 and 2 are used to transmit and receive electromagnetic wave signals. Mobile communication module 150 can provide wireless communication solutions, including 2G / 3G / 4G / 5G, for use in electronic device 100. Wireless communication module 160 can provide wireless communication solutions, including wireless local area networks (WLANs) (such as Wireless Fidelity (Wi-Fi) networks), Bluetooth (BT), Global Navigation Satellite System (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies, for use in electronic device 100.

[0091] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0092] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Mini LED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 100 may include one or N displays 194, where N is an integer greater than 1.

[0093] Electronic device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0094] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's image sensor. The light signal is converted into an electrical signal, and the image sensor transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimizations on image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be integrated into the camera 193.

[0095] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the electronic device 100 may include one or N cameras 193, where N is an integer greater than 1.

[0096] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when electronic device 100 selects a frequency, the DSP performs Fourier transforms on the frequency energy.

[0097] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. Thus, electronic device 100 can play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG 2, MPEG 3, MPEG 4, etc.

[0098] An NPU (Neural Processing Unit) is a computational processor for neural networks (NNs). By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0099] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions, such as saving music, video, and other files on the external memory card.

[0100] Internal memory 121 can be used to store computer-executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of electronic device 100 by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created by electronic device 100 during use (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0101] Electronic device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D and application processor.

[0102] Pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 180A can be disposed on display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may include at least two parallel plates with conductive material. When force is applied to pressure sensor 180A, the capacitance between the electrodes changes. Electronic device 100 determines the pressure intensity based on the change in capacitance. When a touch operation is applied to display screen 194, electronic device 100 detects the touch operation intensity based on pressure sensor 180A. Electronic device 100 can also calculate the touch position based on the detection signal from pressure sensor 180A. In some embodiments, touch operations applied to the same touch position but with different touch operation intensities can correspond to different operation commands. For example, when a touch operation with an intensity less than the pressure threshold is applied to the SMS application icon, a command to view an SMS message is executed. When a touch operation with an intensity greater than or equal to the pressure threshold is applied to the SMS application icon, a command to create a new SMS message is executed.

[0103] The gyroscope sensor 180B can be used to determine the motion attitude of the electronic device 100. In some embodiments, the gyroscope sensor 180B can determine the angular velocity of the electronic device 100 about three axes (i.e., the x, y, and z axes). The gyroscope sensor 180B can be used for image stabilization. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the shake of the electronic device 100, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to counteract the shake of the electronic device 100 by moving in the opposite direction, thus achieving image stabilization. The gyroscope sensor 180B can also be used in navigation and motion-sensing game scenarios.

[0104] The accelerometer 180E can detect the magnitude of acceleration of electronic device 100 in various directions (generally three axes). When electronic device 100 is stationary, it can detect the magnitude and direction of gravity. The accelerometer 180E can also be used to identify the attitude of electronic device 100, and can be applied to applications such as screen orientation switching and pedometers.

[0105] A distance sensor 180F is used to measure distance. Electronic device 100 can measure distance via infrared or laser. In some embodiments, during a shooting scenario, electronic device 100 can utilize the distance sensor 180F for distance measurement to achieve fast focusing.

[0106] The ambient light sensor 180L is used to sense the brightness of ambient light. The electronic device 100 can adaptively adjust the brightness of the display screen 194 based on the sensed ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also work with the proximity sensor 180G to detect whether the electronic device 100 is in a pocket to prevent accidental touches.

[0107] Touch sensor 180K, also known as a "touch panel," can be located on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a "touch display." Touch sensor 180K detects touch operations applied to or near it. Touch sensor 180K can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be located on the surface of electronic device 100, in a different position than display screen 194.

[0108] The software system of electronic device 100 will be described next.

[0109] An operating system runs on top of these components. Examples include Apple's iOS, Google's Android, and Microsoft's Windows. Applications can be installed and run on this operating system.

[0110] The operating system of electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment uses the layered architecture Android system as an example to exemplify the software structure of electronic device 100.

[0111] Figure 3 This is a block diagram of a software system for an electronic device 100 provided in an embodiment of this application. See also... Figure 3 A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the hardware abstraction layer, and the driver layer.

[0112] The application layer can include a series of application packages. For example... Figure 3 As shown, the application layer can include a camera application. Additionally, the application layer can also include applications such as gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and SMS.

[0113] In some instances, the video preview interface provided by a camera application may include effect bias adjustment controls. These controls are used to adjust the effect bias, which indicates the balance between image stabilization effectiveness and field of view. Different effect biases correspond to different image stabilization parameters, which include at least the range of motion and may also include other relevant parameters used for image stabilization processing, such as smoothing parameters.

[0114] By adjusting the effect bias, you can change stabilization parameters such as range of motion, thereby altering the stability and field of view of the recording. Users can then adjust the effect bias using the effect bias adjustment controls, allowing them to choose whether the recording leans towards high stability or a wide field of view.

[0115] In some embodiments, the stabilization parameters that favor a particular effect can be a set of stabilization parameters, which may include stabilization parameters corresponding to various motion states. For example, the various motion states may include at least two of tripod support, handheld stillness, walking, running, and stationary rotation. The stabilization parameters corresponding to each motion state include at least the range of motion, and may also include other stabilization parameters such as stabilization path parameters. Stabilization path parameters may be smoothing parameters, etc.

[0116] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0117] For example, such as Figure 3 As shown, the application framework layer may include a camera access interface. The camera access interface may include a camera management interface and a camera device interface. The camera management interface provides an interface for managing the camera; the camera device interface provides an interface for accessing the camera.

[0118] A hardware abstraction layer (HAL) is used to abstract hardware. For example, a HAL can include a camera abstraction layer and other hardware device abstraction layers. The camera hardware abstraction layer can call camera algorithms from the camera algorithm library.

[0119] For example, such as Figure 3 As shown, the hardware abstraction layer includes the camera hardware abstraction layer and the camera algorithm library. The camera algorithm library may include the EIS algorithm module, and may also include other software algorithm modules.

[0120] The camera hardware abstraction layer can receive IMU data sent by the IMU signal processing driver in the driver layer and sensor images sent by the image processor driver, and send the IMU data and sensor images to the EIS algorithm module for processing.

[0121] In one embodiment, the EIS algorithm module stores multiple sets of image stabilization parameters, each corresponding to a different motion state. Each set of stabilization parameters includes at least the range of motion. When the camera is recording video, the EIS algorithm module can perform motion detection based on IMU data to detect the current motion state of the electronic device and determine the target stabilization parameter corresponding to the current motion state from the multiple sets of stabilization parameters. After determining the target stabilization parameter, the EIS algorithm module can calculate image transformation information of the sensor image based on the IMU data, the sensor image, and the target stabilization parameter. This image transformation information is then sent to the camera hardware abstraction layer (HAL), which performs image transformation on the sensor image based on the image transformation information to obtain a stabilized image, thus completing the image stabilization process for the sensor image.

[0122] In one possible implementation, after the EIS algorithm module determines the target image stabilization parameters, it can calculate the image transformation parameters of sensor image 1 based on the IMU data, the input sensor image 1, and the target image stabilization parameters. The image transformation information is then sent to the camera hardware abstraction layer, which performs image transformation on sensor image 1 based on the image transformation information to obtain the image stabilization image corresponding to sensor image 1 (i.e., the target image after image stabilization).

[0123] In another embodiment, after determining the target stabilization parameters, the EIS algorithm module can also determine the image size change of the next input image based on the target stabilization parameters. Based on this image size change, it sends a FOV change request to the camera hardware abstraction layer to request that the FOV of the next frame sensor image to be output be adjusted to the target FOV. The camera hardware abstraction layer, based on this FOV change request, can obtain the sensor image (sensor image 2) at the target FOV and send it to the EIS algorithm module. Upon receiving sensor image 2, the EIS algorithm module can calculate the image transformation parameters of sensor image 2 based on the IMU data, sensor image 2, and the target stabilization parameters. It then sends this image transformation information to the camera hardware abstraction layer, which performs image transformation on sensor image 2 based on this information to obtain the stabilized image corresponding to sensor image 2 (i.e., the target image after stabilization).

[0124] For example, after receiving the FOV change request, the camera hardware abstraction layer can send the FOV change request to the image processor driver, instructing the image processor driver to digitally zoom the original sensor image output by the image sensor according to the FOV change request, obtaining the sensor image of the target FOV (sensor image 2), and then sending sensor image 2 to the EIS algorithm module via the camera hardware abstraction layer. Alternatively, after receiving the FOV change request, the camera hardware abstraction layer itself can also digitally zoom the original sensor image output by the image processor driver to obtain the sensor image of the target FOV (sensor image 2), and then send sensor image 2 to the EIS algorithm module. This application embodiment does not limit the method by which the camera hardware abstraction layer obtains the sensor image of the target FOV according to the FOV change request.

[0125] In another embodiment, the EIS algorithm module stores electronic image stabilization parameters corresponding to different effect preferences. Before recording begins, the user can adjust the effect preference using the effect preference adjustment button in the recording preview interface. The application layer can then send the user-adjusted target effect preference to the EIS algorithm module via the application framework layer and the camera hardware abstraction layer. Upon receiving the target effect preference, the EIS algorithm module can determine the corresponding electronic image stabilization parameter from the electronic image stabilization parameters corresponding to different effect preferences. When the camera device is recording, the EIS algorithm module can calculate the image transformation information of the sensor image based on the IMU data, the input sensor image, and the electronic image stabilization parameter. This image transformation information is then sent to the camera hardware abstraction layer, which performs image transformation on the sensor image based on the image transformation information to obtain a stabilized image, thus completing the image stabilization process for the sensor image.

[0126] The electronic image stabilization (EIS) parameters corresponding to the target effect can include a single set of stabilization parameters or multiple sets of stabilization parameters corresponding to various motion states. When the EIS parameters corresponding to the target effect include a single set of stabilization parameters, the EIS algorithm module calculates the image transformation information of the sensor image based on IMU data, the sensor image, and the set of stabilization parameters, regardless of the motion state of the electronic device during video recording. Alternatively, when the EIS parameters corresponding to the target effect include multiple sets of stabilization parameters corresponding to various motion states, the EIS algorithm module can first perform motion detection based on IMU data to detect the current motion state of the electronic device, then determine the target stabilization parameters corresponding to the current motion state from the EIS parameters corresponding to the target effect, and finally calculate the image transformation information of the sensor image based on IMU data, the sensor image, and the target stabilization parameters.

[0127] The driver layer is used to provide drivers for different hardware devices. For example, the driver layer may include camera device drivers, which may include IMU signal processing drivers and image processor drivers.

[0128] The hardware layer may include a camera device. The camera device may include an IMU sensor and an image sensor. The image sensor may include one or more. The image sensor may be a complementary metal-oxide-semiconductor (CMOS) image sensor, etc. Additionally, the hardware layer may also include other sensors, such as a time-of-flight (TOF) depth sensor, a multispectral sensor, etc., but this application does not limit the specific sensors used.

[0129] In this embodiment of the application, by calling the hardware abstraction layer interface in the hardware abstraction layer, the connection between the application layer and application framework layer above the hardware abstraction layer and the driver layer and hardware layer below can be realized, thereby realizing camera data transmission and function control.

[0130] It should be noted that, Figure 3 The layers in the illustrated software architecture and the components contained within each layer do not constitute a specific limitation on the electronic device. In other embodiments, the electronic device may include more layers than illustrated, such as a system library layer and a kernel layer. Furthermore, each layer may include more or fewer components than illustrated, which is not limited in this application.

[0131] It is understood that, in order to implement the electronic image stabilization method in the embodiments of this application, an electronic device includes hardware and / or software modules that perform various functions. Based on the algorithm steps of the examples described in the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is implemented in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in conjunction with the embodiments.

[0132] It should be noted that although the embodiments of this application are described using the Android system as an example, the basic principles are also applicable to electronic devices based on operating systems such as iOS or Windows.

[0133] The following example, using a shooting scenario, illustrates the workflow of the software and hardware of electronic device 100.

[0134] When a user taps on the touch sensor 180K, the camera app is activated and invokes various camera devices in the camera hardware abstraction layer via the camera access interface. For example, the camera hardware abstraction layer can send a command to the camera device driver to invoke a specific camera, while the camera algorithm library begins loading the EIS algorithm used in this embodiment. When the image sensor in the hardware layer is invoked, for example, image sensor 1 in a specific camera is invoked to acquire a raw image. This raw image is returned to the hardware abstraction layer, where the EIS algorithm in the loaded camera algorithm library is used to perform image stabilization on the raw image to obtain the target image. The obtained target image is then sent back to the camera application via the camera access interface for display and storage.

[0135] Next, the electronic image stabilization method involved in the embodiments of this application will be described in detail with reference to the video recording scenario.

[0136] Currently, when electronic devices record video using cameras, the device's electronic image stabilization function typically performs image stabilization on the video frames captured by the camera according to a fixed margin, and the image stabilization capability is fixed.

[0137] For example, after the camera app is launched, the device's electronic image stabilization (EIS) function can initialize the margin, which is a preset margin. Once the camera starts recording, the margin is fixed. Regardless of the movement of the electronic device, the EIS function performs image stabilization on the video frames captured by the camera according to the preset margin. The stabilization capability is fixed, and the field of view of each video frame in the recorded video is also fixed.

[0138] However, the aforementioned method of image stabilization based on a fixed margin offers limited stabilization capability and flexibility, potentially failing to meet the diverse needs of various shooting scenarios. For example, in practical applications, users may use electronic devices to shoot under different motion conditions, such as handheld shooting while stationary, walking, or running. The stabilization requirements differ depending on the motion condition. For instance, handheld shooting with relatively little device shake doesn't require strong stabilization, while shooting while running involves more significant shake, necessitating stronger stabilization. Applying a fixed margin to image stabilization under all motion conditions may not satisfy the varying stabilization needs of different shooting scenarios.

[0139] To address the aforementioned issues and improve the flexibility of image stabilization, this application provides two electronic image stabilization methods: one is an adaptive adjustment method for the stabilization range, and the other is a method for adjusting the stabilization range according to the user's wishes. These two electronic image stabilization methods are described below.

[0140] The first type of electronic image stabilization method: an image stabilization method that adaptively adjusts the range of motion of the stabilization.

[0141] In this electronic image stabilization method, various stabilization parameters corresponding to different motion states are pre-configured. These parameters include the range of motion, and may also include other stabilization parameters such as smoothing parameters. The range of motion indicates the difference in the field of view of the image before and after stabilization. The smoothing parameter is used to smooth the shake path, which indicates the image shake trajectory during the shooting process.

[0142] During recording, the current motion state of the electronic device can be detected. Based on the first motion state, the first stabilization parameter corresponding to the first motion state can be determined from the stabilization parameters corresponding to various motion states. Then, based on the first stabilization parameter, the video frames captured by the camera device are stabilized, and the stabilized video frames are displayed or stored as the recorded video frames.

[0143] In this way, the motion state of the electronic device can be detected in real time during recording, and the recorded video frames can be stabilized according to the stabilization parameters such as the range of motion adapted to the real-time motion state. This achieves the effect of adaptively adjusting the range of motion of the stabilization and the field of view of the output image according to the real-time motion state, which can meet the stabilization needs of diverse shooting scenarios, thereby improving the flexibility and accuracy of electronic image stabilization, and thus improving the recording effect and user experience.

[0144] In some embodiments, the multiple motion states may include at least two of the following: tripod state, handheld stationary state, stationary rotation state, walking state, and running state. The tripod state refers to the state in which the electronic device is mounted on a tripod, and the handheld stationary state refers to the state in which the user holds the electronic device and remains stationary.

[0145] In some embodiments, the stabilization parameter refers to the stabilization processing parameter used to control the intensity of stabilization.

[0146] As an example, image stabilization parameters can include range of motion, with different ranges of motion corresponding to different motion states. For instance, the more intense the motion (such as greater amplitude or greater shaking), the larger the corresponding range of motion.

[0147] As another example, image stabilization parameters can also include smoothing parameters. Smoothing parameters are used to smooth the shake path, which indicates the trajectory of image shake during shooting. For example, the more intense the motion, the stronger the smoothing capability indicated by the corresponding smoothing parameter. For instance, smoothing parameters can be filter coefficients (such as filter order). The more intense the motion, the stronger the filtering intensity indicated by the corresponding filter coefficients. For example, the more intense the motion, the higher the corresponding filter order. The filter order can be 5th order, 10th order, etc.

[0148] As an example, the stabilization parameters for various motion states are shown in Table 1 below.

[0149] Table 1

[0150] motion state Activity margin Smoothing parameters tripod 0％ Smoothing parameter 1 Handheld still 5％ Smoothing parameter 2 walk 20％ Smoothing parameter 3 running 30％ Smoothing parameter 4 Spinning in place 10％ Smoothing parameter 5

[0151] It should be noted that Table 1 above does not constitute a limitation on motion state and image stabilization parameters. Image stabilization parameters corresponding to different motion states can also be in other forms, and this application embodiment does not limit them.

[0152] The second type of electronic image stabilization method is one that adjusts the range of motion of the stabilization according to the user's wishes.

[0153] In this electronic image stabilization method, after switching to the video preview interface, the preview interface can include an effect preference adjustment control. This control indicates the tendency of the recording effect between stability and field of view. Different effect preferences correspond to different stabilization parameters, including range of motion and other stabilization parameters such as smoothing parameters. Users can adjust the effect preference using the control, and the target effect preference specified by the adjustment operation can be determined. Then, in response to the recording start operation, during recording, the video frames captured by the camera are stabilized according to the target stabilization parameters corresponding to the target effect preference, and the stabilized video frames are displayed or stored as the recorded video frames. For example, the recorded video frames can be displayed in real time on the video display interface.

[0154] By setting up effect preference adjustment controls in the video preview interface, users can easily choose the effect preference according to their needs, that is, whether the image stabilization effect leans more towards high stability or a wider field of view. The range of motion of the image stabilization can be adjusted according to the user's selected effect preference, thereby changing the field of view of the output image. This improves the flexibility of electronic image stabilization, meets the user's self-adjustment needs, and enhances recording quality and user experience.

[0155] In some embodiments, the effect tendency adjustment control can be infinitely adjusted within a specific effect tendency range, or it can be adjusted between multiple effect tendency levels. This application embodiment does not limit this.

[0156] Next, taking the effect preference adjustment control as a slider control as an example, we will provide an exemplary description of the effect preference adjustment control in the video preview interface with reference to the attached diagram.

[0157] Please refer to Figure 4 , Figure 4 This is a schematic diagram of the video recording preview interface in portrait and landscape modes provided in the embodiments of this application. The video recording preview interface refers to the preview interface in recording mode. Figure 4As shown in Figure (a), the video preview interface in portrait mode includes a vertical effect bias adjustment control 10 and a record button 20 at the bottom. The effect bias adjustment control 10 is located on the side of the video preview interface and is a slider control, including a slider and a track. The track is parallel to the side of the electronic device and extends from the top to the bottom. One end of the track represents maximum stability, and the other end represents the maximum field of view; for example, the top represents maximum stability, and the bottom represents the maximum field of view. Users can drag the slider to adjust the effect bias, that is, adjust the bias of the jitter effect between high stability and a large field of view. For example, dragging the slider upwards adjusts the effect bias to be more biased towards high stability, and dragging the slider downwards adjusts the effect bias to be more biased towards a large field of view. Figure 4 As shown in Figure (b), the video preview interface in landscape mode includes a horizontal effect preference adjustment control 10 and a record button 20. The effect preference adjustment control 10 includes a slider and a track, which is parallel to the side of the electronic device. The leftmost end of the track represents maximum stability, and the rightmost end represents the maximum field of view. Users can drag the slider to the left to adjust the effect preference to a higher stability, and drag the slider to the right to adjust the effect preference to a larger field of view.

[0158] It should be noted that the track in the effect preference adjustment control 10 can also be perpendicular to the side, and the effect preference adjustment control 10 can also be located in other positions on the video preview interface. This application embodiment does not limit the direction and position of the effect preference adjustment control 10. For example, the video preview interface in portrait mode can also include the horizontal effect preference adjustment control 10, and the video preview interface in landscape mode can also include the vertical effect preference adjustment control 10.

[0159] It should also be noted that, Figure 4 The example given is a slider control for adjusting the effect orientation. It should be understood that the effect orientation adjustment control can also be a progress bar control, a scroll bar control, an adjustment knob control, or an adjustment arrow control, etc. This application embodiment will not provide examples of each of these.

[0160] Next, we will illustrate the process of users' self-adjustment of their preferred effects with specific examples.

[0161] Please refer to Figure 5 , Figure 5 This is a schematic diagram illustrating a user's adjustment of the desired effect, provided in an embodiment of this application. The user can click the camera application icon to launch the camera application. In response to the user's click, the electronic device can display, as shown below... Figure 5Figure (a) shows the preview interface for the photo mode. At the bottom of this preview interface are icons for other shooting modes such as video recording mode and portrait mode. The user can click the video recording mode icon; in response to the user's click, the electronic device can switch the shooting mode from photo mode to video recording mode and display something like... Figure 5 The preview interface for video recording mode shown in Figure (b) includes a vertical effect preference adjustment control 10 located on the side. The effect preference adjustment control 10 may include a slider and a track. After switching to the preview interface for video recording mode, the slider can be in its initial position (e.g., the middle of the track) or in the position previously adjusted by the user; this embodiment does not limit this. The user can drag the slider in the effect preference adjustment control 10 to adjust the effect preference, such as... Figure 5 As shown in Figure (b), users can drag the slider down to adjust the effect towards a wider field of view; for example, the slider can be dragged down to... Figure 5 The position of the slider shown in Figure (c) is as follows. Furthermore, as the user adjusts the effect preference, the preview interface can also adjust the field of view of the preview image accordingly. For example, as the user drags the slider downwards, the field of view of the preview image displayed on the preview interface also increases. In this way, the user can intuitively determine whether the current effect preference adjustment is appropriate based on the field of view of the preview image in the preview interface, thereby achieving precise adjustment. After the effect preference adjustment is completed, as shown... Figure 5 As shown in Figure (d), the user can click the recording button 20 in the preview screen to start the recording function. In response to the user's operation, the electronic device can start recording and, during the recording process, perform image stabilization on each frame captured by the camera device according to the shake parameters corresponding to the effect preference selected by the user.

[0162] It should be noted that electronic devices can also hide or show effect preference adjustment controls. For example, effect preference adjustment controls can be hidden or redisplayed based on user operation, thereby improving the flexibility of displaying effect preference adjustment controls.

[0163] It should also be noted that the two electronic image stabilization methods described above can be combined as needed. For example, in the second electronic image stabilization method, the target stabilization parameter corresponding to the target effect tendency can also include stabilization parameters corresponding to various motion states. After determining the target effect tendency specified by the adjustment operation, in response to the recording start operation, during the recording process, the first motion state of the electronic device can also be detected. Based on the first motion state, the first stabilization parameter corresponding to the first motion state is determined from the stabilization parameters corresponding to various motion states. Then, based on the first stabilization parameter, the video frames captured by the camera device are stabilized, and the stabilized video frames are displayed or stored as the recorded video frames.

[0164] In some embodiments, different effect tendencies correspond to different sets of image stabilization parameters, and each set of parameters includes image stabilization parameters corresponding to various motion states. For example, please refer to Table 2 below, which shows the sets of image stabilization parameters corresponding to two different effect tendencies.

[0165] Table 2

[0166]

[0167] It should be noted that Table 2 above does not constitute a limitation on the stabilization parameter group corresponding to the effect tendency. Different effect tendencies may correspond to different stabilization parameter groups in other forms, and this application embodiment does not limit this.

[0168] It should be noted that the first type of electronic image stabilization method mentioned above, namely the adaptive adjustment of the stabilization range of motion, can also include two implementation methods. One is to adaptively adjust the stabilization range of motion and FOV, that is, to change the FOV size of the output image as the range of motion is adjusted. The other is to adaptively adjust the stabilization range of motion while maintaining a constant FOV size, that is, to maintain a constant FOV size as the range of motion is adjusted, without changing the FOV size of the output image.

[0169] Next, we will introduce in detail the active range of adaptive image stabilization and the implementation method of FOV.

[0170] Please refer to Figure 6 , Figure 6 This is a schematic diagram illustrating an adaptive adjustment process for the active range and field of view (FOV) of image stabilization, as provided in an embodiment of this application, and is applied to electronic devices, such as... Figure 6 As shown, the implementation of adaptive adjustment of the active range and FOV of image stabilization can include the following steps:

[0171] A1. During the recording process of the camera device, the first video frame captured by the camera device is acquired, and the first video frame is used as the input image for the current image stabilization process.

[0172] The first video frame can be the original video frame captured by the camera device, or it can be a video frame obtained by processing the original video frame captured by the camera device.

[0173] For example, the first video frame can be a video frame obtained by processing the original video frame captured by the camera device using image processing techniques such as digital zoom. For instance, the magnification of the original video frame captured by the camera device is generally 1x, and the first video frame can be a video frame with other magnifications obtained by processing the original video frame using digital zoom technology, such as a 1.1x video frame or a 1.2x video frame, etc.

[0174] It should be noted that the magnification of an image is inversely proportional to the field of view (FOV); the higher the magnification, the smaller the FOV.

[0175] A2. Acquire motion status data of electronic devices.

[0176] Motion state data refers to data used to determine the motion state of an electronic device, such as inertial sensor data. For example, motion state data may include acceleration data and angular velocity data, such as acceleration data and angular velocity data in three-dimensional space. In addition, motion state data may also include other motion state data such as direction data, which are not limited in this embodiment.

[0177] As an example, motion state data can be IMU data measured by an IMU. An IMU includes multiple inertial sensors, such as accelerometers and gyroscopes, and may also include magnetometers, etc. Accordingly, IMU data includes acceleration data measured by the accelerometer, angular velocity data measured by the gyroscope, and may also include orientation data measured by the magnetometer, etc.

[0178] A3. Based on motion state data, detect the first motion state currently in which the electronic device is located.

[0179] Based on motion state data, information such as the position, attitude, speed, and rotational motion of electronic devices can be analyzed, and the motion state of the electronic devices can be assessed based on this information.

[0180] A4. Based on the first motion state, determine the first stabilization parameter corresponding to the first motion state from the target stabilization parameter group. The target stabilization parameter group includes stabilization parameters corresponding to various motion states.

[0181] The target stabilization parameter group can be a fixed stabilization parameter group or a stabilization parameter group matched from multiple stabilization parameter groups. For example, the stabilization parameter group corresponding to the target effect tendency specified by the user through the effect tendency adjustment control is not limited in this embodiment.

[0182] A5. Based on the first anti-shake parameter, perform anti-shake processing on the input image to obtain the output image.

[0183] In other words, after determining the first stabilization parameter corresponding to the first motion state, the current input image can be stabilized based on the first stabilization parameter.

[0184] As an example, stabilizing an input image based on a first stabilization parameter can include the following two possible implementations.

[0185] The first possible implementation is to perform image stabilization on the input image according to the first stabilization parameter.

[0186] In the first possible implementation, after determining the first stabilization parameter corresponding to the current first motion state, the input image can be directly stabilized according to the first stabilization parameter.

[0187] In some embodiments, image transformation parameters of the input image can be determined first based on the first stabilization parameter. Then, the input image is transformed according to the image transformation parameters to obtain the output image.

[0188] Image transformation parameters are parameters used to transform the input image to compensate for jitter. Image transformation parameters can include one or more of geometric transformation parameters, scale transformation parameters, and pixel transformation parameters.

[0189] Geometric transformations can include one or more of the following: translation, rotation, scaling, affine transformation, and perspective transformation. Scaling transformations can change the resolution, granularity, or level of detail of an image, and can include one or more of the following: cropping, pixel interpolation, and magnification. Pixel transformations operate on the pixel values ​​in an image.

[0190] In some embodiments, a stabilization path can be calculated based on first stabilization parameters, and image transformation parameters of the input image can be determined based on the stabilization path. The stabilization path indicates a series of operational paths taken to compensate for camera shake.

[0191] In some embodiments, calculating the image stabilization path also requires one or more of the following data: motion state data of the electronic device, input image, and camera intrinsic parameters. The input image may include the current input image to be stabilized, as well as multiple consecutive frames of input images preceding it. For example, the image stabilization path can be calculated based on a first stabilization parameter, motion state data of the electronic device, camera intrinsic parameters, and multiple consecutive frames of input images.

[0192] By performing image transformation on the input image according to the image transformation parameters, the jitter of the input image can be compensated, and the blur or offset caused by device jitter can be canceled, thereby obtaining a relatively clear and stable output image.

[0193] The second possible implementation is as follows: Based on the first stabilization parameter, adjust the current second stabilization parameter according to a preset adjustment step size to obtain the third stabilization parameter. Then, perform stabilization processing on the current input image according to the third stabilization parameter to obtain the output image.

[0194] In the second possible implementation, to prevent large changes in the stabilization parameters from causing excessive changes in the field of view (FOV) of the output image between two consecutive frames, thus affecting the user experience, the current stabilization parameters can be gradually adjusted according to a preset adjustment step size to gradually adjust the stabilization parameters from the current stabilization parameters to the first stabilization parameters. For example, the stabilization parameters can be adjusted from the current stabilization parameters to the first stabilization parameters within N frames.

[0195] The second stabilization parameter can be either the initial stabilization parameter or the stabilization parameter corresponding to the previous input frame, i.e., the stabilization parameter used when stabilizing the previous input frame. For example, if the current input image is the first video frame captured during recording, the second stabilization parameter is the initial stabilization parameter; if the current input image is not the first video frame captured during recording, the second stabilization parameter is the stabilization parameter used when stabilizing the previous input image.

[0196] In some embodiments, a third stabilization parameter (i.e., a third stabilization parameter used to perform stabilization processing on the current input image) can be determined based on the current second stabilization parameter and the first stabilization parameter.

[0197] For example, the adjustment direction can be determined based on the first and second stabilization parameters, and the second stabilization parameter can be adjusted according to the adjustment direction and the preset adjustment step size to obtain the third stabilization parameter.

[0198] A6. Display or store the output image as a single video frame after recording.

[0199] Next, we will introduce in detail how to adaptively adjust the range of motion for image stabilization while maintaining a constant FOV size.

[0200] In one embodiment, during the adaptive adjustment of the stabilization range, the FOV of the output image can be kept constant by changing the FOV of the input image.

[0201] To facilitate understanding, the following example will be used to illustrate the process of adaptively adjusting the range of motion and field of view (FOV) of image stabilization, specifically in the scenario where the motion state of an electronic device switches from the second motion state to the first motion state.

[0202] Please refer to Figure 7 , Figure 7 This is a schematic diagram illustrating another adaptive adjustment process for the active range and field of view (FOV) of image stabilization provided in an embodiment of this application. For example... Figure 7 As shown, the implementation of adaptive adjustment of the active range and FOV of image stabilization can include the following steps:

[0203] S701. During the recording process of the camera device, the first video frame captured by the camera device is acquired, and the first video frame is used as the input image for the current image stabilization process.

[0204] S702. Motion detection is performed based on the motion status data of the electronic device.

[0205] S703. If the electronic device is detected to switch from the second motion state to the first motion state, the first stabilization parameter corresponding to the first motion state is determined from the target stabilization parameter group, and the second motion state corresponds to the second stabilization parameter.

[0206] As an example, before starting recording, the target image stabilization parameter set can be determined based on the user's desired effect. Different effect preferences correspond to different image stabilization parameter sets, and the target image stabilization parameter set refers to the image stabilization parameter set corresponding to the user's desired effect. For example, the target parameter set can be as shown in Table 1 above.

[0207] S704. Determine whether the current image stabilization parameters have reached the first image stabilization parameter.

[0208] S705. If not, then adjust the current image stabilization parameters according to the first image stabilization parameter and the preset adjustment step size to obtain the adjusted image stabilization parameters.

[0209] For example, if the current image stabilization parameter is the second image stabilization parameter during the first adjustment, the second image stabilization parameter can be adjusted according to the first image stabilization parameter and the preset adjustment step size to obtain the third image stabilization parameter.

[0210] It should be noted that when the image stabilization parameters include multiple parameters, different parameters correspond to different adjustment step sizes. For each parameter in the current image stabilization parameters, the current parameter can be adjusted according to the corresponding target parameter in the first image stabilization parameters, according to the corresponding adjustment step size, to obtain the adjusted parameter.

[0211] The adjustment step size for different parameters can be preset and set as needed. For example, the adjustment step size for the activity range can be 1% or 2%. Similarly, when the smoothing parameter is the smoothing intensity order, the adjustment step size can be order 1 or order 2, etc.

[0212] For example, referring to Table 1, assuming the second motion state is walking and the first motion state is running, i.e., the electronic device switches from walking to running, the second stabilization parameters corresponding to the second motion state include an activity range of 20% and a smoothing parameter of 3, and the first stabilization parameters corresponding to the first motion state include an activity range of 30% and a smoothing parameter of 4. In this case, for the activity range parameter, the current activity range can be increased by 1% each time until it reaches 30%. For the smoothing parameter of 3, the current smoothing parameter can be adjusted according to the corresponding adjustment step size each time until it is adjusted to the smoothing parameter of 4.

[0213] S706. Calculate the stabilization path based on the adjusted stabilization parameters and motion status data.

[0214] S707. Determine the image transformation parameters of the input image based on the anti-shake path.

[0215] S708. Based on the image transformation parameters, perform image transformation on the input image to obtain the output image.

[0216] The output image is the stabilized frame obtained by stabilizing the first video frame.

[0217] As an example, the image transformation parameter can be a warp parameter, which can be used to process the input image to obtain the output image.

[0218] Next, we will introduce in detail how to adaptively adjust the range of motion for image stabilization while maintaining a constant FOV size.

[0219] Please refer to Figure 8 , Figure 8 This is a schematic diagram illustrating a process for adaptively adjusting the range of motion for image stabilization while maintaining a constant field of view (FOV), as provided in an embodiment of this application, and applied to electronic devices. For example... Figure 8 As shown, the method of adaptively adjusting the range of motion of image stabilization while maintaining a constant FOV size can include the following steps:

[0220] B1. During the recording process of the camera device, the first motion state of the electronic device is detected based on the motion state data.

[0221] B2. Based on the first motion state, determine the first stabilization parameter corresponding to the first motion state from the target stabilization parameter group. The target stabilization parameter group includes stabilization parameters corresponding to various motion states.

[0222] B3. Based on the first stabilization parameter and the current second stabilization parameter, calculate the change in the input image size of the next input image.

[0223] The image size change refers to the change in the input image size of the next input image relative to the current input image, provided that the field of view of the output image remains unchanged. In other words, it is the change in the input image size of the next input image relative to the current input image, provided that the field of view of the output image after stabilizing the next input image according to the first stabilization parameter remains unchanged compared to the field of view of the output image after stabilizing the current input image according to the current second stabilization parameter.

[0224] As an example, the first stabilization parameter includes a first range of motion, and the second stabilization parameter includes a second range of motion. The change in input image size can be calculated based on the first and second ranges of motion. The change in input image size can be the ratio of the image size of the next input image to the image size of the current input image.

[0225] For example, a second image size can be determined based on a first active range, a second active range, and a first image size of the current input image, and then the ratio of the second image size to the first image size can be determined. The second image size must meet the following requirement: the field of view of the output image after image stabilization of the input image with the second image size based on the second active range is the same as the field of view of the output image after image stabilization of the input image with the first image size based on the first active range.

[0226] For example, assume the first active area is 30%, the second active area is 20%, and the first image size is 4000*3000 (width*height). Then the field of view corresponding to the first active area is 70%, and the field of view corresponding to the second active area is 80%. The width of the second image size = (4000*80%) / 70% = 4571. The ratio of the second image size to the first image size = 4571 / 4000.

[0227] B4. Based on the change in the size of the input image, adjust the FOV of the next raw video frame captured by the camera to obtain the input image to be processed.

[0228] The input image to be processed needs to meet the following FOV requirement: the ratio of the field of view of the input image to the field of view of the previous input image is equal to the ratio of the size of the second image to the size of the first image. This ensures that after stabilizing the input image according to the first stabilization parameter, an output image with an unchanged FOV can be obtained, thus maintaining a constant FOV for the output image.

[0229] As an example, the field of view (FOV) of the next raw video frame can be adjusted by changing the magnification. For instance, the magnification of the input image to be processed can be determined based on the change in the size of the input image, and the next raw video frame can be zoomed according to this magnification to obtain an input image with that magnification.

[0230] For example, the magnification of the original video frame is generally 1x. Assuming that the magnification of the current input image is 1.2x, the magnification of the next input image (the input image to be processed) is (4000 / 4571)*1.2x.

[0231] It should be noted that the embodiments of this application are only used as an example of obtaining an input image that meets the FOV requirements by adjusting the FOV of the original video frame captured by the camera. It should be understood that an FOV adjustment request can also be sent to the camera device to request the camera device to capture and output a video frame that meets the FOV requirements when capturing the next video frame, and the video frame that meets the FOV requirements output by the camera device can be used as the input image.

[0232] B5. Apply image stabilization to the input image according to the first stabilization parameter to obtain the output image.

[0233] In some embodiments, image transformation parameters of the input image can be determined first based on the first stabilization parameter. Then, the input image is transformed according to the image transformation parameters to obtain the output image.

[0234] In addition, image stabilization is also related to camera intrinsics. Before performing image stabilization on the input image to be processed according to the first image stabilization parameter, the current first camera intrinsics can be transformed according to the change in the size of the input image to obtain the second camera intrinsics. Then, image stabilization is performed on the input image to be processed according to the first image stabilization parameter and the second camera intrinsics to obtain the output image.

[0235] For example, the ratio between the first image size and the second image size can be multiplied by the first camera intrinsics to obtain the second camera intrinsics. For instance, assuming the ratio between the first image size and the second image size is 4000 / 4571, and the first distance in the first camera intrinsics is 2000, then the second distance in the second camera intrinsics is (4000 / 4571)*2000.

[0236] In another possible implementation, to prevent large changes in the stabilization parameters from affecting the user experience, the current stabilization parameters can be gradually adjusted according to a preset adjustment step size, so as to gradually adjust the stabilization parameters from the current stabilization parameters to the first stabilization parameters. For example, the stabilization parameters can be adjusted from the current stabilization parameters to the first stabilization parameters within N frames.

[0237] In this case, the second stabilization parameter can be adjusted according to the first stabilization parameter and a preset adjustment step size to obtain the third stabilization parameter. Based on the third stabilization parameter and the current second stabilization parameter, the change in the input image size of the next input image is calculated. Based on the change in the input image size, the FOV of the next raw video frame captured by the camera is adjusted to obtain the input image to be processed. Then, according to the third stabilization parameter, the input image to be processed is stabilized to obtain the output image.

[0238] For example, the adjustment direction can be determined based on the first and second stabilization parameters, and the second stabilization parameter can be adjusted according to the adjustment direction and the preset adjustment step size to obtain the third stabilization parameter.

[0239] As an example, the third stabilization parameter includes a third active range, and the second stabilization parameter includes a second active range. The input image size change can be calculated based on the third and second active ranges. Here, the image size change refers to the change in the input image size of the next input image relative to the current input image, while ensuring that the field of view of the output image remains unchanged. The input image size change can be the ratio of the image size of the next input image to that of the current input image.

[0240] For example, the second image size can be determined based on the third active range, the second active range, and the first image size of the current input image, and then the ratio of the second image size to the first image size can be determined. The second image size needs to meet the following requirement: the field of view of the output image after image stabilization of the input image with the second image size based on the third active range is the same as the field of view of the output image after image stabilization of the input image with the first image size based on the second active range.

[0241] For example, assuming the first active range is 30%, the second active range is 20%, the first image size is 4000*3000 (width*height), and the adjustment step size corresponding to the active parameters is 1%, then the adjusted active range (third active range) is 21%. Furthermore, the field of view corresponding to the second active range is 80%, the field of view corresponding to the third active range is 79%, and the width of the second image size = (4000*80%) / 79% = 4050. The ratio of the second image size to the first image size = 4050 / 4000.

[0242] Assuming that after determining the change in the size of the input image, the FOV of the next original video frame is adjusted by adjusting the magnification. Further assuming the magnification of the original video frame is 1x and the magnification of the current input image is 1.2x, then the magnification of the next input image (the input image to be processed) is (4000 / 4050)*1.2x.

[0243] It should be noted that the embodiments of this application are only used as an example of obtaining an input image that meets the FOV requirements by adjusting the FOV of the original video frame captured by the camera. It should be understood that an FOV adjustment request can also be sent to the camera device to request the camera device to capture and output a video frame that meets the FOV requirements when capturing the next video frame, and the video frame that meets the FOV requirements output by the camera device can be used as the input image.

[0244] In addition, image stabilization is also related to camera intrinsics. Before performing image stabilization on the input image to be processed according to the third image stabilization parameter, the current first camera intrinsics can be transformed according to the change in the size of the input image to obtain the second camera intrinsics. Then, image stabilization is performed on the input image to be processed according to the third image stabilization parameter and the second camera intrinsics to obtain the output image.

[0245] For example, the ratio between the first image size and the second image size can be multiplied by the first camera intrinsics to obtain the second camera intrinsics. For instance, assuming the ratio between the first image size and the second image size is 4000 / 4050, and the first distance in the first camera intrinsics is 2000, then the second distance in the second camera intrinsics is (4000 / 4050)*2000.

[0246] B6. Display or store the output image as a single video frame after recording.

[0247] To facilitate understanding, the following example will be used to illustrate the specific process of adaptively adjusting the range of motion of image stabilization while maintaining a constant FOV, taking the scenario of an electronic device switching from a second motion state to a first motion state as an example.

[0248] Please refer to Figure 9 , Figure 9 This is a schematic diagram illustrating another processing step provided in this application embodiment that adaptively adjusts the active range of image stabilization while maintaining a constant FOV size. For example... Figure 9 As shown, the method of adaptively adjusting the range of motion of image stabilization while maintaining a constant FOV size can include the following steps:

[0249] S901. Determine the baseline FOV before recording.

[0250] The baseline FOV is used to indicate the required FOV of the output image after image stabilization. In other words, the FOV of the output image after subsequent image stabilization remains unchanged and is always the baseline FOV.

[0251] In one embodiment, the baseline FOV can be determined after switching to recording mode, such as when switching to recording mode and displaying the recording mode preview interface.

[0252] As an example, a baseline FOV can be determined based on user-defined effect preferences. For instance, a target stabilization parameter set can be determined based on user-defined effect preferences, the current motion state can be detected based on the motion state data of the electronic device, and the baseline FOV can be determined based on the stabilization parameters corresponding to the current motion state.

[0253] As another example, a preset FOV can be used as a baseline FOV. The preset FOV is obtained in advance, for example, through initialization.

[0254] S902. During the recording process, motion detection is performed based on the motion status data of the electronic device.

[0255] S903. If it is detected that the electronic device switches from the second motion state to the first motion state, the first stabilization parameter corresponding to the first motion state is determined from the target stabilization parameter group, and the second motion state corresponds to the second stabilization parameter.

[0256] As an example, before starting recording, the target image stabilization parameter set can be determined based on the user's desired effect. Different effect preferences correspond to different image stabilization parameter sets, and the target image stabilization parameter set refers to the image stabilization parameter set corresponding to the user's desired effect. For example, the target parameter set can be as shown in Table 1 above.

[0257] S904. Determine whether the current image stabilization parameters have reached the first image stabilization parameter.

[0258] S905. If not, then adjust the current image stabilization parameters according to the first image stabilization parameter and the preset adjustment step size to obtain the adjusted image stabilization parameters.

[0259] For example, if the current image stabilization parameter is the second image stabilization parameter during the first adjustment, the second image stabilization parameter can be adjusted according to the first image stabilization parameter and the preset adjustment step size to obtain the third image stabilization parameter.

[0260] It should be noted that when the image stabilization parameters include multiple parameters, different parameters correspond to different adjustment step sizes. For each parameter in the current image stabilization parameters, the current parameter can be adjusted according to the corresponding target parameter in the first image stabilization parameters, according to the corresponding adjustment step size, to obtain the adjusted parameter.

[0261] The adjustment step size for different parameters can be preset and set as needed. For example, the adjustment step size for the activity range can be 1% or 2%. Similarly, when the smoothing parameter is the smoothing intensity order, the adjustment step size can be order 1 or order 2, etc.

[0262] For example, referring to Table 1, assuming the second motion state is walking and the first motion state is running, i.e., the electronic device switches from walking to running, the second stabilization parameters corresponding to the second motion state include an activity range of 20% and a smoothing parameter of 3, and the first stabilization parameters corresponding to the first motion state include an activity range of 30% and a smoothing parameter of 4. In this case, for the activity range parameter, the current activity range can be increased by 1% each time until it reaches 30%. For the smoothing parameter of 3, the current smoothing parameter can be adjusted according to the corresponding adjustment step size each time until it is adjusted to the smoothing parameter of 4.

[0263] S906. Based on the first stabilization parameter and the adjusted stabilization parameter, calculate the change in the input image size of the next input image.

[0264] S907. Based on the change in the size of the input image, adjust the FOV of the next raw video frame captured by the camera to obtain the input image to be processed.

[0265] S908. Based on the change in the size of the input image, transform the current intrinsic parameters of the first camera to obtain the intrinsic parameters of the second camera.

[0266] S909: Calculates the image stabilization path based on the adjusted image stabilization parameters, motion state data, and the intrinsic parameters of the second camera.

[0267] S910. Determine the image transformation parameters of the input image based on the anti-shake path.

[0268] S911. Based on the image transformation parameters, perform image transformation on the input image to obtain the output image.

[0269] The output image is the stabilized frame obtained by stabilizing the first video frame.

[0270] As an example, the image transformation parameter can be a warp parameter, which can be used to process the input image to obtain the output image.

[0271] Next, combined Figure 3 The electronic image stabilization method provided in the embodiments of this application will be described.

[0272] Figure 10 This is a flowchart illustrating an electronic image stabilization method provided in this application. The method is applied in an electronic device, which includes an IMU sensor, an image sensor, an IMU signal processing driver, an image processor driver, a camera hardware abstraction layer, an EIS algorithm module, and a camera application. This application embodiment uses a scenario combining user-defined effect preferences with adaptive adjustment of the range of motion and field of view (FOV) based on motion state as an example. The method includes the following steps:

[0273] S1001: The user opens the camera application and switches to video recording mode.

[0274] S1002: In response to the user's operation, the camera application displays a preview interface for the recording mode, which includes controls for adjusting the effect preference.

[0275] For example, a user can open the camera app by clicking the camera app icon. In response to the user's click, the camera app launches and displays a preview interface of the shooting mode, which includes video recording toggle controls. The user can click the video recording toggle controls, and in response, the camera app switches to video recording mode and displays a preview interface for that mode.

[0276] S1003: The camera application sends a start command to the IMU sensor.

[0277] S1004: The IMU sensor sends the measured IMU data to the camera hardware abstraction layer.

[0278] S1005: The camera hardware abstraction layer sends IMU data to the EIS algorithm module.

[0279] S1006: The user adjusts the effect preference control.

[0280] S1007: In response to the user's adjustment operation, the camera application determines the target effect tendency specified by the adjustment operation.

[0281] S1008: The camera application sends the target effect tendency identifier to the EIS algorithm module via the camera hardware abstraction layer.

[0282] S1009: The EIS algorithm module determines the target image stabilization parameter group corresponding to the target effect tendency from the image stabilization parameter groups corresponding to different effect tendencies. The target image stabilization parameter group includes the image stabilization parameters corresponding to various motion states.

[0283] The stabilization parameters for each motion state include range of motion and smoothing parameters.

[0284] S1010: The EIS algorithm module detects that the electronic device is in motion based on IMU data.

[0285] S1011: The EIS algorithm module determines the stabilization parameter 1 corresponding to motion state 1 from the target stabilization parameter group.

[0286] Among them, image stabilization parameter 1 includes range of motion and smoothing parameter.

[0287] S1012: User starts recording.

[0288] For example, users can start recording by clicking the record button on the preview screen.

[0289] S1013: In response to the user's action, the camera application sends a recording command to the image sensor.

[0290] S1014: The image sensor sends the acquired video frame 1 to the camera hardware abstraction layer.

[0291] S1015: The camera hardware abstraction layer sends video frame 1 to the EIS algorithm module.

[0292] S1016: After receiving video frame 1 from the camera hardware abstraction layer, the EIS algorithm module determines the image transformation parameter 1 of video frame 1 based on the image stabilization parameter 1, IMU data, and video frame 1.

[0293] The EIS algorithm module can determine the image transformation parameter 1 of video frame 1 based on the image stabilization parameter 1, IMU data, video frame 1, and multiple consecutive video frames acquired before video frame 1.

[0294] In addition, the EIS algorithm module can also determine the image transformation parameter 1 of video frame 1 based on data such as image stabilization parameter 1, IMU data, camera intrinsic parameters, video frame 1, and multiple consecutive video frames preceding video frame 1.

[0295] S1017: The EIS algorithm module sends image transformation parameter 1 to the camera hardware abstraction layer.

[0296] S1018: The camera hardware abstraction layer performs image transformation on video frame 1 according to image transformation parameter 1 to obtain the corresponding image stabilization frame 1.

[0297] Among them, the anti-shake frame refers to the video frame after the anti-shake processing.

[0298] S1019: The camera hardware abstraction layer sends the image stabilization frame 1 to the camera application.

[0299] S1020: The camera app displays frame 1 of the image stabilization in the video recording interface.

[0300] Without changing motion, the EIS algorithm module continues to determine the image transformation parameters of the next video frame input by the camera hardware abstraction layer based on the stabilization parameter 1, so as to perform stabilization processing on the next video frame according to the stabilization parameter 1.

[0301] S1021: The image sensor sends the acquired video frame i to the camera hardware abstraction layer.

[0302] S1022: The camera hardware abstraction layer sends video frame i to the EIS algorithm module.

[0303] S1023: The EIS algorithm module detects that the electronic device has switched from motion state 1 to motion state 2 based on IMU data.

[0304] S1024: The EIS algorithm module determines the stabilization parameter 2 corresponding to motion state 2 from the target stabilization parameter group.

[0305] Among them, image stabilization parameter 2 includes range of motion and smoothing parameters.

[0306] S1025: The EIS algorithm module adjusts the current image stabilization parameter 1 according to the preset adjustment step size based on the image stabilization parameter 2, and obtains the image stabilization parameter 3.

[0307] S1026: After receiving video frame i, the EIS algorithm module determines the image transformation parameters i of video frame i based on the anti-shake parameter 3, IMU data and video frame i.

[0308] S1027: The EIS algorithm module sends the image transformation parameter i to the camera hardware abstraction layer.

[0309] S1028: The camera hardware abstraction layer performs image transformation on video frame i based on image transformation parameter i to obtain the corresponding image stabilization frame i.

[0310] S1029: The camera hardware abstraction layer sends the stabilization frame i to the camera application.

[0311] S1030: The camera app displays the stabilized frame in the video recording interface.

[0312] The EIS algorithm module can continue to adjust the current stabilization parameters according to the preset adjustment step size based on stabilization parameter 2. It then determines the image transformation parameters for the next video frame based on the adjusted stabilization parameters, performs image transformation on the next frame, and sends these parameters to the camera hardware abstraction layer. The camera hardware abstraction layer then performs image transformation on the next video frame based on the received parameters, generating a stabilized frame for the next video frame, which is then sent to the camera application for display or storage. This process continues until the current stabilization parameter equals stabilization parameter 2, at which point parameter adjustment stops, allowing subsequent video frames acquired by the image sensor to be stabilized based on stabilization parameter 2.

[0313] In this embodiment, the user can independently select the desired effect on the video preview interface, and the electronic device can determine the target stabilization parameter set based on the user's selected effect. During recording, the electronic device can detect the device's motion state in real time and perform stabilization processing on the recorded video frames based on stabilization parameters such as the range of motion adapted to the real-time motion state in the target stabilization parameter set. This satisfies the user's need for independent adjustment, and the stabilization range and the field of view of the output image can be adaptively adjusted according to the real-time motion state, improving the flexibility and accuracy of electronic stabilization, thereby enhancing the recording effect and user experience.

[0314] Next, combined Figure 3 The electronic image stabilization method provided in the embodiments of this application will be described.

[0315] Figure 11 and Figure 12 This is a flowchart illustrating another electronic image stabilization method provided in this application. The method is applied to an electronic device, which includes an IMU sensor, an image sensor, an IMU signal processing driver, an image processor driver, a camera hardware abstraction layer, an EIS algorithm module, and a camera application. This application embodiment uses a scenario where the user customizes the desired effect and adaptively adjusts the range of motion while maintaining a constant field of view (FOV) as an example.

[0316] Please refer to Figure 11 , Figure 11 The electronic image stabilization method shown includes the following steps S1111-S1123.

[0317] S1101: The user opens the camera app and switches to video recording mode.

[0318] S1102: In response to the user's operation, the camera application displays a preview interface for the recording mode, which includes controls for adjusting the effect preference.

[0319] S1103: The camera application sends a start command to the IMU sensor.

[0320] S1104: The IMU sensor sends the measured IMU data to the camera hardware abstraction layer.

[0321] S1105: The camera hardware abstraction layer sends IMU data to the EIS algorithm module.

[0322] S1106: The user adjusts the effect tendency on the effect tendency adjustment control.

[0323] S1107: In response to the user's adjustment operation, the camera application determines the target effect tendency specified by the adjustment operation.

[0324] S1108: The camera application sends the target effect tendency identifier to the EIS algorithm module via the camera hardware abstraction layer.

[0325] S1109: The EIS algorithm module determines the target image stabilization parameter group corresponding to the target effect tendency from the image stabilization parameter groups corresponding to different effect tendencies. The target image stabilization parameter group includes the image stabilization parameters corresponding to various motion states.

[0326] The stabilization parameters for each motion state include range of motion and smoothing parameters.

[0327] S1110: The EIS algorithm module detects that the electronic device is in motion based on IMU data.

[0328] S1111: The EIS algorithm module determines the stabilization parameter 1 corresponding to motion state 1 from the target stabilization parameter group.

[0329] Among them, image stabilization parameter 1 includes range of motion and smoothing parameter.

[0330] S1112: The EIS algorithm module determines the baseline FOV based on the stabilization parameter 1.

[0331] The baseline FOV is used to indicate the required FOV of the output image after image stabilization. In other words, the FOV of the output image after subsequent image stabilization remains unchanged and is always the baseline FOV.

[0332] It should be noted that the embodiment of this application takes the determination of the reference FOV based on the anti-shake parameters corresponding to the motion state of the device before recording as an example. In other embodiments, the reference FOV may also be a preset FOV, such as the initial FOV after initialization. This embodiment of the application does not limit this.

[0333] S1113: User starts recording.

[0334] S1114: In response to the user's action, the camera application sends a recording command to the image sensor.

[0335] S1115: The image sensor sends the acquired raw video frame 1 to the image processor driver.

[0336] S1116: The image processor driver processes the original video frame 1 to obtain the video frame 1 to be processed.

[0337] The FOV of video frame 1 can be the same as or different from that of the original video frame 1.

[0338] For example, the image processor driver can digitally zoom the original video frame 1 to obtain a video frame 1 with a different FOV than the original video frame 1.

[0339] S1117: The image processor driver sends video frame 1 to the camera hardware abstraction layer.

[0340] S1118: The camera hardware abstraction layer sends video frame 1 to the EIS algorithm module.

[0341] S1119: After receiving video frame 1 from the camera hardware abstraction layer, the EIS algorithm module determines the image transformation parameter 1 of video frame 1 based on the image stabilization parameter 1, IMU data, current camera intrinsic parameters 1, and video frame 1.

[0342] The EIS algorithm module can determine the image transformation parameter 1 of video frame 1 based on the image stabilization parameter 1, IMU data, the current camera intrinsic parameter 1, video frame 1, and multiple consecutive video frames acquired before video frame 1.

[0343] S1120: The EIS algorithm module sends image transformation parameter 1 to the camera hardware abstraction layer.

[0344] S1121: The camera hardware abstraction layer performs image transformation on video frame 1 according to image transformation parameter 1 to obtain the corresponding image stabilization frame 1.

[0345] S1122: The camera hardware abstraction layer sends the stabilized frame 1 to the camera application.

[0346] S1123: Camera app displays frame 1 with image stabilization.

[0347] Without changing motion, the EIS algorithm module can continue to determine the image transformation parameters of the next video frame input by the camera hardware abstraction layer based on the image stabilization parameter 1, and send the image transformation parameters of the next video frame to the camera hardware abstraction layer. The hardware abstraction layer then performs image transformation on the next video frame based on these image transformation parameters, thereby achieving image stabilization processing of the next video frame based on the image stabilization parameter 1.

[0348] It should be noted that each video frame input from the camera hardware abstraction layer to the EIS algorithm module can be processed according to S1115-S1117 described above. For example, the image sensor can send the acquired next video frame (original video frame 2) to the image processor driver. The image processor driver processes the original video frame 2 to obtain the video frame to be processed. The image processor driver sends video frame 2 to the camera hardware abstraction layer, which then sends it to the EIS algorithm module. The EIS algorithm module further determines the image transformation parameters of video frame 2 based on the image stabilization parameter 1 and sends these parameters to the camera hardware abstraction layer for image transformation, thereby completing the image stabilization processing for the next video frame.

[0349] It should also be noted that during the process of performing image stabilization on the video frames driven by the image processor according to the stabilization parameter 1 corresponding to motion state 1, the motion state of the subsequent electronic devices may change. The following will combine... Figure 12 The image stabilization process after changes in motion state is explained in detail.

[0350] Please refer to Figure 12 , Figure 12 The electronic image stabilization method shown includes the following steps S1124-S1145.

[0351] S1124: After receiving video frame i from the camera hardware abstraction layer, the EIS algorithm module detects that the electronic device has switched from motion state 1 to motion state 2 based on the IMU data.

[0352] Here, video frame i can be any video frame that the camera hardware abstraction layer inputs to the EIS algorithm module after video frame 1 in motion state 1.

[0353] S1125: The EIS algorithm module determines the image transformation parameters i of video frame i based on the current image stabilization parameters i, the current camera intrinsic parameters i, and the IMU data and video frame i.

[0354] S1126: The EIS algorithm module sends the image transformation parameter i to the camera hardware abstraction layer.

[0355] S1127: The camera hardware abstraction layer performs image transformation on video frame i according to the image transformation parameter i to obtain the corresponding image stabilization frame 1.

[0356] S1128: The camera hardware abstraction layer sends the stabilization frame i to the camera application.

[0357] S1129: The camera application displays the stabilized frame in the video recording interface.

[0358] Upon receiving video frame i from the camera hardware abstraction layer, if the electronic device detects a switch from motion state 1 to motion state 2, the EIS algorithm module can, on one hand, continue to determine the image transformation parameters of video frame i based on the current stabilization parameter 1, and continue to perform stabilization processing on video frame i based on the current stabilization parameter 1. On the other hand, the EIS algorithm module can determine the stabilization parameter 2 corresponding to motion state 2, and adjust the FOV of the next video frame based on the stabilization parameter 2. By adjusting the FOV of the next input frame, the activity range can be adaptively adjusted while maintaining a constant FOV.

[0359] S1130: The EIS algorithm module determines the stabilization parameter 2 corresponding to motion state 2 from the target stabilization parameter group.

[0360] S1131: The EIS algorithm module adjusts the current image stabilization parameters according to the preset adjustment step size based on the image stabilization parameter 2, and obtains the image stabilization parameter 3.

[0361] It should be noted that during the first adjustment, the current image stabilization parameter was set to 1.

[0362] S1132: The EIS algorithm module determines the change in image size of the video frame i+1 to be input based on the anti-shake parameter 3, the anti-shake parameter 1, and the image size of video frame i.

[0363] S1133: The EIS algorithm module generates a FOV change request based on the change in image size.

[0364] The FOV adjustment request is used to request the output of video frame i+1 that meets the FOV requirement. The FOV requirement means that the change in the field of view of the input video frame i+1 and the current video frame i corresponds to the change in image size, so that the field of view of the output stabilized frame after stabilizing the input video frame i+1 according to stabilization parameter 3 is the same as the field of view of the output stabilized frame after stabilizing the current video frame i according to stabilization parameter 1.

[0365] In this way, it can be ensured that the FOV of the output stabilized frame remains unchanged after the next video frame (video frame 3) is stabilized according to the adjusted stabilization parameter 3. This allows for adaptive adjustment of the range of motion according to the motion state while maintaining a constant FOV of the output image.

[0366] As an example, an FOV change request may include FOV change amount information or target FOV information. FOV change amount information indicates the amount of change between the FOV of the input video frame i+1 and the FOV of video frame i, for example, it could be the ratio of the FOV of the input video frame i+1 to the FOV of video frame i. Target FOV information indicates the FOV of the input video frame i+1.

[0367] As another example, an FOV change request can also be used to instruct an adjustment of the magnification, thereby adjusting the FOV. For instance, an FOV change request may include magnification change amount information or target magnification information. The magnification change amount indicates the amount of change between the magnification of the input video frame i+1 and that of video frame i, such as the ratio of the magnification of the input video frame i+1 to that of video frame i. The target magnification indicates the magnification of the input video frame i+1.

[0368] S1134: The EIS algorithm module sends a FOV change request to the camera hardware abstraction layer.

[0369] It should be noted that the embodiments of this application are only illustrated by the example of the EIS algorithm module generating an FOV change request based on the image size change and sending the FOV change request to the camera hardware abstraction layer. It should be understood that the EIS algorithm module can also send the image size change to the camera hardware abstraction layer, and the camera hardware abstraction layer can generate an FOV change request based on the image size change. The embodiments of this application do not limit this.

[0370] S1135: The EIS algorithm module transforms the current camera intrinsic parameter 1 based on the change in image size to obtain camera intrinsic parameter 2.

[0371] S1136: The camera hardware abstraction layer sends the FOV change request to the image processor driver.

[0372] S1137: The image sensor will acquire raw video frame i+1 and send it to the image processor driver.

[0373] S1138: The image processor driver processes the original video frame i+1 according to the FOV change request to obtain a video frame i+1 that meets the FOV requirements.

[0374] S1139: The image processor driver sends video frame i+1 to the camera hardware abstraction layer.

[0375] S1140: The camera hardware abstraction layer sends video frame i+1 to the EIS algorithm module.

[0376] S1141: The EIS algorithm module determines the image transformation parameter i+1 of video frame i+1 based on the image stabilization parameter 3, IMU data, camera intrinsic parameter 2, and video frame i+1.

[0377] S1142: The EIS algorithm module sends the image transformation parameter i+1 to the camera hardware abstraction layer.

[0378] S1143: The camera hardware abstraction layer performs image transformation on video frame i+1 based on image transformation parameter i+1 to obtain the corresponding image stabilization frame i+1.

[0379] S1144: The camera hardware abstraction layer sends the stabilization frame i+1 to the camera application.

[0380] S1145: The camera app displays the stabilization frame i+1.

[0381] After that, S1131-S1145 can be executed repeatedly to adjust the current stabilization parameters according to the preset adjustment step size, and to perform stabilization processing on the next video frame according to the adjusted stabilization parameters, until the current stabilization parameters are equal to stabilization parameter 2, at which point the adjustment of the stabilization parameters is stopped, and subsequent video frames are stabilized according to stabilization parameter 2.

[0382] In this embodiment, the user can adjust the desired effect on the video preview interface as needed, and the electronic device can determine the target stabilization parameter set based on the user's adjusted effect. During recording, the electronic device can detect the device's motion state in real time. Then, based on the stabilization parameters such as the range of motion adapted to the real-time motion state in the target stabilization parameter set, it first adjusts the FOV of the next input frame in reverse, and then performs stabilization processing on the next input frame based on the stabilization parameters such as the range of motion adapted to the real-time motion state, thereby ensuring that the FOV of the output frame remains constant. In this way, the user's self-adjustment needs can be met, and a constant FOV can be maintained while adaptively adjusting the range of motion stabilization. This avoids the impact of changes in the FOV of the output video frame on the user experience, meets the needs of diverse shooting scenarios, and further improves the recording effect and user experience.

[0383] This application also provides a chip coupled to a memory, which is used to read and execute computer programs or instructions stored in the memory to perform the methods in the above embodiments.

[0384] This application also provides an electronic device including a chip for reading and executing computer programs or instructions stored in a memory, causing the methods in the various embodiments to be performed.

[0385] This embodiment also provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed on an electronic device, the electronic device performs the aforementioned method steps to implement the electronic image stabilization method in the above embodiment.

[0386] This embodiment also provides a computer program product, which is a computer-readable storage medium storing program code. When the computer program product is run on a computer, it causes the computer to perform the above-mentioned related steps to implement the electronic image stabilization method in the above embodiment.

[0387] In addition, embodiments of this application also provide an apparatus, which may specifically be a chip, component or module. The apparatus may include a connected processor and a memory. The memory is used to store computer execution instructions. When the apparatus is running, the processor can execute the computer execution instructions stored in the memory to cause the chip to execute the electronic image stabilization methods in the above-described method embodiments.

[0388] In this embodiment, the electronic device, computer-readable storage medium, computer program product or chip are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects of the corresponding methods provided above, and will not be repeated here.

[0389] This application does not specifically limit the structure of the execution subject of the method provided in this application embodiment. As long as a program containing the code of the method provided in this application embodiment can be run to perform video processing according to the method provided in this application embodiment, it is acceptable. For example, the execution subject of the method provided in this application embodiment can be an electronic device, or a functional module in an electronic device that can call and execute a program.

[0390] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0391] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0392] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0393] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium and includes several instructions that cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0394] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An electronic image stabilization method, characterized in that, Applied to an electronic device, including a camera device, the method includes: During the recording process, the first motion state of the electronic device is detected based on the motion state data of the electronic device; Based on the first motion state, a first stabilization parameter corresponding to the first motion state is determined from the target stabilization parameter group. The target stabilization parameter group includes stabilization parameters corresponding to various motion states. The stabilization parameter includes a range of motion, which is used to indicate the difference in field of view of the image before and after stabilization processing. Based on the first image stabilization parameter, the first video frame captured by the camera device is subjected to image stabilization processing to obtain the second video frame; The second video frame is displayed or stored.

2. The method as described in claim 1, characterized in that, Before detecting the first motion state of the electronic device based on its motion state data during recording, the method further includes: The preview interface for the video recording mode includes an effect tendency adjustment control. The effect tendency adjustment control is used to adjust the effect tendency. The effect tendency is used to indicate the tendency of the image stabilization effect between stability and field of view. Different effect tendencies correspond to different image stabilization parameter groups. Each image stabilization parameter group includes image stabilization parameters corresponding to the various motion states. In response to an adjustment operation on the effect tendency adjustment control, the target effect tendency specified by the adjustment operation is determined; From the different effect tendencies corresponding to their respective image stabilization parameter groups, determine the target image stabilization parameter group corresponding to the target effect tendency.

3. The method as described in claim 1 or 2, characterized in that, The image stabilization parameters also include smoothing parameters, which are used to smooth the shake path, and the shake path is used to indicate the image shake trajectory during the shooting process.

4. The method according to any one of claims 1-3, characterized in that, The step of performing image stabilization processing on the first video frame captured by the camera device according to the first image stabilization parameter to obtain the second video frame includes: If the first stabilization parameter is different from the current second stabilization parameter, the second stabilization parameter is adjusted according to the first stabilization parameter to obtain the third stabilization parameter; wherein, the second stabilization parameter is the initial stabilization parameter or the stabilization parameter used when performing stabilization processing on the previous video frame of the first video frame; Based on the third stabilization parameter, the first video frame is subjected to stabilization processing to obtain the second video frame.

5. The method as described in claim 4, characterized in that, The step of adjusting the second image stabilization parameter based on the first image stabilization parameter to obtain the third image stabilization parameter includes: The adjustment direction is determined based on the first stabilization parameter and the second stabilization parameter; The second stabilization parameter is adjusted according to the adjustment direction and preset adjustment step size to obtain the third stabilization parameter.

6. The method as described in claim 4, characterized in that, The step of performing image stabilization processing on the first video frame according to the third image stabilization parameter to obtain the second video frame includes: Based on the third anti-shake parameter, determine the image transformation parameters of the first video frame; The first video frame is transformed according to the image transformation parameters to obtain the second video frame.

7. The method as described in claim 4, characterized in that, The second image stabilization parameter includes a second range of motion, and the third image stabilization parameter includes a third range of motion; Before performing image stabilization processing on the first video frame according to the third image stabilization parameter, the method further includes: The second image size is determined based on the second active range, the third active range, and the first image size of the previous video frame; wherein, the first field of view of the output image after the input image of the second image size is stabilized based on the third active range is the same as the second field of view, and the second field of view refers to the field of view of the output image after the input image of the first image size is stabilized based on the second active range; The first video frame is obtained based on the second image size and the first image size, wherein the image size of the first video frame is the first image size, and the ratio of the field of view of the first video frame to the field of view of the previous video frame is equal to the ratio of the second image size to the first image size.

8. The method as described in claim 7, characterized in that, The step of obtaining the first video frame based on the second image size and the first image size includes: Acquire the original video frames captured by the camera device; Based on the second image size and the first image size, the original video frame is processed to obtain the first video frame.

9. The method as described in claim 7, characterized in that, The step of obtaining the first video frame based on the second image size and the first image size includes: Based on the second image size and the first image size, the camera device is controlled to capture and output a video frame with an image size equal to the first image size and a field of view ratio equal to the ratio of the second image size to the first image size. The first video frame captured and output by the camera device is acquired.

10. The method according to any one of claims 7-9, characterized in that, Before performing image stabilization processing on the first video frame according to the third image stabilization parameter, the method further includes: The second camera intrinsics are determined based on the second image size, the first image size, and the current first camera intrinsics of the camera device. The step of performing image stabilization processing on the first video frame according to the third image stabilization parameter to obtain the second video frame includes: Based on the third stabilization parameter and the second camera intrinsic parameter, the first video frame is subjected to stabilization processing to obtain the second video frame.

11. The method according to any one of claims 1-10, characterized in that, After performing image stabilization processing on the first video frame captured by the camera device according to the first image stabilization parameter, the method further includes: When the electronic device is detected to switch from the first motion state to the second motion state based on the motion state data of the electronic device, a fourth stabilization parameter corresponding to the second motion state is determined from the target stabilization parameter group; Based on the fourth stabilization parameter, the third video frame captured by the camera device is subjected to stabilization processing to obtain the fourth video frame, wherein the third video frame is the next video frame after the first video frame. The fourth video frame is displayed or stored.

12. The method as described in claim 11, characterized in that, The step of performing image stabilization processing on the third video frame captured by the camera device according to the fourth image stabilization parameter includes: Based on the fourth stabilization parameter, the first stabilization parameter is adjusted to obtain the fifth stabilization parameter; Based on the fifth stabilization parameter, the third video frame is subjected to stabilization processing to obtain the fourth video frame.

13. The method as described in claim 12, characterized in that, The step of adjusting the first image stabilization parameter according to the fourth image stabilization parameter to obtain the fifth image stabilization parameter includes: The adjustment direction is determined based on the first stabilization parameter and the fourth stabilization parameter; The first stabilization parameter is adjusted according to the adjustment direction and preset adjustment step size to obtain the fifth stabilization parameter.

14. An electronic image stabilization method, characterized in that, Applied to an electronic device, including a camera device, the method includes: The preview interface for the video recording mode includes an effect tendency adjustment control. The effect tendency adjustment control is used to adjust the effect tendency. The effect tendency is used to indicate the tendency of the image stabilization effect between stability and field of view. Different effect tendencies correspond to different image stabilization parameters. The image stabilization parameters include a range of motion. The range of motion is used to indicate the difference in field of view of the image before and after image stabilization. In response to an adjustment operation on the effect tendency adjustment control, the target effect tendency specified by the adjustment operation is determined; In response to the recording start operation, the first video frame captured by the camera device is subjected to image stabilization processing according to the target image stabilization parameters corresponding to the target effect tendency, so as to obtain the second video frame; The second video frame is displayed.

15. The method as described in claim 14, characterized in that, The anti-shake parameters corresponding to the target effect tend to include anti-shake parameters corresponding to various motion states; Before performing image stabilization processing on the first video frame captured by the camera device according to the target image stabilization parameters corresponding to the target effect tendency, the method further includes: Based on the motion state data of the electronic device, the current first motion state of the electronic device is determined; The step of performing image stabilization processing on the first video frame captured by the camera device according to the target image stabilization parameters corresponding to the target effect tendency includes: From the anti-shake parameters corresponding to each of the various motion states, determine the first anti-shake parameter corresponding to the first motion state; Based on the first stabilization parameters, the first video frame is subjected to stabilization processing to obtain the second video frame.

16. The method as described in claim 14 or 15, characterized in that, The image stabilization parameters also include smoothing parameters, which are used to smooth the shake path, and the shake path is used to indicate the image shake trajectory during the shooting process.

17. The method according to any one of claims 14-16, characterized in that, The effect adjustment controls are slider controls, progress bar controls, scroll bar controls, adjustment knob controls, or adjustment arrow controls.

18. An electronic device, characterized in that, The electronic device includes: one or more processors, and a memory; the memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, and the one or more processors invoking the computer instructions to cause the electronic device to perform the method as claimed in any one of claims 1 to 13 or claims 14 to 17.

19. A computer program product containing instructions, characterized in that, When it is run on a computer, it causes the computer to perform the method as claimed in any one of claims 1 to 13 or claims 14 to 17.

20. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes instructions that, when executed on an electronic device, cause the electronic device to perform the method as claimed in any one of claims 1 to 13 or claims 14 to 17.

Images