An image processing method, a terminal device, and a computer-readable storage medium

Abstract

This application provides an image processing method, a terminal device, and a computer-readable storage medium, relating to the field of image processing technology. The method is applied to a terminal device including a first camera and a second camera. The method includes: responding to a first operation by a user opening a camera application, displaying a preview interface of the camera application, the preview interface including shooting controls, and the camera application having enabled background blur and image stabilization functions; responding to a second operation by the user on the shooting controls, storing a target image; wherein the target image is an image obtained by background blur processing of a first image based on depth information of the first image, the depth information being based on first calibration data, the first image and the second image being an image captured by the first camera at a first moment, the second image being an image captured by the second camera at the first moment, and the first calibration data being used to indicate the pose relationship between the first camera and the second camera at the first moment.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image processing method, a terminal device, and a computer-readable storage medium. Background Technology

[0002] With the development of technology, portable smart devices such as smartphones and tablets are being used more and more in people's daily lives. For example, people are more accustomed to using portable smart devices to take pictures and record wonderful moments in life.

[0003] Currently, the background blur and image stabilization functions of smart devices cannot be enabled simultaneously. Otherwise, the image stabilization process may cause lens movement, making it impossible to accurately calculate depth information and thus affecting the blurring effect. Summary of the Invention

[0004] In view of this, this application provides an image processing method, a terminal device, and a computer-readable storage medium that can achieve image stabilization while ensuring image blurring in user shooting scenarios.

[0005] In a first aspect, this application provides an image processing method applied to a terminal device including a first camera and a second camera. The method includes: in response to a first operation by a user opening a camera application, displaying a preview interface of the camera application, the preview interface including shooting controls, and the camera application having enabled background blur and image stabilization functions; in response to a second operation by the user on the shooting controls, storing a target image; wherein the target image is an image obtained by performing background blur processing on a first image based on depth information of a first image, the depth information being determined based on first calibration data, the first image and the second image being an image captured by the first camera at a first moment, the second image being an image captured by the second camera at the first moment, and the first calibration data being used to indicate the pose relationship between the first camera and the second camera at the first moment.

[0006] The second operation of the user on the shooting control is the user pressing the shutter button.

[0007] Based on the above process, with the background blur function and image stabilization function enabled, the terminal device responds to the user's shutter operation by determining the target image based on the image stabilization data, the first image, and the second image. This ensures that the terminal device can perform image stabilization when shooting the target image, and determines the depth information used for blurring based on the image stabilization data, thereby ensuring the blurring effect of the target image.

[0008] In one possible implementation of the first aspect, the method further includes: acquiring first image stabilization data corresponding to a first moment, the first image stabilization data being the movement parameters of the lens of the moving target camera at the first moment, the target camera including a first camera and / or a second camera; determining first calibration data based on the first image stabilization data and second calibration data; the second calibration data being used to indicate the initial pose relationship between the first camera and the second camera.

[0009] Since the first image stabilization data is the movement parameter of the lens of the target camera at the first moment, it means that the lens of the target camera has moved. Therefore, based on the second calibration data used to indicate the initial pose relationship between the first camera and the second camera, and the first image stabilization data, the pose relationship between the first camera and the second camera after the lens of the target camera moves at the first moment can be accurately calculated. In this way, the accuracy of the first calibration data can be guaranteed, thereby improving the accuracy of depth information calculation based on the first calibration data, and thus improving the bokeh effect.

[0010] In one possible implementation of the first aspect, obtaining the first image stabilization data at the first moment includes:

[0011] The target dual-camera image is determined. The target dual-camera image is the dual-camera image captured by the terminal device at the first moment. The target dual-camera image includes the first image and the second image.

[0012] Based on the timestamp of the target dual-camera image, the first image stabilization data is obtained; the time difference between the timestamp of the first image stabilization data and the timestamp of the target dual-camera image is within a preset time range.

[0013] The first image stabilization data represents the direction and distance of lens movement in the camera of the control terminal device. Since the movement control process is continuous, the image stabilization data will not experience abrupt changes within a certain time period. Therefore, if the time difference between the timestamp of the first image stabilization data and the timestamp of the target dual-camera image is within a preset time range, and the change in the image stabilization data is small within this preset time range, then the time difference between the timestamp of the first image stabilization data and the timestamp of the target dual-camera image is acceptable. In this case, the image stabilization data acquired at the timestamp of the target dual-camera image can be considered the first image stabilization data.

[0014] Furthermore, the terminal device can determine the blurred image based on the target dual-camera image. Therefore, it can accurately determine the first image stabilization data based on the timestamp of the target dual-camera image, ensuring the correlation between the first image stabilization data and the target dual-camera image. That is, the time difference between the timestamp of the first image stabilization data and the timestamp of the target dual-camera image is within a preset time range.

[0015] In the above implementation process, after the target dual-camera image is determined, the first image stabilization data can be accurately determined based on the timestamp of the target dual-camera image to ensure the accuracy of the first image stabilization data, thereby ensuring the accuracy of the first calibration data and improving the bokeh effect.

[0016] In one possible implementation of the first aspect, determining the target dual-camera image includes: acquiring multiple sets of dual-camera images, each set of dual-camera images including an image captured by a first camera and an image captured by a second camera at the same time, wherein the time difference between the timestamp corresponding to each set of dual-camera images and the first time is within a preset time range; and determining the target dual-camera image from the multiple sets of dual-camera images.

[0017] In scenarios involving snapshots or regular shooting, the quality of the dual-camera images acquired at the first moment may be poor due to camera shake or misalignment with the subject. Therefore, when determining the target dual-camera image, multiple sets of dual-camera images with timestamps within a preset time range from the first moment can be acquired first. Then, the target dual-camera image that meets the requirements can be determined from these multiple sets of dual-camera images.

[0018] The target dual-camera image that meets the requirements can be a dual-camera image with a resolution greater than a preset resolution, or a dual-camera image with a similarity between the first image and the second image greater than a preset similarity, etc.

[0019] "Same moment" can mean that the moment when the first camera captures the image is exactly the same as the moment when the second camera captures the image, or it can mean that the difference between the moment when the first camera captures the image and the moment when the second camera captures the image is within a preset range.

[0020] In this case, if the difference between the time when the first camera captures the image and the time when the second camera captures the image is within a preset range, it can be indicated that although the time when the first camera captures the image and the time when the second camera captures the image are not exactly the same, the difference is small. At this time, the similarity between the first camera captures the image and the second camera captures the image is high, so they can be considered to be at the same time.

[0021] In the above implementation process, it can be ensured that a target dual-camera image is selected from dual-camera images with timestamps closer to the first moment, so as to ensure image quality.

[0022] In one possible implementation of the first aspect, obtaining first image stabilization data based on the timestamp of the target dual-camera image includes: obtaining second image stabilization data corresponding to each of the multiple sets of dual-camera images; and determining the first image stabilization data based on the timestamp of the target dual-camera image and the multiple sets of second image stabilization data.

[0023] After determining multiple sets of dual-camera images, multiple second-level image stabilization data can be determined. To ensure the accuracy of the final first-level image stabilization data, the first-level image stabilization data can be further determined by filtering and interpolation calculations based on the timestamp of the target dual-camera image and multiple second-level image stabilization data, thereby improving the bokeh effect based on dual-camera images while maintaining image stabilization.

[0024] In one possible implementation of the first aspect, determining the first image stabilization data based on the timestamp of the target dual-camera image and multiple second image stabilization data includes: determining the third image stabilization data corresponding to the timestamp of the target dual-camera image from the multiple second image stabilization data based on the timestamp of the target dual-camera image; and, if there are multiple third image stabilization data, performing interpolation processing on the multiple third image stabilization data to obtain the first image stabilization data.

[0025] Since image stabilization data may not exist at the timestamp of the target dual-camera image, the terminal device can first determine the third image stabilization data corresponding to the timestamp of the target dual-camera image from multiple second image stabilization data sets. There is a time difference between the timestamp of the third image stabilization data and the timestamp of the target dual-camera image, and this time difference is within a preset range. Then, the terminal device performs interpolation processing on the third image stabilization data to obtain the image stabilization data for the timestamp of the target dual-camera image. This ensures accurate calculation of depth information based on the image stabilization data, thereby improving the bokeh effect of the dual-camera image.

[0026] In one possible implementation of the first aspect, the multiple sets of dual-camera images are valid dual-camera images among dual-camera images whose time difference between the corresponding timestamp and the first moment is within a preset time range; the multiple second image stabilization data are valid image stabilization data among the image stabilization data corresponding to each set of dual-camera images in the multiple sets of dual-camera images.

[0027] A valid dual-camera image can be a dual-camera image with a resolution greater than a preset resolution, or a dual-camera image where the similarity between the first and second images is greater than a preset similarity. It should be noted that the process for determining a valid dual-camera image is similar to, but not entirely the same as, the process for determining a target dual-camera image that meets the above conditions. The preset resolution and preset similarity involved in the determination process can be the same or different, and the specific values ​​are configured according to the actual application.

[0028] Effective image stabilization data can be image stabilization data with stabilization parameters lower than the preset stabilization parameters.

[0029] By determining the effectiveness of multiple sets of dual-camera images and multiple secondary image stabilization data using the above methods, the quality of the target image can be guaranteed.

[0030] In one possible implementation of the first aspect, the terminal device includes a memory, the memory including at least two storage areas; acquiring second image stabilization data corresponding to each of the multiple sets of dual-camera images includes: reading the second image stabilization data from a first storage area of ​​the memory; wherein, when reading the image stabilization data from the first storage area, the terminal device stores the image stabilization data to be stored in the second storage area, the first storage area being any one of the at least two storage areas, and the second storage area being a storage area other than the first storage area among the at least two storage areas.

[0031] The terminal device can read the second image stabilization data from the first storage area. While reading the second image stabilization data, the terminal device can continue to store other image stabilization data in the second storage area. This avoids the problem of missing image stabilization data due to insufficient storage space or the terminal device's reading process, ensuring accurate acquisition of image stabilization data and thus improving the bokeh effect.

[0032] In a second aspect, this application provides a terminal device, the terminal device including a display screen, a memory, and one or more processors; the display screen, the memory, and the processor are coupled; the display screen is used to display an image generated by the processor, the memory is used to store computer program code, the computer program code including computer instructions; when the processor executes the computer instructions, it causes the terminal device to perform the method as described in the first aspect and any possible design of the method.

[0033] Thirdly, this application provides a computer-readable storage medium including computer instructions that, when executed on a terminal device, cause the terminal device to perform the method described in the first aspect above and any possible design of the method thereof.

[0034] Fourthly, this application provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the method described in the first aspect above and any possible design of the method thereof.

[0035] Fifthly, this application provides an apparatus included in a terminal device, which has the function of implementing the terminal device behavior in any of the above aspects and possible implementations. This function can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes at least one module or unit corresponding to the above function. For example, an allocation module or unit, a scanning module or unit, a recycling module or unit, a moving module or unit, and a storage module or unit, etc.

[0036] Sixthly, embodiments of this application provide a chip system including a processor and potentially a memory, for implementing any of the methods provided in the first aspect above. The chip system may be composed of chips or may include chips and other discrete devices.

[0037] Understandably, the terminal device described in the second aspect and any possible design of the above-mentioned device, the computer-readable storage medium described in the third aspect, and the computer program product described in the fourth aspect are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here. Attached Figure Description

[0038] Figure 1 A schematic diagram illustrating a process of taking a blurred photo, provided as an embodiment of this application;

[0039] Figure 2 A schematic diagram of a blurred photograph provided for an embodiment of this application;

[0040] Figure 3 A schematic diagram illustrating a photo stabilization shooting process provided in an embodiment of this application;

[0041] Figure 4 A schematic diagram illustrating an image processing procedure provided in an embodiment of this application;

[0042] Figure 5 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application;

[0043] Figure 6 A software structure block diagram of a terminal device provided in an embodiment of this application;

[0044] Figure 7 A flowchart illustrating an image processing method provided in this application embodiment. Figure 1 ;

[0045] Figure 8 A flowchart illustrating an image processing method provided in this application embodiment. Figure 2 ;

[0046] Figure 9 A schematic diagram illustrating a dual-camera image storage process provided in an embodiment of this application;

[0047] Figure 10 A schematic diagram illustrating a shake-stabilized data storage process provided in an embodiment of this application;

[0048] Figure 11 This is a schematic diagram of a dual-camera image and image stabilization data provided in an embodiment of this application;

[0049] Figure 12 This is a schematic diagram illustrating the correspondence between dual-camera images and image stabilization data, provided in an embodiment of this application.

[0050] Figure 13 A schematic diagram of a data smoothing filter for anti-shake purposes provided in an embodiment of this application;

[0051] Figure 14 This is a schematic diagram of a stabilization data interpolation method provided in an embodiment of this application;

[0052] Figure 15 This is a schematic diagram of depth calculation provided in an embodiment of this application. Detailed Implementation

[0053] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this embodiment, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships may exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0054] It should be noted that, in the embodiments of this application, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design scheme described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0055] Before introducing the embodiments of this application, the technologies involved in the embodiments of this application will be described in detail.

[0056] 1. Image stabilization technology is a technique designed to address the issue of blurry photos taken when a user holds a handheld device and the hand shakes.

[0057] In a typical shooting scenario, before the shutter button is pressed, the light reflected from the subject passes through the lens in the camera and forms a clear image on the plane of the light sensor. If the shutter button is pressed at this moment, and the device remains steady, a clear photo can be taken. However, if the device shakes due to the user's hand movement when the shutter button is pressed, the lens in the camera will shift. This change in the position between the lens and the subject will cause the light reflected from the subject to form a blurry image on the plane of the light sensor, resulting in a blurry photo.

[0058] To address the aforementioned issues, a related technology proposes an optical image stabilizer (OIS). OIS monitors the displacement information caused by camera shake. The device then calculates the amount of displacement that needs to be compensated based on this displacement information. Subsequently, the device moves the lens in the camera according to the required compensation amount to counteract the displacement caused by the shake, thereby preventing blurry photos.

[0059] For example, in a Cartesian coordinate system with the x-axis as the horizontal axis and the y-axis as the vertical axis, if the center of the lens was originally at (0, 0), and the OIS detects a camera shake, with the displacement information indicating that the camera moved 1 unit in the positive x-axis direction, it can be understood that if the camera moves 1 unit in the positive x-axis direction, then the center of the lens will be at (1, 0). Therefore, the terminal device can determine, based on the displacement information of the camera shake, that the lens needs to be moved 1 unit in the opposite x-axis direction from its current position after the shake. The direction of movement can be represented by uppercase letters and symbols, for example, X+ represents the positive x-axis direction, Y- represents the opposite y-axis direction, and the number represents the length of movement. Based on the OIS detection results, the terminal device can record the displacement information as X+1, indicating that when the lens shakes, it moves 1 unit in the positive x-axis direction. The terminal device then calculates the required compensation displacement as X-1, meaning that the OIS can control the center position of the lens in the camera to move 1 unit in the opposite x-axis direction from (1, 0), returning the lens's center position to (0, 0). This avoids the problem of blurry photos caused by camera shake.

[0060] In the above example, the displacement information of the shake is used to represent the amount of movement of the lens in the terminal device or the terminal device camera, which includes the direction of movement and the distance of movement. The amount of displacement to be compensated calculated by the terminal device is used to represent the amount of movement of the lens in the terminal device camera that needs to be controlled. For ease of description, this embodiment of the application refers to it as image stabilization data or OIS data. The image stabilization data also includes the direction of movement and the distance of movement accordingly.

[0061] In subsequent embodiments of this application, the direction of movement and the distance of movement can be the movement parameters in this application.

[0062] Understandably, the direction of movement in the jitter displacement information is opposite to the direction of movement in the anti-shake data, and the distance of movement in the jitter displacement information is equal to the distance of movement in the anti-shake data.

[0063] 2. Blur effect is a photography technique that produces photos where the foreground is sharp while the background is blurred.

[0064] Generally, photographers adjust the shooting parameters of the lens to change the path of light in the lens, making the shape or outline of the background other than the subject blurry or unclear, thus creating a bokeh effect.

[0065] Portable devices, due to cost or technological limitations, cannot be equipped with lenses that allow for adjustable shooting parameters, thus preventing the use of the aforementioned methods to capture bokeh photos. To meet users' needs for bokeh photos, algorithmic bokeh techniques are currently commonly used, employing image processing technology to achieve the bokeh effect. Specifically, the device first captures a non-blurred image, then uses algorithms to process it, ensuring the subject remains sharp while blurring the background, thereby achieving a bokeh effect.

[0066] When using algorithms to blur backgrounds in photography, the device first needs to determine the areas within the image that need to remain sharp and the areas that need to be blurred. This achieves the blurring effect. The device can determine the foreground and background—that is, the areas that need to be sharp and the areas that need to be blurred—by measuring the distance between the subject and the camera within these areas. In other words, the device needs to utilize the image's depth information to blur the image.

[0067] Currently, the distance between the object being photographed and the camera can be calculated using the principle of human eye triangulation. Therefore, a dual-camera algorithm is proposed to achieve bokeh (bokeh effect). For example... Figure 1As shown, in the process of capturing blurred photos using a dual-camera algorithm, the terminal device simultaneously acquires images using both cameras in the dual-camera module. Since the main and auxiliary cameras are positioned differently, when the same object is photographed from both positions, the object's position in the main camera's field of view differs from its position in the auxiliary camera's field of view. An angle exists between the lines connecting the main camera and the object, and between the auxiliary camera and the object; this angle is called the parallax angle. Therefore, the object, the main camera, and the auxiliary camera can form a triangle. Furthermore, given the positions of the main and auxiliary cameras, the positions of their internal lenses, and optical parameters such as focal length, the distance between the object and the cameras can be calculated using the principle of triangle similarity. Specific calculation formulas can be found in related technical documents and will not be elaborated here.

[0068] In other words, given a fixed pose relationship between the two cameras in a dual-camera module, the terminal device can obtain the distance between the object and the camera corresponding to each pixel in each image based on the two images captured by the dual-camera module. This distance is also known as the depth information of the image.

[0069] Generally, terminal devices blur the main image captured by the main camera based on depth information to obtain a blurred photo. Specifically, the terminal device can use the depth information of the image to determine the main subject that needs to be highlighted and the background that needs to be blurred, and then blur the background to obtain a blurred photo. Figure 2 The image shown is a schematic diagram of a blurred photograph. In this blurred photograph, the main part of region 1 is clear, while the part in region 2 is blurred.

[0070] Currently, when users take photos with portable devices, there is still a problem of unclear images due to camera shake. For example, if a user's hand shakes while taking a picture, causing the device to move, the image sensor may receive different light sources at the same location. The superposition of these different light sources at the same location results in a blurry image. To address this issue, OIS (Optical Image Stabilizer) technology, mentioned above, can be used to solve the problem of unclear images caused by camera shake. Figure 3 As shown, OIS is configured in the terminal device to control the movement of the camera and avoid blurry photos due to shaking.

[0071] However, current terminal devices cannot simultaneously enable background blur and image stabilization functions implemented through dual-camera algorithms. If both are enabled, and the user's hands shake when pressing the shutter button, the device will adjust the lens of at least one camera in the dual-camera module based on the detected shake to ensure a clear image. However, because the lens changes, the device cannot determine the positional relationship between the two cameras after the lens change, and therefore cannot accurately calculate depth information from the two images acquired by the dual-camera module. This results in poor background blur, or even no blurring at all.

[0072] Therefore, this application provides an image processing method applied to a terminal device. The terminal device can detect displacement information caused by camera shake and perform image stabilization control on the lens based on the detected displacement information to ensure the acquisition of clear dual-camera images. Subsequently, the terminal device compensates for the parameters of the lens capturing the dual-camera images based on the image stabilization control information to determine the lens's position when capturing the dual-camera images. This ensures accurate calculation of the image's depth information based on the lens's position and the resulting blurred image, thus guaranteeing a blurred effect.

[0073] In some examples, such as Figure 4 As shown, the image stabilization module of the terminal device can detect the displacement information caused by shaking of the terminal device, and control the lens in the camera of the dual-camera module to move in the opposite direction according to the displacement information to cancel out the displacement caused by shaking, ensuring that the dual-camera module outputs two clear images. Then, the dual-camera compensation module obtains the image stabilization data of the movement of the lens in the dual-camera module controlled by the image stabilization module, and compensates the calibration position of the lens in the dual-camera module based on the image stabilization data to accurately obtain the current position of the lens, i.e., the compensated position of the lens. Finally, the depth blur module of the terminal device can accurately determine the depth information in the image based on the compensated position of the lens and the two images, and performs accurate blur processing on the image based on the depth information to obtain a blurred photo.

[0074] To better understand the embodiments of this application, the terminal device provided in the embodiments of this application will be introduced first.

[0075] The image processing method provided in this application can be applied to terminal devices. Specifically, terminal devices can be mobile phones, tablets, smart screens, laptops, in-vehicle devices, wearable devices (such as smartwatches), ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), artificial intelligence devices, and other terminal devices with camera functions. This application does not limit the specific type of terminal device or the operating system installed.

[0076] The following describes the hardware structure of the terminal device.

[0077] Figure 5 A schematic diagram of the structure of terminal device 100 is shown. Terminal device 100 may include processor 110, external memory interface 120, internal memory 121, universal serial bus (USB) interface 130, charging management module 140, power management module 141, battery 142, antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194, and subscriber identification module (SIM) card interface 195, image stabilization module 196, etc.

[0078] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the terminal device 100. In other embodiments of this application, the terminal device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0079] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0080] The controller can serve as the central nervous system and command center of the terminal device 100. The controller can generate operation control signals based on the instruction opcode and timing signals to control the fetching and execution of instructions.

[0081] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0082] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the terminal device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to perform data storage functions.

[0083] Internal memory 121 can be used to store computer executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of terminal device 100 by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of terminal device 100 (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0084] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0085] The charging management module 140 receives charging input from a charger, which can be either a wireless or wired charger. While charging the battery 142, the charging management module 140 can also supply power to the terminal device via the power management module 141. The power management module 141 connects to the battery 142, the charging management module 140, and the processor 110.

[0086] The wireless communication function of the terminal device 100 can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor, and baseband processor.

[0087] Antennas 1 and 2 are used to transmit and receive electromagnetic wave signals. Each antenna in terminal device 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with a tuning switch.

[0088] The mobile communication module 150 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the terminal device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1.

[0089] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through audio devices (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through the display screen 194.

[0090] The wireless communication module 160 can provide solutions for wireless communication applications on the terminal device 100, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR) technology, etc.

[0091] The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signal, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0092] In some embodiments, antenna 1 of terminal device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling terminal device 100 to communicate with networks and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc.

[0093] Terminal device 100 implements display functions through a GPU, display screen 194, and application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0094] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. In some embodiments, the terminal device 100 may include one or N display screens 194, where N is a positive integer greater than 1.

[0095] Terminal device 100 can perform shooting functions through ISP, camera 193, video image codec, GPU, display screen 194 and application processor.

[0096] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization of image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0097] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the terminal device 100 may include one or N cameras 193, where N is a positive integer greater than 1.

[0098] In some embodiments, the terminal device includes two cameras, one of which may be referred to as the main camera and the other as the auxiliary camera or secondary camera. In this embodiment, the two cameras can be considered to form a module, such as a dual-camera module.

[0099] The image stabilization module 196 is used to detect the displacement information of the terminal device body shaking, and move the lens in the camera 193 according to the displacement amount that needs to be compensated calculated from the shaking displacement information, so as to avoid the blurring of the captured photos due to the shaking of the terminal device.

[0100] A digital signal processor (DSP) is used to process digital signals. Besides digital image signals, it can also process other digital signals. For example, when terminal device 100 selects a frequency, the DSP can perform Fourier transforms on the frequency energy.

[0101] Video image codecs are used to compress or decompress digital video or images. Terminal device 100 may support one or more video image codecs. Thus, terminal device 100 can play or record video in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG 2, MPEG 3, MPEG 4, etc. Alternatively, terminal device 100 can display or capture images in various encoding formats, such as Joint Photographic Experts Group (JPEG) and Portable Network Graphics (PNG).

[0102] NPU stands for Neural Network (NN) Computing Processor. By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in terminal devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0103] Terminal device 100 can implement audio functions such as music playback and recording through audio module 170, speaker, receiver, microphone, headphone jack, and application processor.

[0104] The audio module converts digital audio information into analog audio signals for output, and also converts analog audio input into digital audio signals. The audio module can also be used for encoding and decoding audio signals. The speaker converts audio electrical signals into sound signals. The receiver converts audio electrical signals into sound signals. The microphone converts sound signals into electrical signals. The headphone jack is used to connect wired headphones. The headphone jack can be a USB 130 interface, or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface, or other interfaces.

[0105] The sensor module 180 may include pressure sensors, gyroscope sensors, barometric pressure sensors, magnetic sensors, accelerometers, distance sensors, proximity sensors, fingerprint sensors, temperature sensors, touch sensors, ambient light sensors, bone conduction sensors, etc.

[0106] A gyroscope sensor can be used to determine the motion attitude of the terminal device 100. In some embodiments, the gyroscope sensor can determine the angular velocity of the terminal device 100 around three axes (i.e., the x, y, and z axes). The gyroscope sensor can be used for image stabilization. For example, when the shutter is pressed, the gyroscope sensor detects the angle of the terminal device 100's shake, calculates the distance the lens module needs to compensate based on the angle, and allows the lens to counteract the shake of the terminal device 100 through reverse movement, thus achieving image stabilization. The gyroscope sensor can also be used in navigation and motion-sensing gaming scenarios.

[0107] Buttons 190 include a power button, volume buttons, etc. A motor 191 can generate vibration feedback. An indicator 192 can be an indicator light, used to indicate charging status, battery level changes, messages, missed calls, notifications, etc. A SIM card interface 195 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to make contact with and separate from the terminal device 100. The terminal device 100 can support one or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 is compatible with external memory cards. The terminal device 100 interacts with the network through the SIM card to achieve functions such as calls and data communication.

[0108] The following section introduces the software structure of the terminal device.

[0109] The software system of terminal device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This embodiment of the invention uses... Figure 6 The software structure of the terminal device 100 is illustrated using the software structure block diagram shown below as an example.

[0110] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the operating system is divided into four layers, from top to bottom: the application layer, the application framework layer, the hardware abstraction layer (HAL), and the kernel layer.

[0111] For ease of explanation, Figure 6 The diagram also shows the hardware layer that connects to the software architecture.

[0112] It should be noted that the embodiments of this application are only illustrated using a common operating system as an example. In other operating systems, the solution of this application can also be implemented as long as the functions implemented by each functional module are similar to those in the embodiments of this application.

[0113] The application layer can include a series of application packages. For example... Figure 6 As shown, the application package can include applications such as camera, gallery, etc.

[0114] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0115] like Figure 6As shown, the application framework layer may include a camera service. The camera service connects the camera application in the application layer to the relevant hardware in the hardware layer. In some embodiments, the camera service can establish the connection between the camera application and the hardware layer by calling a hardware interface.

[0116] Understandably, the application framework layer may also include a window manager, content provider, view system, phone manager, resource manager, notification manager, etc. The window manager manages window programs. The content provider stores and retrieves data, making this data accessible to the application. The view system includes visual controls, such as controls for displaying text and images. The view system can be used to build the application. The display interface can consist of one or more views. The phone manager provides communication functionality for the terminal device 100. The resource manager provides the application with various resources, such as localized strings, icons, images, layout files, video files, etc. The notification manager allows the application to display notification information in the status bar, which can be used to convey informative messages and can disappear automatically after a short pause without user interaction.

[0117] The HAL layer is located above the kernel layer. The HAL layer can abstract the hardware of the hardware layer, hide the differences between different hardware, and provide a unified hardware interface for the upper layers.

[0118] like Figure 6 As shown, the HAL layer includes a camera device, an image processing module, and a storage module.

[0119] Camera devices are used to connect camera services to the underlying camera driver and camera.

[0120] The image processing module is used to acquire images and related data, and to process them. In some embodiments, the image processing module can perform operations such as format conversion, content processing, and encoding on the images.

[0121] In this embodiment of the application, the image processing module can determine the blurred image based on the two images acquired by the dual-camera module and the image stabilization data of the image stabilization module, according to the image processing method provided in this embodiment of the application.

[0122] The kernel layer is the layer between the hardware layer and the software layer. For example... Figure 6 As shown, the kernel layer contains a camera driver, which is used to control the camera in the hardware layer to perform shooting actions.

[0123] Understandably, the kernel layer can also contain display drivers, audio drivers, sensor drivers, etc., but we will not go into detail about these drivers here.

[0124] The following example, using the shooting scenario, illustrates the workflow of the software and hardware of terminal device 100.

[0125] The camera application at the application layer displays a shooting interface on the terminal device. If the shooting interface includes shooting controls, the user can click on the shooting controls. When the touch sensor receives a touch operation, the corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the touch operation into raw input events (including touch coordinates, timestamps, etc.). The raw input events are stored in the kernel layer. The framework layer obtains the raw input events from the kernel layer and can identify the corresponding control as a shooting control. Then, the camera calls the interface provided by the camera service in the framework layer. After the interface provided by the camera service is called, the camera device in the HAL layer can call the camera driver in the kernel layer. The camera driver can drive the dual-camera module to capture two images. Then, the dual-camera module sends the two images to the image processing module in the HAL layer through the camera driver. The image processing module can also obtain image stabilization data from the image stabilization module. The image stabilization data is used to record the direction and distance of lens movement controlled by the image stabilization module in the dual-camera module. Finally, the image processing module can process the image stabilization data and the two images to obtain a blurred photo.

[0126] In this embodiment of the application, the user's operation of opening the camera application is the first operation. In response to the first operation, the terminal device can display the shooting interface, which is also referred to as the preview interface mentioned above.

[0127] The following describes an image processing method provided by an embodiment of this application.

[0128] In scenarios where both image stabilization and bokeh effects are enabled, the terminal device can capture bokeh photos using the image processing method provided in this application. For example, the terminal device's shooting configuration page may include image stabilization and bokeh options. When both image stabilization and bokeh options are selected, it indicates that the terminal device simultaneously enables both image stabilization and bokeh effects.

[0129] In some scenarios, when the terminal device is in a shooting mode that requires both image stabilization and bokeh effects to be enabled simultaneously, bokeh photos can also be taken using the image processing method provided in this application. Examples include portrait shooting mode and sports shooting mode. The shooting modes described above are just one possible application scenario; the actual shooting mode used in a real-world application scenario can be determined based on actual shooting needs. No restrictions are placed on the shooting modes here.

[0130] The following is combined Figure 7 The flowchart of the image processing method shown is as follows. Figure 1This application introduces an image processing method provided by an embodiment of the present application. The method is executed by a terminal device and includes the following steps:

[0131] S701, The terminal device acquires dual-camera images captured by the dual-camera module and image stabilization data at the moment the dual-camera images were captured.

[0132] like Figure 4 As shown, the dual-camera module includes a main camera and an auxiliary camera. When the user presses the shutter button, both the main camera and the auxiliary camera capture images, thus obtaining a dual-camera image. The dual-camera image includes the main image captured by the main camera and the auxiliary image captured by the auxiliary camera.

[0133] In this embodiment of the application, the user's action of pressing the shutter button, that is, the user's action of clicking the shooting control, is referred to as the second action, and the moment when the user presses the shutter button can be the first moment in this application.

[0134] In some embodiments, due to technical issues such as instruction transmission latency, there may be a difference between the time when the main camera acquires the main image and the time when the auxiliary camera acquires the auxiliary image. Therefore, if the time difference between the main camera acquiring the main image and the auxiliary camera acquiring the auxiliary image is within an acceptable error range, it can be considered that the main camera and the auxiliary camera acquire images simultaneously. For example, if the acceptable error range is ±0.15 milliseconds of the main camera acquiring the main image, and the main camera acquires the main image at time T0, while the auxiliary camera acquires the auxiliary image at time T0+0.1 milliseconds, then the main image acquired by the main camera and the auxiliary image acquired by the auxiliary camera can be considered to be acquired simultaneously. In this case, the main image acquired by the main camera and the auxiliary image acquired by the auxiliary camera can also be considered to correspond. The main image acquired by the main camera and the auxiliary image acquired by the auxiliary camera are referred to as a set of dual-camera images.

[0135] like Figure 4 As shown, the image stabilization module in this embodiment controls the movement of the lens in the dual-camera module based on the displacement information of the terminal device caused by shaking. In other words, if the position of the lens in the dual-camera module moves during the capture of a dual-camera image, the terminal device needs to acquire image stabilization data to determine the new position of the lens based on the image stabilization data and the original position of the lens. Thus, the depth information of the image can be accurately calculated based on the new position of the lens and the dual-camera image.

[0136] The image stabilization module can control the movement of the lens of the main camera in the dual-camera module, or it can control the movement of the lens of both the main camera and the auxiliary camera in the dual-camera module.

[0137] In this embodiment of the application, the image stabilization data acquired at the moment of capturing dual-camera images can be the first image stabilization data in this application.

[0138] In some embodiments, the hardware structure corresponding to the image stabilization module can be determined according to the actual situation. For example, if the OIS can detect the displacement information of the terminal device caused by shaking and control the lens movement based on the displacement information, the image stabilization module may only include the OIS. Alternatively, if the OIS cannot detect the displacement information of the terminal device caused by shaking, but can only control the lens movement based on the displacement information, the image stabilization module may include the OIS and a sensor for detecting the displacement information of the terminal device caused by shaking. Yet another example, if the OIS can detect the displacement information of the terminal device caused by shaking and can control the lens movement, but cannot determine how to control the lens movement based on the displacement information, the image stabilization module may include the OIS and a processor for determining how to control the lens movement based on the displacement information. Alternatively, the image stabilization module may only include the OIS, and the image stabilization module may work in conjunction with other processors that determine how to control the lens movement based on the displacement information to achieve image stabilization control.

[0139] S702: The terminal device compensates the calibration data of the dual-camera module based on the image stabilization data to obtain the compensated calibration data.

[0140] The calibration data for the dual-camera module is used to record the initial positions of the lenses within the module. Specifically, if only the lens of the main camera in the dual-camera module is movable, the calibration data can be used only to record the initial positions of the lenses in the main camera. If both the lenses of the main camera and the auxiliary camera in the dual-camera module are movable, the calibration data can be used only to record the initial positions of the lenses in the main camera.

[0141] In this embodiment, the compensated calibration data can be the first calibration data in this application, and the calibration data used to record the initial position of the lens in the dual-camera module can be the second calibration data in this application.

[0142] In some examples, where the lens in the main camera is movable but the lens in the secondary camera is immovable, the image stabilization module can only control the movement of the lens in the main camera module. In this case, the image stabilization data records the direction and distance of the main camera lens's movement. The terminal device compensates for the calibration data of the dual-camera module based on the image stabilization data, obtaining compensated calibration data. In other words, the terminal device can calculate the position of the lens in the main camera after its movement based on the image stabilization data and the calibration data used to record the initial position of the lens in the main camera.

[0143] In other examples, where the lens in the main camera is movable, and the lens in the auxiliary camera is also movable, the image stabilization module can control the movement of only the lens in the main camera of the dual-camera module, or only the lens in the auxiliary camera, or it can control the movement of both the main camera lens and the auxiliary camera lens. The image stabilization data records the direction and distance of movement of the main camera lens and the auxiliary camera lens. The terminal device compensates for the calibration data of the dual-camera module based on the image stabilization data to obtain compensated calibration data. In other words, the terminal device can calculate the position of the main camera lens after movement and the position of the auxiliary camera lens after movement based on the image stabilization data and the calibration data used to record the initial positions of the lenses in the main camera and the auxiliary camera.

[0144] S703: The terminal device calculates the depth information of the dual-camera images based on the compensated calibration data and the dual-camera images.

[0145] S704: The terminal device acquires blurred photos based on depth information and dual-camera images.

[0146] As can be seen from the above embodiments, the compensated calibration data represents the position of the lenses in the dual-camera module after movement. Therefore, the terminal device calculates the depth information of the dual-camera image based on the compensated calibration data and the dual-camera image. Then, the terminal device can blur the image captured by the dual-camera module according to the depth information. For specific calculations, please refer to the aforementioned techniques for implementing blurring using dual-camera algorithms, which will not be elaborated here.

[0147] In some embodiments, the terminal device blurs the main image captured by the main camera in the dual-camera module based on depth information. For details on the blurring process, please refer to the above description of the blurring technology, which will not be repeated here.

[0148] Furthermore, in a dual-camera setup, one of the two images is used as the final photograph, while the other is used to assist in calculating the depth information of the image. Therefore, in general, the main image captured by the primary camera is used as the final photograph, and the clarity of the main image captured by the primary camera is ensured to be higher than that of the secondary image captured by the secondary camera. In this way, a clear photograph can be taken.

[0149] In this embodiment, the main camera can be the first camera in this application, the main image captured by the main camera is the first image in this application, and the auxiliary camera can be the second camera in this application, the auxiliary image captured by the auxiliary camera is the second image in this application.

[0150] In some embodiments, after the terminal device executes S704 to obtain a blurred photo, it can store the blurred photo so that the user can view the blurred photo on the terminal device. Furthermore, the terminal device can perform operations such as storing, transmitting, and editing the blurred photo in response to the user's operation.

[0151] In this application embodiment, the blurred photo may refer to the target image in this application, and the dual-camera image in this application embodiment may be the target dual-camera image in this application.

[0152] The following is combined Figure 8 The flowchart of the image processing method shown is as follows. Figure 2 This application introduces an image processing method provided by an embodiment of the present application. The method is executed by a terminal device and includes the following steps:

[0153] S801, The terminal device stores multiple sets of dual-camera images A output by the dual-camera module and the timestamp of each set of dual-camera images A in the image buffer memory.

[0154] When a user opens the camera app installed on their device and enables the background blur function, and the device uses a dual-camera algorithm for blurring, the dual-camera module begins working. The main and auxiliary cameras in the dual-camera module continuously capture images and output them in the order they are captured; this sequence of images is called an image sequence. The image sequence output by the dual-camera module is sent to both the preview stream processing module and the data stream processing module. After processing by the preview stream processing module, the image sequence is displayed sequentially on the device's screen, providing the user with a preview of the shooting process. At the moment the user presses the shutter button, the device selects a set of dual-camera images from the image sequence processed by the data stream processing module and determines the photograph based on this set of images. The timestamp of this set of dual-camera images corresponds to the moment the shutter button is pressed. In other words, after the shooting action is completed, the set of dual-camera images processed by the preview stream processing module is saved as an image file on the device for the user to view. Understandably, the above process only briefly introduces the processing of the preview stream and data stream during shooting. For a more detailed processing procedure, please refer to the relevant technical documentation.

[0155] In snapshot scenarios, if the terminal device determines the image from the image sequence of the input data stream processing module at the moment the user presses the shutter button as the captured photo, it will result in an unclear image. Therefore, after the user opens the camera application, the terminal device can store multiple sets of dual-camera images A from the image sequence output by the dual-camera module, along with the timestamps of dual-camera images A, in the image buffer memory. This allows the terminal device to select the appropriate dual-camera image from the various dual-camera images A.

[0156] In some embodiments, the terminal device may include an image buffer memory. Since the set of dual-camera images output by the dual-camera module includes a main image captured by the main camera and a secondary image captured by the secondary camera, the terminal device may store the main image captured by the main camera and its timestamp, as well as the secondary image captured by the secondary camera and its timestamp, in the image buffer memory. Alternatively, the terminal device may store only the main image captured by the main camera, its timestamp, and the secondary image captured by the secondary camera in the image buffer memory.

[0157] In some examples, the timestamp for each set of dual-camera images A can be two timestamps. That is, the terminal device stores the timestamp of each image in each set of dual-camera images A in the image buffer. For example, if the main image is acquired at 0.05ms and the secondary image is acquired at 0.09ms, then the timestamps for this set of dual-camera images A can be both 0.05ms and 0.09ms. Subsequently, the terminal device stores the dual-camera image along with its timestamps of 0.05ms and 0.09ms in the image buffer.

[0158] The terminal device will store the dual-camera image A and its timestamp in real time output by the dual-camera module. Since the storage space in the image buffer is limited, the terminal device can delete the earliest stored dual-camera image A and its timestamp in the image buffer, and overwrite the most recently acquired dual-camera image A and its timestamp in the image buffer.

[0159] In other examples, the timestamp for each set of dual-camera images A can be a single timestamp. Specifically, when the time difference between the acquisition times of the main image and the auxiliary image in a set of dual-camera images A can be ignored—for example, if the time difference between the acquisition times of the main image and the auxiliary image in each set of dual-camera images is within 0.05ms—the terminal device can acquire a bokeh photo with good blurring effect based on this set of dual-camera images. In this case, the acquisition time of the main image can be used as the timestamp for this set of dual-camera images, so that the corresponding image stabilization data can be determined based on this timestamp later.

[0160] In some other embodiments, the image buffer memory of the terminal device may include a main image buffer memory and a secondary image buffer memory, such as... Figure 9 As shown. In this embodiment, the image sequence output by the main camera can be stored in the main image buffer memory, and the timestamp of each image output by the main camera can also be stored in the main image buffer memory. The image sequence output by the auxiliary camera can be stored in the auxiliary image buffer memory, and the timestamp of each image output by the auxiliary camera can also be stored in the auxiliary image buffer memory.

[0161] In some examples, the terminal device can first obtain a main image and its timestamp from the main image buffer, and then determine the auxiliary image corresponding to the main image from the auxiliary image buffer based on the timestamp of the main image, and use the main image and the auxiliary image as a set of dual-camera images.

[0162] S802, The terminal device stores multiple anti-shake data A output by the anti-shake module and the timestamp of each anti-shake data A in the anti-shake data buffer memory.

[0163] Image stabilization data is used to record the direction and distance of lens movement controlled by the image stabilization module in the dual-camera module. When the camera application is running and image stabilization is enabled, the image stabilization module begins outputting image stabilization data. To ensure accurate calculation of image depth information based on the dual-camera images output by the dual-camera module, the terminal device can first determine the position of the lens in each camera within the dual-camera module when outputting the dual-camera images. Therefore, the terminal device can store multiple image stabilization data points A output by the image stabilization module, along with the timestamp of each data point A, in the image stabilization data buffer.

[0164] In snapshot scenarios, users may frequently press the shutter button. Each time the user presses the shutter, the terminal device acquires a dual-camera image and image stabilization data. As described above, the terminal device stores multiple dual-camera images A and the timestamp of each image in an image buffer, and multiple image stabilization data A and their timestamps in an image stabilization data buffer. This ensures that the terminal device can retrieve the corresponding dual-camera image from the image buffer and the image stabilization data from the image stabilization data buffer each time the user presses the shutter, preventing the loss of dual-camera images and image stabilization data, and avoiding situations where the image cannot be blurred.

[0165] In some embodiments, the image stabilization data buffer storage may include multiple buffers, each independent. When the user does not press the shutter button, the terminal device simultaneously stores multiple image stabilization data points into the multiple buffers respectively. This ensures that if one buffer has a problem, another buffer can store the image stabilization data, guaranteeing the accuracy of the image stabilization data.

[0166] In some examples, when the user presses the shutter, the terminal device stops storing image stabilization data in one of the buffers and retrieves the corresponding image stabilization data from that buffer. Meanwhile, the terminal device continues to store image stabilization data in other buffers. For instance, when the user presses the shutter, the terminal device can select one of multiple buffers as the read target and configure that buffer into read mode, allowing the terminal device to read data only from that buffer, while the other buffers can continue storing image stabilization data.

[0167] In some examples, the stabilization data buffer memory includes buffer 1 and buffer 2, such as... Figure 10 As shown, when the user does not press the shutter button, the image stabilization data output by the image stabilization module is simultaneously stored in buffer 1 and buffer 2. At this time, if both buffers can store the image stabilization data normally, the image stabilization data stored in buffer 1 and buffer 2 should be the same.

[0168] The buffer has limited storage space, and the terminal device can overwrite the oldest stabilization data stored in the stabilization module with the most recently output stabilization data to ensure that the data in the buffer is updated in real time.

[0169] Furthermore, upon the user's first shutter press, the corresponding dual-camera compensation module in the terminal device responds to the shutter press by starting to read stabilization data from buffer 1, while the stabilization module continues to store stabilization data into buffer 2. Moreover, to prevent the stabilization data in buffer 1 from being overwritten by the most recently output stabilization data from the stabilization module, the stabilization module stops storing new stabilization data into buffer 1 at the moment the dual-camera supplementary module reads stabilization data from buffer 1.

[0170] Furthermore, if the user presses the shutter again within a certain time, and the dual-camera compensation module has already read the image stabilization data from buffer 1, and the image stabilization module continues to store new image stabilization data into buffer 1, the corresponding dual-camera compensation module in the terminal device responds to the user's shutter press operation by starting to read image stabilization data from buffer 2, while the image stabilization module continues to store image stabilization data into buffer 1. This avoids the terminal device being unable to read the image stabilization data corresponding to the current shutter press from buffer 1 because buffer 1 did not store the most recently output image stabilization data in a timely manner. Furthermore, to prevent the image stabilization data in buffer 2 from being overwritten by new image stabilization data, the image stabilization module stops storing new image stabilization data into buffer 2.

[0171] Understandably, if the user presses the shutter again within a certain period of time after the above embodiment, the dual-camera compensation module can read the image stabilization data from buffer 1, and the image stabilization module continues to store the image stabilization data into buffer 2. Through the above-mentioned ping-pong mechanism, it can be ensured that the image stabilization module continuously and stably provides the image stabilization data to the subsequent image processing process.

[0172] It should be noted that the dual-camera compensation module selects the buffer to read the image stabilization data according to the order of the buffer arrangement. In practice, the buffer can also be selected to read the image stabilization data in other ways.

[0173] In the above S801 and S802, since the dual-camera images and their timestamps, as well as the image stabilization data and the timestamp of each image stabilization data, are all stored in the corresponding buffer memory, the reading pressure of the dual-camera images and image stabilization data can be alleviated in scenarios where the user presses the shutter multiple times in a short period of time, so as to ensure that the subsequent image processing process can obtain the dual-camera images and image stabilization data corresponding to each time the shutter is pressed.

[0174] After storing the dual-camera images and image stabilization data in the corresponding buffer memory through the above S801 and S802, the terminal device can execute the following S803 in response to the user pressing the shutter button:

[0175] S803, the terminal device obtains multiple sets of dual-camera images B within the time period corresponding to the moment the user presses the shutter, as well as multiple sets of image stabilization data B corresponding to each set of dual-camera images B, from the image buffer memory and the image stabilization data buffer memory.

[0176] "Corresponding to the moment the user presses the shutter" means that the time difference between the timestamps of multiple sets of dual-camera images B and the moment the user presses the shutter is within a preset range. In other words, it refers to the multiple sets of dual-camera images B and multiple image stabilization data B within the time period corresponding to the moment the user presses the shutter. For example, if the time period corresponding to the moment the user presses the shutter is within 0.15 milliseconds before and after the moment the user presses the shutter, such as... Figure 11 As shown, time T0 corresponds to dual-camera image 1 and image stabilization data 1; time T1 corresponds to image stabilization data 2; time T2 corresponds to dual-camera image 2; time T3 corresponds to dual-camera image 3 and image stabilization data 3; time T4 corresponds to no dual-camera image and image stabilization data; and time T5 corresponds to dual-camera image 4 and image stabilization data 4. If the user presses the shutter at time T3, T3-0.15ms is a time between T0 and T1, and T3+0.15ms is a time after T5. Then, the terminal device can select image stabilization data 2 corresponding to time T1, dual-camera image 2 corresponding to time T2, dual-camera image 3 and image stabilization data 3 corresponding to time T3, and dual-camera image 4 and image stabilization data 4 corresponding to time T5 as multiple sets of dual-camera images B and multiple sets of image stabilization data B. It is understandable that the time the user presses the shutter is not necessarily the same as the timestamp for acquiring the dual-camera images and image stabilization data, as in the example above. Figure 11 The moment the user presses the shutter can also be T3+0.05 milliseconds.

[0177] Understandably, the above embodiments only illustrate one situation. The time period corresponding to the moment the user presses the shutter can also be a period of time after the moment the user presses the shutter, a period of time before the moment the user presses the shutter, or a relatively long period of time after the moment the user presses the shutter and a relatively short period of time before the moment the user presses the shutter.

[0178] It should be noted that the maximum acquisition frequency of image stabilization data is greater than the maximum acquisition frequency of dual-camera images. Therefore, the maximum amount of image stabilization data that can be acquired in a unit of time is greater than the maximum amount of dual-camera image data that can be acquired in a unit of time.

[0179] In some embodiments, a set of dual-camera images can correspond to multiple image stabilization data. Specifically, the terminal device can first acquire multiple dual-camera images B, and for each dual-camera image B, acquire multiple image stabilization data B corresponding to that dual-camera image B. For example, the terminal device acquires five sets of dual-camera images B, and the timestamps of the five sets of dual-camera images B are: Timestamp1, Timestamp2, Timestamp3, Timestamp4, and Timestamp5, respectively. Then, based on the five timestamps, the terminal device finds the timestamps closest to the above timestamps among the timestamps of the multiple image stabilization data A: Timestamp_OIS1, Timestamp_OIS2, Timestamp_OIS3, Timestamp_OIS4, and Timestamp_OIS5, respectively. Then, based on the exposure time of the dual-camera module or a preset time length, the terminal device determines the timestamp range of the image stabilization data B corresponding to the dual-camera image within the exposure time. With an exposure time of 2t ms, where t is a known number, the timestamp ranges of the image stabilization data B corresponding to each set of dual-camera images are: [Timestamp_OIS1 - t ms Timestamp_OIS1 + t ms], [Timestamp_OIS2 - t ms Timestamp_OIS2 + t ms], [Timestamp_OIS3 - t ms Timestamp_OIS3 + t ms], [Timestamp_OIS4 - t ms Timestamp_OIS4 + t ms], [Timestamp_OIS5 - t ms Timestamp_OIS5 + t ms]. In other words, the image stabilization data B within the range of [Timestamp_OIS1 - t ms Timestamp_OIS1 + t ms] corresponds to the set of dual-camera images B with timestamp 1.

[0180] In some examples, a set of dual-camera images has multiple image stabilization data points. For example... Figure 12 As shown, the dual-camera image B1 corresponds to 17 stabilization data points B, from B1 to B17, and the dual-camera image B2 corresponds to 17 stabilization data points B, from B18 to B34.

[0181] In some embodiments, since the image stabilization module only outputs stabilization data when shaking occurs, the amount of stabilization data B corresponding to different dual-camera images B may vary. For example, as... Figure 12 As shown, dual-camera images B1 and B2 each correspond to 17 image stabilization data points B, while dual-camera image B3 corresponds to 6 image stabilization data points B from B35 to B40, and dual-camera image B4 corresponds to 3 image stabilization data points B from B41 to B43.

[0182] In some embodiments, since the image stabilization module only outputs stabilization data when shaking occurs, and there may be a large or small difference between the timestamps of the two dual-camera images, there may be a discontinuity in the stabilization data B corresponding to two consecutively acquired dual-camera images B. For example, as... Figure 12 As shown, dual-camera image B4 corresponds to three image stabilization data Bs, from image stabilization data B41 to image stabilization data B43, while dual-camera image B5 corresponds to image stabilization data B60 and image stabilization data B61.

[0183] Through S803, the terminal device can obtain multiple sets of dual-camera images B and multiple anti-shake data B, as well as the timestamp of each dual-camera image B and the timestamp of each anti-shake data B. Optionally, the terminal device can execute S804 to filter out invalid data from the dual-camera images B and the multiple anti-shake data B to ensure the accuracy of the data.

[0184] In some embodiments, among the multiple sets of dual-camera images B and multiple anti-shake data B selected by the terminal device in S803, there may be invalid dual-camera images B and / or anti-shake data B. Therefore, the terminal device can determine the valid data through the following S804.

[0185] S804, The terminal device determines the valid sets of dual-camera images C and multiple sets of image stabilization data C from multiple sets of dual-camera images B and multiple sets of image stabilization data B.

[0186] The terminal device can determine whether the dual-camera image C is valid based on the image content in the dual-camera image B.

[0187] In some embodiments, the terminal device can identify and compare the main image and the auxiliary image in the dual-camera image B, calculate the similarity between the main image and the auxiliary image, and if the similarity between the main image and the auxiliary image is less than the preset similarity, it means that the main image and the auxiliary image may not have been captured at the same time or at similar times, and then the dual-camera image B can be regarded as an invalid dual-camera image B.

[0188] In some other embodiments, the terminal device can identify the clarity of the main image and the auxiliary image in the dual-camera image B. If the clarity of both the main image and the auxiliary image is less than the preset clarity, it indicates that the main image and the auxiliary image may be unclear. In this case, the dual-camera image B can be regarded as an invalid dual-camera image B.

[0189] The terminal device can determine whether the image stabilization data C is valid based on the data content in the image stabilization data B.

[0190] In some embodiments, the terminal device can determine whether the movement direction and movement distance in the image stabilization data B are valid image stabilization data B.

[0191] In some examples, the terminal device can consider image stabilization data B, whose direction of movement differs from the direction in which the lens can move, as invalid image stabilization data B. For example, in a Cartesian coordinate system with the x-axis as the horizontal axis and the y-axis as the vertical axis, the lens can only move within this Cartesian coordinate system. If image stabilization data B contains data in the z-direction, then that image stabilization data B is invalid.

[0192] In some examples, the terminal device may consider image stabilization data B with a movement distance greater than the maximum possible movement distance of the lens as invalid. For instance, in a Cartesian coordinate system with the x-axis as the horizontal axis and the y-axis as the vertical axis, the lens can only move along the x-axis and y-axis, and the maximum distance moved in each direction is 5 units. If image stabilization data B shows a movement of 6 units along a certain direction, such as the x-axis, then that image stabilization data B is invalid.

[0193] In other embodiments, the terminal device may also use filtering and noise reduction to determine the effective anti-shake data C.

[0194] In some examples, filtering is performed by selecting one of a variety of window function filters, such as Gaussian filtering, within the exposure time of a set of dual-camera images or within the exposure time of the main image in a set of dual-camera images.

[0195] In other examples, within the exposure time of a set of dual-camera images or within the exposure time of the main image in a set of dual-camera images, the filter window is smoothly moved across the time domain of the stabilization data B to select the stabilization data B corresponding to each filter window. For example... Figure 13As shown, there are five image stabilization data points B: B1, B2, B3, B4, and B5. A filtering window can have three image stabilization data points B. Therefore, during the time-domain smoothing process, the terminal device can determine B1' based on B1, B2, and B3; B2' based on B2, B3, and B4; and B3' based on B3, B4, and B5. The terminal device can calculate either the average of B1, B2, and B3 as B1', or the maximum value of B1, B2, and B3 as B1'; the specific calculation method is not limited here. Ultimately, the terminal device determines B1', B2', and B3' as the filtered image stabilization data B, i.e., determines the effective image stabilization data C.

[0196] In some examples, the terminal device can use multiple filtering methods to filter the stabilized data B multiple times to determine the effective stabilized data C. For example, the terminal device can first use Gaussian filtering to determine the stabilized data B' from the stabilized data B, and then smoothly move the filtering window over the time domain of the stabilized data B' to select the stabilized data B" corresponding to each filtering window, that is, determine the effective stabilized data C.

[0197] Understandably, other methods can also be used to filter the stabilized data B to select the stabilized data C. In this way, filtering can reduce the impact of noise on the stabilized data and improve the accuracy of subsequent depth information calculations based on the stabilized data.

[0198] After the terminal device determines the valid sets of dual-camera images C and multiple sets of image stabilization data C from multiple sets of dual-camera images B and multiple sets of image stabilization data B, the terminal device can configure invalid dual-camera images B and invalid image stabilization data B with invalid flag bits, or assign invalid values ​​to their corresponding flag bits. For example, assigning a value of 0 to the flag bit corresponding to invalid image stabilization data B indicates that the image stabilization data is invalid, and assigning a value of 1 to the flag bit corresponding to valid image stabilization data B indicates that the image stabilization data is valid.

[0199] Furthermore, the terminal device can identify valid image stabilization data C based on whether the image stabilization data B contains a flag indicating invalidity.

[0200] In the above embodiments, when the terminal device executes S803 but not S804, the dual-camera image B can be multiple sets of dual-camera images in this application, and the image stabilization data B can be the second image stabilization data in this application. When the terminal device executes both S803 and S804, the image stabilization data C can be the second image stabilization data in this application.

[0201] S805. The terminal device determines the target dual-camera image D from multiple sets of dual-camera images C, and determines the timestamp a of the target dual-camera image D.

[0202] When a user presses the shutter button, the device can select from multiple sets of dual-camera images A stored in the image buffer that correspond to the moment the user presses the shutter, based on the timestamp. "Corresponding to the moment the user presses the shutter" means that the time difference between the timestamp of the selected sets of dual-camera images B and the moment the user presses the shutter is within a preset range. Thus, the terminal device can determine a valid dual-camera image C from the selected sets of dual-camera images B. Then, it selects one set of dual-camera images from the valid sets of dual-camera images C as the target dual-camera image D to ensure the sharpness of the final captured photo.

[0203] In the embodiments of this application, the target dual-camera image D can be the target dual-camera image in this application.

[0204] After processing, the target dual-camera image D is the final photo that needs to be presented to the user and stored in the terminal device. In some embodiments, the terminal device can determine the clearest image from multiple sets of dual-camera images C as the target dual-camera image D, or the terminal device can determine the most striking image from multiple sets of dual-camera images C as the target dual-camera image D. The most striking image can be defined as a dual-camera image with a clarity higher than a certain value and containing the most faces, as determined by the terminal device. Specifically, the most striking image can be determined based on the user's needs or the actual shooting scene; no further restrictions are imposed here.

[0205] In addition, after the terminal device determines the target dual-camera image D from multiple sets of dual-camera images C, it can determine the timestamp of the target dual-camera image D, or the timestamp of the main image in the target dual-camera image D as timestamp a, so as to ensure the accuracy of timestamp a.

[0206] Optionally, the terminal device can execute S806 to ensure accurate determination of the stabilization data corresponding to the timestamp.

[0207] S806. The terminal device determines multiple anti-shake data D corresponding to timestamp a from multiple anti-shake data C.

[0208] As is known in the embodiments related to S803 described above, the terminal device can determine the corresponding image stabilization data B based on the timestamp of the dual-camera image B. In some scenarios, the amount of image stabilization data B determined by the terminal device based on the timestamp of the dual-camera image B in S803 is greater than a preset amount of data; or, after the terminal device determines the image stabilization data B in S803, the terminal device executes S804 and determines that the amount of valid image stabilization data C is greater than a preset amount of data. In this case, the terminal device can execute S806 to ensure that multiple image stabilization data D corresponding to the timestamp a can be accurately determined.

[0209] If the amount of image stabilization data B determined by the terminal device based on the timestamp of the dual-camera image B in S803 is less than the preset amount of data, or if the terminal device determines the image stabilization data B in S803 and then executes S804 to determine that the amount of effective image stabilization data C is less than the preset amount of data, then the terminal device may or may not execute S806.

[0210] In the above embodiments, when the terminal device executes S806, the multiple anti-shake data D corresponding to the timestamp a can be the third anti-shake data in this application.

[0211] Optionally, if the terminal device obtains multiple anti-shake data D by executing S806, it can execute S807:

[0212] S807, The terminal device obtains the image stabilization data E with the timestamp a by interpolating multiple image stabilization data D.

[0213] Because the sampling frequency of the image stabilization data differs from that of the dual-camera images (generally, the sampling frequency of the image stabilization data is higher), the generation time of the image stabilization data cannot be exactly the same as the generation time of the dual-camera images. In other words, there is a high probability that image stabilization data D will not exist at timestamp a. Based on this, the terminal device can obtain image stabilization data E corresponding to timestamp a by taking the difference between image stabilization data D and timestamp a within a preset time difference range. Here, image stabilization data E corresponding to timestamp a means that the acquisition time of image stabilization data E is timestamp a.

[0214] like Figure 14 As shown, the terminal device acquires dual-camera image 1 and image stabilization data 1 at time T0, image stabilization data 2 at time T1, dual-camera image 2 at time T2, and image stabilization data 3 at time T3. The terminal device can obtain image stabilization data 4 based on the difference between image stabilization data 2 and image stabilization data 3, and image stabilization data 4 corresponds to time T2. In this way, the terminal device can accurately acquire image stabilization data 4 at time T2 when dual-camera image 2 is acquired, and then blur the dual-camera image 2 based on image stabilization data 4, effectively improving the blurring effect.

[0215] In some embodiments, such as Figure 14 The image stabilization data 2 and image stabilization data 3 are two image stabilization data D obtained by the terminal device executing S806. The time interval between the timestamps of image stabilization data 2 and image stabilization data 3 and the timestamp of the dual-camera image 2 is less than the preset time interval. Then the terminal device can use a function of the form interpolation(image stabilization data 2, T1, image stabilization data 3, T3, T2) to obtain image stabilization data 4 at time T2.

[0216] The interpolation method used by the terminal device can be linear interpolation or other methods. For example, the terminal device can also use a larger amount of image stabilization data for interpolation, such as using the two most recent image stabilization data before time T2 and the two most recent image stabilization data after time T2, and using the Bicubic interpolation algorithm to obtain the image stabilization data at time T2. This application will not elaborate on this further; please refer to related technologies.

[0217] S808 and the terminal device compensate for the dual-camera calibration data of the dual-camera module based on the image stabilization data E to obtain new dual-camera calibration data.

[0218] Dual-camera calibration data is used to record the initial position of the movable lens in the dual-camera module, while image stabilization data records the direction and distance of the movement of the lens in the dual-camera module controlled by the image stabilization module. Therefore, the terminal device compensates for the dual-camera calibration data of the dual-camera module based on the image stabilization data to obtain new dual-camera calibration data, which is the position of the lens in the dual-camera module after movement.

[0219] In some embodiments, the image stabilization module pushes the lens of the main lens in the dual-camera module to translate in the X and Y directions perpendicular to the optical axis. The corresponding image stabilization data is (x_offset_ois, y_offset_ois), where x_offset_ois represents the direction and distance of movement of the main lens in the X-axis direction, and y_offset_ois represents the direction and distance of movement of the main lens in the Y-axis direction. The dual-camera calibration data of the dual-camera module is (dual_cam_x_offset, dual_cam_y_offset). After the terminal device executes S808, the new dual-camera calibration data obtained is (dual_cam_x_offset_new, dual_cam_y_offset_new). Where dual_cam_x_offset_new = dual_cam_x_offset + x_offset_ois, and dual_cam_y_offset_new = dual_cam_y_offset + y_offset_ois.

[0220] In the above embodiments, when the terminal device executes S806 but not S807 and S808, the image stabilization data D in this application embodiment can be the first image stabilization data in this application. When the terminal device executes S807 and S808, or executes S806-S808, the image stabilization data E in this application embodiment can be the first image stabilization data in this application.

[0221] S809, the terminal device calculates depth information based on the new dual-camera calibration data and the target dual-camera image D.

[0222] like Figure 15 As shown, the depth calculation module of the terminal device can calculate the depth information based on the new dual-camera calibration data and the main image and auxiliary image in the target dual-camera image D. For details, please refer to the binocular depth parallax and depth calculation methods in related technologies, which will not be elaborated here.

[0223] After the terminal device calculates the depth information, it can execute S810 to blur the main image:

[0224] S810, the terminal device blurs the main image in the dual-camera image D of the target based on depth information.

[0225] The terminal device can set areas within a preset depth range in the main image of the target dual-camera image D to be sharp, while setting areas outside the preset depth range to be blurry, thereby obtaining an image like... Figure 2 The blurred image shown.

[0226] This application also provides a computer-readable storage medium including computer instructions that, when executed on the terminal device, cause the terminal device to perform various functions or steps in the above method embodiments.

[0227] This application also provides a computer program product, including a computer program that, when run on a terminal device, causes the terminal device to perform various functions or steps in the above method embodiments.

[0228] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0229] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0230] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0231] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0232] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially or in other words, the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0233] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image processing method, characterized in that, Applied to a terminal device including a first camera and a second camera, the method includes: In response to the user's first operation of opening the camera application, a preview interface of the camera application is displayed, the preview interface including shooting controls, and the camera application has enabled background blur and image stabilization functions; Acquire first image stabilization data corresponding to the first moment, wherein the first image stabilization data is the movement parameters of the lens of the target camera moving at the first moment, and the target camera includes the first camera and / or the second camera; Based on the first image stabilization data and the second calibration data, the first calibration data is determined; the first calibration data is used to indicate the pose relationship between the first camera and the second camera at the first moment, and the second calibration data is used to indicate the initial pose relationship between the first camera and the second camera. In response to a second operation by the user on the shooting control, the target image is stored; The target image is an image obtained by blurring the background of the first image based on the depth information of the first image. The depth information is determined based on the first calibration data, the first image, and the second image. The first image is an image captured by the first camera at the first moment, and the second image is an image captured by the second camera at the first moment.

2. The method according to claim 1, characterized in that, The step of obtaining the first image stabilization data at the first moment includes: The target dual-camera image is determined, wherein the target dual-camera image is the dual-camera image captured by the terminal device at the first moment, and the target dual-camera image includes the first image and the second image; The first image stabilization data is obtained based on the timestamp of the target dual-camera image; the time difference between the timestamp of the first image stabilization data and the timestamp of the target dual-camera image is within a preset time range.

3. The method according to claim 2, characterized in that, The determination of the target dual-camera image includes: Multiple sets of dual-camera images are acquired. Each set of dual-camera images includes an image captured by the first camera and an image captured by the second camera at the same time. The time difference between the timestamp of each set of dual-camera images and the first time is within a preset time range. The target dual-camera image is determined from the plurality of dual-camera images.

4. The method according to claim 3, characterized in that, The step of obtaining the first image stabilization data based on the timestamp of the target dual-camera image includes: Obtain the second image stabilization data corresponding to each of the multiple sets of dual-camera images; The first image stabilization data is determined based on the timestamp of the target dual-camera image and multiple second image stabilization data.

5. The method according to claim 4, characterized in that, The step of determining the first image stabilization data based on the timestamp of the target dual-camera image and multiple second image stabilization data includes: Based on the timestamp of the target dual-camera image, a third image stabilization data corresponding to the timestamp of the target dual-camera image is determined from a plurality of second image stabilization data; In the case where there are multiple third image stabilization data, interpolation is performed on the multiple third image stabilization data to obtain the first image stabilization data.

6. The method according to claim 4 or 5, characterized in that, The multiple sets of dual-camera images are valid dual-camera images among dual-camera images whose time difference between the corresponding timestamp and the first time is within a preset time range. The sharpness of the valid dual-camera images is greater than a preset sharpness, or the similarity between the two images included in the valid dual-camera images is greater than a preset similarity. The multiple sets of second image stabilization data are valid image stabilization data among the image stabilization data corresponding to each set of dual-camera images in the multiple sets of dual-camera images. The image stabilization parameters of the valid image stabilization data are less than preset image stabilization parameters.

7. The method according to claim 4 or 5, characterized in that, The terminal device includes a memory, which includes at least two storage areas; The step of obtaining the second image stabilization data corresponding to each of the multiple sets of dual-camera images includes: Read the second anti-shake data from the first storage area of ​​the memory; In the case of reading image stabilization data from the first storage area, the terminal device stores the image stabilization data to be stored in the second storage area. The first storage area is any one of the at least two storage areas, and the second storage area is a storage area other than the first storage area among the at least two storage areas.

8. A terminal device, characterized in that, The terminal device includes a display screen, a memory, and one or more processors; the display screen, the memory, and the processors are coupled; the display screen is used to display an image generated by the processor, the memory is used to store computer program code, the computer program code including computer instructions; when the processor executes the computer instructions, the terminal device performs the method as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, Includes computer instructions that, when executed on a terminal device, cause the terminal device to perform the method as described in any one of claims 1-7.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the method as described in any one of claims 1-7.

Images