Image processing method, electronic device and storage medium

Abstract

The present application provides an image processing method, electronic device, and storage medium, relating to the field of image processing technology. In this method, after a user starts a camera application, the ambient brightness can be identified to obtain a brightness index, which is negatively correlated with the ambient brightness. Then, in response to the user triggering the shooting function of the camera application, when it is determined that the brightness index is less than or equal to a first threshold, the ambient brightness is high, and the first image obtained by the first camera and the second image obtained by the second camera can be blurred to obtain a first photo. When it is determined that the brightness index is greater than the first threshold, the ambient brightness is low, and the first image can be blurred based on the first image to obtain a second photo. In this way, based on different ambient brightness, a more suitable blurring processing method can be adopted to achieve adaptive switching between different blurring processing methods, which can improve the blurring effect of the photo and thus enhance the user experience.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, electronic equipment, and storage medium. Background Art

[0002] With the widespread adoption of electronic devices like mobile phones and tablets, and the rapid development of photography technology, the camera functions of these devices have become increasingly diverse, with more and more users choosing to use them to capture the wonderful moments of life. The methods used for capturing images have also become more diverse. For example, to highlight the subject in a scene, the background can be blurred to clearly highlight the subject.

[0003] However, the photos after blurring may have abnormal blurring effects, such as poor blurring of the background, or mistaken blurring of the subject, which makes it impossible to highlight the subject and easily affects the user experience. Summary of the Invention

[0004] In order to solve the above problems, the present application provides an image processing method, an electronic device and a storage medium, the purpose of which is to improve the blurring effect of photos taken by the electronic device and enhance the user experience.

[0005] In the first aspect, the present application provides an image processing method. Exemplarily, the method can be applied to electronic devices, which can be mobile phones, tablet computers, laptop computers, and other devices that include camera applications. In this method, the user can start the camera application, for example, the user clicks on the icon of the camera application, or the user issues gesture commands and voice commands to start the camera application. In response to the user's start-up operation of the camera application, the electronic device can identify the ambient brightness and obtain a brightness index. Exemplarily, the electronic device can sense the ambient brightness through a camera, and obtain a brightness index based on the sensed ambient brightness. The brightness index is negatively correlated with the ambient brightness, indicating that the higher the ambient brightness, the smaller the brightness index, and the lower the ambient brightness, the larger the brightness index. Then, when the user triggers the shooting function of the camera application, for example, the user clicks on the shooting control of the camera application, or the user issues gesture commands and voice commands to trigger the photo function, the electronic device responds to the user's shooting operation of the camera application. The triggering operation of the shooting function first compares the brightness index with the first threshold value. The first threshold value is a pre-set brightness index threshold value. When the electronic device determines that the brightness index is less than or equal to the first threshold value, it indicates that the ambient brightness is relatively high and is more suitable for binocular blur processing. The electronic device can blur the first image based on the first image captured by the first camera and the second image captured by the second camera to obtain a first photo; after the user triggers the shooting function of the camera application, when the electronic device determines that the brightness index is greater than the first threshold value, it indicates that the ambient brightness is relatively low and is more suitable for monocular blur processing. The electronic device can blur the first image based on the first image captured by the first camera itself to obtain a second photo.

[0006] In this way, under different ambient brightness conditions, the present application adopts a more applicable blur processing method. When the ambient brightness is high, the output quality of the first camera and the second camera is not affected, and a binocular blur processing method can be used to obtain a first photo with better quality; when the ambient brightness is low, the output quality of the camera may be affected. In order to avoid the superposition of the influences on the first image and the second image captured by the first camera and the second camera, or to bring greater influence to each other, resulting in abnormal blur effect, a monocular blur processing method can be used. Compared with the single blur processing method used in the prior art, the present application can adaptively switch between different blur processing methods, improve the blur effect of the captured photos, improve the photo quality, and thereby improve the user experience.

[0007] In one possible implementation, the image processing method may further include: when a user triggers a shooting function of a camera application, the electronic device, in response to the user's triggering operation of the shooting function of the camera application, may compare the brightness index with a second threshold value, the second threshold value being also a pre-set brightness index threshold value, and the second threshold value being greater than the first threshold value. For example, the first threshold value may be 355 and the second threshold value may be 370. When the electronic device determines that the brightness index is greater than the second threshold value, it indicates that the ambient brightness is lower, that is, the shooting scene may be in an extremely dark environment. The electronic device may perform photo retouching on the first image to obtain a third photo. For example, the photo retouching may include portrait enhancement processing, beauty processing, and other processing unrelated to blur processing. That is, there is no need to blur the first image, and the obtained third photo does not have a background blur effect. For example, when taking a portrait photo, the background other than the person in the third photo is not blurred and does not have a blur effect.

[0008] In this way, when the shooting scene is in an extremely dark environment, the image captured by the camera is also greatly affected. In order to avoid blurring the image at this time, resulting in an unclear or abnormal blurring effect, the image can be not blurred. This application can select a more suitable processing method based on the ambient brightness, which can avoid damage to the photo quality and improve the user experience.

[0009] In one possible implementation, the step of determining the first threshold for comparing the brightness index may include: the electronic device may determine a first zoom ratio applied by the camera, and based on a correspondence between multiple zoom ratios and multiple thresholds, determine a first threshold corresponding to the first zoom ratio. For example, when the first zoom ratio is 2.5x, the first threshold may be 300, and when the first zoom ratio is 5x, the first threshold may be 355. In this way, at different zoom ratios, the characteristics of the image vary, and based on different first thresholds, it is beneficial to accurately determine a more appropriate blur processing method under different zoom ratios.

[0010] In one possible implementation, the electronic device may perform a blurring process on the first image based on the first image and the second image to obtain the first photograph, which may include: the electronic device first performs a depth calculation based on the first image and the second image to obtain first depth information, and then the electronic device uses the first depth information to blur the first image to obtain the first photograph. In this way, the depth calculation is performed based on the first image captured by the first camera and the second image captured by the second camera, that is, based on the richer image information captured by the two cameras. The depth information obtained is more accurate and has higher precision, which can enhance the blurring effect of the first photograph.

[0011] In a possible embodiment, the above-mentioned electronic device performs depth calculation based on the first image and the second image to obtain the first depth information, which may include: the electronic device performs depth calculation based on the first image, the second image in the first image format and the second image in the second image format to obtain the first depth information. Exemplarily, the first image format may be the RAW format of the original image captured by the camera, and the second image format may be the converted YUV format. In this way, based on the second image in different image formats, richer information can be extracted, and then more accurate depth information can be calculated, which is conducive to further improving the blurring effect of the first photo.

[0012] In a possible embodiment, the first image and the second image both include a first object, that is, the shooting scene of the electronic device includes the first object. For example, the first object can be a person, a plant such as a flower or a tree, an animal, or any other object such as a star. Accordingly, the electronic device performs depth calculation based on the first image, the second image in the first image format, and the second image in the second image format to obtain the first depth information. This may include: the electronic device first extracts the contour information of the first object based on the first image, such as the contour information of a human body; then, the electronic device can perform depth calculation based on the first image, the second image in the first image format, the second image in the second image format, and the contour information of the first object to obtain the first depth information. Since the image is blurred in order to highlight the subject, that is, the first object, more accurate first depth information can be calculated based on the contour information of the first object when performing the depth calculation, which is conducive to further improving the blurring effect of the first photo.

[0013] In one possible embodiment, the electronic device defocusing the first image based on the first image to obtain the second photo may include: the electronic device performing a depth calculation based on the first image to obtain second depth information; and then using the second depth information to defocus the first image to obtain the second photo. In this way, when the ambient brightness is low, the two cameras are not affected by the ambient brightness, which would reduce the quality of the images captured, thereby causing errors in the depth calculation and obtaining abnormal depth information. Instead, only the image captured by one camera is used for depth calculation, which improves the accuracy of the depth calculation and thus enhances the defocusing effect of the second photo.

[0014] In one possible embodiment, both the first and second images include people, that is, the captured scene includes people, and the images need to be blurred to highlight the people. When the ambient brightness is high, the electronic device uses binocular blur processing to obtain the first photo, and objects other than the people have a background blur effect. When the ambient brightness is low, the electronic device uses monocular blur processing to obtain the second photo, and objects other than the people have a background blur effect. In this way, the image processing method provided by this application can select a more suitable blur processing method based on changes in ambient brightness, thereby obtaining photos of higher quality.

[0015] In a second aspect, the present application provides an electronic device comprising a memory and a processor; the memory stores computer program code, the computer program code comprising computer instructions; one or more processors call the computer instructions so that the electronic device executes the image processing method of the first aspect above.

[0016] In a third aspect, the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the image processing method of the first aspect described above is implemented.

[0017] It can be seen from the above technical solution that this application has the following beneficial effects:

[0018] After the user starts the camera application, the electronic device can identify the ambient brightness and obtain a brightness index. The brightness index is negatively correlated with the ambient brightness, that is, the larger the brightness index, the lower the ambient brightness; then, after the user triggers the shooting function of the camera application, the brightness index can be compared with the first threshold. When the brightness index is less than or equal to the first threshold, it indicates that the ambient brightness is high. A binocular blur processing method can be used to blur the first image based on the first image captured by the first camera and the second image captured by the second camera to obtain a first photo; when the brightness index is greater than the first threshold, it indicates that the ambient brightness is low. A monocular blur processing method can be used to blur the first image based on the first image captured by the first camera to obtain a second photo.

[0019] In this way, when the ambient brightness changes, it is possible to adaptively switch between the binocular blur processing method and the monocular blur processing method. When the ambient brightness is high, the image quality captured by the camera is not affected, and the binocular blur processing method with better blur processing is adopted. When the ambient brightness is low, the image quality captured by the camera is affected. In order to avoid the superposition of the effects of the first image captured by the first camera and the second image captured by the second camera, or to avoid a greater impact on each other and affecting the blur effect, the monocular blur processing method can be adopted. This application can adopt a blur processing method with better blur effect in shooting environments with different brightness, which can improve the blur effect, obtain better quality photos, and thus enhance the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A schematic diagram of a blur effect for taking a photo provided by an embodiment of the present application;

[0021] Figure 2 An application scenario of an image processing method provided in an embodiment of the present application;

[0022] Figure 3 A schematic diagram of the system structure of an electronic device provided in an embodiment of the present application;

[0023] Figure 4a A signaling interaction diagram of the preview phase of an image processing method provided in an embodiment of the present application;

[0024] Figure 4b A signaling interaction diagram of the shooting phase of an image processing method provided in an embodiment of the present application;

[0025] Figure 5 A schematic diagram of a switching blur processing method provided in an embodiment of the present application;

[0026] Figure 6 A schematic diagram of a blur photography path provided in an embodiment of the present application;

[0027] Figure 7 A signaling interaction diagram of an image processing method provided in an embodiment of the present application;

[0028] Figure 8 A signaling interaction diagram of another image processing method provided in an embodiment of the present application;

[0029] Figure 9 A schematic diagram of the composition of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to make the description of the following embodiments clear and concise, the vocabulary involved in the embodiments of the present application is first explained. It should be understood that this explanation is for a clearer understanding of the embodiments of the present application and does not necessarily constitute a limitation of the embodiments of the present application.

[0031] Original image: Also known as a RAW image, it is an image obtained by converting an electrical signal obtained by an image sensor and then pre-processing it by an image signal processor. In some embodiments, the camera of an electronic device may include an image sensor and an image signal processor.

[0032] Zoom ratio: refers to the focal length range of a camera, also known as focal length. In some embodiments, the zoom ratio of an electronic device ranges from 0.5x to 30x, where x represents the fixed focal length of the electronic device.

[0033] Lux index: In this embodiment of the present application, it is used to indicate the ambient brightness of the shooting scene. A larger lux index indicates a darker environment. In this embodiment of the present application, the camera of the electronic device can sense the ambient brightness and then identify the lux index.

[0034] Primary camera: In the embodiments of this application, the raw image captured by the primary camera is primarily used to extract depth information and display it to the user. In some embodiments, it can also be referred to as the first camera, and the raw image captured by the primary camera can be referred to as the first image.

[0035] Auxiliary camera: In embodiments of this application, the raw image captured by the auxiliary camera is primarily used to extract depth information. Based on the extracted depth information, the electronic device can blur the raw image captured by the primary camera to produce a photo displayed to the user. In some embodiments, this can also be referred to as a second camera, and the raw image captured by the auxiliary camera can be referred to as the second image.

[0036] The following is a comparative explanation of the technical advantages of an image processing method, electronic device, and storage medium provided by this application, in conjunction with related technologies. For ease of understanding, an example scenario is used for illustration. In this example scenario, the electronic device may be a mobile phone.

[0037] In related technologies, to enhance the blur effect of photos taken with mobile phones, binocular blurring is often used to blur images captured by cameras (also known as photos). Binocular blurring refers to blurring based on the original images captured by two cameras. Generally speaking, binocular blurring can produce better blurring effects than monocular blurring (that is, blurring based on the original image captured by a single camera).

[0038] However, users may also need to use the blur function in shooting scenarios such as low-light scenes or sports scenes. In low-light scenes, the brightness of the shooting environment is relatively low. In sports scenes, in order to take clear photos, the phone will reduce the exposure to highlight the subject, and the brightness will also be relatively low. In the above scenarios, if binocular blur is used, for example, the original images collected by camera 1 and camera 2 are used for binocular blur processing, when the brightness is low, the image quality of camera 1 and camera 2 will be affected. The impact on the two may be superimposed or affect each other, which can easily affect the blur effect of the binocular blur processing.

[0039] For example, the background blur effect may be poor, such as Figure 1 As shown in (a), the blur effect of the photo is poor, there is no obvious difference between the background and the person in the photo, and the person in the photo cannot be highlighted; or the blur effect may be abnormal, such as the subject being blurred by mistake, etc. Figure 1 As shown in (b), the resulting photo has an abnormal blur effect. While the background is blurred, half of the subject's face is also mistakenly blurred, failing to clearly highlight the subject. As can be seen from the above, photos processed using blurring techniques in related technologies may exhibit poor or abnormal blurring, failing to highlight the subject and impacting the user experience.

[0040] Therefore, in order to solve the above problems, the embodiments of the present application provide an image processing method, an electronic device and a storage medium. In this method, a user starts a camera application, and the ambient brightness can be identified to obtain a brightness index. The lower the ambient brightness, the greater the brightness index. When the user triggers the shooting function of the camera application, it is determined that the identified brightness index is less than or equal to a first threshold value, indicating that the ambient brightness is high. Based on the first image captured by the first camera and the second image captured by the second camera, the first image is blurred to obtain a first photo, that is, the first image is blurred using a binocular blur processing method; when the user triggers the shooting function of the camera application, it is determined that the identified brightness index is greater than the first threshold value, indicating that the ambient brightness is low, and the image quality of the camera may be affected. In order to avoid the superposition of the influences on the original images captured by the two cameras, or to bring a greater influence to each other, resulting in an abnormal blur effect, the first image can be blurred based on the first image captured by the first camera to obtain a second photo, that is, the first image is blurred using a monocular blur processing method. In this way, based on the ambient brightness, the present application can adopt a more applicable blur processing method. Compared with the single blur processing method used in the prior art, the present application can improve the blur effect of the first photo or the second photo taken, improve the photo quality, and thus improve the user experience.

[0041] In order to make the technical personnel in this field understand the solution of this application more clearly, Figure 2 First, the application scenario of the technical solution of this application is described.

[0042] Still taking a mobile phone as an example, the application scenario of the image processing method provided in the embodiments of the present application is exemplified. In this scenario, the mobile phone may include a camera application, and the user can use the shooting function of the camera application to take pictures. The mobile phone may include multiple cameras, such as a main camera, a telephoto camera, and an ultra-wide-angle camera.

[0043] like Figure 2 As shown, after the mobile phone starts the camera application, the camera preview interface 210 is displayed. The camera preview interface 210 includes a zoom control 211, a shooting control 212, a shooting mode selection control 213 and a thumbnail display area 214. The zoom control 211 is used to adjust the zoom ratio of the camera application, the shooting control 212 is used to implement the shooting function of the camera application, the shooting mode selection control 213 is used to adjust the shooting mode of the camera application, and the thumbnail display area 214 is used to display thumbnails of the taken photos to the user.

[0044] In some embodiments, as Figure 2 As shown, the user can slide the shooting mode selection control 213 to the right to select the "large aperture" shooting mode. Using the mobile phone's "large aperture" shooting mode to shoot photos can highlight the subject and blur out irrelevant background objects, that is, blur processing can be performed in this shooting mode. The user can then slide the zoom control 211 to the right to increase the zoom ratio from 1x to 2.5x. In response to the user's operation of increasing the zoom ratio, the mobile phone's preview image's framing range becomes smaller.

[0045] It should be noted that the aforementioned "large aperture" shooting mode is only an example. The phone can also blur photos in other shooting modes, such as blurring portrait photos in "portrait" shooting mode. This application does not limit this. The aforementioned adjustment of the zoom ratio is also an example. The user can adjust the zoom ratio to another zoom ratio, or the phone can automatically adjust the zoom ratio of the camera application in certain specific shooting modes. This application does not limit this.

[0046] In some embodiments, while the camera application is running, the mobile phone can sense the ambient brightness and identify a brightness index. A larger brightness index represents a darker ambient brightness.

[0047] It should be understood that when the ambient brightness is bright, the camera's photosensitivity is almost unaffected. Therefore, compared with the monocular blurring method, the binocular blurring method can achieve a better blurring effect for the photo.

[0048] In a possible implementation, if the mobile phone recognizes that the ambient brightness is less than or equal to the brightness index threshold A (also referred to as the first threshold), it indicates that the ambient brightness is relatively bright. After the user presses the shooting control 212, the mobile phone can use binocular blurring to perform blurring. The mobile phone uses the main camera as the main camera and the telephoto camera as the auxiliary camera, and blurs the original images collected by the dual cameras to obtain the taken photos.

[0049] It should be noted that the above-mentioned use of the main camera as the main camera and the telephoto camera as the auxiliary camera is only an example. The telephoto camera can also be used as the main camera and the main camera as the auxiliary camera.

[0050] Understandably, in low ambient light conditions, camera image quality will be affected, and cameras with lower light sensitivity will be more affected. For example, if the telephoto camera has slightly inferior hardware and lower light sensitivity than the main camera, in such cases, if the phone still uses binocular blurring, the image quality of the telephoto camera will affect the depth information extracted from both the primary and auxiliary images. Inaccuracies in the extracted depth information will result in poor or abnormal blurring.

[0051] Therefore, in one possible implementation, if the mobile phone recognizes that the brightness index of the ambient brightness is greater than the brightness index threshold A, it indicates that the ambient brightness is dark. After the user presses the shooting control 212, the blurring photo path can use a monocular blurring method for blurring, that is, the mobile phone only extracts depth information based on the original image captured by the main camera, avoiding the abnormal influence of the original image captured by the telephoto camera. Compared with binocular blurring, photos with better blurring effect can be obtained.

[0052] like Figure 2 As shown, in some embodiments, the user can click on the shooting control 212 to trigger the shooting function of the camera application. In response to the user's click operation on the shooting control 212, the mobile phone recognizes that the brightness index is greater than the brightness index threshold A, then based on the original image captured by the main camera, it uses a monocular blur method to blur it to obtain a taken photo. After the monocular blur processing is completed, the mobile phone can display a thumbnail of the taken photo in the thumbnail display area 214. The user can then click on the thumbnail of the taken photo, and the mobile phone responds to the user's click operation to display the image viewing interface 220, which displays the taken photo to the user. It can be seen that compared to Figure 1 In (a), the blur effect of the photo is better, which can clearly highlight the characters. Figure 1 For example, in (b), the person is not blurred when taking the photo, and a clear portrait can be obtained.

[0053] It should be understood that in darker shooting environments, the camera's image quality will be more affected. In such cases, regardless of whether the mobile phone uses binocular or monocular blurring, the blurring effect may not be obvious. The image quality may also affect the extracted depth information, resulting in abnormal blurring effects.

[0054] Therefore, in one possible implementation, if the mobile phone recognizes that the ambient brightness is greater than the brightness index threshold B (also referred to as the second threshold), after the user presses the shooting control 212, the blurring photo path may not perform blurring processing, that is, the mobile phone does not need to extract depth information based on the original image, thereby avoiding abnormal effects caused by the original image captured by the telephoto camera or the main camera. Compared with the blurring processing used by the mobile phone, it can avoid obtaining photos with unclear or abnormal blurring effects.

[0055] As can be seen from the above, different blur processing methods have their own advantages in shooting environments with different brightness. The embodiment of the present application can adaptively switch between binocular blur, monocular blur, and no blur processing methods according to changes in external environment such as ambient brightness, thereby obtaining photos with better image quality and improving the user experience.

[0056] Next, the software structure of the electronic device is described by taking an electronic device running an Android system having a layered architecture as an example.

[0057] like Figure 3 As shown, the layered architecture divides the software structure into several layers, each with a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the Android system consists of the application layer, application framework layer, hardware abstraction layer, driver layer, and hardware layer, from top to bottom.

[0058] The application layer may include a series of application packages. In the embodiment of the present application, the application package may include application packages for applications such as camera and gallery.

[0059] The application framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions. In an embodiment of the present application, the application framework layer may include a camera access interface, which may include camera management and camera devices. The camera access interface is used to provide an application programming interface and programming framework for camera applications.

[0060] The hardware abstraction layer is an interface layer located between the application framework layer and the driver layer, providing a virtual hardware platform for the operating system. In the embodiment of the present application, the hardware abstraction layer may include a multi-camera module, a perception engine module, a decision module, and a processing engine module.

[0061] In some embodiments, the multi-camera module can provide virtual hardware for camera device 1, camera device 2, camera device 3, or more camera devices, with each camera device corresponding to a camera. In the embodiment of the present application, camera device 1 can correspond to the main camera, camera device 2 can correspond to the telephoto camera, and camera device 3 can correspond to the ultra-wide-angle camera. The multi-camera module can receive the brightness index sent by the perception engine module and then determine whether to obtain the original image from one camera or two cameras based on the brightness index.

[0062] In some embodiments, the perception engine module is used to send a brightness perception request to the camera device driver, obtain the brightness index recognized by the camera, and send the brightness index to the decision module and the multi-camera module respectively.

[0063] In some embodiments, the decision module is used to receive the brightness index sent by the perception engine module, decide on a blur processing method based on the brightness index, and send the decided blur processing method to the processing engine module.

[0064] In some embodiments, the processing engine module is used to perform image processing on the original image based on the blur processing method sent by the decision module to obtain a photograph.

[0065] Exemplarily, the processing engine module may include a RawUnpack encapsulation module, an AIRaw encapsulation module, a portrait enhancement encapsulation module, a depth algorithm encapsulation module, a blur algorithm encapsulation module, a beauty algorithm encapsulation module, and a JPEG encoding encapsulation module. In some embodiments, each module included in the processing engine module may be encapsulated with an algorithm for implementing the functions corresponding to each module.

[0066] The usage of each of the above modules can be found in the following embodiments and will not be described in detail here. It should be noted that the names of the modules are only examples and may be other names as long as the functions corresponding to the modules are realized. This application does not limit this.

[0067] The driver layer is the layer between hardware and software. The driver layer includes drivers for various hardware devices. The driver layer can include camera device drivers, etc.

[0068] In some embodiments, the camera device driver is used to drive the camera's photosensitive element to sense the ambient brightness and identify the brightness index. The camera device driver is also used to drive the camera's sensor (also known as an image sensor) to convert the image light signal into an image electrical signal, and drive the image signal processor to pre-process the image electrical signal to obtain the original image. In the embodiment of the present application, sensor 1 can be the sensor of the main camera, sensor 2 can be the sensor of the telephoto camera, and sensor 3 can be the sensor of the ultra-wide-angle camera.

[0069] Next, let's take a mobile phone as an electronic device. The mobile phone includes multiple cameras such as a telephoto camera and a main camera. The user uses the "large aperture" shooting mode of the mobile phone to shoot, and the main camera is the main camera and the telephoto camera is the auxiliary camera. Figure 3 The system structure of the electronic equipment shown, Figure 4a-Figure 8 The image processing method provided in the embodiment of the present application is introduced.

[0070] It should be noted that the above Figure 3 The camera access interface of the application framework layer and the camera device driver of the driver layer are described in the following embodiments in the form of text, not in Figure 4a 、 Figure 4b 、 Figure 7 and Figure 8 Shown in.

[0071] Example 1:

[0072] The image processing method may include a preview phase and a capture phase.

[0073] like Figure 4a As shown, the preview phase may include the following steps:

[0074] S401: The camera application receives a user's start-up operation for the camera application.

[0075] In some embodiments, the launch operation refers to a trigger operation in which the user launches a camera application of the mobile phone.

[0076] Exemplarily, the startup operation may include a user clicking on the icon of the camera application included in the main interface of the mobile phone, or a user clicking on the icon of the camera application on the lock screen interface of the mobile phone. As another example, the user has pre-set gesture instructions. For example, when the mobile phone is in the screen-off state, the startup operation may be a user drawing a circle on the mobile phone screen or double-clicking the screen. As another example, the user has pre-set voice instructions, and the startup operation includes the user making sounds such as "open camera" and "take a selfie". This application does not limit this.

[0077] S402: The camera application sends a camera application startup request to the multi-camera module.

[0078] In response to the user's start-up operation on the camera application, the camera application of the application layer calls the camera access interface of the application framework layer to start the camera application. That is, the camera application first sends a start-up request for the camera application to the camera access interface, and then the camera access interface sends the start-up request of the camera application to the multi-camera module of the hardware abstraction layer.

[0079] S403: The multi-camera module sends a call request to the perception engine module.

[0080] In some embodiments, after receiving a start request from a camera application, the multi-camera module may call the perception engine module to obtain a brightness index corresponding to the ambient brightness.

[0081] S404: The perception engine module sends a brightness perception request to the main camera.

[0082] In response to the call request sent by the multi-camera module, the perception engine module first sends a brightness perception request to the camera device driver of the driver layer, and the camera device driver then sends the brightness perception request to the main camera of the hardware layer, thereby driving the main camera to recognize the ambient brightness.

[0083] In some embodiments, the perception engine module may be encapsulated with an algorithm, which is used to drive the camera to recognize the ambient brightness through the camera device driver, that is, to perceive the changes in the ambient brightness in real time.

[0084] S405: The main camera identifies and obtains a brightness index based on the ambient brightness.

[0085] In some embodiments, the main camera is driven by a camera device, and its photosensitive element can sense the ambient brightness. The photosensitive element then obtains a brightness index corresponding to the environment based on the sensed ambient brightness.

[0086] In some other embodiments, the mobile phone may include an ambient light sensor that can sense the ambient brightness. The main camera interacts with the ambient light sensor to obtain the ambient brightness, and then obtains the brightness index based on the ambient brightness recognition.

[0087] S406: The main camera sends the brightness index to the perception engine module.

[0088] The main camera first sends the brightness index to the camera device driver, and the camera device driver then sends the brightness index corresponding to the environment to the perception engine module.

[0089] It should be noted that when the camera application is running, the main camera continuously senses the ambient brightness. Therefore, when the ambient brightness changes, the latest recognized brightness index can be continuously sent to the perception engine module. In this way, the perception engine module can obtain the corresponding brightness of the environment in real time, providing a basis for subsequent selection of blur processing methods.

[0090] S407: The perception engine module sends the brightness index to the multi-camera module.

[0091] S408: The multi-camera module determines that the brightness index is less than or equal to the brightness index threshold A, and sends a request to obtain the original image to the main camera and the telephoto camera respectively.

[0092] The multi-camera module first sends a request to the camera device driver for obtaining the original images of the two cameras, so that the camera device driver drives the main camera and the telephoto camera to collect the original images respectively.

[0093] It should be understood that when the ambient brightness is high, the image quality of the camera is higher. At this time, the binocular blur processing method can obtain an image with a better blur effect.

[0094] In some embodiments, the multi-camera module may be configured to compare the brightness index with a brightness index threshold A.

[0095] The brightness index threshold A is a preset index threshold. The larger the brightness index, the darker the ambient brightness.

[0096] In some embodiments, as Figure 5 As shown, when the zoom ratio of the camera application is 2.5x, the brightness index threshold A can be 300. At this time, the multi-camera module determines that the brightness index sent by the perception engine module is less than or equal to 300, indicating that binocular blur processing is required, and the original images captured by the main camera and the auxiliary camera need to be obtained.

[0097] In some embodiments, as Figure 5 As shown, when the zoom ratio of the camera application is 5x, the brightness index threshold A is 355. At this time, the multi-camera module needs to determine whether the brightness index is less than or equal to 355 before continuing to execute subsequent steps.

[0098] It should be noted that the different values ​​of the brightness index threshold A under different zoom ratios are only examples and can also be the same value. This application does not limit this.

[0099] S409: The main camera captures and obtains an original image stream A.

[0100] In response to the drive of the camera device, that is, the camera device drives the sensor of the main camera to convert the image light signal into an image electrical signal, and drives the image signal processor to pre-process the image electrical signal to obtain the original image. During this process, the main camera continues to work, so the main camera can capture the original image stream A including multiple frames of original images.

[0101] S410: The telephoto camera captures and obtains an original image stream B.

[0102] The camera device drives the sensor and image signal processor of the telephoto camera to obtain a raw image. The telephoto camera can collect a raw image stream B including multiple frames of raw images.

[0103] In some embodiments, there is no restriction on the execution order of S409 and S410.

[0104] S411: The main camera sends the original image stream A to the processing engine module.

[0105] In some embodiments, as Figure 6 As shown, the processing engine module can be understood as the blur photography channel, which is used to perform various image processing on the original image output by the camera, such as image format conversion, image enhancement, and image blur, to obtain the captured photo that can be displayed to the user. Therefore, the main camera needs to send the captured original image stream A to the processing engine module.

[0106] S412: The telephoto camera sends the original image stream B to the processing engine module.

[0107] In some embodiments, there is no restriction on the execution order of S411 and S412.

[0108] S413: The perception engine module sends the brightness index to the decision module.

[0109] In some embodiments, the decision module is configured to decide a blurring processing method to be used based on a brightness index of the ambient brightness.

[0110] It should be noted that this application does not limit the execution order of S407-S412 and S413. S407 and S413 may be executed simultaneously, and then S408-S412 may be executed, or S413 may be executed first, and then S407-S412 may be executed.

[0111] like Figure 4b As shown, the shooting stage may include the following steps:

[0112] S414: The camera application receives a user trigger operation for a shooting function.

[0113] After the user confirms that the camera of the mobile phone is aimed at the object to be photographed through the preview image displayed on the mobile phone screen, the shooting function of the camera application can be triggered.

[0114] In some embodiments, as Figure 2 As shown, the triggering operation of the shooting function may be a click operation of the user on the shooting control 212 .

[0115] In some embodiments, the triggering operation of the photo-taking function may also be a user's voice command or gesture command to the camera application, which can trigger the camera application to take a photo. This application does not limit this.

[0116] S415: The camera application sends a shooting request to the decision module.

[0117] The camera application sends a shooting request to the decision module through the camera access interface.

[0118] S416: The decision module determines that the brightness index is less than or equal to the brightness index threshold A, and sends a binocular blur processing request to the processing engine module.

[0119] In some embodiments, the decision module may be configured to compare the brightness index with a brightness index threshold A.

[0120] Based on the above introduction, when the ambient brightness is high, a binocular blur processing method can be adopted. That is, in response to a shooting request, when the decision module determines that the received brightness index is less than or equal to the brightness index threshold A, the decision module can decide that the blur processing method to be used is a binocular blur processing method, so the decision module can send a binocular blur processing request to the processing engine module.

[0121] In some embodiments, as Figure 5 As shown, when the camera application's zoom ratio is 2.5x, the decision module determines that the brightness index is less than or equal to 300, and then a binocular blur processing request can be sent to the processing engine module. When the camera application's zoom ratio is 5x, the decision module determines that the brightness index is less than or equal to 355, and then a binocular blur processing request can also be sent to the processing engine module.

[0122] S417: The processing engine module performs binocular blur processing based on the received original image stream A and the received original image stream B to obtain a photograph.

[0123] Based on the introduction of the above steps, the processing engine module receives the original image stream A captured by the main camera and the original image stream B captured by the telephoto camera, and responds to the binocular blur processing request sent by the decision module. At this time, the processing engine module can perform binocular blur processing based on the original image stream A and the original image stream B to obtain the taken photo.

[0124] In some embodiments, as Figure 3 As shown, the processing engine module may include a RawUnpack encapsulation module, an AIRaw encapsulation module, a portrait enhancement encapsulation module, a depth algorithm encapsulation module, a blur algorithm encapsulation module, a beauty algorithm encapsulation module, and a jpeg encoding encapsulation module. Figure 6 As shown, after the shooting function of the camera application is triggered, the blurring photography path starts to work, that is, the various modules included in the processing engine module start to process the original image.

[0125] In a possible implementation, the subject of the shooting scene may be a person. Take a frame image 1 in the original image stream A and a frame image 2 in the original image stream B as an example. Figure 6 The process of binocular blur processing is introduced.

[0126] The original image is a compressed image, so image 1 and image 2 must first be input into the RawUnpack encapsulation module, which can decompress image 1 and output image 3, and also decompress image 2 and output image 4.

[0127] Image 3 and Image 4 are input into the AIRaw packaging module, which can convert the image formats, respectively converting the image formats of Image 3 and Image 4 from RAW (which can be called the first image format) to YUV (which can be called the second image format), and outputting Image 5 and Image 6. The AIRaw packaging module can also extract and output the character contour information from Image 3.

[0128] Subsequent depth calculations require the original image captured by the auxiliary camera, so subsequent transmissions must include decompressed Image 4. Images 4, 5, 6, and the person's outline information are input into the portrait enhancement encapsulation module, which performs character enhancement processing on Image 5 based on the outline information and outputs Image 7.

[0129] Image 4, image 6, image 7 and the character contour information are input into the depth algorithm encapsulation module, which is used to use the encapsulated depth algorithm to perform alignment and depth calculation based on image 4, image 6, image 7 and the character contour information to output binocular depth information (also called first depth information).

[0130] The binocular depth information and image 7 are input into the blurring algorithm encapsulation module, which is used to perform binocular blurring processing on image 7 based on the binocular depth information using the encapsulated blurring algorithm to obtain image 8.

[0131] The image 8 is input to the beautification algorithm encapsulation module, which is used to use the encapsulated beautification algorithm, such as the skin beautification algorithm, the eye enlargement algorithm, etc., to beautify the person in the image 8 and output the image 9.

[0132] The image 9 is then input into the jpeg encoding and packaging module, which is used to convert the image format of the image 9 from YUV to jpeg, and output the captured photo so as to be displayed to the user.

[0133] It should be noted that the various modules included in the above-mentioned processing engine module are only examples. The processing engine module can also include other modules. When there are no people in the shooting scene, it can also pass through the above-mentioned portrait enhancement encapsulation module and beauty algorithm encapsulation module and other modules related to the person, but no processing is required, and it can also not pass through the modules related to the person. This application does not limit this.

[0134] In some embodiments, the algorithms supported by the AIRaw package module may be different at different zoom ratios, and a specific algorithm may be used for a specific zoom ratio to further improve image quality.

[0135] Exemplarily, the AIRaw encapsulation module can encapsulate the ellip algorithm, the pLite algorithm, the Mef algorithm, the QuadraSR algorithm, and the QuadraSRLite algorithm.

[0136] In one possible implementation, when the camera's zoom ratio is 2.5x, the AIRaw package module can support the ellip algorithm, the pLite algorithm, and the Mef algorithm. The AIRaw package module can determine the brightness level of the shooting environment based on the brightness index. If the ambient brightness is determined to be medium, the Mef algorithm can be used to improve image quality. If the ambient brightness is determined to be low (i.e., in a dark environment), the ellip algorithm and the pLite algorithm can be used to improve image quality.

[0137] In one possible implementation, when the zoom ratio of the camera application is 5x, the AIRaw encapsulation module can support the ellip algorithm, the pLite algorithm, the Mef algorithm, the QuadraSR algorithm, and the QuadraSRLite algorithm. The usage of the ellip algorithm, the pLite algorithm, and the Mef algorithm can be found in the above examples and will not be repeated here. The AIRaw encapsulation module can also determine whether the QuadraSR algorithm and the QuadraSRLite algorithm need to be used based on the zoom ratio of the camera application. For example, if a magnification threshold is set, and the zoom ratio of the camera application is greater than or equal to the magnification threshold, these two methods are used. In an embodiment of the present application, when the zoom ratio is 5x, the QuadraSR algorithm and the QuadraSRLite algorithm can be used to improve image quality.

[0138] In some embodiments, the above-mentioned method of improving image quality may include performing denoising processing, improving clarity processing, etc. on the image, which is not limited in this application. Based on the image characteristics under different ambient brightness and different zoom factors, different algorithms can be used to process the image in a targeted manner to obtain higher quality images.

[0139] S418: The processing engine module sends the captured photo to the camera application.

[0140] The processing engine module sends the captured photos to the camera application through the camera access interface.

[0141] S419: The camera application displays thumbnails of the captured photos.

[0142] The camera application displays a thumbnail of the captured photo (also referred to as the first photo) to the user.

[0143] In some embodiments, as Figure 2 As shown, after the user clicks the shooting control 312, the mobile phone displays a thumbnail of the taken photo in the thumbnail display area 214. When the user clicks the thumbnail, the image viewing interface 220 can display the taken photo.

[0144] Example 2:

[0145] like Figure 7 As shown, the image processing method provided in the embodiment of the present application may include the following steps: S707-S717.

[0146] It should be noted that the image processing method provided in the embodiment of the present application also includes the following steps: S701-S706, which are not included in Figure 7 As shown in FIG, the implementation of S701 to S706 can refer to the implementation of S401 to S406 in the above embodiment, and will not be repeated here.

[0147] S707: The perception engine module sends the brightness index to the multi-camera module.

[0148] The implementation of S707 can refer to the implementation of S407 in the above embodiment, and will not be repeated here.

[0149] S708: The multi-camera module determines that the brightness index is greater than the brightness index threshold A, and sends a request to obtain the original image to the main camera.

[0150] It should be understood that when the ambient brightness is low, the image quality of the camera will be affected. When the binocular blur processing method is still used, the influences received by the main camera and the auxiliary camera will be superimposed or interfere with each other, which may easily result in poor or abnormal blur effects. Therefore, it is more advantageous to use a monocular blur processing method at this time. Compared with the binocular blur processing method, the monocular blur processing method can obtain an image with better blur effect.

[0151] In an embodiment of the present application, the larger the brightness index, the darker the ambient brightness. When the multi-camera module determines that the brightness index is greater than the brightness index threshold A, a request to obtain the original image can be sent only to the main camera, that is, the main camera.

[0152] In some embodiments, as Figure 5 As shown, when the camera's zoom ratio is 2.5x, the multi-camera module determines that the brightness index sent by the perception engine module is greater than 300, indicating that monocular defocusing is required and the original image captured by the primary camera needs to be obtained. When the camera's zoom ratio is 5x, the multi-camera module determines that the brightness index sent by the perception engine module is greater than 355, and the subsequent steps can be continued.

[0153] S709: The main camera captures and obtains an original image stream A.

[0154] S710: The main camera sends the original image stream A to the processing engine module.

[0155] S711: The perception engine module sends the brightness index to the decision module.

[0156] The implementation of S709-S711 can refer to the implementation of S409, S411 and S413 in the above embodiments respectively, and will not be repeated here.

[0157] It should be noted that this application does not limit the execution order of S707-S710 and S711. S707 and S711 may be executed simultaneously, and then S708-S710 may be executed, or S711 may be executed first, and then S707-S710 may be executed.

[0158] S712: The camera application receives a user trigger operation for a shooting function.

[0159] S713: The camera application sends a shooting request to the decision module.

[0160] The implementation of S712-S713 can refer to the implementation of S414-S415 in the above embodiment respectively, and will not be repeated here.

[0161] S714: The decision module determines that the brightness index is greater than the brightness index threshold A, and sends a monocular blur processing request to the processing engine module.

[0162] Based on the above introduction, when the ambient brightness is low, a monocular blur processing method can be adopted. That is, in response to a shooting request, when the decision module determines that the received brightness index is greater than the brightness index threshold A, the decision module can decide that the blur processing method to be used is a monocular blur processing method, so the decision module can send a monocular blur processing request to the processing engine module.

[0163] In some embodiments, as Figure 5 As shown, when the camera's zoom ratio is 2.5x, the decision module determines that the brightness index is greater than 300, and then a monocular defocusing processing request can be sent to the processing engine module. When the camera's zoom ratio is 5x, the decision module determines that the brightness index is greater than 355, and then a monocular defocusing processing request can be sent to the processing engine module.

[0164] S715: The processing engine module performs monocular blur processing based on the received original image stream A to obtain a photograph.

[0165] Based on the introduction of the above steps, the processing engine module receives the original image stream A captured by the main camera. In response to the monocular blur processing request sent by the decision module, the processing engine module can perform monocular blur processing based on the original image stream A to obtain the taken photo.

[0166] Based on the above example, the subject of the photo is a person, and the processing engine module is as follows: Figure 3 Taking the various modules shown as an example, the blurring photographing path starts working at this time. The difference from the above example is that the blurring photographing path no longer needs to process the original image stream B collected by the telephoto camera.

[0167] Taking a frame of image in the original image stream A as an example, the RawUnpack encapsulation module, AIRaw encapsulation module, portrait enhancement encapsulation module, beauty algorithm encapsulation module, and JPEG encoding encapsulation module perform the same work on the frame of image, which will not be repeated here.

[0168] Different from the above embodiments, the depth algorithm encapsulation module is used to use the encapsulated depth algorithm to perform segmentation and depth calculation based on the input image to obtain monocular depth information (also called second depth information); the blurring algorithm encapsulation module is used to use the encapsulated blurring algorithm to perform monocular blurring on the input image based on the monocular depth information.

[0169] In some embodiments, when executing binocular blur processing and monocular blur processing, the depth algorithms used may be different, that is, the depth algorithm encapsulation module may encapsulate depth algorithm 1 suitable for binocular blur and depth algorithm 2 suitable for monocular blur.

[0170] It should be noted that other implementations of S715 can refer to the implementation of S417 in the above embodiment, and will not be repeated here.

[0171] S716: The processing engine module sends the captured photo to the camera application.

[0172] S717: The camera app displays thumbnails of captured photos.

[0173] In this embodiment of the present application, the taken photo may also be referred to as a second photo.

[0174] The implementation of S716-S717 can refer to the implementation of S418-S419 in the above embodiment respectively, and will not be repeated here.

[0175] Example 3:

[0176] like Figure 8 As shown, the image processing method provided in the embodiment of the present application may include the following steps: S807-S817.

[0177] It should be noted that the image processing method provided in the embodiment of the present application also includes the following steps: S801-S806, which are not included in Figure 8 As shown in FIG, the implementation of S801 to S806 can refer to the implementation of S401 to S406 in the above embodiment, and will not be repeated here.

[0178] S807: The perception engine module sends the brightness index to the multi-camera module.

[0179] S808: The multi-camera module determines that the brightness index is greater than the brightness index threshold B, and sends a request to obtain the original image to the main camera.

[0180] It should be understood that when the ambient brightness is extremely low, that is, in an extremely dark environment, the quality of the camera's image output will be greatly affected. At this time, if the image is blurred, the blurring effect may not be obvious and the subject cannot be highlighted. It is also easy to cause abnormal blurring effects. Therefore, it is not appropriate to blur the image at this time.

[0181] Based on the introduction of the above embodiment, the larger the brightness index, the darker the ambient brightness. When the multi-camera module determines that the brightness index is greater than the brightness index threshold B, a request to obtain the original image can be sent only to the main camera, that is, the main camera.

[0182] In some embodiments, as Figure 5 As shown, when the zoom ratio of the camera application is 2.5x or 5x, the multi-camera module determines that the brightness index sent by the perception engine module is greater than 370, indicating that it is not appropriate to blur the image at this time. It is necessary to obtain the original image captured by the main camera and perform normal image processing to obtain the photo displayed to the user.

[0183] It should be noted that, when the camera application has different zoom ratios, different brightness index thresholds B may also be set, and this application does not limit this.

[0184] S809: The main camera captures and obtains an original image stream A.

[0185] S810: The main camera sends the original image stream A to the processing engine module.

[0186] S811: The perception engine module sends the brightness index to the decision module.

[0187] The implementation of S809-S811 can refer to the implementation of S409, S411 and S413 in the above embodiments respectively, and will not be repeated here.

[0188] It should be noted that this application does not limit the execution order of S807-S810 and S811. S807 and S811 may be executed simultaneously, and then S808-S810 may be executed, or S811 may be executed first, and then S807-S810 may be executed.

[0189] S812: The camera application receives a user trigger operation for a shooting function.

[0190] S813: The camera application sends a shooting request to the decision module.

[0191] The implementation of S812-S813 can refer to the implementation of S414-S415 in the above embodiment respectively, and will not be repeated here.

[0192] S814: The decision module determines that the brightness index is greater than the brightness index threshold B, and sends a processing request without blurring to the processing engine module.

[0193] Based on the above introduction, when the ambient brightness is extremely low, a processing method that does not require blurring can be adopted. That is, in response to a shooting request, when the decision module determines that the received brightness index is greater than the brightness index threshold B, the decision module can decide that the blurring processing method to be used is a processing method that does not require blurring. Therefore, the decision module can send a processing request that does not require blurring to the processing engine module.

[0194] In some embodiments, as Figure 5As shown, when the zoom ratio of the camera application is 2.5x or 5x, the decision module determines that the brightness index is greater than 370, and a processing request without blurring can be sent to the processing engine module.

[0195] S815: The processing engine module performs image processing based on the received original image stream A to obtain a photograph.

[0196] Based on the introduction of the above steps, the processing engine module receives the original image stream A captured by the main camera, and responds to the processing request without blurring sent by the decision module. At this time, the processing engine module can perform image processing based on the original image stream A, that is, perform image processing without blurring to obtain the taken photo.

[0197] Based on the above example, the subject of the photo is a person, and the processing engine module is as follows: Figure 3 The blurring photographing path starts working. Unlike the above example, the blurring photographing path does not need to process the original image stream B captured by the telephoto camera, nor does it need to perform blurring processing on the original image stream A.

[0198] Taking a frame from raw image stream A as an example, the RawUnpack, AIRaw, portrait enhancement, beauty, and JPEG encoding modules perform the same tasks for this frame, and are not detailed here. The difference is that the image still passes through the depth and blur algorithms, but these two modules do not process the input image.

[0199] It should be noted that other implementations of S815 can refer to the implementation of S417 in the above embodiment, and will not be repeated here.

[0200] S816: The processing engine module sends the captured photo to the camera application.

[0201] S817: The camera application displays thumbnails of the captured photos.

[0202] In this embodiment of the present application, the taken photo may also be referred to as a third photo.

[0203] The implementation of S816-S817 can refer to the implementation of S418-S419 in the above embodiment respectively, and will not be repeated here.

[0204] In addition, in some embodiments, when the zoom ratio of the camera application is 1x, the main camera can be used as the primary camera, and the ultra-wide-angle camera can be used as the auxiliary camera to perform the various steps described in the above embodiments.

[0205] In other embodiments, a magnification threshold can be set in advance. When the zoom magnification of the camera application is greater than the magnification threshold, for example, when the zoom magnification is 5x, the telephoto camera can be used as the main camera and the main camera can be used as the auxiliary camera to execute the various steps described in the above embodiments. This application does not limit this.

[0206] Next, the composition of the electronic device is introduced.

[0207] It should be noted that the electronic device in the above embodiment is a mobile phone for illustrative purposes only. In some embodiments, the electronic device may be a tablet computer, a wearable device, an in-vehicle device, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), or other terminal device. This application does not impose any particular restrictions on the specific form of the above electronic device, as long as it can realize the camera function.

[0208] like Figure 9 As shown, the electronic device 900 may include a processor 910, an internal memory 920, a camera 930, a display screen 940, and a sensor module 950. In some embodiments, the sensor module 950 may include an ambient light sensor.

[0209] It should be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device. In other embodiments, the electronic device may include more or fewer components than shown, or may combine or separate certain components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0210] The processor 910 may include one or more processing units. For example, the processor 910 may include an application processor (AP), a graphics processing unit (GPU), an image signal processor (ISP), a digital signal processor (DSP), etc. The different processing units may be independent devices or integrated into one or more processors.

[0211] The processor 910 may also be provided with a memory for storing instructions and data.

[0212] The internal memory 920 can be used to store computer executable program codes, which include instructions. The processor 910 executes the instructions stored in the internal memory 920 to execute various functional applications and data processing of the electronic device 900.

[0213] In some embodiments, the internal memory 920 stores instructions for executing the image processing method. The processor 910 can implement the image processing method provided in the embodiment of the present application by executing the instructions stored in the internal memory 920.

[0214] Electronic device 900 implements display functions through an image processor, display screen 940, and an application processor. The image processor is a microprocessor for image processing and is connected to display screen 940 and the application processor. The image processor is used to perform mathematical and geometric calculations for graphics rendering. Processor 910 may include one or more image processors that execute program instructions to generate or modify display information. Display screen 940 is used to display images, videos, etc.

[0215] In some embodiments, the display screen 940 is used to display the running interface of the camera application of the electronic device 900, such as a camera preview interface, an image viewing interface, etc.

[0216] The electronic device 900 can implement a shooting function through an ISP, a camera 930, a video codec, an image processor, a display screen 940, and an application processor.

[0217] The ISP processes data fed back by the camera 930. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and converted into a visible image. The ISP can also perform algorithmic optimization for image noise, brightness, and skin tone. It can also optimize parameters such as exposure and color temperature of the captured scene.

[0218] In some embodiments, the ISP can be set in the camera 930. The camera 930 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element (i.e., sensor). The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the image light signal into an image electrical signal, and then passes the image electrical signal to the ISP for conversion into a digital image signal (i.e., the original image). The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in standard RGB, YUV, and other formats.

[0219] In some embodiments, the electronic device 900 may include N cameras 930 , where N is a positive integer greater than 1. For example, the electronic device 900 may include a telephoto camera, a main camera, an ultra-wide-angle camera, and the like.

[0220] The ambient light sensor is used to sense ambient brightness. In some embodiments, the ambient light sensor is used to interact with the camera 930 so that the camera 930 can obtain a brightness index based on ambient brightness recognition.

[0221] An embodiment of the present application further provides a computer-readable storage medium storing a computer program, which, when executed by a computer, can implement one or more steps in any of the above-mentioned image processing methods.

[0222] The computer readable storage medium may be a non-transitory computer readable storage medium, for example, a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0223] Another embodiment of the present application further provides a computer program product comprising instructions, which, when executed by a computer, can implement one or more steps in any of the above-mentioned image processing methods.

[0224] The electronic device, computer-readable storage medium, and computer program product provided in this embodiment are all used to execute the corresponding image processing methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding image processing methods provided above, and will not be repeated here.

[0225] The terms "first", "second" and "third" in the specification, claims and drawings of this application are used to distinguish different objects rather than to limit a specific order.

[0226] In the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0227] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An image processing method, characterized in that: include: In response to a user's operation of starting a camera application, identifying an ambient brightness to obtain a brightness index; wherein the brightness index is negatively correlated with the ambient brightness; Determining a first zoom ratio of the camera application, and determining, based on a correspondence between multiple zoom ratios and multiple thresholds, a threshold corresponding to the first zoom ratio as a first threshold; In response to the user adjusting the first zoom ratio, determining an adjusted zoom ratio for the camera application, and setting a threshold value corresponding to the adjusted zoom ratio as the first threshold value based on a correspondence between the plurality of zoom ratios and a plurality of threshold values; In response to the user triggering an operation of a shooting function of the camera application, determining that the brightness index is less than or equal to the first threshold, decompressing the first image and the second image respectively through the RawUnpack encapsulation module, format-converting the decompressed first image and the decompressed second image respectively through the AIRaw encapsulation module to obtain a first image in a second image format and a second image in a second image format, extracting object contour information from the decompressed first image through the AIRaw encapsulation module, and encapsulating the decompressed second image, the first image in the second image format, the second image in the second image format, and the object through the enhanced encapsulation module. performing object enhancement processing on the contour information to obtain an enhanced image, performing depth calculation on the second image in the first image format, the second image in the second image format, the enhanced image, and the object contour information through a depth algorithm encapsulation module to obtain first depth information, performing binocular defocusing processing on the enhanced image and the first depth information through a defocusing algorithm encapsulation module to obtain a defocused image, performing beautification processing on the defocused image through a beautification algorithm encapsulation module to obtain a beautified image, and performing format conversion on the beautified image through a jpeg encoding encapsulation module to obtain a first photo; the first image is obtained based on the first camera, and the second image is obtained based on the second camera; In response to the user triggering an operation of a shooting function of the camera application, determining that the brightness index is greater than the first threshold, decompressing the first image through the RawUnpack encapsulation module, performing format conversion on the decompressed first image through the AIRaw encapsulation module to obtain a first image in a second image format, extracting object contour information from the decompressed first image through the AIRaw encapsulation module, performing object enhancement processing on the first image in the second image format and the object contour information through the enhancement encapsulation module to obtain an enhanced image, performing depth calculation on the enhanced image and the object contour information through the depth algorithm encapsulation module to obtain second depth information, performing monocular defocusing processing on the enhanced image and the second depth information through the defocusing algorithm encapsulation module to obtain a defocused image, performing beauty processing on the defocused image through the beauty algorithm encapsulation module to obtain a beautified image, and converting the format of the beautified image through the JPEG encoding encapsulation module to obtain a second photo; In response to the user triggering the shooting function of the camera application, it is determined that the brightness index is greater than a second threshold, and the first image is retouched to obtain a third photo; the second threshold is greater than the first threshold; the third photo does not have a background blur effect.

2. The method according to claim 1, characterized in that The first image and the second image both include a person; other objects except the person in the first photo have a background blur effect; other objects except the person in the second photo have a background blur effect.

3. An electronic device, characterized in that: including memory and processor; The memory is coupled to the processor, and the memory is used to store computer program code, where the computer program code includes computer instructions. One or more of the processors call the computer instructions to enable the electronic device to execute the image processing method according to any one of claims 1-2.

4. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the image processing method according to any one of claims 1 to 2 is implemented.

Images