Image processing method and electronic device

Abstract

The application discloses an image processing method and an electronic device, and belongs to the technical field of image processing. The method comprises the following steps: in a preview state, a highlight algorithm is used to identify whether a shooting scene is a preset typical scene in real time; a first shooting channel is used for shooting in the typical scene, and a second shooting channel is used for shooting in a non-typical scene, so that dynamic switching of the shooting channel can be realized. Compared with a traditional snapshot mode which cannot realize variable exposure processing, the embodiment of the application can realize variable exposure processing on the premise that the snapshot mode has the snapshot function, so that better shooting functions are provided for dealing with complex shooting environments (for example, high dynamic shooting scenes or dark scenes), the image effect of a non-highlight picture in the snapshot mode is improved, and the shooting experience of a user is improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and more specifically, to an image processing method and an electronic device. Background Technology

[0002] With the rapid development of smart devices, camera functionality has become an essential feature. Users' demands and experiences regarding camera (photos and / or videos) capabilities on smart devices are constantly increasing. Smart devices offer users different shooting modes, such as snapshot mode. Currently, the image quality obtained in snapshot mode is poor, resulting in unsatisfactory images and negatively impacting the user's shooting experience. Summary of the Invention

[0003] In view of this, this application provides an image processing method, an electronic device, a computer-readable storage medium, and a computer program product, which can improve the image effect in snapshot mode and help improve the user's shooting experience.

[0004] In a first aspect, an image processing method is provided, including: displaying a photo preview interface;

[0005] The content in the photo preview interface is identified to obtain the identification result, which is used to indicate whether the current shooting scene is a preset typical scene;

[0006] Receive the user's first operation, which is used to trigger taking a picture;

[0007] In response to the first operation, when the recognition result is a first scene, a first image-taking path is used to take a picture, where the first scene is a preset typical scene;

[0008] In response to the first operation, when the recognition result is a second scene, a second photographing path is used to take a picture. The second scene is an atypical scene, and the first photographing path is different from the second photographing path.

[0009] The above method can be executed by an electronic device or a chip within an electronic device. Based on the above scheme, by real-time identification of whether the shooting scene is a preset typical scene, and by using a first shooting path to take pictures in typical scenes and a second shooting path to take pictures in atypical scenes, dynamic switching of the shooting path can be achieved. Furthermore, compared to traditional snapshot modes that cannot achieve variable exposure processing, the embodiments of this application enable the snapshot mode to achieve variable exposure processing while retaining the snapshot function, thus handling high dynamic range shooting scenes or low-light scenes. This provides better shooting capabilities for complex shooting environments (such as high dynamic range shooting scenes or low-light scenes), improves the image quality of non-highlight scenes in snapshot mode, and helps to enhance the user's shooting experience.

[0010] In some embodiments, the photo preview interface is a photo preview interface in the snapshot mode. Optionally, displaying the photo preview interface includes: displaying the photo preview interface in response to the user's selection of the snapshot mode, wherein the photo preview interface is a preview interface in the snapshot mode.

[0011] Therefore, in capture mode, the image processing method of this application embodiment can dynamically switch the shooting path to achieve better shooting results. For example, in eagle eye capture mode, if the current shooting scene is a low-light scene, the user does not need to manually switch to another shooting mode; that is, a high-quality photo can be obtained while still in the current eagle eye capture mode, thus improving the user experience.

[0012] In this application embodiment, different shooting paths are used for different shooting scenarios, namely typical scenarios and atypical scenarios, in order to improve the shooting effect. The specific implementation methods of different shooting paths are described below.

[0013] In one possible implementation, a first image capture path is used to capture an image, including: obtaining first image data from a first cache, wherein the first image data is data of a selected image frame cached in the first cache when the first operation is received; performing image processing on the first image data through a preprocessing algorithm module to obtain second image data; and performing image processing on the second image data through a first postprocessing algorithm module to output the first captured image frame.

[0014] Therefore, when a photo request is received, if the shooting scene is determined to be a typical scene, then the photo is taken based on the first photo path, the cached high-quality frames are retrieved from the cache, and the high-quality frames are sent to the photo algorithm module for further image processing, and finally the first photo image frame is obtained, which helps to improve the image quality.

[0015] In the embodiments of this application, for cases where the shooting scene is a typical scene, a corresponding post-processing algorithm module is also designed, such as a first post-processing algorithm module.

[0016] Optionally, image processing of the second image data is performed by the first post-processing algorithm module, including: when using a wide-angle camera or a telephoto camera, calling a first algorithm link to perform image processing on the second image data; or, when using a main camera, calling a first algorithm link, a second algorithm link, or a third algorithm link to perform image processing on the second image data; wherein, in the first algorithm link, image fusion is performed based on multiple standard frames; and in the second algorithm link, image fusion is performed based on multiple standard frames and multiple short frames.

[0017] Therefore, when taking photos using different cameras, the corresponding algorithm chain is invoked to perform post-processing on the images, thereby further optimizing image quality and improving image effects.

[0018] In one possible implementation, a second image capture path is used to take a picture, including: obtaining third image data from a first buffer, wherein the third image data is image data corresponding to a variable exposure image frame determined when the first operation is received, the variable exposure image frame refers to an image frame generated based on a first exposure parameter issued by the auto exposure (AE) module, the first exposure parameter being different from the exposure parameter corresponding to the preview frame; performing image processing on the third image data through a preprocessing algorithm module to obtain fourth image data; and performing image processing on the fourth image data through a second postprocessing algorithm module to output a second captured image frame.

[0019] Therefore, when a photo request is received, if the shooting scene is determined to be atypical, the second shooting path is used to take the photo. When there is a need for variable exposure, variable exposure processing is performed to obtain the second photo image frame, which helps to improve image quality.

[0020] For example, multiple image frames are selected from the image capture queue of the preview image frames. These multiple image frames are the image frames in the capture queue that are closest to the capture command. The AE module sends variable exposure parameters (e.g., a first exposure parameter) to the lower layers (e.g., sequentially passed from the frame layer, HAL layer, and driver layer to the sensor). The first exposure parameter is different from the exposure parameter corresponding to the preview image frame. The sensor applies the variable exposure parameters sent by the AE module and outputs multiple variable exposure frames. The sensor sends the multiple variable exposure frames to the image capture algorithm module (e.g., a second post-processing algorithm module) and finally outputs the second captured image frame.

[0021] In one possible implementation, recognizing the content in the photo preview interface and obtaining the recognition result includes: performing real-time recognition of the content in the photo preview interface using the Excellent Algorithm module to obtain the recognition result.

[0022] In one possible implementation, a typical scenario can be an exciting scenario; an atypical scenario is a non-exciting scenario. Optionally, the preset typical scenario includes any of the following scenarios: shooting a basketball, splashing water, smiling, jumping, flowers, and pets.

[0023] In a second aspect, an electronic device is provided, including a unit for performing any of the methods in the first aspect. The electronic device may be a terminal or a chip within a terminal. The electronic device includes a communication unit, a display unit, and a processing unit.

[0024] When the electronic device is a terminal, the processing unit may be a processor, the communication unit may be a communication interface, and the display unit may be a graphics processing module and a screen; the terminal may also include a memory for storing computer program code, which, when the processor executes the computer program code stored in the memory, causes the terminal to perform any of the methods in the first aspect.

[0025] When the electronic device is a chip within a terminal, the processing unit can be a logic processing unit inside the chip, the communication unit can be a communication interface, pins, or circuits, and the display unit can be a graphics processing unit inside the chip. The chip may also include a memory, which can be memory within the chip (e.g., registers, caches, etc.) or memory located outside the chip (e.g., read-only memory, random access memory, etc.). The memory is used to store computer program code, and when the processor executes the computer program code stored in the memory, it causes the chip to execute any of the methods in the first aspect.

[0026] Thirdly, a computer-readable storage medium is provided that stores computer program code, which, when executed by an electronic device, causes the electronic device to perform any of the methods in the first aspect.

[0027] Fourthly, a computer program product is provided, the computer program product comprising: computer program code, which, when executed by an electronic device, causes the electronic device to perform any of the methods in the first aspect. Attached Figure Description

[0028] Figure 1 This is an example diagram illustrating an application scenario of an embodiment of this application;

[0029] Figure 2 This is a schematic flowchart of an image processing method according to an embodiment of this application;

[0030] Figure 3 This is a comparison image of the photographic effect in an embodiment of this application;

[0031] Figure 4 This is another comparison image showing the photographic effects of an embodiment of this application;

[0032] Figure 5 This is a schematic diagram of a software architecture applied in an embodiment of this application;

[0033] Figure 6 This is another software architecture diagram used in the embodiments of this application;

[0034] Figure 7This is a schematic diagram of the structure of an electronic device applicable to this application. Detailed Implementation

[0035] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0036] In the embodiments of this application, unless otherwise stated, "multiple" can mean two or more.

[0037] This application applies to electronic devices. This application does not limit the specific type of electronic device. The electronic device can be a mobile phone, smart screen, tablet computer, wearable electronic device, in-vehicle electronic device, augmented reality (AR) device, virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), projector, etc.

[0038] The electronic device in this application embodiment is equipped with an image acquisition device (such as a camera) to realize the shooting function.

[0039] In some embodiments, the electronic device has multiple different shooting modes to provide users with a rich photography experience. Optionally, the multiple shooting modes of the electronic device include a snapshot mode. In snapshot mode, the electronic device can automatically capture photos of exciting moments. This application does not limit the specific naming of the snapshot mode; for example, the snapshot mode can be called a "Exciting Moment Snapshot Mode," "Exciting Snapshot Mode," "Exciting Automatic Snapshot Mode," "Eagle Eye Automatic Snapshot Mode," or "Eagle Eye Snapshot Mode," etc.

[0040] For ease of description, the image processing method of this application embodiment is described below using a mobile phone as an example of an electronic device. The following is combined with... Figure 1 The following are examples of scenarios. Figure 1 This is an example diagram of an application scenario in an embodiment of this application.

[0041] like Figure 1 In the mobile phone interface shown in (1), the interface can display multiple applications: Application 1, Application 2, ..., Application 7, and the camera application. The phone responds to the user clicking the camera application to launch the camera. After the camera is running, the phone interface displays as shown... Figure 1 The interface shown in (2) is called the camera's shooting preview interface. The shooting preview interface may include a viewfinder 11, settings controls 12, album icon 13, photo control 14, and camera rotation control, etc.

[0042] The viewfinder 11 is used to capture a preview image, which can be displayed in real time. The settings control 12 provides camera-related settings. Figure 1 In the interface shown in (2), the album icon 13 displays a thumbnail. The phone can take a picture in response to the user clicking the camera control 14. The camera rotation control can be used to switch cameras.

[0043] In some embodiments, the phone displays the camera settings interface in response to the user clicking the settings control 12, for example... Figure 1 The interface shown in (3). Figure 1 The interface shown in (3) includes settings related to taking photos and settings related to video.

[0044] Optionally, the settings panel related to photography includes a photo aspect ratio control, a smart shooting control 15, and a filter recommendation switch. The photo aspect ratio control is used to set the aspect ratio (or size) of the captured photo. The smart shooting control 15 is used to set various shooting modes based on the smart shooting algorithm. The filter recommendation switch is used to recommend suitable filters to the user based on their filter usage habits when AI photography is enabled.

[0045] Using the smart camera control 15, users can enable the Eagle Eye Capture Mode in the camera settings interface. For example, in response to the user clicking the smart camera control 15, the phone displays something like... Figure 1 The interface shown in (4). Figure 1 As shown in (4), the intelligent photo setting interface includes a voice-controlled photo control, a gesture photo switch option, a smile capture switch option, and an eagle eye automatic capture option 16.

[0046] Optionally, the video-related settings panel includes options for video resolution, video frame rate, efficient video format switching, and multiple recording switching. The video resolution option is used to adjust or set the resolution of the captured video. The video frame rate option is used to adjust or set the frame rate of the video.

[0047] It should be understood that Figure 3 The various function options or controls shown in (3) are merely illustrative descriptions and the embodiments of this application are not limited thereto.

[0048] In some embodiments, the mobile phone activates the Eagle Eye Capture Mode in response to the user's action of activating the mode. For example, Figure 1 The Eagle Eye Auto Capture option 16 shown in (4) is enabled. When the Eagle Eye Auto Capture option 16 is enabled, the preview interface displayed on the phone can be the photo preview interface in Eagle Eye Capture mode, for example, Figure 1 The interface shown in (5).

[0049] exist Figure 1 In the interface shown in (5), window 17 can be displayed in the viewfinder 11. Window 17 is used to prompt the user that "Eagle Eye Snapshot is now enabled." Furthermore, Figure 1 The interface shown in (5) displays icon 181, which also indicates that the Eagle Eye Capture mode is currently enabled. Optionally, Figure 1 The interface shown in (5) can also display icon 182 at the top, which indicates that the current mode is Eagle Eye Capture.

[0050] It is understood that window 17 may also be other forms of prompt box, or the text content displayed in window 17 may be replaced with other text. This application embodiment does not specifically limit this.

[0051] It is also understandable that window 17 can automatically disappear after a period of time (for example, 3 seconds) to avoid obstructing the viewfinder.

[0052] It's understandable. Figure 1 Middle (2) to Figure 1 The example shown in (4) is only one way to enable the Eagle Eye Capture Mode, and the embodiments of this application are not limited to this. For example, the mobile phone enables or disables the Eagle Eye Capture Mode in response to the user's click on icon 182.

[0053] Of course, you can turn the Eagle Eye capture mode on or off by clicking icon 182. Figure 1 The Eagle Eye automatic capture option 16 in the interface shown in (4) will also be updated synchronously. For example, the phone responds to the user's click on icon 182 and turns on the Eagle Eye capture mode; at this time, Figure 1 The Eagle Eye auto-capture option 16 in the interface shown in (4) will also be updated to be enabled. For example, in response to the user clicking icon 182, the phone disables the Eagle Eye capture mode; at this time, Figure 1 The Eagle Eye Auto Capture option 16 in the interface shown in (4) will also be updated to be off.

[0054] It should be understood that Figure 1 The photographing scenario shown is merely illustrative of one application scenario of this application and does not constitute a limitation on the embodiments of this application. In fact, the embodiments of this application can also be applied to other scenarios using cameras, such as video recording scenarios, video call scenarios, and live video streaming scenarios.

[0055] It should also be understood that Figure 1 Middle (2) to Figure 1 The interface shown in (5) is a schematic diagram of an interface for a user to operate in portrait mode on a mobile phone, but this application is not limited to this. For example, a user can take pictures or perform other operations in landscape mode on a mobile phone.

[0056] It should also be understood that Figure 1 The position of icon 181 shown in (5) in the interface is only an example, and the embodiments of this application are not limited thereto.

[0057] It is understood that this application does not specifically limit how the camera is activated. For example, as Figure 1 As shown in (1), the phone activates the camera in response to the user's tapping of the camera application.

[0058] Currently, mobile phones respond to users' requests to enter target capture mode (such as eagle eye capture mode) and then enter target capture mode to take photos. However, in target capture mode, the shooting effect is poor in high dynamic range scenes or low-light shooting scenes, which affects the user's shooting experience.

[0059] In view of this, embodiments of this application provide an image processing method that, by real-time recognition of the content of the photo preview interface, determines whether the current shooting scene is a typical scene and obtains the recognition result; upon receiving a user's photo request, it dynamically switches the photo shooting path based on the recognition result, specifically including: when the recognition result indicates that the current shooting scene is a typical scene (e.g., a spectacular scene), a first photo shooting path is used for taking a photo; when the recognition result indicates that the current shooting scene is an atypical scene (e.g., a low-light scene), a second photo shooting path is used for taking a photo. This allows for the selection of a suitable photo shooting path, or in other words, automatically switching to a suitable photo shooting path to improve the photo quality. Furthermore, under the same capture mode, the image processing method of this application embodiment can dynamically switch the photo shooting path to achieve better photo quality. For example, in Eagle Eye capture mode, if the current shooting scene is a low-light scene, the user does not need to manually switch to another photo shooting mode; a high-quality photo can be obtained while still under the current Eagle Eye capture mode, improving the user experience.

[0060] The following combination Figure 2 This application describes an image processing method according to an embodiment. Figure 2 This is a schematic flowchart of an image processing method according to an embodiment of this application. Figure 2 As shown, the image processing method includes:

[0061] Step 410: The photo preview interface is displayed.

[0062] The photo preview interface displays a real-time image of the target object to be photographed. It's understandable that the content displayed in the photo preview interface can change depending on the objects included in the actual shooting scene.

[0063] This application does not specifically limit the method of entering the photo preview interface.

[0064] In some embodiments, in response to a user's selection of a target shooting mode, the mobile phone invokes the camera (including the front camera and / or the rear camera) and displays a photo preview interface. Optionally, the target shooting mode is a snapshot mode. This application does not specifically limit the naming of the snapshot mode; for information on naming snapshot modes, please refer to the preceding description, which will not be repeated here.

[0065] In some embodiments, in response to the user selecting the Eagle Eye Capture mode, the mobile phone displays a preview interface for taking photos under the Eagle Eye Capture mode, for example... Figure 1 The interface shown in (5).

[0066] After the phone displays the camera preview interface, the phone's perception engine perceives the content in the camera preview interface in real time in order to determine whether the current shooting scene is a typical scene.

[0067] Step 420: Recognize the content in the photo preview interface to obtain the recognition result. The recognition result is used to indicate whether the current shooting scene is a preset typical scene.

[0068] In other words, in preview mode, the phone can determine whether the current shooting scene is a preset typical scene by recognizing or sensing the content in the photo preview interface in real time, i.e., the recognition result.

[0069] It should be noted that in preview mode, the phone will continuously monitor the preview interface in real time to determine which shooting path to use when it receives a user's photo request.

[0070] In some embodiments, the mobile phone uses a sophisticated algorithm (or a sophisticated algorithm module) to recognize or perceive the content in the photo preview interface in real time to determine whether the current shooting scene is a typical scene or an atypical scene, and outputs the recognition result based on the content of the current photo preview interface. The sophisticated algorithm is used to identify whether the current shooting scene is a preset typical scene.

[0071] This application does not specifically limit the type of the "highlight algorithm" in the embodiments. The image of the current shooting scene can be identified based on the "highlight algorithm" or detection algorithm in related technologies.

[0072] Optionally, the recognition result can be used to indicate that the current shooting scene is a preset typical scene, or to indicate that the current shooting scene is an atypical scene.

[0073] Optionally, the preset typical scenario is a highlight scenario. Correspondingly, the atypical scenario is a non-highlight scenario.

[0074] For example, the preset typical scenarios include, but are not limited to, the following scenarios: playing basketball, splashing water, smiling, jumping, flowers, pets, etc.

[0075] It is understood that the preset typical scenarios can be adjusted or set according to actual needs, and this application embodiment does not specifically limit them. For example, preset typical scenarios also include dance scenarios, looking back scenarios, parabolic scenarios, table tennis scenarios, badminton scenarios, running scenarios, etc.

[0076] For example, when a user turns on the Eagle Eye capture mode to capture a person's jumping action, the recognition result corresponding to the time period from the moment the person is detected to appear in the preview screen and starts jumping to the moment the person finishes jumping and lands on the ground is judged as an exciting scene.

[0077] It should be noted that when a mobile phone determines that the current scene is a preset typical scene, it will pre-cache the image data of the highlights in the cache. The following description takes a typical scene as an example of a highlight scene.

[0078] In some embodiments, when the current shooting scene is identified as a highlight scene, the phone will pre-select and cache frames in the cache. The image data cached in the cache is the image data corresponding to the highlight frame (or highlight image frame).

[0079] Optionally, the best frames can be determined according to different rules or preset conditions in different best scenes.

[0080] For example, in a shooting environment where a basketball game is being filmed, the phone will select the frame corresponding to the moment of the shot as the highlight frame.

[0081] For example, if the current shooting scene is a water splashing scene, the frame corresponding to the moment the phone splashes the water is determined as the highlight frame.

[0082] For example, if the current shooting scene includes a human face, the phone will select the frame corresponding to the smiling face of the person as the best frame.

[0083] For example, if the current shooting scene is a person jumping, the phone will select the frame corresponding to the highest point of the jump as the highlight frame.

[0084] For example, if the current shooting scene includes a landscape scene, the phone will select the frame corresponding to the flowers as the best frame.

[0085] For example, if the current shooting scene includes animals, the phone will select the frame corresponding to the pet as the highlight frame.

[0086] It should be understood that the above examples of highlights are merely illustrative descriptions, and the embodiments of this application do not impose any specific limitations on them.

[0087] It should also be understood that the embodiments of this application do not specifically limit the rules for determining highlight frames, and the rules for determining highlight frames can be determined based on related technologies. For example, determining highlight frames can also combine one or more other factors used to evaluate image quality, such as image composition evaluation, image sharpness, and image quality.

[0088] In other embodiments, when the current shooting scene is identified as a less than ideal scene, the phone acquires image data using the normal shooting mode. That is, if the current shooting scene is determined to be unsuitable, there is no need for pre-selecting and caching frames; instead, image data is acquired according to the normal shooting process, such as acquiring the image data corresponding to the current preview frame for taking a picture. The purpose of this is that, for scenarios where variable exposure is required (e.g., high dynamic range scenes, night scenes), variable exposure queuing processing can be performed on the image data to select the variable exposure frame to improve the shooting effect and enhance photo quality. Variable exposure queuing processing can be understood as: changing the exposure of an image frame and inserting the image frame with the changed exposure into the queue to output a better-quality image frame.

[0089] In some embodiments, the variable exposure queueing process specifically includes: the AE module issuing a set of variable exposure parameters. Optionally, the total exposure achievable by a set of exposure parameters can be quantified as the product of exposure time and ISO.

[0090] Optionally, the purpose of performing variable exposure processing on image data includes, but is not limited to: (1) covering high dynamic range scenes, for example, reducing the exposure value of the image for overexposed scenes, or increasing the exposure value of the image for underexposed scenes; (2) denoising multi-frame image data in order to adjust the signal-to-noise ratio of the image.

[0091] The above describes how a mobile phone processes image data in different shooting scenarios. The following describes how to dynamically switch shooting paths to take a picture after receiving a user's photo-taking request.

[0092] Step 430: Receive the user's first operation, which is used to trigger taking a picture.

[0093] The first operation can be understood as the user triggering the phone to take a picture. This application embodiment does not limit the specific form of the first operation. The first operation can be the user clicking the camera control in the camera preview interface, the user pressing the camera button on the phone (e.g., a physical shortcut key on the phone), or the user pressing the camera button on a phone accessory (e.g., headphones) (e.g., a physical shortcut key on the phone accessory).

[0094] It should be understood that the above description is only based on the example of the first operation triggering the mobile phone to take a picture, and the embodiments of this application are not limited thereto. In fact, the mobile phone can also take pictures in response to other user commands. For example, the mobile phone can take pictures in response to a voice command input by the user.

[0095] For example, the first action is a user click. Figure 1 The operation of the camera control 14 shown in (5).

[0096] In this embodiment of the application, upon obtaining the recognition result, the mobile phone responds to the user's first operation and can dynamically switch the shooting path according to the recognition result of the excellent algorithm, and automatically select the appropriate shooting path. The following describes the steps 441 and 442.

[0097] Step 441: In response to the first operation, when the recognition result is a first scene, take a picture using the first image capture path, where the first scene is a preset typical scene.

[0098] In other words, when the current scene is identified as a typical scene, the system will switch to the first image capture path to take a picture. The first image capture path can also have other names, such as the "Spectacular Scene Image Capture Path," etc.

[0099] Optionally, taking photos using the first photographing path includes:

[0100] First image data is obtained from the first cache, wherein the first image data is the data of the highlights frame cached in the first cache when the first operation is received;

[0101] The first image data is processed by the preprocessing algorithm module to obtain the second image data;

[0102] The second image data is processed by the first post-processing algorithm module to output the first captured image frame.

[0103] The first buffer is used to cache image data. Specifically, the first buffer is used to guarantee the data of the captured frames. After detecting a capture request, the mobile phone retrieves the image data of the captured frame from the first buffer and returns it to the user. This application embodiment does not limit the specific type of the first buffer. For example, the first buffer can be a regular buffer or a buffer specific to a certain capture mode, such as the buffer in the Eagle Eye capture mode. For example, the first buffer is a zero shutter lag (ZSL) buffer. The ZSL buffer is used to store the image data directly output from the sensor. In ZSL mode, after the capture command is issued, the system selects the image data of the corresponding frame from the ZSL buffer and sends it to the capture stream and thumbnail stream for processing.

[0104] When using the first image capture path, the image data obtained from the first cache can be: data corresponding to the best frame within a preset time period, where the preset time is related to the moment the user's image capture request is detected. The preset time being related to the moment the user's image capture request is understood as either a period of time before the moment the user's image capture request is detected, or a period of time after the moment the user's image capture request is detected.

[0105] For example, the "highlighted frames" within a preset time period refer to the best image frames within 0.5 seconds after the user's photo request is detected. Or, for another example, the "highlighted frames" within a preset time period refer to the best image frames within 0.5 seconds before the user's photo request is detected.

[0106] It should be understood that the above examples of preset time are merely illustrative descriptions, and the duration of the preset time is not specifically limited in the embodiments of this application, but may depend on actual needs.

[0107] After obtaining the first image data from the first cache, the first image data can be sent to the image capture algorithm module for processing. Optionally, the image capture algorithm module includes a pre-processing algorithm module and a first post-processing algorithm module.

[0108] Optionally, the preprocessing algorithm module is used to preprocess the image based on image processing algorithms. For example, preprocessing includes, but is not limited to, format conversion, binarization, etc. It should be understood that the specific processing of the preprocessing algorithm module in this application embodiment is not limited, and the processing methods in related technologies can be referred to.

[0109] The first post-processing algorithm module is used to perform image post-processing on the image data obtained when taking a picture using the first imaging path. Optionally, the first post-processing algorithm module is used to perform offline processing on the image to improve the image quality. For example, the first post-processing algorithm module is used to perform one or more of the following processing on the image: multi-frame fusion noise reduction processing, multi-frame HDR processing, etc.

[0110] In some embodiments, the second image data is processed by a first post-processing algorithm module to output a first captured image frame, including:

[0111] When taking photos using a wide-angle or telephoto camera, a first algorithm chain is invoked to process the second image data. In this first algorithm chain, image fusion is performed based on multiple frames (e.g., four normal frames) without any exposure adjustment to obtain the captured image frame. This first algorithm chain is implemented by invoking an enhanced high dynamic range (HDRP) algorithm. The essence of the HDRP algorithm is to denoise underexposed images. For example, the first algorithm chain is an enhanced high dynamic range lightweight algorithm chain (HDRPlite).

[0112] When taking photos using either a wide-angle or telephoto camera, the output image mode is always binning mode. Binning mode is a sensor output method. It involves adding the charges sensed by adjacent pixels together and reading them out as a single pixel. The image data output through binning mode is in Bayer raw format. For example, in binning mode, the sensor can sum and average the values ​​of four pixels to obtain the value of a single pixel, which is then output as the processed pixel value.

[0113] Optionally, in response to the user switching the zoom level of the photo to below 1X (e.g., 0.5X), the phone switches to the wide-angle camera to take the photo.

[0114] The above description of HDRPlite can be found in related technical descriptions, and this application does not impose specific limitations on it. This application does not limit the capitalization of the letters in HDRPlite; for example, HDRPlite can also be written as hdrplite, and this is not specifically limited.

[0115] Alternatively, when taking a picture using the main camera, the first algorithm link, the second algorithm link, or the third algorithm link can be invoked to process the second image data. A description of the first algorithm link can be found above and will not be repeated here.

[0116] In the second algorithm link, image fusion processing is performed based on two long frames (e.g., normal frames) and two short frames, without any exposure adjustment, to obtain the first captured image frame. This second algorithm link is implemented by calling a multi-exposure fusion (MEF) algorithm. For example, the second algorithm link could be MEFlite. MEF is a process that synthesizes a set of low dynamic range (LDR) images with different exposures to obtain an image with a visual effect similar to a high dynamic range (HDR) image.

[0117] Similarly, in the third algorithm link, image fusion processing is performed based on two long frames (e.g., the long frames are normal frames) and two short frames, without any exposure adjustment, to obtain the first captured image frame. For example, the third algorithm link is implemented using a motion capture (MC) algorithm. The MC algorithm can also be understood as a motion-based image capture algorithm. The third algorithm link is MClite.

[0118] When using the main camera to take pictures, different algorithm chains may be invoked due to the different frame output modes (or image formats) of the main camera. For example, if the main camera uses HDR mode, the HDRPlite algorithm chain is invoked; if the main camera uses dual analog gain (DAG) or dual conversion gain (DCG) mode, the MEFlite algorithm chain is invoked; and if the main camera uses staggered high dynamic range (stagger HDR) frame output mode, the MClite algorithm chain is invoked. Stagger HDR, by increasing the sensor's frame rate, is a technique that allows the acquisition of multiple frames with different exposure brightness within a single acquisition cycle. For example, stagger HDR can generate both long and short frames within a single acquisition cycle.

[0119] The DAG mode is a technique that uses simultaneous exposure, employing two different analog gain readouts to obtain two frames with different exposures, and then fusing them to obtain a high dynamic range image. For a description of the DAG mode, please refer to the relevant technical documentation.

[0120] DCG mode is a technique that uses simultaneous exposure to obtain two images with different exposures, read out using two different conversion gains, and then fuses them to obtain a high dynamic range image. DCG mode refers to adding DCG to CMOS pixels, enabling the sensor to simultaneously possess high sensitivity and high dynamic range. For example, DCG includes high conversion gain (HCG) and low conversion gain (LCG). High HCG corresponds to long exposure frames, resulting in good signal-to-noise ratio in dark areas. Low LCG corresponds to short exposure frames, ensuring good highlight detail. When the sensor operates in DCG mode, it obtains both long and short exposure frames separately, then fuses them, and the fused image frame is the sensor's output image. In other words, the image output by the sensor in DCG mode is the fused image frame of long and short exposure frames.

[0121] The processing principles of the first post-processing algorithm module can also be understood by referring to Table 1 below.

[0122] Table 1

[0123]

[0124] Based on Table 1 above, when taking pictures with different cameras, the first post-processing algorithm module uses different algorithm links for post-processing.

[0125] It should be understood that the embodiments of this application do not specifically limit the specific functions or roles of the first post-processing algorithm module. The first post-processing algorithm module may also include other image processing, such as Bayer domain processing, cropping processing, upscaling processing, etc.

[0126] Step 442: In response to the first operation, when the recognition result is a second scene, a second photographing path is used to take a picture, where the second scene is an atypical scene.

[0127] The first photography path and the second photography path are different photography paths.

[0128] Optionally, a second photographing path may be used for taking pictures, including:

[0129] Acquire third image data, wherein the third image data is the image data corresponding to the variable exposure image frame determined when the first operation is received, and the variable exposure image frame refers to the image frame generated based on the first exposure parameters issued by the auto exposure (AE) module, and the first exposure parameters are different from the exposure parameters corresponding to the preview screen frame.

[0130] The third image data is processed by the preprocessing algorithm module to obtain the fourth image data;

[0131] The second post-processing algorithm module performs image processing on the fourth image data and outputs the second captured image frame.

[0132] For example, in a low-light shooting scene, when taking a picture using the second shooting path, upon receiving a shooting request, multiple image frames are selected from the shooting queue of preview image frames. These multiple image frames are the image frames in the shooting queue that are closest to the shooting command. The AE module sends variable exposure parameters (e.g., first exposure parameters) to the lower layers (e.g., sequentially passed from the frame layer, HAL layer, and driver layer to the sensor). The first exposure parameters are different from the exposure parameters corresponding to the preview image frames. The sensor applies the variable exposure parameters sent by the AE module and outputs multiple variable exposure frames. The sensor sends the multiple variable exposure frames to the shooting algorithm module (e.g., the second post-processing algorithm module) for further image processing.

[0133] The first exposure parameter is used to adjust the exposure. Optionally, the first exposure parameter includes the exposure time (e.g., 10 ms) and the ISO sensitivity.

[0134] For a description of the preprocessing algorithm module, please refer to the previous introduction. For the sake of brevity, it will not be repeated here.

[0135] The second post-processing algorithm module provides post-processing for the image data acquired during the second shooting process. For example, the specific processing method of the second post-processing algorithm module can refer to the post-processing algorithm module in the normal shooting mode (or conventional shooting process). For instance, the second post-processing algorithm module performs one or more of the following processing on the image: multi-frame fusion noise reduction, multi-frame HDR processing, etc.

[0136] The processing principles of the second post-processing algorithm module can also be understood by referring to Table 2 below.

[0137] Table 2

[0138]

[0139] Based on Table 2 above, when taking photos using different cameras, the second post-processing algorithm module uses the same algorithm chain for post-processing. The specific process can be referred to the normal photo-taking process. For example, the algorithm chain in the normal photo-taking mode includes any of the following algorithm chains: HDR algorithm chain; MEF algorithm chain, etc.

[0140] The above describes one implementation method for taking photos using a second image capture path in scenarios where variable exposure is required. Of course, for scenarios without variable exposure requirements, upon receiving a photo capture request, the image data corresponding to the preview frame can be sent to the image capture algorithm module for image processing. In some embodiments, taking photos using the second image capture path includes: acquiring the image data corresponding to the preview frame and sending it to the image capture algorithm module for image processing.

[0141] Optionally, in the software design, the first post-processing algorithm module and the second post-processing algorithm module can be different algorithm modules to post-process the image data received from the corresponding shooting path. That is, for the capture mode, compared with the method of using the same post-processing algorithm module (or post-processing algorithm chain) for image processing, the embodiments of this application provide two types of post-processing algorithm modules, namely, a first post-processing algorithm module corresponding to typical scenes (the specific implementation process can be referred to above), and a second post-processing algorithm module corresponding to atypical scenes (the specific implementation process can be referred to above), so as to perform the post-processing process in a targeted manner, thereby improving the image effect.

[0142] Optionally, in the embodiments of this application, regardless of whether it is the first or second image capture path, in addition to the image processing process based on the processing algorithm module described above, one or more of the following processing methods can also be performed on the image: bad pixel correction (BPC), black level correct (BLC), lens shade correction (LSC), automatic white balance (AWB), Bayer domain noise reduction (NR), Demosaic, color correct (CC), YUV domain noise reduction (NR), color enhancer (CE), sharpening, tone mapping, etc., and the embodiments of this application do not specifically limit this. Of course, the order in which the image processing algorithms listed here are executed and the image processing performed by the aforementioned post-processing algorithm module are also not limited.

[0143] In some embodiments, when a user triggers a photo-taking operation in a certain snapshot mode, dynamic switching of the photo-taking path can be achieved based on steps 441 and 442 described above. Compared to traditional snapshot modes that cannot achieve variable exposure processing, this application embodiment not only provides a first photo-taking path but also a second photo-taking path, enabling the snapshot mode to achieve variable exposure processing while maintaining its snapshot function. This allows it to handle high dynamic range shooting scenarios or low-light scenarios, thus providing better shooting capabilities for complex shooting environments and improving the image quality of non-highlight scenes in the snapshot mode. In this application embodiment, after a high dynamic range shooting scenario or low-light scenario, the photo-taking path can be automatically switched without the user having to manually switch from the snapshot mode to other photo-taking modes, avoiding back-and-forth switching and greatly improving the user's shooting experience. Furthermore, this application embodiment provides corresponding post-processing algorithms for the first and second photo-taking paths to further improve image quality.

[0144] The following combination Figure 3 and Figure 4 This application describes a comparison example of the photo-taking effects of embodiments thereof.

[0145] For the same high dynamic range shooting scenario, after the user clicks to take a photo, Figure 3 (1) shows a photo captured in the traditional snapshot mode; Figure 3 Image (2) shows a photograph captured in the snapshot mode of this application embodiment. It can be clearly seen that... Figure 3 The photo shown in (2) is far superior to the image shown in (2). Figure 3 The photograph shown in (1) is an example. Figure 3 The details of the sky and the ground in the photo shown in (2) are clearer. In other words, the image processing method of this application embodiment, or the capture mode provided by this application embodiment, has better image capture effect in high dynamic range scenes.

[0146] For the same night scene shooting scenario, after the user clicks to take a photo, Figure 4 (1) shows a photo captured in the traditional snapshot mode; Figure 4 Image (2) shows a photograph captured in the snapshot mode of this application embodiment. It can be clearly seen that... Figure 4 The photo shown in (2) is far superior to the image shown in (2). Figure 4 The photograph shown in (1) is an example. Figure 4 The details of the sky and the ground in the photo shown in (2) are clearer. In other words, the image processing method of this application embodiment, or the snapshot mode provided by this application embodiment, produces better image quality in night scene scenarios.

[0147] Understandable. Figure 3or Figure 4 The photo comparisons shown are merely illustrative examples, and the embodiments of this application are not limited thereto.

[0148] The following combination Figures 5 to 7 This application describes the software system and hardware architecture used in its embodiments.

[0149] Figure 5 This is a schematic diagram of the architecture (including software system and some hardware) used in the embodiments of this application. Figure 5 As shown, the application architecture is divided into several layers, each with a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the application architecture can be divided into five layers, from top to bottom: the application layer, the application framework layer, the hardware abstraction layer (HAL), the driver layer, and the hardware layer.

[0150] like Figure 5 As shown, the application layer includes the camera and gallery. This is understandable. Figure 5 The examples shown are only a portion of the applications; in fact, the application layer can include other applications as well, and this application does not limit this. For example, the application layer may also include applications such as messaging, alarm clock, weather, stopwatch, compass, timer, flashlight, calendar, and Alipay.

[0151] like Figure 5 As shown, the application framework layer includes a camera access interface. The camera access interface includes camera management and camera devices. The hardware abstraction layer includes a camera hardware abstraction layer and a camera algorithm library. The camera hardware abstraction layer includes multiple camera devices. The camera algorithm library includes a perception engine, a first image capture path, a second image capture path, and an image capture algorithm processing module.

[0152] The perception engine is used to identify the image in preview mode to obtain the recognition result. In some embodiments, such as Figure 6 As shown, the perception engine includes a sophisticated algorithm module.

[0153] After receiving the user's photo-taking command, the choice of which photo-taking path to use depends on the recognition results obtained by the algorithm module. The first photo-taking path is the one used when a typical scene is recognized, for example, Figure 6 The diagram illustrates the process of frame selection based on the user's click time via the chip platform (i.e., the normal photo-taking process). The second photo-taking path is used when atypical scenes are detected, for example... Figure 6 The process of pre-selecting frames through the frame selection module in SIT is shown.

[0154] It should be understood that the perception engine can also be set up in other layers. As one possible implementation, the perception engine can be placed in the application layer or the application framework layer.

[0155] The image processing algorithm module is used to process image data. Optionally, the image processing algorithm module includes a pre-processing algorithm module and a post-processing algorithm module. The post-processing algorithm module includes a first post-processing algorithm module and a second post-processing algorithm module. For example, the image processing algorithm module is... Figure 6 The image capture algorithm module shown.

[0156] The driver layer is used to drive hardware resources. The driver layer can include multiple driver modules. For example... Figure 5 As shown, the driver layer includes camera device drivers, digital signal processor drivers, and graphics processor drivers, etc.

[0157] The hardware layer includes sensors, an image signal processor, a digital signal processor, and a graphics processor. The sensors include multiple sensors, a Time-of-Flight (TOF) camera, and a multispectral sensor. Optionally, the image signal processor may include a first ISP module, a second ISP module, and a third ISP module.

[0158] For example, a user can tap the camera application. When the user taps the camera to launch the camera, the command to launch the camera is sent to the camera hardware abstraction layer (HIP) through the camera access interface. The HIP then calls the camera device driver and the camera algorithm library. The perception engine in the camera algorithm library performs real-time recognition on the preview image to obtain the recognition result (e.g., typical or atypical scene).

[0159] When the user clicks the camera control, the camera command is sent to the camera hardware abstraction layer (HAL) via the camera access interface. The HAL then calls the camera device driver and the camera algorithm library. The perception engine in the camera algorithm library determines whether to use the first or second image capture path based on the recognition results. Further, the image data obtained through the image capture path is sent to the image capture algorithm processing module for further processing.

[0160] Optionally, the camera algorithm library is also used to send digital signals to the digital signal processor (DSP) driver in the driver layer, so that the DSP driver can call the DSP in the hardware layer for digital signal processing. The DSP can then return the processed digital signal to the camera algorithm library through its driver. The camera algorithm library is also used to send digital signals to the graphics signal processor (GSP) driver in the driver layer, so that the GSP driver can call the graphics processor in the hardware layer for digital signal processing. The GSP can then return the processed graphics data to the camera algorithm library through its driver.

[0161] Additionally, the image output from the image signal processor can be sent to the camera device driver. The camera device driver can then send the image output from the image signal processor to the camera hardware abstraction layer. The camera hardware abstraction layer can then send the image to the post-processing algorithm module for further processing, or it can send the image to the camera access interface. The camera access interface can then send the image returned by the camera hardware abstraction layer to the camera.

[0162] The following combination Figure 6 This application describes the detailed architecture of embodiments of the invention. For example, as shown... Figure 6 As shown, when the user taps to activate the camera and enter preview mode, the phone displays a preview interface, for example, Figure 1 As shown in Figure (5), the Highlights algorithm module in the perception engine performs real-time perception of the preview screen to determine whether the current scene is a highlight or a non-highlight scene. When the current shooting scene is a highlight, the Highlights algorithm module will notify SuperImageTurbo (SIT) to pre-select and cache frames. Specifically, the frame selection module in SIT can pre-cache highlight image frames.

[0163] When a user sends a photo request, the phone dynamically switches the photo-taking path based on the recognition results of the "Highlights" algorithm. Specifically, when the "Highlights" algorithm module identifies the current shooting scene as a "highlight" scene, it sends a pre-selected cached frame to the photo-taking algorithm (photo-taking module) for processing. When the current scene is identified as a "lowlight" scene, the chip platform selects a frame based on the user's click time (i.e., the normal photo-taking process). In the snapshot mode, this embodiment of the application, by dynamically switching to the second photo-taking path, can perform variable exposure interpolation processing based on the current shooting scene, selecting variable exposure frames to improve the photo-taking effect.

[0164] Furthermore, in the post-processing workflow of the image-taking algorithm, this application embodiment provides a first post-processing algorithm module for typical scenarios and a second post-processing algorithm module for atypical scenarios. That is, different features are selected for post-processing of photos for different scenarios, which can further improve image quality. The functions of the first and second post-processing algorithm modules can be found in the description of the method embodiments above, and will not be repeated here.

[0165] The software system used in the embodiments of this application has been described in detail above. The following section, in conjunction with... Figure 7 Describe the hardware system of electronic device 1000.

[0166] Figure 7 A schematic diagram of the structure of an electronic device 1000 applicable to this application is shown.

[0167] Electronic device 1000 may include processor 110, external memory interface 120, internal memory 121, universal serial bus (USB) interface 130, charging management module 140, power management module 141, battery 142, antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194, and subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0168] It should be noted that, Figure 7 The structure shown does not constitute a specific limitation on the electronic device 1000. In other embodiments of this application, the electronic device 1000 may include... Figure 7 The components shown may include more or fewer components, or the electronic device 1000 may include... Figure 7 The components shown may be a combination of certain components, or the electronic device 1000 may include... Figure 7 The components shown are sub-components of certain components. For example, Figure 7 The proximity sensor 180G shown is optional. Figure 7 The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0169] Processor 110 may include one or more processing units. For example, processor 110 may include at least one of the following processing units: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, video codec, digital signal processor (DSP), baseband processor, and neural network processing unit (NPU). These different processing units may be independent devices or integrated devices.

[0170] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0171] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0172] In some embodiments, processor 110 may include one or more interfaces. For example, processor 110 may include at least one of the following interfaces: an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a SIM interface, and a USB interface.

[0173] Figure 7 The connection relationships between the modules shown are merely illustrative and do not constitute a limitation on the connection relationships between the modules of the electronic device 1000. Optionally, the modules of the electronic device 1000 may also adopt a combination of various connection methods described in the above embodiments.

[0174] The charging management module 140 receives power from a charger, which can be either a wireless or wired charger. In some wired charging embodiments, the charging management module 140 receives current from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 receives electromagnetic waves (current path shown as dashed lines) via the wireless charging coil of the electronic device 1000. While charging the battery 142, the charging management module 140 can also supply power to the electronic device 1000 via the power management module 141.

[0175] The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, providing power to the processor 110, internal memory 121, display screen 194, camera 193, and wireless communication module 160, etc. The power management module 141 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (e.g., leakage current, impedance). Optionally, the power management module 141 can be located within the processor 110, or the power management module 141 and the charging management module 140 can be located in the same device.

[0176] The wireless communication function of electronic device 1000 can be realized through devices such as antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor.

[0177] Electronic device 1000 can implement display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0178] Display screen 194 can be used to display images or videos. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a mini light-emitting diode (Mini LED), a micro light-emitting diode (Micro LED), a micro OLED, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 1000 may include one or N displays screens 194, where N is a positive integer greater than 1.

[0179] Electronic device 1000 can achieve shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0180] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can perform algorithmic optimization of image noise, brightness, and color. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0181] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into a standard red-green-blue (RGB), YUV, or other image signal format. In some embodiments, the electronic device 1000 may include one or N cameras 193, where N is a positive integer greater than 1.

[0182] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when electronic device 1000 is selecting a frequency, the DSP is used to perform Fourier transforms on the frequency energy.

[0183] Video codecs are used to compress or decompress digital video. Electronic device 1000 may support one or more video codecs. Thus, electronic device 1000 can play or record video in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG 2, MPEG 3, and MPEG 4.

[0184] An NPU (Neural Processing Unit) is a processor that borrows from the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, to rapidly process input information and continuously learn. NPUs can enable intelligent cognitive functions in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0185] Electronic device 1000 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0186] It is understood that the image processing method of this application embodiment can be applied to Figure 7 The specific implementation steps of the electronic device shown can be referred to the description of the method embodiments above, and will not be repeated here.

[0187] This application also provides a computer program product that, when executed by a processor, implements the methods described in any of the method embodiments of this application.

[0188] The computer program product can be stored in memory and, after processes such as preprocessing, compilation, assembly, and linking, is finally converted into an executable object file that can be executed by a processor.

[0189] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a computer, implements the methods described in any of the method embodiments of this application. The computer program may be a high-level language program or an executable object program.

[0190] The computer-readable storage medium can be volatile memory or non-volatile memory, or it can include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0191] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process and technical effects of the above-described apparatus and equipment can be referred to the corresponding processes and technical effects in the foregoing method embodiments, and will not be repeated here.

[0192] In the several embodiments provided in this application, the systems, apparatuses, and methods disclosed can be implemented in other ways. For example, some features of the method embodiments described above can be ignored or not performed. The apparatus embodiments described above are merely illustrative; the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Multiple units or components can be combined or integrated into another system. Furthermore, the coupling between units or components can be direct coupling or indirect coupling, including electrical, mechanical, or other forms of connection.

[0193] It should be understood that in the various embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0194] Furthermore, the terms "system" and "network" are often used interchangeably in this paper. The term "and / or" in this paper merely describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " in this paper generally indicates that the preceding and following related objects have an "or" relationship.

[0195] In summary, the above description is merely a preferred embodiment of the technical solution of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. An image processing method, characterized in that, include: In response to the user's selection of a capture mode, a photo preview interface is displayed, which is the preview interface for the capture mode; The content in the photo preview interface is identified to obtain the identification result. The identification result is used to indicate whether the current shooting scene is a preset typical scene, and the preset typical scene is a wonderful scene. Receive the user's first operation, which is used to trigger taking a picture; In response to the first operation, when the recognition result is a wonderful scene, the first shooting path is used to take a picture by pre-selecting and caching frames. In response to the first operation, when the recognition result is a non-exciting scene and a high dynamic range scene or a low light scene, the second shooting path is used to take a picture by performing variable exposure interpolation processing on the image data to select a variable exposure frame. The first shooting path is different from the second shooting path.

2. The method according to claim 1, characterized in that, The step of taking photos using the first photographing path includes: First image data is obtained from the first cache, wherein the first image data is the data of the best image frames cached in the first cache when the first operation is received; The first image data is processed by the preprocessing algorithm module to obtain the second image data; The second image data is processed by the first post-processing algorithm module to output the first captured image frame.

3. The method according to claim 2, characterized in that, The image processing of the second image data by the first post-processing algorithm module includes: When using a wide-angle camera or a telephoto camera, the first algorithm link is invoked to perform image processing on the second image data; Alternatively, when using the main camera, the first algorithm link, the second algorithm link, or the third algorithm link can be invoked to perform image processing on the second image data; In the first algorithm link, image fusion is performed based on multiple standard frames; in the second algorithm link, image fusion is performed based on multiple standard frames and multiple short frames.

4. The method according to any one of claims 1 to 3, characterized in that, The method of taking pictures using the second imaging path includes: Acquire third image data, wherein the third image data is the image data corresponding to the variable exposure image frame determined when the first operation is received, and the variable exposure image frame refers to the image frame generated based on the first exposure parameters issued by the automatic exposure AE module, and the first exposure parameters are different from the exposure parameters corresponding to the preview screen frame. The third image data is processed by the preprocessing algorithm module to obtain the fourth image data; The second post-processing algorithm module performs image processing on the fourth image data and outputs the second captured image frame.

5. The method according to any one of claims 1 to 3, characterized in that, The step of recognizing the content in the photo preview interface and obtaining the recognition result includes: The content in the photo preview interface is identified in real time by the excellent algorithm module to obtain the identification result.

6. The method according to any one of claims 1 to 3, characterized in that, The preset typical scenarios include any of the following scenarios: shooting a basketball, splashing water, smiling, jumping, flowers, and pets.

7. An electronic device, characterized in that, The device includes a processor and a memory coupled together, the memory being used to store a computer program that, when executed by the processor, causes the electronic device to perform the method of any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the electronic device to perform the method of any one of claims 1 to 6.

9. A chip system, characterized in that, The chip system is applied to an electronic device, the chip system including one or more processors, the processors being configured to invoke computer instructions to cause the electronic device to perform the method as described in any one of claims 1 to 6.

Images