Image processing method and electronic equipment

Abstract

An image processing method and an electronic device are applied to the technical field of image processing. The method comprises the following steps: identifying whether a shooting scene is a preset typical scene in real time through a wonderful algorithm in a preview state; and the first photographing path is used for photographing in the typical scene, and the second photographing path is used for photographing in the atypical scene, so that the dynamic switching of the photographing paths can be realized. Moreover, compared with a traditional snapshot mode in which variable exposure processing cannot be realized, the embodiment of the invention can realize variable exposure processing on the premise that the snapshot mode has a snapshot function, so that a better shooting function is provided for coping with a complex shooting environment (such as a high-dynamic shooting scene or a dark light scene), and the shooting experience of a user is improved. The image effect of the non-wonderful picture in the snapshot mode is improved, and the shooting experience of the user is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular, to an image processing method and an electronic device. Background Art

[0002] With the increasing development of smart terminals, the shooting function has become an essential function of smart terminals. Users' demand and experience for shooting (photographing and / or videography) of smart terminals are also increasing. Smart terminals provide users with different shooting modes, such as snapshot mode. At present, the image effect obtained by users in the snapshot mode is not good, resulting in poor image quality, which affects the user's shooting experience. Summary of the invention

[0003] In view of this, the present application provides an image processing method, an electronic device, a computer-readable storage medium and a computer program product, which can improve the image effect in the snapshot mode and help improve the user's shooting experience.

[0004] In a first aspect, an image processing method is provided, comprising: displaying a photo preview interface;

[0005] Recognize the content in the photo preview interface to obtain a recognition result, wherein the recognition result is used to indicate whether the current shooting scene is a preset typical scene;

[0006] Receiving a first operation of a user, where the first operation is used to trigger taking a photo;

[0007] In response to the first operation, when the recognition result is a first scene, taking a photo using a first photographing path, the first scene being a preset typical scene;

[0008] In response to the first operation, when the recognition result is a second scene, a second photographing path is used to take a photo, the second scene is an atypical scene, and the first photographing path is different from the second photographing path.

[0009] The above method can be executed by an electronic device or a chip in the electronic device. Based on the above scheme, by real-time identification of whether the shooting scene is a preset typical scene; and using the first shooting path to take pictures in a typical scene, and using the second shooting path to take pictures in an atypical scene, dynamic switching of the shooting path can be achieved. In addition, compared with the traditional snapshot mode that cannot achieve variable exposure processing, the embodiment of the present application enables the snapshot mode to have both the snapshot function and the variable exposure processing, and can cope with high-dynamic shooting scenes or dark light scenes, thereby providing better shooting functions for coping with complex shooting environments (for example, high-dynamic shooting scenes or dark light scenes), improving the image effect of non-wonderful pictures in the snapshot mode, and helping to improve the user's shooting experience.

[0010] In some embodiments, the photo preview interface is a photo preview interface in a snapshot mode. Optionally, displaying the photo preview interface includes: in response to a user selecting a snapshot mode, displaying the photo preview interface, the photo preview interface being a preview interface in the snapshot mode.

[0011] Therefore, in the snapshot mode, the image processing method of the embodiment of the present application can dynamically switch the photo path to achieve a better photo effect. For example, in the Eagle Eye snapshot mode, if the current shooting scene is a dark scene, the user does not need to manually switch to another photo mode, that is, the current Eagle Eye snapshot mode can still obtain a photo with high image quality, which improves the user experience.

[0012] In the embodiment of the present application, for different shooting scenes, that is, typical scenes and atypical scenes, different shooting paths are used to shoot pictures, so as to improve the shooting effect. The specific implementation methods of different shooting paths are described below.

[0013] In a possible implementation, taking photos is performed using a first photographing path, including: obtaining first image data from a first cache, wherein the first image data is data of a wonderful image frame cached in the first cache when the first operation is received; performing image processing on the first image data through a pre-processing algorithm module to obtain second image data; performing image processing on the second image data through a first post-processing algorithm module to output a first photographed image frame.

[0014] Therefore, when a photo request is received, if the shooting scene is determined to be a typical scene, the photo is taken based on the first photo path, the wonderful picture frames cached in advance are obtained from the cache, and the wonderful picture frames are sent to the photo algorithm module for further image processing, and finally the first photo image frame is obtained, which helps to improve the image quality.

[0015] In the embodiment of the present application, for the case where the shooting scene is a typical scene, a corresponding post-processing algorithm module is also designed accordingly, for example, a first post-processing algorithm module.

[0016] Optionally, the second image data is processed by a first post-processing algorithm module, including: when a wide-angle camera or a telephoto camera is used, calling the first algorithm link to perform image processing on the second image data; or, when a main camera is used, calling the first algorithm link, the second algorithm link or the third algorithm link to perform image processing on the second image data; wherein, in the first algorithm link, image fusion is performed based on multiple standard frames; and in the second algorithm link, image fusion is performed based on multiple standard frames and multiple short frames.

[0017] Therefore, when taking pictures with different cameras, the corresponding algorithm link is called to post-process the image, which can further optimize the image quality and improve the image effect.

[0018] In a possible implementation, taking a photo using a second photographing path includes: acquiring third image data from a first cache, wherein the third image data is image data corresponding to a variable exposure image frame determined when the first operation is received, and the variable exposure image frame refers to an image frame generated based on a first exposure parameter sent by an automatic exposure (AE) module, and the first exposure parameter is different from an exposure parameter corresponding to a preview screen frame; performing image processing on the third image data through a pre-processing algorithm module to obtain fourth image data; performing image processing on the fourth image data through a second post-processing algorithm module to output a second photographing image frame.

[0019] Therefore, when a photo request is received, if the shooting scene is determined to be an atypical scene, the second photo path is used to take the photo, and variable exposure processing is performed when there is a need for variable exposure to obtain a second photo image frame, which helps to improve image quality.

[0020] Exemplarily, multiple image frames are selected from the photo queue of the preview image frame, and the multiple image frames are the image frames in the photo queue that are closest to the photo command. The AE module sends a variable exposure parameter (for example, a first exposure parameter) to the bottom layer (for example, from the framework layer, the HAL layer, and the driver layer to the sensor in sequence). The first exposure parameter is different from the exposure parameter corresponding to the preview picture frame. The sensor takes effect on the variable exposure parameter sent by the AE module and outputs multiple variable exposure frames. The sensor sends the multiple variable exposure frames to the photo algorithm module (for example, the second post-processing algorithm module), and finally outputs the second photo image frame.

[0021] In a possible implementation, identifying the content in the photo preview interface to obtain a recognition result includes: identifying the content in the photo preview interface in real time by using a wonderful algorithm module to obtain the recognition result.

[0022] In a possible implementation, a typical scene may be a wonderful scene, and an atypical scene may be a non-wonderful scene. Optionally, the preset typical scene includes any one of the following scenes: shooting, splashing water, smiling face, jumping, flowers, and pets.

[0023] In a second aspect, an electronic device is provided, comprising a unit for executing any one of the methods in the first aspect. The electronic device may be a terminal or a chip in a terminal. The electronic device comprises a communication unit, a display unit and a processing unit.

[0024] When the electronic device is a terminal, the processing unit may be a processor, the communication unit may be a communication interface, and the display unit may be a graphics processing module and a screen; the terminal may also include a memory for storing computer program code, and when the processor executes the computer program code stored in the memory, the terminal executes any one of the methods in the first aspect.

[0025] When the electronic device is a chip in a terminal, the processing unit may be a logic processing unit inside the chip, the communication unit may be a communication interface, a pin or a circuit, etc., and the display unit may be a graphics processing unit inside the chip; the chip may also include a memory, which may be a memory inside the chip (for example, a register, a cache, etc.) or a memory located outside the chip (for example, a read-only memory, a random access memory, etc.); the memory is used to store computer program code, and when the processor executes the computer program code stored in the memory, the chip executes any one of the methods of the first aspect.

[0026] According to a third aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program code. When the computer program code is executed by an electronic device, the electronic device executes any one of the methods according to the first aspect.

[0027] According to a fourth aspect, a computer program product is provided, the computer program product comprising: a computer program code, when the computer program code is executed by an electronic device, the electronic device executes any one of the methods according to the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is an example diagram of an application scenario of an embodiment of the present application;

[0029] Figure 2 is a schematic flow chart of an image processing method according to an embodiment of the present application;

[0030] Figure 3 This is a comparison diagram of photographing effects of an embodiment of the present application;

[0031] Figure 4 This is another comparison chart of photographic effects of an embodiment of the present application;

[0032] Figure 5 It is a schematic diagram of a software architecture applied in an embodiment of the present application;

[0033] Figure 6 It is another software architecture diagram of the application embodiment of the present application;

[0034] Figure 7The present invention is a schematic diagram of the structure of an electronic device suitable for the present application. DETAILED DESCRIPTION

[0035] The technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0036] In the embodiments of the present application, unless otherwise specified, “plurality” may mean two or more than two.

[0037] The embodiments of the present application are applicable to electronic devices. The embodiments of the present application do not impose any restrictions on the specific types of electronic devices. The electronic device may be a mobile phone, a smart screen, a tablet computer, a wearable electronic device, an in-vehicle electronic device, an augmented reality (AR) device, a virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), a projector, and the like.

[0038] The electronic device according to the embodiment of the present application is equipped with an image collector (such as a camera) to realize the shooting function.

[0039] In some embodiments, the electronic device has a plurality of different camera modes to provide a rich camera experience for the user. Optionally, the plurality of different camera modes of the electronic device includes a snapshot mode. In the snapshot mode, the electronic device can automatically capture photos of wonderful moments. The specific naming of the snapshot mode is not limited in the embodiment of the present application. For example, the snapshot mode can be a wonderful moment snapshot mode, a wonderful snapshot mode, a wonderful automatic snapshot mode, an eagle eye automatic snapshot mode or an eagle eye snapshot mode, etc.

[0040] For ease of description, the following describes the image processing method of the present application embodiment by taking a mobile phone as an example of an electronic device. Figure 1 Example description of the scenario in . Figure 1 This is an example diagram of an application scenario of an embodiment of the present application.

[0041] like Figure 1 In the mobile phone interface shown in (1), the interface can display multiple applications: application 1, application 2, ..., application 7 and the camera application. In response to the user clicking the camera application, the mobile phone starts the camera. After the camera is running, the mobile phone interface displays the following Figure 1 The interface shown in (2) in the figure. This interface can be called the shooting preview interface of the camera. The shooting preview interface can include a viewfinder 11, a setting control 12, an album icon 13, a photo control 14, and a camera rotation control.

[0042] The viewfinder 11 is used to obtain the preview image of the shooting, and can display the preview image in real time. The setting control 12 is used to provide the setting functions related to the camera. Figure 1 In the interface shown in (2), the album icon 13 displays thumbnails. The mobile phone can take a photo in response to the user clicking the photo control 14. The camera rotation control can be used to switch cameras.

[0043] In some embodiments, in response to the user clicking the setting control 12, the mobile phone displays a camera setting interface, such as: Figure 1 The interface shown in (3) is shown in the figure. Figure 1 In the interface shown in (3), the camera device includes a setting bar related to taking pictures and a setting bar related to videos.

[0044] Optionally, the setting bar related to taking pictures includes a photo ratio control, a smart photo control 15, and a filter recommendation switch option. The photo ratio control is used to set the aspect ratio (or size) of the taken photo. The smart photo control 15 is used to set various photo modes involved in the smart photo algorithm. The filter recommendation switch option is used to recommend suitable filters to users based on their filter usage habits when AI photography is turned on.

[0045] Through the smart photo control 15, the user can turn on the eagle eye snapshot mode in the camera setting interface. For example, in response to the user clicking the smart photo control 15, the mobile phone displays the following Figure 1 The interface shown in (4) is shown in the figure. Figure 1 As shown in (4), the smart photo setting interface includes a voice-controlled photo control, a gesture photo switch option, a smile capture switch option, and an eagle-eye automatic capture option 16.

[0046] Optionally, the video-related setting bar includes a video resolution option, a video frame rate option, a high-efficiency video format switch option, and a one-shot multiple-recording switch option. The video resolution option is used to adjust or set the resolution of the captured video. The video frame rate option is used to adjust or set the frame rate of the video.

[0047] It should be understood that Figure 3 The various functional options or controls shown in (3) are merely exemplary descriptions, and the embodiments of the present application are not limited thereto.

[0048] In some embodiments, the mobile phone turns on the Hawkeye capture mode in response to the user's operation of turning on the Hawkeye capture mode. For example, Figure 1 The Hawkeye automatic snapshot option 16 shown in (4) is turned on. When the Hawkeye automatic snapshot option 16 is turned on, the preview interface displayed by the mobile phone can be a photo preview interface in the Hawkeye snapshot mode, for example, Figure 1 The interface shown in (5).

[0049] exist Figure 1 In the interface shown in (5), a window 17 may be displayed in the viewfinder 11. The window 17 is used to prompt the user that "Eagle Eye Wonderful Snapshot is turned on". Figure 1 The interface shown in (5) shows icon 181, which may also indicate that the Hawkeye snapshot mode is currently turned on. Figure 1 An icon 182 may also be displayed above the interface shown in (5), and the icon 182 may indicate that the current mode is the Hawkeye snapshot mode.

[0050] It is understandable that window 17 may also be a prompt box in other forms, or the text content displayed in window 17 may be replaced with other text, and the embodiment of the present application does not specifically limit this.

[0051] It can also be understood that the window 17 can automatically disappear after a period of time (for example, 3 seconds) to avoid blocking the framing image.

[0052] It is understandable. Figure 1 Middle (2) to Figure 1 (4) is only one implementation of turning on the Hawkeye snapshot mode, and the embodiments of the present application are not limited thereto. For example, in response to the user clicking the icon 182, the mobile phone turns on or off the Hawkeye snapshot mode.

[0053] Of course, by clicking icon 182, the eagle eye snapshot mode can be turned on or off. Figure 1 The Hawkeye automatic snapshot option 16 in the interface shown in (4) will also be updated synchronously. For example, in response to the user clicking the icon 182, the mobile phone turns on the Hawkeye snapshot mode; at this time, Figure 1 The Hawkeye automatic snapshot option 16 in the interface shown in (4) will also be synchronously updated to the on state. For another example, the mobile phone responds to the user clicking the icon 182 and turns off the Hawkeye snapshot mode; at this time, Figure 1 The Hawkeye automatic capture option 16 in the interface shown in (4) will also be synchronously updated to a closed state.

[0054] It should be understood that Figure 1 The photo-taking scene in the embodiment is only a schematic illustration of an application scenario of the present application, which does not constitute a limitation on the embodiments of the present application. In fact, the embodiments of the present application can also be applied to other scenes using cameras, such as video recording scenes, video call scenes, video live broadcast scenes, etc.

[0055] It should also be understood that Figure 1 Middle (2) to Figure 1 The interface shown in (5) is a schematic diagram of an interface for a user to operate in a vertical screen of a mobile phone, but the present application is not limited thereto. For example, a user can take a photo or perform other operations in a horizontal screen of a mobile phone.

[0056] It should also be understood that Figure 1 The position of the icon 181 shown in (5) in the interface is only an example, and the embodiments of the present application are not limited to this.

[0057] It is understood that the embodiments of the present application do not specifically limit how to start the camera. Figure 1 As shown in (1), the mobile phone starts the camera in response to the user clicking the camera application.

[0058] Currently, in response to a user entering a target capture mode (eg, eagle eye capture mode), a mobile phone enters the target capture mode to take a photo. However, in the target capture mode, for high dynamic range scenes or low light shooting scenes, the shooting effect is poor, affecting the user's shooting experience.

[0059] In view of this, an embodiment of the present application provides an image processing method, which determines whether the current shooting scene is a typical scene by real-time recognition of the content of the photo preview interface, and obtains a recognition result; after receiving the user's photo request, the photo path is dynamically switched based on the recognition result, specifically including: when the recognition result indicates that the current shooting scene is a typical scene (such as a wonderful scene), the first photo path is used to take a photo; when the recognition result indicates that the current shooting scene is an atypical scene (such as a dark light scene), the second photo path is used to take a photo, and a suitable photo path can be selected to take a photo, or automatically to a suitable photo path to take a photo, thereby improving the photo effect. In addition, in the same capture mode, the image processing method of the embodiment of the present application can dynamically switch the photo path to achieve a better photo effect. For example, in the Eagle Eye capture mode, if the current shooting scene is a dark light scene, there is no need for the user to manually switch to another photo mode, that is, a photo with high image quality can be obtained in the current Eagle Eye capture mode, thereby improving the user experience.

[0060] The following combination Figure 2 The image processing method according to the embodiment of the present application is described. Figure 2 FIG. 1 is a flow chart of an image processing method according to an embodiment of the present application. Figure 2 As shown, the image processing method includes:

[0061] Step 410: display the photo preview interface.

[0062] The photo preview interface can display a real-time picture of the target object to be photographed by the user in the preview state. It can be understood that the content displayed in the photo preview interface can change with the change of the objects included in the actual shooting scene.

[0063] The embodiment of the present application does not specifically limit how to enter the photo preview interface.

[0064] In some embodiments, the mobile phone responds to the user's operation of selecting the target photo mode, calls the camera (including the front camera and / or the rear camera), and displays the photo preview interface. Optionally, the target photo mode is a snapshot mode. The specific naming of the snapshot mode is not specifically limited in the embodiment of the present application. For the naming of the snapshot mode, please refer to the previous description, which will not be repeated here.

[0065] In some embodiments, in response to the user selecting the Hawkeye snapshot mode, the mobile phone displays a photo preview interface in the Hawkeye snapshot mode, for example, Figure 1 The interface shown in (5).

[0066] After the mobile phone displays the photo preview interface, the perception engine in the mobile phone perceives the content in the photo preview interface in real time to know whether the current shooting scene is a typical scene.

[0067] Step 420: Recognize the content in the photo preview interface to obtain a recognition result, where the recognition result is used to indicate whether the current shooting scene is a preset typical scene.

[0068] That is to say, in the preview state, the mobile phone can know whether the current shooting scene is a preset typical scene, that is, the recognition result, by real-time recognition or perception of the content in the photo preview interface.

[0069] It should be noted that in the preview state, the mobile phone will always use the photo preview interface for real-time perception, so as to decide which photo channel to use for taking photos after receiving the user's photo request.

[0070] In some embodiments, the mobile phone uses a wonderful algorithm (or a wonderful algorithm module) to identify or perceive the content in the photo preview interface in real time to determine whether the current shooting scene is a typical scene or an atypical scene, and outputs a recognition result based on the screen content in the current photo preview interface. Among them, the wonderful algorithm is used to identify whether the current shooting scene is a preset typical scene.

[0071] The embodiment of the present application does not specifically limit the specific algorithm type of the wonderful algorithm, and the picture of the current shooting scene can be identified based on the wonderful algorithm or detection algorithm in the relevant technology.

[0072] Optionally, the recognition result is used to indicate that the current shooting scene is a preset typical scene, or is used to indicate that the current shooting scene is an atypical scene.

[0073] Optionally, the preset typical scene is a wonderful scene. Correspondingly, the atypical scene is a non-wonderful scene.

[0074] Exemplarily, the preset typical scenes include but are not limited to the following scenes: playing basketball, splashing water, smiling faces, jumping, flowers, pets, etc.

[0075] It is understandable that the preset typical scenes can be adjusted or set according to actual needs, and the present application embodiment does not specifically limit this. For example, the preset typical scenes also include dance scenes, looking back scenes, parabolic scenes, table tennis scenes, badminton scenes, running scenes, etc.

[0076] For example, when a user turns on the Eagle Eye capture mode and prepares to capture a person's jumping action, the recognition result corresponding to the time period from the moment the person is detected jumping in the preview screen to the moment the person finishes the jump and falls to the ground is determined as a wonderful scene.

[0077] It should be noted that when the mobile phone determines that the current scene is a preset typical scene, it will cache the image data of the wonderful picture frame in advance in the cache. The following description takes the typical scene as an example of a wonderful scene.

[0078] In some embodiments, when the current shooting scene is identified as a wonderful scene, the mobile phone will pre-select frames in the cache. The image data cached in the cache is the image data corresponding to the wonderful picture frame (or wonderful image frame).

[0079] Optionally, the wonderful picture frames in different wonderful scenes may be determined according to different rules or preset conditions.

[0080] For example, when the current shooting environment is a basketball scene, the mobile phone determines the picture corresponding to the action at the moment of shooting as a wonderful picture frame.

[0081] For another example, when the current shooting scene is a water splashing scene, the picture corresponding to the moment when the mobile phone splashes the water is determined as a wonderful picture frame.

[0082] For another example, when the current shooting scene is a scene containing a face, the mobile phone determines the picture corresponding to the smiling face of the person as a wonderful picture frame.

[0083] For another example, when the current shooting scene is a scene of a person jumping, the mobile phone determines the picture corresponding to the position where the person jumps to the highest point as a wonderful picture frame.

[0084] For another example, when the current shooting scene is a scene containing scenery, the mobile phone determines the picture corresponding to the flowers as the wonderful picture frame.

[0085] For another example, when the current shooting scene includes an animal scene, the mobile phone determines the picture corresponding to the pet as a wonderful picture frame.

[0086] It should be understood that the above examples of wonderful picture frames are merely exemplary descriptions, and the embodiments of the present application do not specifically limit this.

[0087] It should also be understood that the present application embodiment does not specifically limit the determination rule of the wonderful picture frame, and the determination rule of the wonderful picture frame can be determined based on relevant technologies. For example, the determination of the wonderful picture frame can also be combined with one or more factors for evaluating image quality such as image composition evaluation, image clarity, image quality, etc.

[0088] In other embodiments, when it is identified that the current shooting scene is a non-exciting scene, the mobile phone will obtain image data in accordance with the normal camera mode. That is to say, if the current shooting scene is judged to be a non-exciting scene, there is no need to select the frame cache in advance, but to obtain image data according to the normal camera process, for example, obtain the image data corresponding to the current preview screen frame to take a photo. The purpose of doing this is that if the current shooting scene is a non-exciting scene, if there is a scene with variable exposure requirements (for example, high dynamic scenes, night scenes, etc.), the image data can be subjected to variable exposure queue-jumping processing, so as to select variable exposure frames to improve the shooting effect and improve the photo quality. The variable exposure queue-jumping processing can be understood as: by changing the exposure of the image frame, and inserting the image frame after the exposure is changed into the queue, so as to output image frames with better effects.

[0089] In some embodiments, the variable exposure queue-jumping process specifically includes: the AE module sends a set of variable exposure parameters. Optionally, the total exposure amount that can be achieved by a set of exposure parameters can be quantified as the product of exposure time and sensitivity.

[0090] Optionally, the purpose of performing variable exposure processing on image data includes but is not limited to: (1) covering high dynamic range scenes, for example, reducing the exposure value of the image for an overexposed scene, or increasing the exposure value of the image for an underexposed scene; (2) performing noise reduction on multiple frames of image data to adjust the signal-to-noise ratio of the image.

[0091] The above describes how the mobile phone processes image data in different shooting scenarios. The following describes how to dynamically switch the shooting path to take a photo after receiving a user's photo request.

[0092] Step 430: Receive a first operation of the user, where the first operation is used to trigger taking a photo.

[0093] The first operation can be understood as an operation in which a user triggers a mobile phone to take a photo. The specific form of the first operation is not limited in the embodiment of the present application. The first operation can be an operation in which a user clicks a photo control in a photo preview interface, or an operation in which a user presses a photo button of a mobile phone (for example, a physical shortcut key located on a mobile phone), or an operation in which a user presses a photo button of a mobile phone accessory (for example, a headset) (for example, a physical shortcut key located on a mobile phone accessory).

[0094] It should be understood that the above description is only based on the example of the first operation triggering the mobile phone to take a photo, and the embodiments of the present application are not limited to this. In fact, the mobile phone can also take a photo in response to other instructions of the user. For example, the mobile phone takes a photo in response to a voice command input by the user.

[0095] For example, the first action is when the user clicks Figure 1 The operation of the photo taking control 14 shown in (5).

[0096] In an embodiment of the present application, when the recognition result is obtained, the mobile phone responds to the user's first operation and can dynamically switch the photo-taking path according to the recognition result of the wonderful algorithm, and automatically selects the appropriate photo-taking path, which is described below in combination with step 441 and step 442.

[0097] Step 441, in response to the first operation, when the recognition result is a first scene, taking a photo using a first photographing path, the first scene being a preset typical scene.

[0098] That is to say, when the current scene is identified as a typical scene, the first photographing channel will be switched to take a photo. The first photographing channel may also have other names, such as a wonderful scene photographing channel, and so on.

[0099] Optionally, taking a photo using a first photo-taking path includes:

[0100] Acquire first image data from the first cache, wherein the first image data is data of a wonderful picture frame cached in the first cache when the first operation is received;

[0101] Performing image processing on the first image data by a pre-processing algorithm module to obtain second image data;

[0102] The second image data is processed by a first post-processing algorithm module to output a first photographic image frame.

[0103] Among them, the first cache is used to cache image data. Specifically, the first cache is used to ensure the data of the photo frame. After detecting the photo request, the mobile phone obtains the image data of the photo frame from the first cache and returns it to the user. The embodiment of the present application does not limit the specific type of the first cache. For example, the first cache can be an ordinary buffer, or it can be a buffer unique to a certain photo mode, such as a buffer in the Eagle Eye snapshot mode. For example, the first cache is a zero-shutter lag (ZSL) buffer. The ZSL buffer is used to store image data directly from the sensor. In ZSL mode, after the photo command is issued, the system will select the image data of the corresponding frame from the ZSL buffer and send it to the photo stream and thumbnail stream for processing.

[0104] When the first photographing path is adopted, the image data obtained from the first cache may be: data corresponding to wonderful picture frames within a preset time, and the preset time is related to the moment when the user's photographing request is detected. The preset time is related to the moment when the user's photographing request is detected, which can be understood as: the preset time is a period of time before the moment when the user's photographing request is detected; or the preset time is a period of time after the moment when the user's photographing request is detected.

[0105] For example, the wonderful picture frames within the preset time refer to the wonderful image frames within 0.5 seconds after the user's photo request is detected. For another example, the wonderful picture frames within the preset time refer to the wonderful image frames within 0.5 seconds before the user's photo request is detected.

[0106] It should be understood that the above examples about the preset time are only illustrative descriptions, and the embodiments of the present application do not specifically limit the length of the preset time, which may depend on actual needs.

[0107] After obtaining the first image data from the first cache, the first image data may be sent to a photographing algorithm module for processing. Optionally, the photographing algorithm module includes a pre-processing algorithm module and a first post-processing algorithm module.

[0108] Optionally, the pre-processing algorithm module is used to pre-process the image based on the image processing algorithm. For example, the pre-processing includes but is not limited to: format conversion, binarization processing, etc. It should be understood that the embodiment of the present application does not limit the specific processing of the pre-processing algorithm module, and reference may be made to the processing methods in the relevant technology.

[0109] The first post-processing algorithm module is used to perform image post-processing on the image data obtained when taking pictures using the first photographing path. Optionally, the first post-processing algorithm module is used to perform offline processing on the image to improve the image effect. For example, the first post-processing algorithm module is used to perform one or more of the following processing on the image: multi-frame fusion noise reduction processing, multi-frame HDR processing, etc.

[0110] In some embodiments, performing image processing on the second image data by a first post-processing algorithm module to output a first photographic image frame includes:

[0111] When taking a photo with a wide-angle camera or a telephoto camera, the first algorithm link is called to perform image processing on the second image data; wherein, in the first algorithm link, image fusion is performed based on multiple frames (for example, 4 frames) of standard (normal) frames, and no variable exposure processing is performed to obtain a photographed image frame. The first algorithm link is implemented by calling an enhanced high dynamic range (HDRP) algorithm. The essence of the HDRP algorithm is to denoise underexposed images. For example, the first algorithm link is an enhanced high dynamic range lightweight algorithm link (HDRPlite).

[0112] When taking photos with a wide-angle camera or a telephoto camera, the output mode is the binning mode. The binning mode is a way for the sensor to output images. The binning mode refers to adding the charges induced by adjacent pixels together and reading them out in a pixel mode. The image data output by the binning mode is in Bayer format (bayerraw). For example, in the binning mode, the sensor can sum and average the values ​​of four pixels to obtain the value of one pixel, which is output as the processed pixel value.

[0113] Optionally, in response to the user switching the zoom ratio of the photo to below 1X (for example, 0.5X), the mobile phone switches to the wide-angle camera to take pictures.

[0114] The introduction of the above HDRPlite can refer to the description in the related art, and the present embodiment does not specifically limit this. The present embodiment does not limit the uppercase and lowercase letters of HDRPlite. For example, HDRPlite can also be written as hdrplite, and there is no specific limitation on this.

[0115] Alternatively, when taking a photo with the main camera, the first algorithm chain, the second algorithm chain or the third algorithm chain is called to perform image processing on the second image data. The description of the first algorithm chain can be referred to above and will not be repeated here.

[0116] In the second algorithm link, image fusion processing is performed based on 2 long frames (for example, the long frame is a normal frame) and 2 short frames, and no exposure change processing is performed to obtain a first photographed image frame. The second algorithm link is implemented by calling the multi-exposure fusion (MEF) algorithm. For example, the second algorithm link is MEFlite. MEF is a process of synthesizing a set of low dynamic range (LDR) images with different exposures to obtain a visual effect image similar to a high dynamic range (HDR) image.

[0117] Similarly, in the third algorithm link, image fusion processing is performed based on two long frames (for example, the long frame is a normal frame) and two short frames, and no variable exposure processing is performed to obtain a first photographic image frame. For example, the third algorithm link is implemented by a motion capture (MC) algorithm. The MC algorithm can also be understood as a motion photography algorithm. The third algorithm link is MClite.

[0118] When taking pictures with the main camera, since the main camera has different frame output modes (or image output forms), different algorithm links will be called. For example, if the main camera uses HDR mode to output images, the HDRPlite algorithm link is called; if the main camera uses dual analog gain (DAG) or dual conversion gain (DCG) mode to output images, the MEFlite algorithm link is called; if the main camera uses staggered high dynamic range (stagger HDR) frame output mode, the MClite algorithm link is called. Among them, stagger HDR is a technology that can collect multiple frames of images with different exposure brightness in one acquisition cycle by increasing the frame rate of the sensor. For example, stagger HDR can generate long frames and short frames in one acquisition cycle.

[0119] The DAG mode is a technology that uses simultaneous exposure and two different analog gain readouts to obtain two frames of images with different exposures, and then fuses them to obtain a high dynamic image. For a description of the DAG mode, please refer to the description in the related art.

[0120] The DCG mode is a technology that uses simultaneous exposure and two different conversion gain readouts to obtain two frames of images with different exposures, and then fuses them to obtain a high dynamic image. The DCG mode refers to adding DCG to a CMOS pixel so that the sensor has both high sensitivity and high dynamic range capabilities. Exemplarily, DCG includes high conversion gain (HCG) and low conversion gain (LCG). The high HCG conversion gain corresponds to a long exposure frame, and the dark area has a good signal-to-noise ratio. The low LCG conversion gain is a short exposure frame, which can ensure good highlight details. When the sensor works in the DCG mode, the sensor obtains a long exposure frame and a short exposure frame respectively, and then fuses the long exposure frame and the short exposure frame, and the fused image frame is used as the image output by the sensor. In other words, the image output by the sensor in the DCG mode is an image frame after the long exposure frame and the short exposure frame are fused.

[0121] The above processing principle of the first post-processing algorithm module can also be understood by referring to the following Table 1.

[0122] Table 1

[0123]

[0124] Based on the above Table 1, when different cameras are used to take photos, the first post-processing algorithm module uses different algorithm links for post-processing.

[0125] It should be understood that the embodiment of the present application does not specifically limit the specific functions or effects of the first post-processing algorithm module. The first post-processing algorithm module may also include other image processing, such as Bayer domain processing, cropping processing, up-scaling processing, etc.

[0126] Step 442, in response to the first operation, when the recognition result is a second scene, taking a picture using a second photographing path, the second scene being an atypical scene.

[0127] Wherein, the first photographing passage and the second photographing passage are different photographing passages.

[0128] Optionally, taking photos using a second photo-taking path includes:

[0129] Acquire third image data, wherein the third image data is image data corresponding to a variable exposure image frame determined when the first operation is received, the variable exposure image frame refers to an image frame generated based on a first exposure parameter sent by an automatic exposure (AE) module, and the first exposure parameter is different from an exposure parameter corresponding to a preview frame;

[0130] Performing image processing on the third image data by a pre-processing algorithm module to obtain fourth image data;

[0131] The fourth image data is processed by a second post-processing algorithm module to output a second photographic image frame.

[0132] Exemplarily, for a dark light shooting scene, when taking a picture using the second photographing path, when receiving a photographing request, multiple image frames are selected from the photographing queue of the preview image frames, and the multiple image frames are the image frames in the photographing queue that are closest to the photographing command. The AE module sends a variable exposure parameter (for example, a first exposure parameter) to the bottom layer (for example, from the framework layer, the HAL layer, and the driver layer to the sensor in sequence). The first exposure parameter is different from the exposure parameter corresponding to the preview picture frame. The sensor takes effect on the variable exposure parameter sent by the AE module and outputs multiple variable exposure frames. The sensor sends the multiple variable exposure frames to the photographing algorithm module (for example, the second post-processing algorithm module) for further image processing.

[0133] The first exposure parameter is used to adjust the exposure. Optionally, the first exposure parameter includes exposure time (eg, 10 ms) and sensitivity (International Organization for Standardization, ISO).

[0134] For the description of the pre-processing algorithm module, please refer to the previous introduction. For the sake of brevity, it will not be repeated here.

[0135] The second post-processing algorithm module is used to provide a post-processing process for the image data obtained when taking pictures through the second photographing path. Exemplarily, the specific processing method of the second post-processing algorithm module can refer to the post-processing algorithm module under the normal photographing mode (or the conventional photographing process). For example, the second post-processing algorithm module is used to perform one or more of the following processing on the image: multi-frame fusion noise reduction processing, multi-frame HDR processing, etc.

[0136] The above processing principle of the second post-processing algorithm module can also be understood by referring to the following Table 2.

[0137] Table 2

[0138]

[0139] Based on Table 2 above, when taking photos with different cameras, the second post-processing algorithm module uses the same algorithm link for post-processing. The specific process can refer to the common photo process. For example, the algorithm link of the common photo mode includes any of the following algorithm links: HDR algorithm link; MEF algorithm link, etc.

[0140] The above description is an implementation method for taking photos using the second photo path in a scenario with a variable exposure requirement. Of course, for a scenario without a variable exposure requirement, when a photo request is received, the image data corresponding to the preview screen frame is sent to the photo algorithm module for image processing. In some embodiments, taking photos using the second photo path includes: obtaining the image data corresponding to the preview screen frame, and sending it to the photo algorithm module for image processing.

[0141] Optionally, during software design, the first post-processing algorithm module and the second post-processing algorithm module may be different algorithm modules, so as to post-process the image data transmitted from the corresponding photographing path. That is to say, for the snapshot mode, compared with the method of using the same post-processing algorithm module (or post-processing algorithm link) for image processing, the embodiment of the present application provides two types of post-processing algorithm modules, namely, a first post-processing algorithm module for typical scenes (the specific implementation process can be referred to in the previous text), and a second post-processing algorithm module for atypical scenes (the specific implementation process can be referred to in the previous text), so as to execute the post-processing process in a targeted manner, thereby improving the image effect.

[0142] Optionally, in the embodiment of the present application, whether it is the first photographing channel or the second photographing channel, in addition to the image processing process based on the processing algorithm module described above, one or more of the following processing can be performed on the image: bad pixel correction (BPC), black level correction (BLC), lens shade correction (LSC), automatic white balance (AWB), Bayer domain noise reduction (NR), Demosaic, color correction (CC), YUV domain noise reduction NR, color enhancer (CE), sharpen, tone mapping, etc., and the embodiment of the present application does not specifically limit this. Of course, the order of executing the image processing algorithms listed here and the image processing performed by the aforementioned post-processing algorithm module is not limited.

[0143] In some embodiments, when the user triggers a photo operation in a certain snapshot mode, based on the above steps 441 and 442, dynamic switching of the photo path can be achieved. Compared with the traditional snapshot mode that cannot achieve variable exposure processing, the embodiment of the present application not only provides a first photo path, but also provides a second photo path, so that the snapshot mode can achieve variable exposure processing while having the snapshot function, and can cope with high-dynamic shooting scenes or dark light scenes, thereby providing a better shooting function for dealing with complex shooting environments, and improving the image effect of non-wonderful pictures in the snapshot mode. In the embodiment of the present application, after a high-dynamic shooting scene or a dark light scene, the user does not need to manually switch from the snapshot mode to other photo modes, and the automatic switching of the photo path can be achieved, avoiding switching back and forth, which greatly improves the user's shooting experience. Furthermore, the embodiment of the present application provides corresponding post-processing algorithm processes for the first photo path and the second photo path, respectively, to further improve the image quality.

[0144] The following combination Figure 3 and Figure 4 Describe the comparative examples of photographing effects of the embodiments of the present application.

[0145] For the same high dynamic range shooting scene, after the user clicks to take a photo, Figure 3 (1) shows the photos captured in the traditional capture mode; Figure 3 (2) shows the photos captured in the snapshot mode of the embodiment of the present application. It can be clearly seen that Figure 3 The photo shown in (2) is much better than Figure 3 The photo shown in (1) is shown in the figure. For example, Figure 3 The details of the sky and the ground in the photo shown in (2) will be clearer. That is to say, the image processing method of the embodiment of the present application, or the snapshot mode provided by the embodiment of the present application, has a better snapshot effect in high-dynamic scenes.

[0146] For the same night scene shooting scene, after the user clicks to take a photo, Figure 4 (1) shows the photos captured in the traditional capture mode; Figure 4 (2) shows the photos captured in the snapshot mode of the embodiment of the present application. It can be clearly seen that Figure 4 The photo shown in (2) is much better than Figure 4 The photo shown in (1) is shown in the figure. For example, Figure 4 The details of the sky and the ground in the photo shown in (2) will be clearer. That is to say, the image processing method of the embodiment of the present application, or the snapshot mode provided by the embodiment of the present application, has a better snapshot effect in night scenes.

[0147] Understandably, Figure 3or Figure 4 The photo comparison shown in the figure is only an example description, and the embodiments of the present application are not limited to this.

[0148] The following combination Figures 5 to 7 Describe the software system and hardware architecture used in the embodiments of the present application.

[0149] Figure 5 It is a schematic diagram of the architecture (including software system and some hardware) of the embodiment of the present application. Figure 5 As shown, the application architecture is divided into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the application architecture can be divided into five layers, from top to bottom, namely, the application layer, the application framework layer, the hardware abstraction layer HAL, the driver layer, and the hardware layer.

[0150] like Figure 5 As shown, the application layer includes the camera and the gallery. It can be understood that Figure 5 The application layer includes some applications, and in fact, the application layer may also include other applications, which are not limited in this application. For example, the application layer also includes information, alarm clock, weather, stopwatch, compass, timer, flashlight, calendar, Alipay and other applications.

[0151] like Figure 5 As shown, the application framework layer includes a camera access interface. The camera access interface includes camera management and camera devices. The hardware abstraction layer includes a camera hardware abstraction layer and a camera algorithm library. Among them, the camera hardware abstraction layer includes multiple camera devices. The camera algorithm library includes a perception engine, a first photo path, a second photo path, and a photo algorithm processing module.

[0152] The perception engine is used to recognize the picture in the preview state to obtain a recognition result. In some embodiments, Figure 6 As shown, the perception engine includes wonderful algorithm modules.

[0153] After receiving the user's photo command, which photo path is used depends on the recognition result obtained by the wonderful algorithm module. The first photo path is the photo path used when a typical scene is recognized, for example, Figure 6 The second photographing path is the photographing path used when an atypical scene is identified, for example, Figure 6 The process of selecting frames in advance by the frame selection module in SIT is shown in FIG.

[0154] It should be understood that the perception engine can also be set in other layers. As a possible implementation, the perception engine can be placed in the application layer or the application framework layer.

[0155] The photographing algorithm processing module is used to perform image processing on the image data. Optionally, the photographing algorithm module includes a pre-processing algorithm module and a post-processing algorithm module. The post-processing algorithm module includes a first post-processing algorithm module and a second post-processing algorithm module. For example, the photographing algorithm module is Figure 6 The photo-taking algorithm module shown in .

[0156] The driver layer is used to drive hardware resources. The driver layer can include multiple driver modules. Figure 5 As shown, the driver layer includes camera device driver, digital signal processor driver and graphics processor driver, etc.

[0157] The hardware layer includes a sensor, an image signal processor, a digital signal processor and a graphics processor. Among them, the sensor includes multiple sensors, a TOF camera and a multi-spectral sensor. Optionally, the image signal processor may include an ISP first module, an ISP second module and an ISP third module.

[0158] For example, a user can click on a camera application. When the user clicks on the camera to start the camera, the command to start the camera can be sent to the camera hardware abstraction layer through the camera access interface. The camera hardware abstraction layer calls the camera device driver and calls the camera algorithm library. The perception engine in the camera algorithm library will recognize the preview image in real time and obtain the recognition result (for example, a typical scene or an atypical scene).

[0159] When the user clicks the photo control, the photo command can be sent to the camera hardware abstraction layer through the camera access interface. The camera hardware abstraction layer calls the camera device driver and the camera algorithm library. The perception engine in the camera algorithm library decides whether to use the first photo path or the second photo path to take a photo based on the above recognition results. Furthermore, the image data obtained through the photo path is sent to the photo algorithm processing module for processing.

[0160] Optionally, the camera algorithm library is also used to send digital signals to the digital signal processor driver in the driver layer, so that the digital signal processor driver calls the digital signal processor in the hardware layer to perform digital signal processing. The digital signal processor can return the processed digital signal to the camera algorithm library through the digital signal processor driver. The camera algorithm library is also used to send digital signals to the graphics signal processor driver in the driver layer, so that the graphics signal processor driver calls the graphics processor in the hardware layer to perform digital signal processing. The graphics processor can return the processed graphics data to the camera algorithm library through the graphics processor driver.

[0161] In addition, the image output by the image signal processor can be sent to the camera device driver. The camera device driver can send the image output by the image signal processor to the camera hardware abstraction layer. The camera hardware abstraction layer can send the image to the post-processing algorithm module for further processing, or send the image to the camera access interface. The camera access interface can send the image returned by the camera hardware abstraction layer to the camera.

[0162] The following combination Figure 6 Describe the detailed architecture of the embodiment of the present application. For example, Figure 6 As shown, when the user clicks to start the camera and enters the preview state, the phone displays the preview interface, for example, Figure 1 In the interface shown in (5), the wonderful algorithm module in the perception engine perceives the picture of the preview interface in real time and determines whether the current scene is a wonderful scene or a non-wonderful scene. When the current shooting scene is a wonderful scene, the wonderful algorithm module will notify the super image turbo (SIT) to select frames in advance for caching. Specifically, the frame selection module in the SIT can cache wonderful image frames in advance.

[0163] When the user's photo request is issued, the mobile phone dynamically switches the photo path according to the recognition result of the wonderful algorithm. Specifically, when the wonderful algorithm module recognizes that the current photo scene is a wonderful scene, it will use the cached frame selected in advance by the wonderful algorithm to send it to the photo algorithm (photo module) for processing; when it is recognized that the current scene is not a wonderful scene, the chip platform will select frames according to the user's click time (that is, the process of ordinary photo taking). In the snapshot mode, the embodiment of the present application can perform variable exposure queue interruption processing according to the current shooting scene by dynamically switching to the second photo path, and select variable exposure frames to improve the photo effect.

[0164] Furthermore, in the post-processing process of the photo-taking algorithm, the embodiment of the present application provides a first post-processing algorithm module for typical scenes and a second post-processing algorithm module for atypical scenes, that is, different features are selected for different scenes for photo post-processing, which can further improve the image effect. For the functions of the first post-processing algorithm module and the second post-processing algorithm module, please refer to the description of the method embodiment above, which will not be repeated here.

[0165] The above describes in detail the software system used in the embodiment of the present application. Figure 7 The hardware system of the electronic device 1000 is described.

[0166] Figure 7 A structural schematic diagram of an electronic device 1000 suitable for the present application is shown.

[0167] The electronic device 1000 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0168] It should be noted that Figure 7 The structure shown does not constitute a specific limitation on the electronic device 1000. In other embodiments of the present application, the electronic device 1000 may include Figure 7 The electronic device 1000 may include more or fewer components than those shown. Figure 7 A combination of some of the components shown, or the electronic device 1000 may include Figure 7 Subcomponents of some of the components shown. For example, Figure 7 The illustrated proximity light sensor 180G may be optional. Figure 7 The components shown may be implemented in hardware, software, or a combination of software and hardware.

[0169] The processor 110 may include one or more processing units. For example, the processor 110 may include at least one of the following processing units: an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and a neural-network processing unit (NPU). Different processing units may be independent devices or integrated devices.

[0170] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0171] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0172] In some embodiments, the processor 110 may include one or more interfaces. For example, the processor 110 may include at least one of the following interfaces: an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a SIM interface, and a USB interface.

[0173] Figure 7 The connection relationship between the modules shown is only a schematic illustration and does not constitute a limitation on the connection relationship between the modules of the electronic device 1000. Optionally, the modules of the electronic device 1000 may also adopt a combination of multiple connection modes in the above embodiments.

[0174] The charging management module 140 is used to receive power from the charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive current from the wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive electromagnetic waves through the wireless charging coil of the electronic device 1000 (the current path is shown as a dotted line). While the charging management module 140 is charging the battery 142, it can also power the electronic device 1000 through the power management module 141.

[0175] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, and battery health status (e.g., leakage, impedance). Optionally, the power management module 141 can be set in the processor 110, or the power management module 141 and the charging management module 140 can be set in the same device.

[0176] The wireless communication function of the electronic device 1000 can be implemented through components such as the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor.

[0177] The electronic device 1000 can realize the display function through the GPU, the display screen 194 and the application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.

[0178] The display screen 194 can be used to display images or videos. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a mini light-emitting diode (Mini LED), a micro light-emitting diode (Micro LED), a micro OLED (Micro OLED) or a quantum dot light emitting diode (QLED). In some embodiments, the electronic device 1000 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0179] The electronic device 1000 can realize the shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194 and the application processor.

[0180] The ISP is used to process the data fed back by the camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can perform algorithm optimization on the noise, brightness and color of the image. The ISP can also optimize the exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0181] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard red green blue (RGB), YUV or other format. In some embodiments, the electronic device 1000 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0182] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 1000 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0183] Video codecs are used to compress or decompress digital videos. The electronic device 1000 may support one or more video codecs. Thus, the electronic device 1000 may play or record videos in a variety of coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0184] NPU is a processor that draws on the structure of biological neural networks, such as the transmission mode between neurons in the human brain to quickly process input information, and can also continuously self-learn. Through NPU, the electronic device 1000 can realize intelligent cognition and other functions, such as image recognition, face recognition, voice recognition and text understanding.

[0185] The electronic device 1000 can implement audio functions, such as music playing and recording, through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0186] It can be understood that the image processing method of the embodiment of the present application can be applied to Figure 7 In the electronic device shown in , the specific implementation steps can refer to the introduction of the method embodiment above, which will not be repeated here.

[0187] The present application also provides a computer program product, which, when executed by a processor, implements the method described in any method embodiment of the present application.

[0188] The computer program product can be stored in a memory, and is finally converted into an executable target file that can be executed by a processor after preprocessing, compiling, assembling and linking.

[0189] The present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a computer, the method described in any method embodiment of the present application is implemented. The computer program can be a high-level language program or an executable target program.

[0190] The computer-readable storage medium may be a volatile memory or a nonvolatile memory, or may include both a volatile memory and a nonvolatile memory. The nonvolatile memory may be a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus RAM (DR RAM).

[0191] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described devices and equipment and the technical effects produced can refer to the corresponding processes and technical effects in the aforementioned method embodiments, and will not be repeated here.

[0192] In several embodiments provided in the present application, the disclosed systems, devices and methods can be implemented in other ways. For example, some features of the method embodiments described above can be ignored or not performed. The device embodiments described above are merely schematic, and the division of units is only a logical function division. There may be other division methods in actual implementation, and multiple units or components may be combined or integrated into another system. In addition, the coupling between the units or the coupling between the components may be direct coupling or indirect coupling, and the above coupling includes electrical, mechanical or other forms of connection.

[0193] It should be understood that in the various embodiments of the present application, the size of the serial number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0194] In addition, the terms "system" and "network" are often used interchangeably in this article. The term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0195] In short, the above is only a preferred embodiment of the technical solution of this application, and is not intended to limit the protection scope of this application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application should be included in the protection scope of this application.

Claims

1. An image processing method, characterized in that: include: Display the photo preview interface; Recognize the content in the photo preview interface to obtain a recognition result, wherein the recognition result is used to indicate whether the current shooting scene is a preset typical scene; Receiving a first operation of a user, where the first operation is used to trigger taking a photo; In response to the first operation, when the recognition result is a first scene, taking a photo using a first photographing path, the first scene being a preset typical scene; In response to the first operation, when the recognition result is a second scene, a second photographing path is used to take a photo, the second scene is an atypical scene, and the first photographing path is different from the second photographing path.

2. The method according to claim 1, characterized in that The method of taking photos by using the first photo taking path includes: Acquire first image data from the first cache, wherein the first image data is data of a wonderful image frame cached in the first cache when the first operation is received; Performing image processing on the first image data by a pre-processing algorithm module to obtain second image data; The second image data is processed by a first post-processing algorithm module to output a first photographic image frame.

3. The method according to claim 2, characterized in that The performing image processing on the second image data by using a first post-processing algorithm module comprises: When a wide-angle camera or a telephoto camera is used, calling a first algorithm link to perform image processing on the second image data; Alternatively, when the main camera is used, the first algorithm link, the second algorithm link or the third algorithm link is called to perform image processing on the second image data; Among them, in the first algorithm link, image fusion is performed based on multiple standard frames; in the second algorithm link, image fusion is performed based on multiple standard frames and multiple short frames.

4. The method according to any one of claims 1 to 3, characterized in that The method of taking photos by using the second photo taking path includes: Acquire third image data, wherein the third image data is image data corresponding to a variable exposure image frame determined when the first operation is received, and the variable exposure image frame refers to an image frame generated based on a first exposure parameter sent by an automatic exposure AE module, and the first exposure parameter is different from an exposure parameter corresponding to a preview screen frame; Performing image processing on the third image data by a pre-processing algorithm module to obtain fourth image data; The fourth image data is processed by a second post-processing algorithm module to output a second photographic image frame.

5. The method according to any one of claims 1 to 4, characterized in that The display photo preview interface: In response to the user selecting the snapshot mode, the photo preview interface is displayed, and the photo preview interface is a preview interface in the snapshot mode.

6. The method according to any one of claims 1 to 5, characterized in that The identifying the content in the photo preview interface to obtain the identification result includes: The content in the photo preview interface is recognized in real time through a wonderful algorithm module to obtain the recognition result.

7. The method according to any one of claims 1 to 6, characterized in that The preset typical scenes include any one of the following scenes: shooting basketball, splashing water, smiling face, jumping, flowers and pets.

8. An electronic device, characterized in that: The electronic device comprises a processor and a memory, wherein the processor and the memory are coupled, and the memory is used to store a computer program. When the computer program is executed by the processor, the electronic device executes the method according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the electronic device executes the method according to any one of claims 1 to 7.

10. A chip system, characterized in that: The chip system is applied to an electronic device, and the chip system includes one or more processors, and the processor is used to call computer instructions so that the electronic device executes the method as described in any one of claims 1 to 7.

Images