Image processing method, electronic equipment and computer readable storage medium

Abstract

The invention provides an image processing method, electronic equipment and a computer readable storage medium, and relates to the technical field of terminals, the method is applied to the electronic equipment comprising an image sensor, and the method comprises the following steps: displaying a shooting preview interface of a camera application, the shooting preview interface comprising a shooting control; in response to a trigger operation of a user on the shooting control, storing the image file; wherein the image file comprises a first coded image and a second coded image which are obtained based on the original image, the first coded image comprises original brightness information in the original image, the first coded image is used for brightening the second coded image when the image file is displayed, and the original image is acquired and acquired by the image sensor in response to the triggering operation.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to an image processing method, an electronic device, and a computer-readable storage medium. Background Art

[0002] With the development of electronic technology, most electronic devices support the function of taking and displaying photos, making it convenient for users to carry electronic devices with them to record their lives.

[0003] However, since most users who currently use electronic devices to take photos are not professional photographers, they do not consider the impact of light on photos when taking photos, and they do not control the aperture, shutter speed (i.e., exposure time) and light sensitivity (light sensibility ordinance, ISO) during the shooting process, which results in the electronic device being unable to take photos with good effects. For example, if the ambient light is dim during shooting, the electronic device is likely to take photos that lose image details. Summary of the invention

[0004] In view of this, the present application provides an image processing method, an electronic device and a computer-readable storage medium, which can retain the brightness information obtained by shooting to improve the display effect of the photo.

[0005] In a first aspect, the present application provides an image processing method, which is applied to an electronic device including an image sensor, the method comprising: displaying a shooting preview interface of a camera application, the shooting preview interface including a shooting control;

[0006] In response to a user triggering operation on a shooting control, storing an image file;

[0007] Among them, the image file includes a first encoded image and a second encoded image obtained based on the original image, the first encoded image includes original brightness information in the original image, the first encoded image is used to brighten the second encoded image when displaying the image file, and the original image is acquired by the image sensor in response to a trigger operation.

[0008] The original brightness information includes brightness contrast information.

[0009] The first coded image includes original brightness information, and the second coded image may be an image that can be displayed on a display screen of an electronic device after the original image is processed. Since the original image includes noise information, etc., the brightness and content of the original image need to be adjusted during the process of obtaining the second coded image through processing.

[0010] When the user clicks the shooting control in the shooting preview interface, the electronic device can obtain the original image through the image sensor, and obtain an image file including a first coded image and a second coded image based on the original image, wherein the first coded image includes the original brightness information in the original image. In this way, the brightness information obtained by shooting can be retained to improve the display effect of the photo.

[0011] In a possible implementation manner of the first aspect, the method further includes: in response to a user opening operation of opening the image file, displaying a brightened image, where the brightened image is obtained by brightening the second encoded image based on the first encoded image.

[0012] After storing the image file, the terminal device can display the image file in response to the user's opening operation. In the case where the electronic device can read the first coded image and the second coded image in the image file, the electronic device can use the first coded image to brighten the second coded image, so that more image details can be displayed when displaying the image file, thereby improving the display quality of the image file.

[0013] In a possible implementation of the first aspect, in response to the current shooting mode being a preset shooting mode, or in response to the preview interface including a preset picture, the step of storing the image file is performed; wherein the preset shooting mode is one or more of the following modes: landscape shooting mode, food shooting mode, night scene shooting mode; the preset picture is a picture that does not include a portrait.

[0014] Based on this, the image file including the first encoded image and the second encoded image may be stored in a preset shooting mode to improve storage efficiency.

[0015] In a possible implementation of the first aspect, determining a first coded image and a second coded image based on an original image includes: acquiring a first image in a YUV format and a second image in a YUV format based on the original image, wherein the first image includes original brightness information; and performing image encoding on the first image and the second image, respectively, to acquire a first coded image and a second coded image.

[0016] The original image includes a large amount of data, while the YUV format image includes a small amount of data. Therefore, the electronic device can first convert the original image into a first image in YUV format and a second image in YUV format, and encode based on the first image and the second image to improve encoding efficiency.

[0017] In a possible implementation of the first aspect, image encoding is performed on the first image and the second image respectively to obtain the first encoded image and the second encoded image, including: obtaining a rotation parameter, where the rotation parameter is determined based on the posture of the electronic device; rotating the first image and the second image based on the rotation parameter; and image encoding is performed on the rotated first image and the rotated second image respectively to obtain the first encoded image and the second encoded image.

[0018] When taking a photo, the posture of the electronic device may be in a rotated state. In order to ensure that the image file obtained by the capture can keep the direction of the actual captured object consistent with the scene viewed by the human eye when displayed, the electronic device can first determine the rotation parameters based on the posture of the electronic device, and then rotate the first image and the second image based on the rotation parameters to obtain the first encoded image and the second encoded image, thereby improving the display effect of the image file.

[0019] In a possible implementation manner of the first aspect, obtaining a first coded image and a second coded image includes:

[0020] When the format of the first image is the first format, the first image is encoded by the first hardware encoding module to obtain a first encoded image; the first format is an encoding format supported by the first hardware encoding module; when the format of the first image is not the first format, the first image is encoded by the first software encoding module to obtain the first encoded image; when the format of the second image is the second format, the second image is encoded by the second hardware encoding module to obtain a second encoded image; the second format is an encoding format supported by the second hardware encoding module; when the format of the second image is not the second format, the second image is encoded by the second software encoding module to obtain a second encoded image.

[0021] Among them, the hardware encoding module can be called an offline encoding module, and the software encoding module can be called an online encoding module.

[0022] If the first image is in the first format supported by the first hardware encoding module, it indicates that the first hardware encoding module can encode the first image to obtain a first encoded image. If the first image is in a format not supported by the first hardware encoding module, the first software encoding module can be used to encode the first image to obtain a first encoded image. Correspondingly, the same is true for the second image. The hardware encoding module has a fast encoding speed and a stable encoding process, which can improve the encoding efficiency, and the software encoding module supports many encoding formats. Therefore, through the above scheme, it can be ensured that the first image and the second image can be encoded to obtain the first encoded image and the second encoded image.

[0023] In a possible implementation manner of the first aspect, the original image is a RAW image, and a first image in YUV format and a second image in YUV format are respectively acquired based on the original image, including: converting the RAW image into the first image in YUV format through a first conversion module, and the first conversion module does not change the original brightness information; converting the RAW image into the second image in YUV format through a second conversion module, and the second conversion module changes the original brightness information.

[0024] In the above process, the first image is processed by the first module that does not change the original brightness information to ensure that the original brightness information can be retained in the first encoded image. In addition, the second image is processed by the second module that can change the original brightness information to ensure the display effect of the image file.

[0025] In a possible implementation manner of the first aspect, the method also includes: obtaining image editing parameters, the image editing parameters corresponding to the current shooting mode, the image editing parameters including a first parameter and a second parameter; the first parameter is used to indicate the modification of the image shape in the image, and the second parameter is used to indicate the modification of the color in the image; converting the RAW image into a first image in YUV format through a first conversion module, including: processing the RAW image based on the first parameter through the first conversion module to obtain the first image in YUV format; converting the RAW image into a second image in YUV format through a second conversion module, including: processing the RAW image based on the first parameter and the second parameter through the second conversion module to obtain the second image in YUV format.

[0026] The modification of the image shape refers to a situation where the image is changed due to operations such as perspective distortion and cropping, for example, the lines are bent or stretched.

[0027] After the first conversion module processes the RAW image based on the first parameter, the image shape in the first image in the YUV format is modified. Then, the corresponding second conversion module also needs to process the RAW image based on the first parameter to obtain the second image with the modified image shape. Since the same first parameter is used, the content of the first image and the second image is still consistent. In addition, there is information such as noise in the original image, so the second conversion module also needs to be in the RAW image based on the second parameter to obtain a second image with a higher display effect.

[0028] In a possible implementation of the first aspect, the original image is a RAW image, and the first image and the second image are respectively acquired based on the original image, including: converting the RAW image of the original image into a first image in a YUV format through a third conversion module, and the first conversion module does not change the original brightness information; converting the first image in a YUV format into a second image through a fourth conversion module, and the second conversion module changes the original brightness information in the first image.

[0029] In a possible implementation manner of the first aspect, the method further includes:

[0030] Acquire an image editing parameter, the image editing parameter corresponds to the current shooting mode, and the image editing parameter includes a first parameter and a second parameter; the first parameter is used to instruct the electronic device to modify the image shape in the image, and the second parameter is used to instruct the electronic device to modify the color in the image;

[0031] The RAW image of the original image is converted into a first image in a YUV format by a third conversion module, including:

[0032] Processing the RAW image based on the first parameter by a third conversion module to obtain a first image in a YUV format;

[0033] Converting the first image in YUV format into the second image in YUV format by the fourth conversion module includes:

[0034] The first image in the YUV format is processed based on the second parameter by the fourth conversion module to obtain the second image in the YUV format.

[0035] After the third conversion module processes the RAW image based on the first parameter, the image shape in the first image in the acquired YUV format is modified, and then the fourth conversion module can directly process the first image based on the second parameter to obtain the second image. In this way, only the third conversion module needs to perform the processing of the first parameter, which can ensure that the image shapes in the first image and the second image are completely consistent and improve the processing efficiency.

[0036] In a possible implementation of the first aspect, the first parameter includes a wide-angle distortion processing parameter and / or a cropping parameter, and the second parameter includes a color beautification parameter. During the shooting process, wide-angle distortion problems are prone to occur due to perspective, and the image size is too large. Therefore, the first parameter may include a wide-angle distortion processing parameter and / or a cropping parameter. There are noise points and color inconsistency problems in the original image. Therefore, the second parameter may include a color beautification parameter.

[0037] In a possible implementation manner of the first aspect, storing an image file includes: storing a first encoded image and a second encoded image in a first buffer, or splicing a second buffer corresponding to the first encoded image and a third buffer corresponding to the second encoded image into a first buffer; and storing the image file in the first buffer.

[0038] By means of the above method, the first encoded image and the second encoded image can be stored in the same first buffer, thereby obtaining a complete image file.

[0039] In a possible implementation manner of the first aspect, determining a first coded image and a second coded image based on an original image includes: using a first thread to determine the first coded image based on the original image; and using a second thread to determine the second coded image based on the original image.

[0040] By using the above method, encoding is performed in a multi-threaded manner, which can improve encoding efficiency.

[0041] In a possible implementation manner of the first aspect, the first coded image is a single-channel coded image, and the second coded image is a three-channel color coded image.

[0042] Since the first coded image retains the original brightness information, the first coded image can be a single-channel coded image to reduce the memory occupied by the image file. The third coded image is a three-channel color coded image, which can make the image file have richer colors when displayed, ensuring the display quality of the image.

[0043] In a second aspect, the present application provides an electronic device, comprising a display screen, a memory and one or more processors; the display screen, the memory and the processor are coupled; the display screen is used to display an image generated by the processor, and the memory is used to store computer program code, and the computer program code includes computer instructions; when the processor executes the computer instructions, the electronic device executes the method described in the first aspect and any possible design method thereof.

[0044] In a third aspect, the present application provides a computer-readable storage medium, comprising computer instructions, which, when executed on an electronic device, enables the electronic device to execute the method described in the first aspect and any possible design thereof.

[0045] In a fourth aspect, the present application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the method described in the first aspect and any possible design thereof.

[0046] In a fifth aspect, the present application provides a device, which is included in an electronic device, and the device has the function of implementing the behavior of the electronic device in any of the above aspects and possible implementation methods. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes at least one module or unit corresponding to the above function. For example, an allocation module or unit, a scanning module or unit, a recycling module or unit, a moving module or unit, and a storage module or unit, etc.

[0047] In a sixth aspect, an embodiment of the present application provides a chip system, which includes a processor and may also include a memory, for implementing any one of the methods provided in the first aspect. The chip system may be composed of a chip, or may include a chip and other discrete devices.

[0048] It can be understood that the electronic device described in the second aspect and any possible design thereof provided above, the computer-readable storage medium described in the third aspect, and the computer program product described in the fourth aspect are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 A schematic diagram of a related technology photo processing provided by an embodiment of the present application;

[0050] Figure 2 A schematic diagram of photo processing provided in an embodiment of the present application;

[0051] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application

[0052] Figure 4 A software structure block diagram of an electronic device provided in an embodiment of the present application;

[0053] Figure 5 A schematic diagram of an image processing method provided in an embodiment of the present application Figure 1 ;

[0054] Figure 6 A schematic diagram of a rotational shooting of an electronic device provided in an embodiment of the present application;

[0055] Figure 7 A schematic diagram of an image file provided in an embodiment of the present application;

[0056] Figure 8 A schematic diagram of an image processing method provided in an embodiment of the present application Figure 2 ;

[0057] Fig. 9 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application Figure 1 ;

[0058] Fig.10 A schematic diagram of an image processing method provided in an embodiment of the present application Figure 3 ;

[0059] Fig.11 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application Figure 2 . DETAILED DESCRIPTION

[0060] In the following, the terms "first" and "second" are used only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this embodiment, "at least one" refers to one or more, and "multiple" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0061] It should be noted that in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0062] Before introducing the embodiments of the present application, the technologies involved in the embodiments of the present application are first introduced in detail.

[0063] 1. Image channels, single-channel images, and multi-channel images

[0064] In different image modes, the image channels have different meanings. Generally, in the red, green, blue (RGB) color mode, the image channels are used to store image color data, and one channel represents the brightness information of one color.

[0065] A single-channel image refers to an image with only one channel. Since one channel represents only one color, each pixel in a single channel has only one pixel value.

[0066] For example, each pixel of a single-channel image uses a value to represent the color, and the pixel value of each pixel ranges from 0 to 255, where a pixel with a value of 0 is black, a pixel with a value of 255 is white, and a pixel with an intermediate value between 0 and 255 is different shades of gray, and the closer the pixel value is to 255, the lighter the gray it is. Therefore, a single-channel image is generally a grayscale image. Since each pixel of a single-channel image has only one value to represent the color, and the size of the value can represent the brightness information, each pixel of a single-channel image can be regarded as an image that only carries brightness information.

[0067] A multi-channel image refers to an image with multiple channels. Since one channel represents only one color, each pixel in a multi-channel image uses multiple pixel values ​​to represent different colors.

[0068] Common three-channel images include RGB images. Each pixel of an RGB image includes three pixel values, each of which represents red, green, and blue, and the size of the pixel value represents the brightness of the color. The pixel value of each channel in a pixel represents a color, and the size of each pixel value represents the brightness of the color. For example, a pixel whose pixel values ​​of the three RGB channels are [0, 0, 255] is blue, a pixel whose pixel values ​​of the three RGB channels are [255, 0, 0] is red, and a pixel whose pixel values ​​of the three RGB channels are [0, 255, 0] is green. Pixels composed of three colors with different brightness can be expressed as different colors. For example, red light, green light, and blue light can be superimposed on white light, so the pixel whose pixel values ​​of the three RGB channels are [255, 255, 255] is white. For another example, in the absence of red light, green light, and blue light, black can be obtained, that is, the pixel whose pixel values ​​of the three RGB channels are [0, 0, 0] is black.

[0069] In the related art, since the display screen of an electronic device generally uses light-emitting diodes of three colors, red, green and blue, to realize the display function, the image displayed on the display screen is generally an RGB image.

[0070] Another common three-channel image is the YUV image. In the YUV image, Y represents luminance (Luminance or Luma), and U and V represent chrominance (Chrominance or Chroma). Each pixel in the YUV image contains Y, U, and V components, where the Y component is independent of the U and V components. If each pixel in the YUV image does not have a U component and a V component, then the YUV image is a grayscale image.

[0071] The Y component is generally completely retained in the YUV image, while the U and V components can be obtained through sampling. Therefore, when the image quality is not significantly reduced, the YUV image can effectively reduce the memory space it occupies. Therefore, in the process of transmission and image processing, YUV images are generally used to improve efficiency.

[0072] 2. Image Coding

[0073] Image coding is also called image compression. The main purpose of image coding format is to reduce redundant information in image data, retain image quality and details, and make the image as small as possible, so as to store and transmit digital images more efficiently.

[0074] At present, the formats of image coding include the JPEG format developed by the Joint Photographic Experts Group, the portable network graphics (PNG) format, the graphics interchange format (GIF), etc. It should be noted that the above only gives some common graphic coding formats, and the details can be found in the introduction of related technologies, which will not be repeated here.

[0075] Since JPEG encoding uses lossy compression to remove redundant image data, better image quality can be obtained with less disk space, and JPEG encoding allows files to be compressed with different compression ratios, so it supports multiple compression levels. In addition, JPEG encoding mainly compresses high-frequency information and can better retain color information. Therefore, JPEG encoding is currently commonly used for photos taken by electronic devices. In this way, the subsequent embodiments introduce the present application scheme with JPEG encoding, but the encoding method involved in the embodiments of the present application is not limited to JPEG encoding, and other encoding methods can also be selected according to actual application requirements.

[0076] The image processing method provided in the embodiments of the present application is introduced below with reference to the accompanying drawings.

[0077] In the scenario where users use electronic devices to take photos on a daily basis, the photosensitive element in the electronic device will output a raw image (RAW) according to the user's photo operation, and the photo processing module in the electronic device can process the RAW to obtain a photo. Generally, users expect the photos they take to be in color. Therefore, if Figure 1 As shown, after the RAW is processed by the photo processing module, the obtained photo is a three-channel image that can be expressed as a color image. In addition, in general, the photo needs to be displayed on a display screen for users to view, so the three-channel image is generally an RGB image that is convenient for display on a display screen.

[0078] Since users who use electronic devices to take photos are generally non-professional photographers, most users do not control the aperture, shutter speed and ISO when taking photos to improve the quality of photos taken by electronic devices. For example, in a low-light shooting environment, when a user directly presses the shutter of an electronic device, the photo may be too dark, with missing details and unclear images.

[0079] Currently, electronic devices generally have built-in photo processing algorithms to address the above-mentioned problems. For example, in a low-light photography environment, when a user presses the shutter of an electronic device to take a photo, the electronic device brightens the photo obtained after the shutter is pressed based on the built-in photo processing algorithm to enhance the missing details and unclear issues in the photo, thereby improving the display effect of the photo.

[0080] However, the processing logic of the built-in photo processing algorithm in electronic devices is generally fixed and cannot be adaptively processed according to the actual conditions of different photos. Therefore, when electronic devices use photo processing algorithms to process different photos, some photos have good display effects while others have poor display effects. For example, the processing logic of the built-in photo processing algorithm is to increase the brightness to achieve the effect of making the details of the photo clear. When the electronic device uses this method to process an image that is darker in nature, a photo with clear details can be obtained. However, when the electronic device uses this method to process an image that is brighter in nature, the image will become brighter, and the brighter area will be blurred, resulting in a photo with unclear details.

[0081] Moreover, when the electronic device processes photos using the built-in photo algorithm, the brightness information of the original photo is changed, and the resulting photo does not contain the original brightness information. Afterwards, the electronic device cannot obtain the original brightness information again to process the photo using other algorithms, thereby failing to improve the quality of the photo.

[0082] For example, the photo processing algorithm brightens the photo by adjusting the brightness, and adjusts the contrast to change the difference between brightness and darkness in the photo, thereby changing the clarity of the photo. Since electronic devices cannot adaptively adjust the brightness and contrast of photos according to the actual content of the photo, inappropriate brightening operations lead to missing details and unclear problems in the photo. For example, electronic devices use built-in photo algorithm processing and filtering to brighten the picture in the photo, which will amplify the originally unnoticeable noise in the photo, making it more prominent, thereby reducing the instruction of the photo. In particular, for darker areas in the photo screen, due to the low signal-to-noise ratio, the electronic device will brighten the area again, and the noise in the area will be more obviously amplified. In other words, electronic devices cannot guarantee the quality of photos using this photo processing method.

[0083] Furthermore, if it is found that the processed photos have missing details and unclear images due to changes in brightness information, the difficulty of restoring the missing details in the photos and improving the clarity of the photos through other image processing algorithms will also increase.

[0084] To this end, the present application provides an image processing method, such as Figure 2 As shown, after the photo processing module of the electronic device processes the original image using this method, a three-channel image and a single-channel image can be obtained, wherein the three-channel image retains the color information of the photo, and the single-channel image retains the brightness information in the original image. In this way, the electronic device can first brighten the three-channel image based on the brightness information in the single-channel image, and then display the brightened image.

[0085] In order to better understand the embodiments of the present application, the electronic device provided by the embodiments of the present application is first introduced.

[0086] The electronic device may specifically be a mobile phone, tablet computer, smart screen, laptop computer, vehicle-mounted device, wearable device (such as smart watch), ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), artificial intelligence (artificial intelligence) device and other terminal devices with image processing function. The embodiments of the present application do not limit the specific type of electronic device and the installed operating system.

[0087] Figure 3 A schematic structural diagram of an electronic device 100 is shown.

[0088] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0089] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0090] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, an image codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0091] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0092] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0093] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0094] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0095] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0096] The charging management module 140 is used to receive charging input from a charger. The charger can be a wireless charger or a wired charger. While the charging management module 140 charges the battery 142, it can also power the electronic device through the power management module 141.

[0097] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to power the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160.

[0098] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor. In some embodiments, the antenna 1 of the electronic device 100 is coupled with the mobile communication module 150, and the antenna 2 is coupled with the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology.

[0099] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0100] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, etc. the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation.

[0101] The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 can be set in the same device as at least some modules of the processor 110.

[0102] The wireless communication module 160 can provide wireless communication solutions for application in the electronic device 100, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication technology (NFC), infrared technology (IR), etc.

[0103] The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 may also receive a signal to be sent from the processor 110, modulate the signal, amplify the signal, and convert it into an electromagnetic wave for radiation via the antenna 2.

[0104] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0105] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0106] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, image codec, GPU, display screen 194 and application processor.

[0107] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0108] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1. In some embodiments, the photosensitive element may also be referred to as an image sensor. For ease of description in subsequent embodiments, the image sensor is directly referred to as a sensor.

[0109] Video codecs are used to compress or decompress digital videos. The electronic device 100 may support one or more video codecs. Thus, the electronic device 100 may play or record videos in a variety of coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0110] The image codec is used to compress or decompress digital images. The electronic device 100 may support one or more image codecs. In this way, the electronic device 100 may display or take photos in a variety of encoding formats, such as JPEG, PNG, GIF, etc.

[0111] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0112] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of electronic device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0113] The electronic device 100 can implement audio functions through the audio module 170, speakers, receivers, microphones, headphone interfaces, and application processors, such as music playback and recording. The speakers are used to convert audio electrical signals into sound signals. The receivers are used to convert audio electrical signals into sound signals. The microphones are used to convert sound signals into electrical signals. The headphone interface is used to connect wired headphones.

[0114] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0115] The sensor module 180 may include an acceleration sensor 180E, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, and the like.

[0116] The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in all directions (generally three axes). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device and is applied to applications such as horizontal and vertical screen switching and pedometers.

[0117] The buttons 190 include a power button, a volume button, etc. The buttons 190 may be mechanical buttons or touch buttons.

[0118] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to or disconnected from the electronic device 100 by inserting the SIM card interface 195 or removing the SIM card interface 195. The electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with an external memory card. The electronic device 100 interacts with the network through the SIM card to implement functions such as calls and data communications.

[0119] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. In the embodiment of the present invention, an operating system of a layered architecture is used as an example to exemplify the software structure of the electronic device 100.

[0120] Figure 4 1 is a software structure block diagram of the electronic device 100 according to an embodiment of the present invention.

[0121] The layered architecture divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the operating system is divided into four layers, from top to bottom: application layer, framework layer, hardware abstraction layer (HAL), and kernel layer.

[0122] For ease of explanation, Figure 4 Also shown is the hardware layer that interfaces with the software architecture.

[0123] It should be noted that the embodiment of the present application is illustrated by taking a common operating system as an example. In other operating systems, as long as the functions implemented by each functional module are similar to those of the embodiment of the present application, the solution of the present application can also be implemented.

[0124] The application layer can include a series of application packages. Figure 4 As shown, the application package may include applications such as a camera, a display application, etc. Among them, the display application may be a common gallery, a photo viewing application, a photo beautification and editing application, etc.

[0125] The framework layer provides application programming interface (API) and programming framework for the application programs in the application layer. The framework layer includes some predefined functions.

[0126] like Figure 4 As shown, the framework layer may include a camera service. The camera service is used to connect the camera application and the related hardware of the hardware layer. In some embodiments, the camera service can realize the connection between the camera application and the hardware layer by calling the hardware interface.

[0127] It is understandable that the framework layer may also include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, etc. Among them, the window manager is used to manage window programs. The content provider is used to store and obtain data and make these data accessible to applications. The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, etc. The phone manager is used to provide communication functions for the electronic device 100. The resource manager provides various resources for the application. The notification manager enables the application to display notification information in the status bar, which can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction.

[0128] The HAL layer is located above the kernel layer. The HAL layer can abstract the hardware of the hardware layer, hide the differences between different hardware, and provide a unified hardware interface for the upper layer.

[0129] like Figure 4As shown, the HAL layer includes camera devices, perception engines, and image processing.

[0130] The camera device is used to connect the camera service to the underlying camera driver and camera.

[0131] The perception engine is used to obtain the current photo scene, which can be the shooting angle of the mobile phone, the content of the mobile phone, etc.

[0132] Image processing is used to perform operations such as format conversion, content processing, and encoding on images.

[0133] The kernel layer is the layer between the hardware layer and the software layer. Figure 4 As shown, the kernel layer includes a camera driver, and the camera driver is used to control the camera of the hardware layer to perform shooting actions.

[0134] It can be understood that the kernel layer may also include display drivers, audio drivers, sensor drivers, etc., and these drivers will not be described in detail here.

[0135] The following is an illustrative description of the workflow of the software and hardware of the electronic device 100 in conjunction with the shooting scene.

[0136] For example, a camera application located in the application layer displays a shooting interface in an electronic device, and the user can click on the shooting control in the shooting interface. When the touch sensor receives a touch operation, the corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the touch operation into a raw input event (including touch coordinates, timestamp of the touch operation, and other information). The raw input event is stored in the kernel layer. The framework layer obtains the raw input event from the kernel layer and can identify that the control corresponding to the input event is a shooting control. Then, the camera calls the interface provided by the camera service of the framework layer. After the interface provided by the camera service is called, the camera device in the HAL layer can call the camera driver of the kernel layer, and the camera driver can capture the raw image through the camera. After that, the camera sends the raw image to the image processing of the HAL layer through the camera driver, and the image processing can process the raw image into a photo including a three-channel image and a single-channel image.

[0137] Figure 5 A schematic diagram of an image processing method provided in an embodiment of the present application Figure 1 The method is performed by an electronic device and comprises the following steps:

[0138] S501, obtaining RAW and image processing parameters.

[0139] RAW is the image data directly output from the image sensor of a digital camera, scanner, film scanner or mobile phone, which is equivalent to a file containing image information. The image sensor of an electronic device can convert light signals into electrical signals when the shutter is pressed. Therefore, the image data output by the image sensor includes all the information of the image. Therefore, the RAW output by the image sensor is also called digital negative.

[0140] In some scenarios, when the camera application starts running, the electronic device can start to acquire RAW in real time, and then the electronic device performs an image processing process of the preview stream on the RAW to obtain the preview stream, and then the processed preview stream can be displayed in the viewfinder on the display screen for the user to view. In other scenarios, when the camera application starts running, the electronic device can start to acquire RAW in real time, but the electronic device does not process the acquired RAW. Afterwards, when the user presses the shutter in the camera application display interface to start running, the electronic device performs an image processing process of the data stream on the RAW acquired when the user presses the shutter, and finally obtains an image file stored in the electronic device, that is, the photo is taken.

[0141] Among them, the interface that displays the processed preview stream in the viewfinder of the display screen in the embodiment of the present application is the shooting preview interface. The shutter in the embodiment of the present application can be called a shooting control in the shooting preview interface.

[0142] In some embodiments, the image processing parameters include cropping parameters, wide-angle distortion processing parameters, beautification parameters, etc. When the electronic device processes the image based on the cropping parameters and the wide-angle distortion processing parameters, the shape of the content in the image will change. For example, when the electronic device processes the image based on the cropping parameters, the field of view (FOV) in the image will change, such as a circular object in the image becomes a semicircle after being cropped. For another example, when the electronic device processes the image based on the wide-angle distortion processing parameters, the distortion caused by perspective in the image will be repaired, such as repairing the edge of the curve presented in the picture due to perspective to a straight line. When the electronic device processes the image based on the beautification parameters, the color in the image will change, and the shape of the content in the image will not change. For example, the beautification parameters may include at least one of the exposure value, brightness value, contrast, saturation, color temperature value, etc. Among them, the beautification parameters in the embodiments of the present application may also be referred to as color beautification parameters.

[0143] Among them, the image processing parameters in the embodiment of the present application may be the image editing parameters in the present application, and the first parameter included in the image editing parameters in the present application may be a cropping parameter and a wide-angle distortion processing parameter, and the second parameter may be a beautification parameter.

[0144] Furthermore, generally speaking, electronic devices cannot display directly based on RAW, and RAW needs to be converted into an image format that can be displayed in the electronic device.

[0145] The RAW image may be converted into a preset image format that can be displayed in the electronic device through the following steps, such as a YUV format or other formats. Specifically, the electronic device converts the RAW image into an image format that can be displayed in the electronic device, including S502-S505.

[0146] S502 : Acquire, based on RAW, a YUV format image A that retains original brightness information and a YUV format image B that does not retain original brightness information.

[0147] Since images in the YUV format occupy less memory space, YUV images are generally used in the process of transmission and image processing. Therefore, electronic devices can first convert RAW images into YUV format images to improve processing efficiency.

[0148] In some embodiments, since the image processing method provided in the embodiments of the present application requires a single-channel image that retains the original brightness information and a three-channel image that retains the color information, the electronic device needs to perform two conversion processes separately when converting RAW into a YUV format image that is easy to process.

[0149] In the one conversion process in the above embodiment, the electronic device converts RAW into a YUV format image A that retains the original brightness information. Specifically, the electronic device does not perform any processing on the original brightness information in RAW, for example, the electronic device does not perform any processing that will change the original brightness information, such as color correction matrix (CCM) and gamma correction, on RAW. In addition, in this process, the YUV format image A obtained by the electronic device processing may retain color information or may not retain color information.

[0150] In another conversion process in the above embodiment, the electronic device converts RAW into a YUV format image B without retaining the original brightness information. Specifically, the electronic device can adjust the original brightness information and color information of RAW according to the beautification parameters corresponding to the shooting mode selected by the user when shooting, or according to the beautification parameters corresponding to the filter selected by the user when shooting, and then convert the RAW with the adjusted original brightness information and color information into the YUV format image B.

[0151] The original brightness information in the original image includes brightness contrast information.

[0152] In this embodiment, the YUV format image A that retains the original brightness information may be the first image of the present application, and the YUV format image B that does not retain the original brightness information may be the second image of the present application.

[0153] S503 , respectively process the YUV format image A and the YUV format image B based on the image processing parameters to obtain the YUV format image A′ and the YUV format image B′.

[0154] When the electronic device processes the YUV format image based on the image processing parameters, the shape of the content in the YUV format image will change. Therefore, the electronic device needs to perform the same processing on the YUV format image A and the YUV format image B based on the image processing parameters to ensure that the processed YUV format image A' and the YUV format image B' have the same image content.

[0155] In some embodiments, the image processing parameters include cropping parameters and wide-angle distortion processing parameters, and the electronic device performs cropping and wide-angle distortion processing on the YUV format image A and the YUV format image B respectively based on the wide-angle distortion processing parameters.

[0156] It should be noted that in the process of an electronic device acquiring a YUV format image based on RAW, in addition to performing the operation of converting RAW into YUV format, the electronic device can also crop RAW based on cropping parameters. In this way, a YUV format image with a size that meets the requirements can be acquired based on RAW, and the storage space occupied can be reduced, thereby improving the efficiency of subsequent transmission and processing.

[0157] S504 , performing image encoding on the YUV format image A′ to obtain a single-channel image, and performing image encoding on the YUV format image B′ to obtain a three-channel image.

[0158] The single-channel image in this embodiment may be the first encoded image in this application, and the three-channel image may be the second encoded image in this application.

[0159] When the electronic device encodes the YUV format image A' and the YUV format image B', the obtained single-channel image and three-channel image have the same encoding format, which facilitates the electronic device or other electronic devices to subsequently decode and process the single-channel image and three-channel image based on the same encoding format.

[0160] In some embodiments, when the hardware structure of the electronic device includes an image codec, the image codec can be used to encode the YUV format image A' and the YUV format image B' to obtain a single-channel image and a three-channel image, respectively.

[0161] In some embodiments, when a software codec module is configured in the electronic device, the electronic device can perform image encoding on the YUV format image A' and the YUV format image B' through the software codec module to obtain a single-channel image and a three-channel image respectively.

[0162] The software encoding and decoding module in the above embodiment may include the first software encoding module and the second software encoding module in this application.

[0163] In some embodiments, when an image codec is included in the hardware structure of the electronic device and a software codec module is configured in the electronic device, the electronic device can perform image encoding on the YUV format image A' through the software codec module to obtain a single-channel image, and perform image encoding on the YUV format image B' through the image codec to obtain a three-channel image. The software codec module can be the first software encoding module in the present application, and the image codec can be the second hardware encoding module in the present application.

[0164] Alternatively, the electronic device may encode the YUV format image A' through an image codec to obtain a single-channel image, and encode the YUV format image B' through a software codec module to obtain a three-channel image. The image codec may be the first hardware encoding module in the present application, and the software codec module may be the second software encoding module in the present application.

[0165] It should be noted that the encoding logic of the image codec is the same as that of the software codec module. The image codec is an additional hardware structure. The process of using the image codec to encode the image does not occupy the computing resources of the processor in the electronic device. However, the process of using the software codec module to encode the image by the electronic device needs to occupy the computing resources of the processor in the electronic device. Therefore, the specific method of using the image codec or the software codec module to implement image encoding can be determined according to the actual situation.

[0166] It can be understood that the image codec used to encode the YUV format image A' is the first hardware encoding module, the software codec module used to encode the YUV format image A' is the first software encoding module, the image codec used to encode the YUV format image B' is the second hardware encoding module, and the software codec module used to encode the YUV format image B' is the second software encoding module.

[0167] In some embodiments, when the electronic device uses the same image encoding method for YUV format image A' and YUV format image B', only the original brightness information can be retained in the YUV format image A corresponding to the YUV format image A'. Therefore, the single-channel image obtained by the electronic device performing image encoding on the YUV format image A' only includes the original brightness information, and the three-channel image obtained by performing image encoding on the YUV format image B' may include brightness information and / or color information.

[0168] When the electronic device encodes only the original brightness information of the YUV format image A' and does not restrict the image encoding method of the YUV format image B', the single-channel image obtained by the electronic device through image encoding of the YUV format image A' includes only the original brightness information, and the three-channel image obtained by image encoding of the YUV format image B' may include brightness information and / or color information. For example, in addition to retaining the original brightness information, the YUV format image A corresponding to the YUV format image A' also retains other color information. In the process of image encoding of the YUV format image A', the electronic device only encodes the original brightness information, and other color information is deleted. Therefore, a single-channel image including only the original brightness information can be obtained.

[0169] In some embodiments, the image encoding process of the image codec and the software encoding module includes an image rotation process and an encoding process. The image rotation process can determine whether rotation is required according to the shooting posture of the electronic device. The rotation is briefly described by taking the center of the electronic device display screen as the rotation center a as an example. When the shutter of the electronic device is pressed, Figure 6 As shown, if the electronic device is in a vertical position, the rotation parameter is 0°, indicating that the image rotation process in the image encoding process needs to rotate the image by 0° or not rotate the image. If the electronic device changes from a vertical position to a horizontal position, at this time, the electronic device in the vertical position takes the rotation center a as the rotation point, and rotates the electronic device 90° counterclockwise on the plane where the display screen is located, and becomes a horizontally placed electronic device, then the rotation parameter is -90°. Then, at this time, in the image encoding process, the image captured by the electronic device needs to be rotated 90° counterclockwise on the display plane with the image center as the rotation point to obtain the rotated image, so as to ensure the viewing effect of the user.

[0170] After the electronic device executes S504 to obtain the three-channel image and the single-channel image, S505 is executed to obtain an image file including the three-channel image and the single-channel image:

[0171] S505: Store the single-channel image and the three-channel image in the same buffer to obtain an image file.

[0172] like Figure 7 As shown, the electronic device stores the single-channel image and the three-channel image in a buffer, thereby presenting them as an image file. The buffer in which the single-channel image and the three-channel image are stored together can be a first buffer.

[0173] Alternatively, the electronic device may concatenate the buffer storing the single-channel image and the buffer storing the three-channel image into one buffer, which may also achieve the following effect: Figure 7 As shown, the effect of including a three-channel image and a single-channel image in one image file.

[0174] In the case of stitching, the buffer storing the single-channel image may be the second buffer, and the buffer storing the three-channel image may be the third buffer.

[0175] Through S505 , the electronic device may store the image file.

[0176] Optionally, after executing S505, the electronic device may further execute the following S506:

[0177] S506: Read and display the image file.

[0178] In some embodiments, after the electronic device reads the image file, if the electronic device only recognizes the three-channel image in the image file, then the electronic device may display the three-channel image in the image file on the display screen of the electronic device.

[0179] In some embodiments, after reading the image file, if the electronic device only recognizes a single-channel image in the image file, the electronic device may display the single-channel image in the image file on a display screen of the electronic device.

[0180] In some embodiments, after reading the image file, if the electronic device recognizes a three-channel image and a single-channel image in the image file, and the electronic device only supports displaying three-channel images, then the electronic device can only display the three-channel image in the image file on the display screen of the electronic device.

[0181] In some embodiments, after the electronic device reads the image file, if the electronic device recognizes a three-channel image and a single-channel image in the image file, and the electronic device only supports displaying single-channel images, then the electronic device can only display the single-channel image in the image file on the display screen of the electronic device.

[0182] In some embodiments, after reading the image file, if the electronic device recognizes a three-channel image and a single-channel image in the image file, and the electronic device can process and display the three-channel image and the single-channel image, then the electronic device can adjust the three-channel image based on the single-channel image and display the adjusted image.

[0183] In the above embodiment, since the original brightness information is retained in the single-channel image, the electronic device can use the original brightness information in the single-channel image to adjust the brightness of the three-channel image. For example, the electronic device brightens the screen of the three-channel image based on the original brightness information. For another example, the electronic device can fuse the single-channel image with the three-channel image, so as to brighten the three-channel image by using the original brightness information retained in the single-channel image. The brightened three-channel image does not change the image content of the three-channel image itself, but superimposes the original brightness information of the single channel based on the image content of the three-channel image itself. In this way, the display effect of the three-channel image can be improved while retaining the details of the three-channel image.

[0184] Specifically, after a user takes a photo with a mobile phone, an image file is obtained such as Figure 7 As shown, when the user views the image file through the gallery, if the display algorithm in the gallery can read the single-channel image and the three-channel image in the image file and fuse the single-channel image and the three-channel image, then when the gallery running in the electronic device reads the image file, it can display the image after the fusion of the single-channel image and the three-channel image. The fused image has a higher brightness range and contrast, and compared with the three-channel image, it can provide more dynamic range and image details, so the display effect is better.

[0185] Figure 8 A schematic diagram of an image processing method provided in an embodiment of the present application Figure 2 The method is performed by an electronic device and comprises the following steps:

[0186] S801, obtaining RAW images captured by the sensor and image processing parameters.

[0187] In some embodiments, the RAW captured by the sensor is packed RAW, so Fig. 9 As shown, the packed RAW (packed raw) needs to be processed into RAW through the image data parsing (rawUnpack) module.

[0188] Packed raw can generally be RAW stored in MIPI format (hereinafter referred to as "MIPI RAW"). Since RAW includes all the information of the image, the RAW directly collected by the general image sensor is usually 10 bits, and 1 byte has 8 bits, so the 10-bit RAW needs to occupy 2 bytes. In this way, 6 bits of the 2 bytes occupied by RAW are empty, resulting in a waste of storage space and reducing the transmission and processing speed. Therefore, the MIPI RAW storage rule of storing 4 10-bit RAW data in 5 bytes (40 bits in total) is adopted to realize lossless packaging of RAW, so as to reduce the storage size of RAW and improve the transmission and processing speed. Therefore, current image sensors generally output packed raw to improve transmission and processing speeds.

[0189] The image processing parameters in S801 may include at least one of beautification parameters and wide-angle distortion processing parameters. The beautification parameters are used to adjust the color and brightness of the image, and the wide-angle distortion processing parameters are used to correct the perspective deformation of the image.

[0190] In some embodiments, the parameters included in the image processing parameters specifically correspond to the setting items according to the shooting of the electronic device. For example, when the electronic device is shooting, the camera is in the portrait shooting mode, and the corresponding image processing parameters only include the beautification parameters, and do not include the wide-angle distortion processing parameters. For another example, when the electronic device is shooting, the camera is in the landscape shooting mode, and the corresponding image processing parameters include not only the beautification parameters, but also the wide-angle distortion processing parameters.

[0191] In other embodiments, when the electronic device uses a wide-angle lens to take pictures, the corresponding image processing parameters include wide-angle distortion processing parameters. For example, in the case of landscape shooting mode, some electronic devices use a wide-angle lens to take pictures. At this time, because the pictures taken with the wide-angle lens will be distorted due to perspective, the image processing parameters of these electronic devices include wide-angle distortion processing parameters when the landscape shooting mode with the wide-angle lens is enabled. In the case of landscape shooting mode, some electronic devices do not use a wide-angle lens to take pictures. Then, the image processing parameters of these electronic devices in the case of landscape shooting mode do not include wide-angle distortion processing parameters.

[0192] For another example, some electronic devices include a wide-angle shooting mode in their shooting modes. When a user selects the wide-angle shooting mode, the electronic device uses a wide-angle lens to shoot photos. At this time, the image processing parameters include wide-angle distortion processing parameters. Alternatively, some electronic devices include a wide-angle shooting mode in their shooting modes. When a user selects the wide-angle shooting mode, the user is prompted whether to perform wide-angle distortion processing on the photos. When the user selects that wide-angle distortion processing is required for the photos, the image processing parameters include wide-angle distortion processing parameters. If the user selects that wide-angle distortion processing is not required for the photos, the image processing parameters do not include wide-angle distortion processing parameters.

[0193] It can be understood that in the above embodiment, the beautification parameters corresponding to different shooting modes may be different. For example, the beautification parameters corresponding to the portrait shooting mode include a saturation of -10 and a brightness of +15, while the beautification parameters corresponding to the landscape shooting mode include a saturation of +15 and a brightness of -15. The beautification parameters in the above examples are only exemplary and do not limit the specific values ​​of the beautification parameters.

[0194] After obtaining the RAW image taken by the sensor and the image processing parameters, it can be determined whether the current shooting requires a single-channel image. If the current shooting does not require a single-channel image, the electronic device can only capture an image file including a three-channel image to avoid redundant single-channel images occupying storage space. If the current shooting requires a single-channel image, the electronic device can capture an image file including a single-channel image and a three-channel image, thereby retaining the original brightness information of the image to improve the image display effect.

[0195] S802: Determine whether the current shooting scene requires a single-channel image.

[0196] In some embodiments, the electronic device can determine whether the current shooting scene requires a single-channel image based on the shooting mode. For example, the portrait shooting mode is a shooting scene that does not require a single-channel image, and the landscape mode is a shooting scene that requires a single-channel image. Then, the electronic device can determine that the current shooting scene does not require a single-channel image when the current shooting mode is the portrait shooting mode, and the electronic device can determine that the current shooting scene requires a single-channel image when the current shooting mode is the landscape shooting mode. For another example, the electronic device can consider that a scene that can obtain a high-dynamic range (HDR) image in the current shooting mode is a shooting scene that requires a single-channel image. The single-channel image obtained by processing can achieve a larger brightness range and contrast display.

[0197] It is understandable that the electronic device can also shoot in other shooting modes, which can be configured according to the actual application. For example, the electronic device can also shoot in the food shooting mode and the night scene shooting mode, which are also shooting scenes that require single-channel images. They will not be listed here one by one. In addition, due to the differences in shooting modes of different mobile phones, only some of these modes are used as examples here, and the shooting situation of a certain mode is not limited to a single-channel image shooting scene.

[0198] In some embodiments, the electronic device can determine whether the current shooting scene requires a single-channel image based on the captured picture. For example, it is assumed that shooting a portrait is a shooting scene that does not require a single-channel image, and shooting a landscape without a portrait is a shooting scene that requires a single-channel image. When the electronic device recognizes that the current shooting scene includes a portrait, the electronic device can determine that the current shooting scene does not require a single-channel image. When the electronic device recognizes that the current shooting scene is a landscape that does not include a portrait, the electronic device can determine that the current shooting scene requires a single-channel image.

[0199] In some embodiments, a single-channel shooting option may be provided in the electronic device. When the single-channel shooting option is selected, the electronic device determines that the current shooting scene requires a single-channel image. When the single-channel shooting option is not selected, the electronic device determines that the current shooting scene does not require a single-channel image.

[0200] In some examples, when the single-channel shooting option of the electronic device is not selected, and the electronic device determines that the current shooting scene requires a single-channel image based on the captured picture, the electronic device can display a prompt message to remind the user whether to select the single-channel shooting option. When the single-channel shooting option of the electronic device is selected, and the electronic device determines that the current shooting scene does not require a single-channel image based on the captured picture, the electronic device can also display a prompt message to remind the user whether to cancel the selected single-channel shooting option.

[0201] In some embodiments, when the shooting mode of the electronic device is a landscape shooting mode, that is, a scene requiring a single-channel image, such as Figure 4 The camera application shown can send the single-channel shooting identification to the perception engine at the HAL layer through the camera service. The perception engine sends the single-channel shooting identification to the image processing so that the image processing can perform image processing on the RAW transmitted from the photosensitive element to obtain an image file including a single-channel image and a three-channel image.

[0202] It is understandable that other third-party camera applications can also send a single-channel shooting identifier so that after image processing, an image file including a single-channel image and a three-channel image can be obtained, thereby improving the display effect of the image file.

[0203] When it is determined that a single channel is not needed at present, the electronic device may continue to execute the following S803-S807.

[0204] S803 , converting the RAW image into a YUV format image C based on the beautification parameters in the image processing parameters.

[0205] In some embodiments, Fig. 9 As shown, in the current scenario where a single channel is not required, RAW is input into the raw image (RAW) to YUV domain module 1, and the processing process of the RAW to YUV domain module 1 includes a format conversion process and a brightness information processing process. The format conversion process may be that the electronic device converts the RAW to an image in YUV format, and then the brightness information processing process may be that the electronic device adjusts the original brightness information and color information based on the beautification parameters to obtain an adjusted image in YUV format.

[0206] In some embodiments, Fig. 9 The RAW to YUV domain module 1 shown in the figure includes a format conversion process and a simple brightness information processing process. The format conversion process may be that the electronic device converts the RAW to a YUV format image, and then the simple brightness information processing process may be that the electronic device adjusts the original brightness information to obtain the adjusted original brightness information. After the RAW to YUV domain module 1 outputs the YUV image with the brightness information simply processed, as shown in FIG. Fig. 9 The YUV domain algorithm processing module shown can further adjust the YUV image using beautification parameters.

[0207] Among them, the RAW to YUV domain module 1 can be the second conversion module in this application.

[0208] In the above embodiment, the beautification parameter may include at least one of an exposure value, a brightness value, a contrast, a saturation, a color temperature value, and the like.

[0209] S804: Determine whether the image processing parameters include wide-angle distortion processing parameters.

[0210] When the image processing parameters include wide-angle distortion processing parameters, the electronic device executes S805:

[0211] S805 . Perform wide-angle distortion processing on the YUV format image C based on the wide-angle distortion processing parameters.

[0212] The wide-angle distortion processing parameters are used to correct the perspective deformation of the image. The process of the electronic device performing wide-angle distortion processing according to the wide-angle distortion processing parameters can be found in related technologies and will not be described in detail here.

[0213] After the electronic device performs wide-angle distortion processing on the YUV format image C to obtain the YUV format image C′, S806 may be executed to perform three-channel encoding on the YUV format image C′.

[0214] S806: Three-channel encoding to obtain a three-channel encoded image.

[0215] In S806, the electronic device may perform three-channel encoding in encoding formats such as JPEG format, PNG format, GIF format, etc. For example, when the electronic device adopts the JPEG format, the electronic device may obtain a three-channel JPEG format image.

[0216] In some embodiments, RGB images are generally used for display in electronic devices, so the three-channel encoded image obtained in S806 may be an RGB three-channel encoded image. It can be understood that when the three-channel encoded image is an RGB image and the electronic device uses the JPEG format, the electronic device can obtain a three-channel RGB JPEG format image.

[0217] When the image processing parameters do not include wide-angle distortion processing parameters, the electronic device may directly execute S806 to perform three-channel encoding on the YUV format image C that has undergone wide-angle distortion processing.

[0218] In some embodiments, when it is determined that a single channel is currently required, the electronic device, in addition to executing S803-S806 above, further executes S807-S810:

[0219] S807 , converting the RAW image into a YUV format image D that retains the original brightness information.

[0220] In some embodiments, Fig. 9 As shown, in the current scenario where a single channel is required, RAW is input into the raw image (RAW) to YUV domain module 2, and the processing process of the RAW to YUV domain module 2 only includes a format conversion process. The format conversion process can be a process in which the electronic device converts RAW into a YUV format image, and after the format conversion process, the obtained YUV format image retains the original brightness information in the RAW.

[0221] In other embodiments, Fig. 9As shown, in the current scenario where a single channel is required, RAW is input into the raw image (RAW) to YUV domain module 2, and the processing process of the RAW to YUV domain module 2 includes a format conversion process and a color information deletion process. The format conversion process is the same as the above embodiment and is not repeated here. The color information deletion process can be that the electronic device deletes the color information in the RAW, so that the converted YUV format image only includes the original brightness information.

[0222] In some embodiments, Fig. 9 The RAW to YUV domain module 1 shown can be an online RAW to YUV module, that is, the RAW to YUV domain module 1 is a software module in a processor or image processor of an electronic device. The RAW to YUV domain module 2 can be an offline RAW to YUV module, that is, the RAW to YUV domain module 2 is a hardware module in an electronic device.

[0223] It should be noted that the RAW to YUV domain module 1 can be an offline RAW to YUV module, and the RAW to YUV domain module 2 can be an online RAW to YUV module, or, the RAW to YUV domain module 1 and the RAW to YUV domain module 2 can both be online RAW to YUV modules, or, the RAW to YUV domain module 1 and the RAW to YUV domain module 2 can both be online RAW to YUV modules and offline RAW to YUV modules.

[0224] By selecting an online RAW to YUV module or an offline RAW to YUV module, the processing process of the original image (RAW) to YUV domain module 1 can include a format conversion process and a brightness information processing process, or include a format conversion process and a simple brightness information processing process. By selecting an online RAW to YUV module or an offline RAW to YUV module, the processing process of the original image (RAW) to YUV domain module 2 can include only a format conversion process, or include a format conversion process and a color deletion process.

[0225] Among them, the RAW to YUV domain module 2 can be the first conversion module in this application.

[0226] In some embodiments, in S803 and S807, the electronic device may also crop the RAW based on cropping parameters during the process of converting the RAW to the YUV format. In this way, a YUV format image with a size that meets the requirements can be obtained based on the RAW, and the storage space occupied can be reduced, thereby improving the efficiency of subsequent transmission and processing.

[0227] The electronic device may determine the cropping parameters according to the shooting mode. Figure 4As shown, the shooting mode of the camera application during shooting is the landscape shooting mode, and its corresponding cropping parameter is 16:9. Then, the camera application can send the cropping parameter to the perception engine through the camera service, and the perception engine sends the cropping parameter to the image processing, so that the image processing can crop the image according to the cropping parameter of 16:9 during the process of converting the RAW format image to the YUV format image, thereby obtaining a YUV format image with a size of 16:9.

[0228] It should be noted that, if the RAW needs to be cropped according to the cropping parameters in the S803 process, then the RAW must also be cropped according to the cropping parameters in the S807 process. In this way, it can be ensured that the final three-channel image has the same size as the single-channel image, which facilitates the subsequent brightening of the three-channel image based on the single-channel image.

[0229] S808: Determine whether the image processing parameters include wide-angle distortion processing parameters.

[0230] When the image processing parameters include wide-angle distortion processing parameters, the next step S809 is executed.

[0231] S809 , performing wide-angle distortion processing on the YUV format image D based on the wide-angle distortion processing parameters.

[0232] For the electronic device to perform wide-angle distortion processing on the YUV format image D, reference may be made to the embodiments and related technologies related to S501 and S805, which will not be described in detail here.

[0233] It should be noted that, after the electronic device performs wide-angle distortion processing according to the wide-angle distortion processing parameters, it will correct the deformation caused by perspective in the YUV format image D, that is, the content in the YUV format image D' changes shape. For example, the curved portion in the YUV format image D is processed by wide-angle distortion, and appears as a straight line in the YUV format image D'. Then, in order to ensure that the final three-channel image has the same shape as the image content in the single-channel image, if the electronic device executes S805, it must execute S809, and S805 and S809 can avoid the straight line portion in the single-channel image from appearing as a curve in the three-channel image, resulting in the inability to merge the three-channel image with the single-channel image.

[0234] After executing S809, the electronic device may execute step S810:

[0235] S810: Single-channel encoding to obtain a single-channel encoded image.

[0236] Similar to S806, the electronic device may perform single-channel encoding in encoding formats such as JPEG format, PNG format, GIF format, etc. For example, when the electronic device adopts the JPEG format, the electronic device may obtain a single-channel JPEG format image.

[0237] In some embodiments, RGB images are generally used for display in electronic devices, so the single-channel encoded image obtained in S806 may be an RGB single-channel encoded image. It can be understood that when the single-channel encoded image is an RGB image and the electronic device uses the JPEG format, the electronic device can obtain a single-channel RGB JPEG format image.

[0238] When the image processing parameters do not include wide-angle distortion processing parameters, the electronic device may directly execute S810 to perform three-channel encoding on the YUV format image C that has undergone wide-angle distortion processing.

[0239] It should be noted that the electronic device can use an image codec to perform S806, or use a software codec module to perform S806. Correspondingly, the electronic device can use an image codec to perform S810, or use a software codec module to perform S810. For details, please refer to the embodiments related to S504.

[0240] In some embodiments, the electronic device implements the encoding process using the image codec and the software encoding module, which can be determined according to the functions of the image codec and the software encoding module. When the image codec only supports three-channel encoding and the software encoding module only supports single-channel encoding, the electronic device can use the image codec for three-channel encoding and the software encoding module for single-channel encoding.

[0241] In some embodiments, after executing S808 and determining that the image processing parameters do not include wide-angle distortion processing parameters, the electronic device may directly execute S810.

[0242] In the above process, the electronic device synchronously executes S803-S806 and S807-S810. For example, when the electronic device determines that the current scene requires a single channel, it can create two threads, one of which is a three-channel encoding thread for executing S803-S806, and the other is a single-channel encoding thread for executing S807-S810.

[0243] Among them, in the embodiment of the present application, the single-channel encoding thread can be the first thread, and the three-channel encoding thread can be the second thread.

[0244] After the electronic device executes S806 and S810, a single-channel encoded image and a three-channel encoded image may be obtained, and then the electronic device continues to execute S811.

[0245] S811. Store the single-channel image and the three-channel image in the same buffer.

[0246] S811 is the same as S505. Please refer to the embodiments related to S505 here, and will not be described in detail here.

[0247] In some embodiments, when the wide-angle distortion processing process is performed on a YUV format image, the electronic device needs to first convert the RAW image into a YUV format image. When the wide-angle distortion processing process can process the RAW image, the electronic device can first perform the wide-angle distortion processing on the RAW image, and then convert the RAW image after the wide-angle distortion processing into a YUV format image.

[0248] It can be understood that the image encoding process in the above embodiment may also include a rotation process, which can be specifically referred to in the above S504 and will not be described in detail here.

[0249] In the above embodiment, when the electronic device does not execute S809, the YUV format image D may be the first image in the present application, and when the electronic device does not execute S805, the YUV format image C may be the second image in the present application.

[0250] When the electronic device executes S809, the YUV format image D after wide-angle distortion processing may be the first image in this application. When the electronic device executes S805, the YUV format image C after wide-angle distortion processing may be the second image in this application.

[0251] Fig.10 A schematic diagram of an image processing method provided in an embodiment of the present application Figure 3 The method is performed by an electronic device and comprises the following steps:

[0252] S1001. Obtain RAW images captured by the sensor and image processing parameters.

[0253] S1001 is the same as S801, and reference may be made to the relevant embodiments of S801, such as Fig.11 As shown, the RAW captured by the sensor is also packed raw, and the packed raw is also processed into RAW through the rawUnpack module, which will not be repeated here.

[0254] After obtaining the RAW, the electronic device may execute S1002:

[0255] S1002, converting RAW to YUV format image E.

[0256] like Fig.11 As shown, the RAW image can be converted into a YUV format image E through the raw image (RAW) to YUV domain conversion module 3, and the YUV format image E retains the original brightness information.

[0257] In some embodiments, the raw image (RAW) to YUV domain conversion module 3 may further include a dynamic range extension (DRE) processing module. After the RAW is processed by the DRE processing module, a YUV format image E may be obtained.

[0258] After the electronic device executes S1002, S1003 may be executed to determine whether the electronic device needs to execute a single-channel image processing process. S1003:

[0259] S1003: Determine whether the current shooting scene requires a single-channel image.

[0260] S1003 is the same as S802, and reference may be made to the relevant embodiments of S802, which will not be described in detail here.

[0261] When it is determined that a single channel is not needed in the current scenario, the electronic device executes the following S1004-S1007 to obtain a three-channel image.

[0262] S1004 , processing the YUV format image E based on the beautification parameters in the image processing parameters.

[0263] like Fig.11 As shown, the YUV format conversion module in the raw image (RAW) to YUV domain module 3 can convert the RAW into a YUV format image E. After that, the brightness information processing module in the raw image (RAW) to YUV domain module 3 can process the brightness information of the YUV format image E. For details about brightness processing based on beautification parameters, please refer to the above embodiment, which will not be repeated here.

[0264] In some embodiments, the raw image (RAW) to YUV domain module 3 may also include a dynamic range extension (DRC) processing module. After the DRE processing module processes the RAW, the raw image (RAW) to YUV domain module 3 may convert the RAW into a YUV format image. Thereafter, the DRC processing module may process the YUV format image E based on the beautification parameters.

[0265] S1005: Determine whether the image processing parameters include wide-angle distortion processing parameters.

[0266] When the image processing parameters include wide-angle distortion processing parameters, the next step S806 is executed.

[0267] S1006 . Perform wide-angle distortion processing on the YUV format image E based on the wide-angle distortion processing parameters.

[0268] S1007: Three-channel encoding to obtain a three-channel encoded image.

[0269] S1005-S1007 are similar to S803-S807, and details can be found in the above-mentioned related embodiments, which will not be repeated here.

[0270] When it is determined that a single channel is currently required, the electronic device executes S1008-S1011 in addition to executing the above S1004-S1007.

[0271] S1008 , converting the YUV format image E into a grayscale image F that only retains original brightness information.

[0272] like Fig.11 As shown, the YUV format conversion module in the raw image (RAW) to YUV domain module 3 can convert the RAW into a YUV format image E, and the YUV format image E retains the original brightness information.

[0273] In some embodiments, Fig.11 The single-channel encoding module in supports image encoding of 8-bit grayscale images, so it is necessary to ensure that the YUV format image E is an 8-bit grayscale image. In the case where the YUV format image E output by the YUV format conversion module in the original image (RAW) to YUV domain module 3 is a 14-bit grayscale image, the grayscale value of the original image can be compressed and requantized to convert the 14-bit grayscale image to an 8-bit grayscale image.

[0274] Specifically, the electronic device can calculate the grayscale range according to the maximum and minimum grayscale values ​​of the 14-bit grayscale image, and then calculate the quantization factor according to the 8-bit grayscale range (0-255). The grayscale range length of the 14-bit grayscale image is divided by the 8-bit grayscale range length to obtain the quantization factor of each grayscale range. Afterwards, the grayscale value of each pixel in the 14-bit grayscale image is requantized using the quantization factor, that is, the grayscale value of each pixel in the 14-bit grayscale image is divided by the quantization factor and rounded. Finally, the calculated result is written as an 8-bit grayscale image.

[0275] In the corresponding embodiments of S1002-S1008, the YUV format conversion module in the raw image (RAW) to YUV domain module 3 can be the third conversion module in the present application, and the brightness information processing module in the raw image (RAW) to YUV domain module 3 can be the fourth conversion module in the present application.

[0276] S1009: Determine whether the image processing parameters include wide-angle distortion processing parameters.

[0277] When the image processing parameters include wide-angle distortion processing parameters, the next step S1009 is executed.

[0278] S1010 . Perform wide-angle distortion processing on the YUV format image F based on wide-angle distortion processing parameters.

[0279] S1011, single-channel encoding to obtain a single-channel encoded image.

[0280] When the image processing parameters do not include wide-angle distortion processing parameters, S1011 may be directly executed.

[0281] After the electronic device executes S1007 and S1011, if a single-channel encoded image and a three-channel encoded image are obtained, S1012 is continued to be executed.

[0282] S1012: Store the single-channel image and the three-channel image in the same buffer.

[0283] S1009-S1011 are similar to S808-S810, and details can be found in the above-mentioned related embodiments, which will not be described in detail here.

[0284] In the above embodiment, the electronic device executes S1004-S1007 and S1008-S1011 synchronously. For example, when the electronic device determines that the current scene requires a single channel, it can create two threads, one of which is a three-channel encoding thread for executing S1004-S1007, and the other is a single-channel encoding thread for executing S1008-S1011.

[0285] The YUV format E obtained by the electronic device through S1002 can be sent to the three-channel encoding thread for executing S1004-S1007 and the single-channel encoding thread for executing S1008-S1011 through a shared memory.

[0286] In the above embodiment, when the electronic device does not execute S1009, the YUV format image F may be the first image in the present application, and when the electronic device does not execute S1005, the YUV format image E may be the second image in the present application.

[0287] When the electronic device executes S1009, the YUV format image F after wide-angle distortion processing can be the first image in this application. When the electronic device executes S1005, the YUV format image E after wide-angle distortion processing can be the second image in this application.

[0288] In some embodiments, the present application Figure 5 The image processing process in can obtain an image file including a three-channel image and a single-channel image. According to the above-mentioned image processing logic, when the electronic device performs image encoding, it can also encode the image in other ways to obtain image files in other forms. For example, in S504, the YUV format image A' is encoded in a single-channel manner in one way, and the YUV format image B' is encoded in a single-channel manner in another way, so as to obtain an image file including two single-channel images of different encoding types. For another example, when the electronic device performs image encoding, such as in S504, the electronic device performs three-channel encoding on the YUV format image A' as a grayscale image, and performs YUV encoding on the YUV format image B' with a filter effect, so as to obtain a three-channel encoded image as a grayscale image and a YUV encoded image with a filter effect. Specifically, the present scheme can be used for encoding according to actual needs, and will not be described one by one here.

[0289] Understandably, Figure 8 as well as Fig.10 The encoding process in the encoding method can also be modified according to actual needs to obtain an image file that meets actual needs.

[0290] An embodiment of the present application also provides a computer-readable storage medium, which includes computer instructions. When the computer instructions are executed on the above-mentioned electronic device, the electronic device executes each function or step in the above-mentioned method embodiment.

[0291] The embodiment of the present application also provides a computer program product, including a computer program. When the computer program runs on an electronic device, the electronic device executes each function or step in the above method embodiment.

[0292] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0293] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0294] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0295] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0296] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0297] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application shall be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An image processing method, characterized in that: Applied to an electronic device including an image sensor, the method comprises: Display a shooting preview interface of a camera application, wherein the shooting preview interface includes shooting controls; In response to a user triggering operation on the shooting control, storing an image file; The image file includes the first encoded image and the second encoded image obtained based on the original image, the first encoded image includes the original brightness information in the original image, the first encoded image is used to brighten the second encoded image when displaying the image file, and the original image is acquired by the image sensor in response to the trigger operation.

2. The method according to claim 1, characterized in that The method further comprises: In response to a user opening operation of opening the image file, a brightened image is displayed, where the brightened image is obtained by brightening the second encoded image based on the first encoded image.

3. The method according to claim 1 or 2, characterized in that: In response to the current shooting mode being a preset shooting mode, or in response to the preview interface including a preset picture, executing the step of storing the image file; Among them, the preset shooting mode is one or more of the following modes: landscape shooting mode, food shooting mode, night scene shooting mode; the preset picture is a picture that does not include a portrait.

4. The method according to any one of claims 1 to 3, characterized in that Determining the first coded image and the second coded image based on the original image includes: Based on the original image, respectively acquire a first image in a YUV format and a second image in a YUV format, wherein the first image includes original brightness information; Image encoding is performed on the first image and the second image respectively to obtain the first encoded image and the second encoded image.

5. The method according to claim 4, characterized in that The performing image encoding on the first image and the second image respectively to obtain a first encoded image and a second encoded image includes: Acquire a rotation parameter, wherein the rotation parameter is determined based on a posture of the electronic device; Rotate the first image and the second image based on the rotation parameter; The rotated first image and the rotated second image are respectively encoded to obtain a first encoded image and a second encoded image.

6. The method according to claim 4, characterized in that The obtaining of the first coded image and the second coded image comprises: When the format of the first image is a first format, encoding the first image through a first hardware encoding module to obtain a first encoded image; the first format is an encoding format supported by the first hardware encoding module; when the format of the first image is not the first format, encoding the first image through a first software encoding module to obtain a first encoded image; When the format of the second image is the second format, the second image is encoded by a second hardware encoding module to obtain a second encoded image; the second format is an encoding format supported by the second hardware encoding module; when the format of the second image is not the second format, the second image is encoded by a second software encoding module to obtain a second encoded image.

7. The method according to any one of claims 4 to 6, characterized in that: The original image is a RAW image, and acquiring a first image in a YUV format and a second image in a YUV format based on the original image respectively includes: Converting the RAW image into a first image in a YUV format by a first conversion module, wherein the first conversion module does not change the original brightness information; The RAW image is converted into a second image in a YUV format by a second conversion module, and the second conversion module changes the original brightness information.

8. The method according to claim 7, characterized in that The method further comprises: Acquire an image editing parameter, the image editing parameter corresponding to the current shooting mode, the image editing parameter including a first parameter and a second parameter; the first parameter is used to indicate the modification of the image shape in the image, and the second parameter is used to indicate the modification of the color in the image; The converting the RAW image into a first image in a YUV format by a first conversion module comprises: Processing the RAW image based on the first parameter by the first conversion module to obtain the first image in YUV format; The converting the RAW image into the second image in YUV format by the second conversion module includes: The second conversion module processes the RAW image based on the first parameter and the second parameter to obtain the second image in YUV format.

9. The method according to any one of claims 4 to 6, characterized in that: The original image is a RAW image, and acquiring the first image and the second image based on the original image respectively includes: Converting the RAW image of the original image into a first image in a YUV format by a third conversion module, wherein the first conversion module does not change the original brightness information; The first image in the YUV format is converted into the second image by a fourth conversion module, and the second conversion module changes original brightness information in the first image.

10. The method according to claim 9, characterized in that The method further comprises: Acquire an image editing parameter, the image editing parameter corresponding to the current shooting mode, the image editing parameter including a first parameter and a second parameter; the first parameter is used to instruct the electronic device to modify the image shape in the image, and the second parameter is used to instruct the electronic device to modify the color in the image; The converting the RAW image of the original image into a first image in a YUV format by a third conversion module comprises: Processing the RAW image based on the first parameter by the third conversion module to obtain the first image in YUV format; The converting the first image in YUV format into the second image in YUV format by the fourth conversion module includes: The first image in the YUV format is processed by the fourth conversion module based on the second parameter to obtain the second image in the YUV format.

11. The method according to claim 8 or 10, characterized in that: The first parameter includes a wide-angle distortion processing parameter and / or a cropping parameter, and the second parameter includes a color beautification parameter.

12. The method according to any one of claims 1 to 11, characterized in that The stored image file includes: storing the first coded image and the second coded image in a first buffer, or concatenating a second buffer corresponding to the first coded image and a third buffer corresponding to the second coded image into a first buffer; The image file in the first buffer is stored.

13. The method according to any one of claims 1 to 12, characterized in that Determining the first coded image and the second coded image based on the original image includes: Determine the first encoded image based on the original image using a first thread; The second encoded image is determined based on the original image using a second thread.

14. The method according to any one of claims 1 to 13, characterized in that The first encoded image is a single-channel encoded image, and the second encoded image is a three-channel color encoded image.

15. An electronic device, characterized in that: The electronic device includes a display screen, a memory and one or more processors; the display screen, the memory and the processor are coupled; the display screen is used to display an image generated by the processor, and the memory is used to store computer program code, and the computer program code includes computer instructions; when the processor executes the computer instructions, the electronic device executes the method described in any one of claims 1-14.

16. A computer-readable storage medium, characterized in that: The method comprises computer instructions, which, when executed on an electronic device, cause the electronic device to execute the method as claimed in any one of claims 1 to 14.

17. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the method according to any one of claims 1 to 14 is implemented.

Images