An image processing method, an electronic device, and a readable storage medium

Abstract

The application provides an image processing method, an electronic device and a readable storage medium, and relates to the technical field of image processing. In the method, the electronic device performs the following operations when shooting: obtaining a RAW image; performing demosaicing processing on the RAW image to obtain a first RGB image; converting the RAW image into a grayscale image; and correcting the first RGB image based on the grayscale image to obtain a second RGB image. The application corrects the RGB image obtained by demosaicing processing by using the grayscale image corresponding to the RAW image. Since the grayscale image corresponding to the RAW image is an image obtained without demosaicing processing, the details in the RAW image can be retained in the grayscale image. Further, the RGB image obtained by demosaicing processing is corrected based on the grayscale image, so that the details of the RGB image can be enhanced, and the definition of the RGB image can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to an image processing method, an electronic device and a readable storage medium. BACKGROUND

[0002] Based on the current shooting algorithm, after the electronic device obtains a RAW image collected by an image sensor, the electronic device usually needs to use a demosaic algorithm to convert the RAW image into an RGB image (i.e. a color image). Wherein, the image sensor is an image sensor covered with a color filter array (CFA), and each pixel in the RAW image output by the image sensor is monochrome; the demosaic algorithm is used to supplement other color information in each pixel of the RAW image by interpolation, so as to convert each pixel into color.

[0003] Since the other color information in each pixel is supplemented by interpolation, these color information is usually not very accurate, which may cause the obtained RGB image to have problems such as moire and loss of details at high frequency positions, resulting in a decrease in the clarity of the RGB image. SUMMARY

[0004] The present application provides an image processing method, an electronic device and a readable storage medium, which can improve the clarity of the RGB image.

[0005] In a first aspect, an embodiment of the present application provides an image processing method, the method is applied to an electronic device, and the method comprises the following steps: detecting a shooting instruction, the shooting instruction is used to request the electronic device to shoot; in response to the shooting instruction, obtaining a RAW image; performing demosaic processing on the RAW image to obtain a first RGB image; converting the RAW image into a grayscale image; and correcting the first RGB image based on the grayscale image to obtain a second RGB image.

[0006] The present application proposes to correct the RGB image obtained by demosaic processing of the RAW image by using the grayscale image corresponding to the RAW image. Since the grayscale image corresponding to the RAW image is an image obtained without demosaic processing, the details in the RAW image can be retained in the grayscale image. Further, the RGB image obtained by demosaic processing is corrected based on the grayscale image, which can enhance the details of the RGB image, reduce the moire caused by demosaic processing, and further improve the clarity of the RGB image.

[0007] In addition, it needs to be explained that the gray-scale image in the prior art is usually obtained based on a black-and-white lens, which means that based on the prior art, if the gray-scale image is used to correct the RGB image, an additional black-and-white lens needs to be configured based on the original camera, which on the one hand causes an increase in configuration cost, and on the other hand, there is a position deviation between the two cameras, and the position deviation needs to be calibrated when shooting, and errors are likely to exist in the calibration process. The present application obtains the gray-scale image and the RGB image by processing the same RAW image differently, so that there is no need to additionally configure a black-and-white lens, and there is no need to calibrate the position deviation, thereby being able to reduce the configuration cost while improving the efficiency of image processing and improving the quality of the final RGB image.

[0008] In combination with the first aspect, in some implementations of the first aspect, the correcting the first RGB image based on the gray-scale image to obtain a second RGB image comprises: converting the first RGB image into a first YUV image; fusing the gray-scale image into a Y channel of the first YUV image to obtain a second YUV image; and converting the second YUV image into a second RGB image.

[0009] The present application fuses the gray-scale image into the Y channel of the YUV image, that is, without changing the color of the output image (that is, without changing the U and V channels), the details in the RAW image are fused into the luminance channel of the YUV image, thereby being able to increase the overall details and textures of the image and improve the final image effect.

[0010] In combination with the first aspect and the above implementations, in some implementations of the first aspect, the converting the RAW image into a gray-scale image comprises: counting the pixel value of each pixel in the RAW image; determining a conversion coefficient of the RAW image according to the pixel value of each pixel; and converting the RAW image according to the conversion coefficient to obtain a gray-scale image.

[0011] Compared with the traditional way of obtaining a gray-scale image by using a black-and-white lens, the present application determines the conversion coefficient of the RAW image based on the pixel value in the RAW image, and then converts the RAW image based on the conversion coefficient to obtain a gray-scale image, which is able to obtain a gray-scale image while reducing the configuration cost.

[0012] In some implementations of the first aspect, the pixel value of each pixel corresponds to an R value, a G value, or a B value, and determining the conversion coefficients of the RAW image according to the pixel value of each pixel includes: calculating an average value R' of all pixel values corresponding to the R value, an average value G' of all pixel values corresponding to the G value, and an average value B' of all pixel values corresponding to the B value in the RAW image respectively; and obtaining the conversion coefficient of the pixel value corresponding to the R value as G' / R', the conversion coefficient of the pixel value corresponding to the G value as 1, and the conversion coefficient of the pixel value corresponding to the B value as G' / B' according to the average values R', G', and B'.

[0013] In some implementations of the first aspect, converting the RAW image according to the conversion coefficients to obtain the grayscale image includes: converting the pixel value corresponding to the R value in the RAW image according to the conversion coefficient G' / R' to obtain a first conversion result; converting the pixel value corresponding to the G value in the RAW image according to the conversion coefficient 1 to obtain a second conversion result; converting the pixel value corresponding to the B value in the RAW image according to the conversion coefficient G' / B' to obtain a third conversion result; and obtaining the grayscale image according to the first conversion result, the second conversion result, and the third conversion result.

[0014] In some implementations of the first aspect, when the pixel value is converted according to the conversion coefficient, the conversion can be implemented by multiplication, that is, each pixel value is multiplied by the conversion coefficient corresponding to the pixel value (for example, the pixel value corresponding to the R value is multiplied by the conversion coefficient G' / R', the pixel value corresponding to the G value is multiplied by the conversion coefficient 1, and the pixel value corresponding to the B value is multiplied by the conversion coefficient G' / B'), so that the pixel value of each pixel is approximately equal to the G value, and the RAW image is converted into the grayscale image.

[0015] In some implementations of the first aspect, converting the pixel value corresponding to the R value in the RAW image according to the conversion coefficient G' / R' includes multiplying the conversion coefficient G' / R' by the pixel value corresponding to the R value in the RAW image; converting the pixel value corresponding to the G value in the RAW image according to the conversion coefficient 1 includes multiplying the conversion coefficient 1 by the pixel value corresponding to the G value in the RAW image; and converting the pixel value corresponding to the B value in the RAW image according to the conversion coefficient G' / B' includes multiplying the conversion coefficient G' / B' by the pixel value corresponding to the B value in the RAW image.

[0016] In a second aspect, an electronic device is provided, including: one or more processors; one or more memories; and the memory stores one or more programs, when the one or more programs are executed by the processor, the electronic device performs any possible method of the first aspect.

[0017] In a third aspect, an apparatus is provided, which is included in an electronic device. The apparatus has functions to implement the behaviors of the electronic device in the above aspects and possible implementation manners of the above aspects. The functions can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the above functions. For example, a detecting module or unit, an obtaining module or unit, a processing module or unit, etc.

[0018] In a fourth aspect, a computer-readable storage medium is provided, which stores instructions. When the instructions are run on a computer, the computer is caused to perform the method in the first aspect.

[0019] In a fifth aspect, a computer program product is provided, which contains instructions. When the instructions are run on a computer, the computer is caused to perform the method in the first aspect.

[0020] The technical effects obtained by the above second aspect, third aspect, fourth aspect and fifth aspect are similar to the technical effects obtained by the corresponding technical means in the first aspect, and thus are not described here again. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 Fig. 1 shows an example of an interface for starting a photographing mode in a camera application according to an embodiment of the present application;

[0022] Figure 2 Fig. 2 shows an example of images before and after demosaicing according to an embodiment of the present application;

[0023] Figure 3 Fig. 3 shows an example of a hardware structure of an electronic device according to an embodiment of the present application;

[0024] Figure 4 Fig. 4 shows an example of a software structure of an electronic device according to an embodiment of the present application;

[0025] Figure 5 Fig. 5 shows an example of a flow of an image processing method according to an embodiment of the present application;

[0026] Figure 6 Fig. 6 shows an example of a method for generating a grayscale image according to an embodiment of the present application;

[0027] Figure 7 Fig. 7 shows another example of a flow of an image processing method according to an embodiment of the present application;

[0028] Figure 8 Fig. 8 shows an example of differences between an RGB image before and after correction according to an embodiment of the present application;

[0029] Figure 9 A structure diagram of an image processing device provided by an embodiment of the present application is shown.

[0030] Figure 10 A structure diagram of a chip provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0031] In order to clearly describe the technical solutions of the embodiments of the present application, in the embodiments of the present application, the same items or similar items with basically the same functions and roles are distinguished by using "first", "second", and the like. For example, the first chip and the second chip are merely used to distinguish different chips, and do not limit the sequence. Those skilled in the art can understand that "first", "second", and the like do not limit the quantity and execution sequence, and "first", "second", and the like do not necessarily mean different.

[0032] It should be noted that in the embodiments of the present application, "exemplary" or "for example" is used to represent an example, illustration, or description. Any embodiment or design scheme described as "exemplary" or "for example" in the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, "exemplary" or "for example" is used to specifically present the relevant concept.

[0033] In the embodiments of the present application, "at least one" means one or more, and "multiple" means two or more. "And / or" describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the following three cases: A exists alone, A and B exist together, and B exists alone, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after it. "At least one of the following" or the like means any combination of these items, including any combination of single item or multiple items. For example, at least one of a, b, or c can represent a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, and c can be single or multiple.

[0034] For the convenience of understanding, the following first explains some terms related to the embodiments of the present application.

[0035] RAW image: refers to the original image obtained by converting the light source signal captured by the image sensor in the electronic device into a digital signal. The image sensor is an image sensor covered with a color filter array (CFA), and the original image output by the image sensor is in Bayer format. Each pixel in the original image is monochromatic (e.g., red, green, or blue).

[0036] Demosaic algorithm: a commonly used image interpolation algorithm in image processing, used to convert the RAW image captured by the sensor into an RGB image (i.e., a color image). Specifically, other color information is supplemented in each pixel in the RAW image through interpolation, thereby converting each pixel into a color image, for example, supplementing red and blue information in a green pixel to convert it into a color image.

[0037] RGB image: refers to a color image, which is an image composed of R, G, and B components. R refers to red, G refers to green, and B refers to blue. It should be understood that any color image can be composed of different proportions of red, green, and blue.

[0038] Gray image: refers to an image in which the information of each pixel is described by a quantized gray scale, and there is no color information in the image. The gray scale, also known as the gray scale or gray scale, refers to the brightness of the light and dark. In other words, the gray scale refers to pure white, pure black, and a series of transition colors from black to white. In RGB mode, each of the three primary colors has 256 levels, and the gray scale is formed when the R, G, and B values of a pixel are equal. The number of gray scales is 256 levels.

[0039] YUV image: YUV is a color encoding model, also known as YCbCr. YUV encoding uses luminance and chrominance to represent the color of each pixel. Y represents luminance (luminance or luma), which is the gray scale value; U and V represent chrominance (chrominance or chroma), which describe the hue and saturation.

[0040] Using the YUV color encoding model can reduce the memory usage of image data and improve the processing efficiency of data. In addition, the image data of the YUV encoding model cannot be directly used for display, and needs to be converted to RGB format for normal display.

[0041] Moiré pattern: a high-frequency irregular stripe that can cause the image to appear colored.

[0042] High dynamic range (HDR): a technology for synthesizing an image with high dynamic range by using multiple frames of continuous images with different exposure amounts. Compared with a normal image, a high dynamic range image has no overexposure in bright parts and clear dark details, and can provide more dynamic range and image details. The "dynamic range" refers to the ratio of signal intensity (e.g., light brightness), which can also be referred to as "light ratio" or "contrast" in photography.

[0043] The above is a brief introduction to the terms related to the embodiments of the present application, which will not be described below.

[0044] The image processing method provided by the embodiments of the present application can be applied to various electronic devices with a shooting function. The shooting function can shoot photos or videos. The electronic device can be, but is not limited to, a mobile phone, a tablet computer, a desktop computer, a laptop computer, a notebook computer, a vehicle-mounted device, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) / virtual reality (VR) device, etc., and the embodiments of the present application do not limit this.

[0045] For example, a mobile phone is taken as an example, Figure 1 An interface diagram for starting shooting in a camera application is shown.

[0046] Referring to Figure 1 , when the user lights up the screen of the mobile phone and controls the mobile phone to be in an unlocked state, the mobile phone can display an interface as shown in Figure 1 (a), which can be the desktop of the mobile phone, and the icons of a plurality of installed application programs are displayed on the desktop of the mobile phone, such as a clock application icon, a calendar application icon, a gallery application icon, a memo application icon, a file management application icon, an email application icon, a music application icon, a calculator application icon, a video application icon, a sports health application icon, a weather application icon, a browser application icon, a smart life application icon, a settings application icon, a sound recorder application icon, an application store application icon, a contacts application icon, a phone application icon, an information application icon, and a camera application icon 10, etc.

[0047] The user can perform a touch operation on the camera application icon 10, which can be a click operation, a long press operation, etc. Correspondingly, the mobile phone receives the touch operation of the user on the camera application icon 10, and in response to the touch operation of the user on the camera application icon 10, the mobile phone starts the camera application.

[0048] After the camera app is launched, the phone can display something like this: Figure 1 The default shooting interface is shown in (b) above. This shooting interface includes the currently captured preview image, shooting control 11, and function controls corresponding to various shooting modes. These function controls may include, for example, night mode controls, portrait mode controls, photo mode controls, video mode controls, aperture mode controls, and additional controls for enabling more functions in the camera application. Shooting control 11 is used to trigger the phone's shooting operation. It should be understood that the arrangement order of the various shooting modes on the phone interface can be set and changed as needed, and this embodiment does not impose any limitations on this.

[0049] When a user needs to take an image using the phone's default mode, the user can directly click the shooting control 11. In response to the user's click, the phone will use the default shooting mode to take the image. When a user needs to take an image using night mode, the user can first click the night mode control to activate night mode, and then click the shooting control 11. In response to the user's click, the phone will use night mode to take the image. When a user needs to take an image using portrait mode, the user can first click the portrait mode control to activate night mode, and then click the shooting control 11. In response to the user's click, the phone will use portrait mode to take the image.

[0050] Based on current imaging algorithms, after acquiring RAW images from image sensors, electronic devices typically need to use a demosaic algorithm to convert the RAW images into RGB images. The demosaic algorithm uses interpolation to add other color information to each monochrome pixel in the RAW image, thereby converting each pixel into color.

[0051] Since other color information in each pixel is supplemented by interpolation, this color information is usually not very accurate, which may cause moiré patterns and loss of detail in high-frequency positions of the resulting RGB image, resulting in a decrease in the clarity of the RGB image. Figure 2 The illustration shows a schematic diagram of images before and after depigmentation according to an embodiment of this application. For example... Figure 2 As shown, the RGB image obtained after processing the RAW image with the demosaic algorithm suffers from problems such as loss of detail texture and the appearance of moiré patterns.

[0052] Based on this, the application proposes to correct the RGB image obtained by demosaicing the RAW image by using the gray image corresponding to the RAW image. Since the gray image corresponding to the RAW image is an image obtained without demosaicing, the details in the RAW image can be retained in the gray image. Further, the RGB image obtained by demosaicing is corrected based on the gray image, which can enhance the details of the RGB image, reduce the moire caused by demosaicing, and further improve the clarity of the RGB image.

[0053] The hardware structure schematic diagram of the electronic device 100 that can implement the scheme of the application is introduced below. An example is shown in the following table. Figure 3 The hardware structure schematic diagram of an example of the electronic device 100 provided by the embodiments of the application is shown.

[0054] The electronic device 100 can include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charge management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a loudspeaker 170A, a receiver 170B, a microphone 170C, a headset interface 170D, a sensor module 180, a key 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 can include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0055] It can be understood that the structure shown in the embodiments of the application does not constitute a specific limitation on the electronic device 100. In other embodiments of the application, the electronic device 100 can include more or fewer components than shown, or combine certain components, or split certain components, or different arrangement of components. The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0056] The processor 110 can include one or more processing units, for example: the processor 110 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units can be independent devices or integrated in one or more processors.

[0057] The controller can generate operation control signals according to the instruction operation code and the timing signal, and complete the control of fetching and executing instructions.

[0058] The processor 110 can also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory can save instructions or data that have just been used or are used repeatedly by the processor 110. If the processor 110 needs to use the instructions or data again, it can directly call from the memory. This avoids repeated access and reduces the waiting time of the processor 110, thereby improving the efficiency of the system.

[0059] In some embodiments, the processor 110 can include one or more interfaces. The interfaces can include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0060] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 can contain multiple sets of I2C bus. The processor 110 can be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces respectively. For example, the processor 110 can be coupled to the touch sensor 180K through an I2C interface, so that the processor 110 and the touch sensor 180K communicate through the I2C bus interface, and the touch function of the electronic device 100 is realized.

[0061] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can contain multiple sets of I2S bus. The processor 110 can be coupled to the audio module 170 through the I2S bus, and communication between the processor 110 and the audio module 170 is realized. In some embodiments, the audio module 170 can deliver audio signals to the wireless communication module 160 through the I2S interface, and the function of answering a phone through a Bluetooth headset is realized.

[0062] The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled through the PCM bus interface. In some embodiments, the audio module 170 can also deliver audio signals to the wireless communication module 160 through the PCM interface, and the function of answering a phone through a Bluetooth headset is realized. Both the I2S interface and the PCM interface can be used for audio communication.

[0063] The UART interface is a universal serial data bus, which is used for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is usually used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface, and the Bluetooth function is realized. In some embodiments, the audio module 170 can deliver audio signals to the wireless communication module 160 through the UART interface, and the function of playing music through a Bluetooth headset is realized.

[0064] The MIPI interface can be used to connect the processor 110 and the display screen 194, the camera 193 and other peripheral devices. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), and the like. In some embodiments, the processor 110 and the camera 193 communicate through the CSI interface to implement the photographing function of the electronic device 100. The processor 110 and the display screen 194 communicate through the DSI interface to implement the display function of the electronic device 100.

[0065] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 and the camera 193, the display screen 194, the wireless communication module 160, the audio module 170, the sensor module 180, and the like. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, and the like.

[0066] The USB interface 130 is an interface that conforms to the USB standard specification, and can be a Mini USB interface, a Micro USB interface, a USB Type C interface, or the like. The USB interface 130 can be used to connect a charger to charge the electronic device 100, or to transmit data between the electronic device 100 and a peripheral device. It can also be used to connect a headset to play audio through the headset. The interface can also be used to connect other electronic devices, such as AR devices and the like.

[0067] It can be understood that the interface connection relationship between the modules shown in the embodiments of the present application is only illustrative and does not constitute a structural limitation of the electronic device 100. In other embodiments of the present application, the electronic device 100 can also use different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0068] The charging management module 140 is used to receive charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive wireless charging input through the wireless charging coil of the electronic device 100. The charging management module 140 can charge the battery 142 while also providing power to the electronic device through the power management module 141.

[0069] The power management module 141 is configured to connect the battery 142 and the charging management module 140 to the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to power the processor 110, the internal memory 121, the display 194, the camera 193, the wireless communication module 160, and the like. The power management module 141 can also be configured to monitor parameters such as battery capacity, battery cycle count, battery health status (leakage, impedance), and the like. In some embodiments, the power management module 141 can also be disposed in the processor 110. In some embodiments, the power management module 141 and the charging management module 140 can also be disposed in the same device.

[0070] The wireless communication function of the electronic device 100 can be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor, and the like.

[0071] The antenna 1 and the antenna 2 are configured to transmit and receive electromagnetic wave signals. Each antenna in the electronic device 100 can be configured to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization of the antennas. For example, the antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some embodiments, the antennas can be used in combination with a tuning switch.

[0072] The mobile communication module 150 can provide a solution for wireless communication including 2G / 3G / 4G / 5G and the like applied to the electronic device 100. The mobile communication module 150 can include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), and the like. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and perform filtering, amplification, and the like on the received electromagnetic waves, and transmit the processed electromagnetic waves to the modem processor for demodulation. The mobile communication module 150 can also amplify signals modulated by the modem processor, and radiate the amplified signals as electromagnetic waves through the antenna 1. In some embodiments, at least part of the function modules of the mobile communication module 150 can be disposed in the processor 110. In some embodiments, at least part of the function modules of the mobile communication module 150 and at least part of the modules of the processor 110 can be disposed in the same device.

[0073] The modem processor can include a modulator and a demodulator. The modulator is configured to modulate a low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is configured to demodulate a received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. The low-frequency baseband signal processed by the baseband processor is transmitted to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a microphone 170B, etc.), or displays an image or a video through the display 194. In some embodiments, the modem processor can be a separate device. In other embodiments, the modem processor can be independent of the processor 110 and disposed in the same device as the mobile communication module 150 or other functional modules.

[0074] The wireless communication module 160 can provide a wireless communication solution including a wireless local area network (WLAN) (such as a wireless fidelity (Wi-Fi) network), Bluetooth (BT), a global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR) technology, and the like, which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives an electromagnetic wave via the antenna 2, performs frequency modulation and filtering processing on the electromagnetic wave signal, and transmits the processed signal to the processor 110. The wireless communication module 160 can also receive a signal to be transmitted from the processor 110, perform frequency modulation and amplification thereon, and radiate the signal as an electromagnetic wave via the antenna 2.

[0075] In some embodiments, the antenna 1 and the mobile communication module 150 of the electronic device 100 are coupled, and the antenna 2 and the wireless communication module 160 are coupled, so that the electronic device 100 can communicate with a network and other devices through wireless communication technology. The wireless communication technology can include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology, etc. The GNSS can include a global positioning system (GPS), a global navigation satellite system (GLONASS), a beidu navigation satellite system (BDS), a quasi-zenith satellite system (QZSS), and / or a satellite based augmentation systems (SBAS).

[0076] The electronic device 100 implements a display function through a GPU, a display screen 194, and an application processor, etc. The GPU is a microprocessor for image processing, which is connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 can include one or more GPUs, which execute program instructions to generate or change display information.

[0077] The display screen 194 is configured to display images, videos, and the like. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flex light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light emitting diodes (QLED), or the like. In some embodiments, the electronic device 100 can include one or N display screens 194, where N is a positive integer greater than 1.

[0078] The electronic device 100 can implement the photographing function through the ISP, the camera 193, the video codec, the GPU, the display screen 194, and the application processor.

[0079] The ISP is configured to process the data fed back by the camera 193. For example, when taking a photo, the shutter is opened, the light is transmitted to the camera photosensitive element through the lens, the light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing to convert it into an image visible to the naked eye. The ISP can also optimize the noise, brightness, and skin color of the image. The ISP can also optimize the exposure, color temperature, and other parameters of the shooting scene. In some embodiments, the ISP can be disposed in the camera 193.

[0080] The camera 193 is configured to capture still images or videos. An object generates an optical image through a lens and projects it onto a photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then transmitted to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV, or the like format. In some embodiments, the electronic device 100 can include one or N cameras 193, where N is a positive integer greater than 1.

[0081] The digital signal processor is used to process digital signals, in addition to being able to process digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy, etc.

[0082] The video codec is used to compress or decompress digital video. The electronic device 100 can support one or more video codecs. In this way, the electronic device 100 can play or record videos in multiple encoding formats, such as: moving picture experts group (MPEG) 1, MPEG 2, MPEG 3, MPEG 4, etc.

[0083] The NPU is a neural-network (NN) calculation processor, which can quickly process input information by drawing on the structure of a biological neural network, such as drawing on the transmission mode between human brain neurons, and can also constantly self-learn. Through the NPU, the electronic device 100 can realize intelligent cognition applications such as image recognition, face recognition, voice recognition, text understanding, etc.

[0084] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to realize data storage functions. For example, music, video, etc. Files are saved in the external memory card.

[0085] The internal memory 121 can be used to store computer executable program codes, which include instructions. The internal memory 121 can include a program storage area and a data storage area. The program storage area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.), etc. The data storage area can store data created during the use of the electronic device 100 (such as audio data, a phonebook, etc.), etc. In addition, the internal memory 121 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various function applications and data processing of the electronic device 100 by running instructions stored in the internal memory 121 and / or instructions stored in the memory disposed in the processor.

[0086] The electronic device 100 can realize audio functions through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor, etc. For example, music playing, recording, voice call, video call, etc.

[0087] The audio module 170 is configured to convert digital audio information into an analog audio signal output, and to convert an analog audio input into a digital audio signal. The audio module 170 can also be configured to encode and decode audio signals. In some embodiments, the audio module 170 can be disposed in the processor 110, or some of the functions of the audio module 170 can be disposed in the processor 110.

[0088] The speaker 170A, also referred to as a "loudspeaker", is configured to convert an audio electrical signal into a sound signal. The electronic device 100 can listen to music or listen to a hands-free call through the speaker 170A.

[0089] The receiver 170B, also referred to as a "earpiece", is configured to convert an audio electrical signal into a sound signal. When the electronic device 100 receives a call or a voice message, the user can listen to the voice by holding the receiver 170B close to the ear.

[0090] The microphone 170C, also referred to as a "microphone", "transducer", is configured to convert a sound signal into an electrical signal. When making a call or sending a voice message, the user can speak into the microphone 170C by holding the mouth close to the microphone 170C, and input the sound signal into the microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, in addition to collecting sound signals, the noise reduction function can also be realized. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C, in addition to collecting sound signals, noise reduction, it can also identify the source of the sound, realize the function of directional recording, etc.

[0091] The earphone interface 170D is configured to connect a wired earphone. The earphone interface 170D can be a USB interface 130, or a 3.5mm open mobile terminal platform (OMTP) standard interface, a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0092] The pressure sensor 180A is configured to sense a pressure signal and convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be disposed on the display screen 194. The pressure sensor 180A can be of various types, such as a resistive pressure sensor, an inductive pressure sensor, a capacitive pressure sensor, etc. The capacitive pressure sensor can include at least two parallel plates of conductive material. When a force is applied to the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure according to the change in capacitance. When a touch operation is applied to the display screen 194, the electronic device 100 detects the intensity of the touch operation according to the pressure sensor 180A. The electronic device 100 can also calculate the position of the touch according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations applied to the same touch position but with different touch operation intensities can correspond to different operation instructions. For example, when a touch operation with a touch operation intensity less than a first pressure threshold is applied to a short message application icon, an instruction to view a short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold is applied to the short message application icon, an instruction to create a new short message is executed.

[0093] The gyroscope sensor 180B can be configured to determine the motion attitude of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor 180B. The gyroscope sensor 180B can be used for anti-shake photography. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of shaking of the electronic device 100, calculates the distance that the lens module needs to compensate according to the angle, and lets the lens offset the shaking of the electronic device 100 by reverse movement to achieve anti-shake. The gyroscope sensor 180B can also be used for navigation and motion sensing game scenarios.

[0094] The barometric pressure sensor 180C is configured to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude, assists positioning and navigation by using the air pressure value measured by the barometric pressure sensor 180C.

[0095] The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can detect the opening and closing of a flip cover by using the magnetic sensor 180D. In some embodiments, when the electronic device 100 is a flip phone, the electronic device 100 can detect the opening and closing of the flip cover according to the magnetic sensor 180D. Further, according to the detected opening and closing state of the cover or the flip cover, the electronic device 100 can set a feature such as automatic unlocking of the flip cover.

[0096] The acceleration sensor 180E can detect the magnitude of acceleration of the electronic device 100 in various directions (generally three axes). When the electronic device 100 is stationary, the acceleration sensor 180E can detect the magnitude and direction of gravity. The acceleration sensor 180E can also be used to identify the attitude of the electronic device, and can be applied to landscape / portrait screen switching and pedometer applications.

[0097] Distance sensor 180F is configured to measure distance. Electronic device 100 can measure distance by infrared or laser. In some embodiments, electronic device 100 can utilize distance sensor 180F to measure distance for fast focusing when taking a picture.

[0098] Proximity light sensor 180G can include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode can be an infrared light emitting diode. Electronic device 100 emits infrared light outwardly through the light emitting diode. Electronic device 100 detects infrared reflected light from nearby objects using the photodiode. When sufficient reflected light is detected, electronic device 100 can determine that there is an object near electronic device 100. When insufficient reflected light is detected, electronic device 100 can determine that there is no object near electronic device 100. Electronic device 100 can utilize proximity light sensor 180G to detect that a user is holding electronic device 100 close to the ear for a phone call, so as to automatically turn off the screen to save power. Proximity light sensor 180G can also be used for automatic unlocking and locking of the screen in a holster mode or a pocket mode.

[0099] Ambient light sensor 180L is configured to sense ambient light brightness. Electronic device 100 can adaptively adjust the brightness of display screen 194 according to the sensed ambient light brightness. Ambient light sensor 180L can also be used to automatically adjust white balance when taking a picture. Ambient light sensor 180L can also cooperate with proximity light sensor 180G to detect whether electronic device 100 is in a pocket to prevent accidental touch.

[0100] Fingerprint sensor 180H is configured to collect a fingerprint. Electronic device 100 can utilize the collected fingerprint characteristics to implement fingerprint unlocking, access application lock, fingerprint picture taking, fingerprint call answering, and the like.

[0101] Temperature sensor 180J is configured to detect temperature. In some embodiments, electronic device 100 utilizes the temperature detected by temperature sensor 180J to implement temperature processing strategies. For example, when the temperature reported by temperature sensor 180J exceeds a threshold, electronic device 100 implements performance reduction of a processor located near temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, electronic device 100 heats battery 142 to avoid abnormal shutdown of electronic device 100 caused by low temperature. In other embodiments, when the temperature is lower than yet another threshold, electronic device 100 implements voltage boosting of the output voltage of battery 142 to avoid abnormal shutdown caused by low temperature.

[0102] Touch sensor 180K, also referred to as "touch device". Touch sensor 180K can be disposed on display screen 194, and touch sensor 180K and display screen 194 form a touch screen, also referred to as "touch panel". Touch sensor 180K is configured to detect touch operations applied to or near the touch sensor 180K. The touch sensor can transmit the detected touch operation to the application processor to determine the touch event type. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K can also be disposed on the surface of electronic device 100, which is different from the position of display screen 194.

[0103] Bone conduction sensor 180M can obtain vibration signals. In some embodiments, bone conduction sensor 180M can obtain vibration signals of the human body's vocal vibration bone block. Bone conduction sensor 180M can also contact the human body pulse to receive blood pressure pulsation signals. In some embodiments, bone conduction sensor 180M can also be disposed in a headset to form a bone conduction headset. Audio module 170 can analyze voice signals based on the vibration signals of the vocal vibration bone block obtained by the bone conduction sensor 180M to realize voice functions. The application processor can analyze heart rate information based on the blood pressure pulsation signals obtained by the bone conduction sensor 180M to realize heart rate detection functions.

[0104] Keys 190 include power on / off keys, volume keys, and the like. Keys 190 can be mechanical keys. They can also be touch keys. Electronic device 100 can receive key input and generate key signal input related to user settings and function control of electronic device 100.

[0105] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts and also for touch vibration feedback. For example, touch operations applied to different applications (such as taking pictures, playing audio, etc.) can correspond to different vibration feedback effects. Touch operations applied to different regions of display screen 194 can also correspond to different vibration feedback effects. Different application scenarios (such as time reminders, received messages, alarms, games, etc.) can also correspond to different vibration feedback effects. Touch vibration feedback effects can also be customizable.

[0106] Indicator 192 can be an indicator light, which can be used to indicate charging status, power changes, and also to indicate messages, missed calls, notifications, and the like.

[0107] The SIM card interface 195 is configured to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to achieve contact and separation with the electronic device 100. The electronic device 100 can support one or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 195 can be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external storage cards. The electronic device 100 interacts with a network through the SIM card to achieve functions such as calling and data communication. In some embodiments, the electronic device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0108] The software system of the electronic device 100 can use a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture. Embodiments of the present application exemplarily illustrate the software structure of the electronic device 100 by taking a layered architecture Android system as an example. It should be noted that in the embodiments of the present application, the operating system of the electronic device can include but is not limited to (Symbian), (Andriod), (iOS), (Blackberry), HarmonyOS, and the like, which are not limited in the present application.

[0109] Figure 4 A software structure block diagram of an example of the electronic device 100 is shown.

[0110] The layered architecture divides the software into several layers, and each layer has a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the Android system includes, from top to bottom, an application layer, an application framework layer (Application Framework), a hardware abstraction layer (Hardware Abstract Layer, HAL), a kernel layer, and a hardware layer.

[0111] The application layer can include a camera application or other applications, including but not limited to music, gallery, Bluetooth, calendar, short message, call, navigation, and the like.

[0112] The application framework layer can provide an application program of the application layer with an application programming interface (API) and a programming framework; the application framework layer can include some predefined functions.

[0113] For example, the application framework layer can include a camera access interface; the camera access interface can include a camera management and a camera device; the camera management can be used to provide an access interface for managing a camera; and the camera device can be used to provide an interface for accessing a camera.

[0114] The hardware abstraction layer is used to abstract hardware. For example, the hardware abstraction layer can include a camera hardware abstraction layer and other hardware device abstraction layers; the camera hardware abstraction layer includes an image processing algorithm, etc. After an image sensor of a camera obtains RAW image data, the RAW image data can be processed by the image processing algorithm to obtain an RGB image. In addition, the camera abstraction layer also stores a captured image into a gallery application program, so as to facilitate a user to view.

[0115] The kernel layer is used to provide a driver for different hardware devices. For example, the driver layer can include a camera driver, a display driver and an image processor driver.

[0116] The hardware layer can include a plurality of image sensors, a plurality of image signal processors, a camera, a display screen and other hardware devices.

[0117] In the present application, by calling a hardware abstraction layer interface in the hardware abstraction layer, the connection between the application layer and the application framework layer above the hardware abstraction layer and the driver layer and the hardware layer below can be realized, and the camera data transmission and the function control can be realized.

[0118] In the camera hardware interface layer in the hardware abstraction layer, a vendor can customize functions according to requirements. Compared with the hardware abstraction layer interface, the camera hardware interface layer is more efficient, flexible, low-latency, and can also more richly call an ISP and a GPU to realize image processing. The input image in the hardware abstraction layer can come from an image sensor or a stored picture.

[0119] The scheduling layer in the hardware abstraction layer includes a general functional interface, which is used to realize management and control.

[0120] The camera service layer in the hardware abstraction layer is used to access an interface of an ISP and other hardware.

[0121] The image processing method provided in the present application will be described in detail below. Figures 5 to 8 The image processing method provided in the present application will be described in detail below. Figure 5 A flowchart of an example of the image processing method provided in the embodiment of the present application is shown. Figure 5The image processing method shown can be performed by an electronic device shown in Figure 3 or by a chip configured in the electronic device shown in Figure 3 The image processing method shown includes steps S510 to S560, which are described in detail below. Figure 5

[0122] S510, the electronic device starts a camera application and opens a shooting mode.

[0123] When the user wants to start the camera application, the user can click the camera application icon 10 shown in (a) of FIG. 1B, so that the electronic device can receive the touch operation of the user on the camera application icon 10, and the electronic device starts the camera application in response to the touch operation, and displays the shooting interface shown in (b) of FIG. 1B after the camera application is started, and the user can further select the required shooting mode according to the interface shown in (b) of FIG. 1B. Figure 1 Figure 1 Figure 1 For example, the user can select the default shooting mode of the camera, or can select the night scene, portrait, or HDR mode, and the present application does not limit this. The opening control of the HDR mode can be displayed on the interface of the electronic device when the user clicks the "more" control on the interface shown in (b) of FIG. 1B.

[0124] Figure 1

[0125] It can be understood that there are many ways to start the camera application, in addition to the above-mentioned touch operation on the camera application icon to start the camera application, the camera application can also be started by voice triggering or sliding triggering and the like. For example, when the electronic device is in a locked state, the user can indicate the electronic device to start the camera application by sliding rightward on the display screen of the electronic device. Alternatively, the electronic device is in a locked state, and the icon of the camera application is included on the lock screen, and the user can indicate the electronic device to start the camera application by clicking the icon of the camera application. Alternatively, the electronic device is running other applications, and the application has the permission to call the camera application; the user can indicate the electronic device to start the camera application program by clicking the corresponding control. For example, when the electronic device is running an instant messaging application, the user can indicate the electronic device to start the camera application program by using the control of the camera function, and the like. The specific operation mode of starting the camera application is not limited in the present application.

[0126] S520, the electronic device detects a shooting instruction, and the shooting instruction is used to request the electronic device to shoot.

[0127] ​​​​​After the shooting mode of the electronic device is confirmed, the user can issue a shooting instruction to request the electronic device to shoot. Exemplarily, the shooting instruction can be a click operation of the user on the shooting control, which can be the shooting control 11 shown in (b) of FIG. 1A. It should be understood that the user can also trigger the electronic device to shoot through a voice instruction, or a gesture instruction, etc., which are not limited in the present application. Figure 1

[0128] S530, in response to the shooting instruction, the electronic device acquires a RAW image.

[0129] The RAW image is a bayer format image obtained based on an image sensor covered with a CFA, and each pixel in the RAW image is a monochrome pixel, that is, each pixel corresponds to one color (for example, R, G or B), in other words, the pixel value of each pixel can correspond to an R value, a G value or a B value. Exemplarily, the RAW image can be obtained by the image sensor 110 shown in (a) of FIG. 1A. Figure 6 (a) and (c) of FIG. 1A show an arrangement of a bayer array:

[0130]

[0131] It should be understood that the bayer array shown in (a) and (c) of FIG. 1A is only an example, and other arrangements can also be used in practice, which are not limited in the present application. Figure 6

[0132] Optionally, in the case of the default shooting mode, portrait mode or night scene mode of the camera, a single frame image obtained by the sensor can be used as the RAW image; in the HDR shooting mode, a plurality of images obtained by the sensor can be fused, and the image obtained after fusion can be used as the RAW image, and the plurality of images correspond to different exposure amounts.

[0133] S540, the electronic device performs demosaicing on the RAW image to obtain a first RGB image.

[0134] That is, other color information is supplemented in each pixel in the RAW image by interpolation to convert each pixel into a color, so as to convert the RAW image into a color image (i.e., the first RGB image).

[0135] For example, red and blue information is supplemented in the green pixel to convert it into a color. For example, the monochrome pixel G Figure 6 in the bayer array shown in (a) of FIG. 1A 23 , the interpolation is to insert the information of two colors of R value and B value in the monochrome pixel G 23 . The inserted R value can be determined based on the R values of the surrounding pixels, for example, can be determined based on R 13 and R​​33 The average value is determined; the inserted B value can be determined based on the B values ​​of pixels surrounding that location, for example, it can be based on B... 22 and B 24 The average value is determined. It should be understood that the above interpolation method is only an example, and other interpolation methods can be used in practice. This application does not limit this method.

[0136] The S550 is an electronic device that converts RAW images into grayscale images.

[0137] It should be noted that grayscale images in current technology are usually acquired based on black and white lenses. This means that if you want to use grayscale images to correct RGB images based on current technology, you need to add a black and white lens to the existing camera. This increases the configuration cost, and there is a positional difference between the two cameras. The positional difference needs to be calibrated during shooting, and errors are easy to occur during the calibration process.

[0138] Based on this, this application proposes to statistically analyze the pixel value of each pixel in a RAW image using an electronic device; then, to determine the conversion coefficients of the RAW image based on the pixel value of each pixel; and finally, to convert the RAW image using the conversion coefficients to obtain a grayscale image. Compared to the traditional method of obtaining grayscale images using a black and white lens, this method can reduce configuration costs while obtaining grayscale images.

[0139] In a specific implementation, the electronic device can calculate the average value R' of all pixel values ​​corresponding to the R value, the average value G' of all pixel values ​​corresponding to the G value, and the average value B' of all pixel values ​​corresponding to the B value in the RAW image; and then obtain the conversion coefficient G' / R' of the pixel value corresponding to the R value, the conversion coefficient 1 of the pixel value corresponding to the G value, and the conversion coefficient G' / B' of the pixel value corresponding to the B value based on the average value R', the average value G', and the average value B'.

[0140] For example Figure 6 Taking the Bayer array shown in (a) as an example, the average value R' can be R 11 R 13 R 31 and R 33 The average value, the average value G' can be G 12 G 14 G 21 G 23 G 32 G 34 G 41 and G 43 The average value, the average value B' can be B 22 B 24 B 42 and B44 The average value.

[0141] Converting a RAW image to a grayscale image based on conversion coefficients can include: converting the pixel values ​​corresponding to the R value in the RAW image according to the conversion coefficient G' / R' to obtain a first conversion result; converting the pixel values ​​corresponding to the G value in the RAW image according to the conversion coefficient 1 to obtain a second conversion result; converting the pixel values ​​corresponding to the B value in the RAW image according to the conversion coefficient G' / B' to obtain a third conversion result; and obtaining a grayscale image based on the first conversion result, the second conversion result, and the third conversion result.

[0142] For example, such as Figure 6 As shown in (b), when converting pixel values ​​according to the conversion coefficient, it can be achieved by multiplication, that is, multiplying the pixel value of each pixel with its corresponding conversion coefficient (for example, multiplying the pixel value corresponding to the R value with the conversion coefficient G' / R', multiplying the pixel value corresponding to the G value with the conversion coefficient 1, and multiplying the pixel value corresponding to the B value with the conversion coefficient G' / B'), so that the pixel value of each pixel is approximately equal to the G value, thereby making Figure 6 The RAW image shown in (c) is converted to Figure 6 The grayscale image shown in (d) is shown in the image.

[0143] It should be understood that since the grayscale image corresponding to the RAW image is an image obtained without demosaic processing, the details in the RAW image can be preserved in the grayscale image.

[0144] It should be understood that the above methods for converting RAW images to grayscale images are merely examples. In practice, other methods can also be used to convert RAW images to grayscale images, and this application does not limit this.

[0145] S560, the electronic device corrects the first RGB image based on the grayscale image to obtain the second RGB image.

[0146] Specifically, such as Figure 7 As shown, the first RGB image can be converted into a first YUV image first; then, the grayscale image is fused into the Y channel of the first YUV image to obtain a second YUV image. This application fuses a grayscale image that preserves the details of the RAW image into the Y channel of the YUV image. That is, without changing the output image color (i.e., without changing the U and V channels), the details in the RAW image are fused into the luminance channel of the YUV image, thereby increasing the overall detail and texture of the image and improving the final image effect.

[0147] In addition, since YUV image data generally cannot be used directly for display, it needs to be converted to RGB format in order to display the image correctly. That is, the second YUV image also needs to be converted to the second RGB image in order to display it correctly.

[0148] Figure 8 This is a schematic diagram illustrating the difference between an RGB image before and after correction, as provided in an embodiment of this application. It can be seen that correcting the RGB image obtained through de-mosaic processing based on a grayscale image can enhance the overall detail and texture of the image, improving the final image clarity.

[0149] In summary, this application uses the grayscale image corresponding to the RAW image to correct the RGB image obtained by de-mosaicing the RAW image. Since the grayscale image corresponding to the RAW image is an image obtained without de-mosaicing, the details in the RAW image can be preserved in the grayscale image. Furthermore, by correcting the RGB image obtained by de-mosaicing based on the grayscale image, the details of the RGB image can be enhanced, the moiré pattern caused by de-mosaicing can be reduced, and the clarity of the RGB image can be improved.

[0150] Furthermore, this application obtains grayscale and RGB images by performing different processing on the same RAW image, eliminating the need for additional black and white lenses and positional calibration. This reduces configuration costs while improving image processing efficiency and the quality of the final RGB image.

[0151] The above combination Figures 5 to 8 The embodiments of the method described in this application have been explained. The apparatus for performing the above method provided in the embodiments of this application is described below.

[0152] like Figure 9 As shown, Figure 9 A schematic diagram of an image processing apparatus provided in an embodiment of this application is shown. Figure 9 As shown, the device 900 includes a detection unit 910, an acquisition unit 920, and a processing unit 930.

[0153] The detection unit 910 is used to detect a shooting command, which requests the electronic device to take a picture; in response to the shooting command, the acquisition unit 920 is used to acquire a RAW image; the processing unit 930 is used to: perform de-mosaic processing on the RAW image to obtain a first RGB image; convert the RAW image to a grayscale image; and correct the first RGB image based on the grayscale image to obtain a second RGB image.

[0154] In a possible implementation, the processing unit 930 is further configured to: convert the first RGB image into a first YUV image; fuse the grayscale image into a Y channel of the first YUV image to obtain a second YUV image; and convert the second YUV image into a second RGB image.

[0155] In a possible implementation, the processing unit 930 is further configured to: count pixel values of each pixel in the RAW image; determine a conversion coefficient of the RAW image according to the pixel value of each pixel; and convert the RAW image according to the conversion coefficient to obtain the grayscale image.

[0156] In a possible implementation, the pixel value of each pixel corresponds to an R value, a G value, or a B value, and the processing unit 930 is further configured to: calculate an average value R' of all pixel values corresponding to the R value, an average value G' of all pixel values corresponding to the G value, and an average value B' of all pixel values corresponding to the B value in the RAW image respectively; and obtain a conversion coefficient G' / R' of the pixel value corresponding to the R value, a conversion coefficient 1 of the pixel value corresponding to the G value, and a conversion coefficient G' / B' of the pixel value corresponding to the B value according to the average value R', the average value G', and the average value B'.

[0157] In a possible implementation, the processing unit 930 is further configured to: convert the pixel value corresponding to the R value in the RAW image according to the conversion coefficient G' / R' to obtain a first conversion result; convert the pixel value corresponding to the G value in the RAW image according to the conversion coefficient 1 to obtain a second conversion result; convert the pixel value corresponding to the B value in the RAW image according to the conversion coefficient G' / B' to obtain a third conversion result; and obtain the grayscale image according to the first conversion result, the second conversion result, and the third conversion result.

[0158] In a possible implementation, the processing unit 930 is further configured to: multiply the conversion coefficient G' / R' with the pixel value corresponding to the R value in the RAW image; multiply the conversion coefficient 1 with the pixel value corresponding to the G value in the RAW image; and multiply the conversion coefficient G' / B' with the pixel value corresponding to the B value in the RAW image.

[0159] In a possible implementation, the apparatus 900 can further include a storage unit. The storage unit, the detection unit 910, the acquisition unit 920, and the processing unit 930 are connected through a communication bus. The storage unit can include one or more memories, and the memory can be a device for storing programs or data in one or more devices or circuits. The storage unit can exist independently and be connected to the processing unit through the communication bus. The storage unit can also be integrated with the detection unit 910, the acquisition unit 920, and the processing unit 930.

[0160] The storage unit can store computer execution instructions of the method in the device 900, so that the device 900 executes the method in the above-described embodiments. The storage unit can be a register, a cache memory, or a random access memory (RAM), etc. The storage unit can be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions.

[0161] Figure 10 A structure diagram of a chip provided by an embodiment of the present application is shown. As shown in the figure, the chip 1000 includes one or more than two (including two) processors 1001, a communication line 1002, and a communication interface 1003. Optionally, the chip 1000 further includes a memory 1004. Figure 10

[0162] In some embodiments, the memory 1004 stores the following elements: executable modules or data structures, or a subset thereof, or an extended set thereof.

[0163] The method described in the above embodiments of the present application can be applied to the processor 1001 or implemented by the processor 1001. The processor 1001 can be an integrated circuit chip with a processing capability of signals. In the implementation process, the steps of the above method can be completed by integrated logic circuits or instructions in the form of software in the processor 1001. The processor 1001 described above can be a general-purpose processor (for example, a microprocessor or a conventional processor), a digital signal processor, an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, or discrete hardware components.

[0164] The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. Among them, the software module can be located in a mature storage medium in the field, such as a random access memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable read-only memory (EEPROM). The storage medium is located in the memory 1004, and the processor 1001 reads the information in the memory 1004 and combines the hardware to complete the steps of the above method.

[0165] ​The processor 1001, the memory 1004, and the communication interface 1003 can communicate through the communication line 1002.

[0166] In the above embodiments, the instructions stored in the memory for the processor to execute can be implemented in the form of a computer program product. The computer program product can be written in the memory in advance, or downloaded and installed in the memory in the form of software.

[0167] The embodiments of the present application also provide a computer program product including one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the flow or function according to the embodiments of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that the computer can store or the data storage device such as a server, data center, etc. integrated with one or more available media sets. For example, the available media can include magnetic media (such as floppy disk, hard disk or magnetic tape), optical media (such as digital versatile disc (DVD)), or semiconductor media (such as solid state disk (SSD)) and the like.

[0168] The present application provides an image processing apparatus, which is a first electronic device or is included in the first electronic device, and the apparatus includes one or more processors, one or more memories, and the memory stores one or more programs, when the one or more programs are executed by the processor, the device executes the technical solutions in the above method embodiments.

[0169] The embodiments of the present application provide a chip. The chip includes a processor, and the processor is configured to invoke a computer program in a memory to execute the technical solutions in the above method embodiments. The implementation principles and technical effects are similar to the above related embodiments, and will not be repeated here.

[0170] The embodiments of the present application further provide a computer readable storage medium. The computer readable storage medium stores computer programs or instructions. The computer programs or instructions are executed by a processor to implement the above method. The method described in the above embodiments can be implemented by software, hardware, firmware or any combination thereof, in whole or in part. If implemented by software, the functions can be stored in or transmitted as one or more instructions or codes on a computer readable medium. The computer readable medium can include computer storage medium and communication medium, and can further include any medium that can transfer computer programs from one place to another. The storage medium can be any target medium that can be accessed by a computer.

[0171] As a possible design, the computer readable medium can include a compact disc read-only memory (CD-ROM), RAM, ROM, EEPROM or other optical disk storage; the computer readable medium can include magnetic disk storage or other magnetic disk storage devices. Moreover, any connection line can also be appropriately referred to as a computer readable medium. For example, if software is transmitted from a website, server or other remote source using a coaxial cable, optical fiber cable, twisted pair, DSL or wireless technology (such as infrared, radio and microwave), the coaxial cable, optical fiber cable, twisted pair, DSL or wireless technology such as infrared, radio and microwave is included in the definition of the medium. As used herein, the disk and the optical disk include compact discs (CD), laser discs, optical discs, DVDs, floppy disks and Blu-ray discs, in which the disk usually reproduces data in a magnetic manner, and the optical disk reproduces data optically with a laser. The above combinations should also be included in the scope of the computer readable medium.

[0172] The embodiments of the present application are described with reference to flowcharts and / or block diagrams of the method, device (system) and computer program product according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general purpose computer, a special purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the computer or other programmable data processing device produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The device that implements the functions specified in one flow or multiple flows and / or blocks Figure 1 The device that implements the functions specified in one flow or multiple flows and / or blocks

[0173] The above detailed description is merely descriptive of the application and specific embodiments thereof. It is not intended as a limitation on the scope of the application. Changes, equivalent substitutions, improvements, combinations, and the like, which are apparent to a skilled artisan, are covered by the following claims.

Claims

1. An image processing method, characterized by, The method is applied to an electronic device, and the method comprises: detecting a shooting instruction, the shooting instruction being used to request the electronic device to shoot; in response to the shooting instruction, acquiring a RAW image; performing demosaicing processing on the RAW image to obtain a first RGB image; converting the RAW image into a grayscale image, the grayscale image being an image obtained without demosaicing processing on the RAW image; based on the grayscale image, correcting the first RGB image to obtain a second RGB image.

2. The method of claim 1, wherein, The method of correcting the first RGB image based on the grayscale image to obtain a second RGB image comprises: converting the first RGB image into a first YUV image; fusing the grayscale image into a Y channel of the first YUV image to obtain a second YUV image; converting the second YUV image into the second RGB image.

3. The method according to claim 1 or 2, characterized in that, The method of converting the RAW image into a grayscale image comprises: counting pixel values of each pixel in the RAW image; determining a conversion coefficient of the RAW image according to the pixel values of each pixel; converting the RAW image according to the conversion coefficient to obtain the grayscale image.

4. The method of claim 3, wherein, The pixel values of each pixel correspond to R values, G values or B values, and the method of determining the conversion coefficient of the RAW image according to the pixel values of each pixel comprises: respectively calculating an average value R' of all pixel values corresponding to R values, an average value G' of all pixel values corresponding to G values and an average value B' of all pixel values corresponding to B values in the RAW image; obtaining a conversion coefficient G' / R' of pixel values corresponding to R values, a conversion coefficient 1 of pixel values corresponding to G values and a conversion coefficient G' / B' of pixel values corresponding to B values according to the average value R', the average value G' and the average value B'.

5. The method of claim 4, wherein, The method of converting the RAW image according to the conversion coefficient to obtain the grayscale image comprises: converting pixel values corresponding to R values in the RAW image according to the conversion coefficient G' / R' to obtain a first conversion result; converting pixel values corresponding to G values in the RAW image according to the conversion coefficient 1 to obtain a second conversion result; converting pixel values corresponding to B values in the RAW image according to the conversion coefficient G' / B' to obtain a third conversion result; obtaining the grayscale image according to the first conversion result, the second conversion result and the third conversion result.

6. The method of claim 5, wherein, The method of converting pixel values corresponding to R values in the RAW image according to the conversion coefficient G' / R' comprises: multiplying the conversion coefficient G' / R' by pixel values corresponding to R values in the RAW image. The method of converting pixel values corresponding to G values in the RAW image according to the conversion coefficient 1 comprises: multiplying the conversion coefficient 1 by pixel values corresponding to G values in the RAW image. The method of converting pixel values corresponding to B values in the RAW image according to the conversion coefficient G' / B' comprises: multiplying the conversion coefficient G' / B' by pixel values corresponding to B values in the RAW image.

7. An electronic device, comprising: The method comprises: one or more processors; one or more memories; The memory stores one or more programs, which, when executed by the processor, cause the electronic device to perform the method of any one of claims 1-6.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions, which, when executed on a computer, cause the computer to perform the method of any one of claims 1-6.

Images