Image processing method and electronic device

Abstract

An image processing method and electronic device, relating to the field of image processing, solves the problem of texture detail loss and color deviation in the process of converting a high dynamic range image into a low dynamic range image, and improves the accuracy of the obtained low dynamic range image. In the method, a global mapping matrix and a local mapping matrix of a high dynamic range image are first obtained, and brightness correction is performed on the high dynamic range image based on the global mapping matrix to obtain a second image; color correction is performed on the high dynamic range image based on the local mapping matrix and the second image to obtain a low dynamic range image after the final color and brightness are restored.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image processing, and in particular to an image processing method and electronic device. Background Art

[0002] With the development of electronic technology, the shooting function of electronic devices such as mobile phones has also developed rapidly. In order to improve the shooting quality, some mobile phones have integrated the High-Dynamic Range (HDR) function. However, the HDR image obtained by the mobile phone using the HDR function has a relatively large dynamic range of brightness, which means that the difference between the brightest and darkest parts of the image is very large, such as generally within 0.001cd / m 2 -100000cd / m 2 In other words, the bright parts are overexposed and the dark parts are too dark, resulting in an unusable image. In order to make the image presented by the electronic device truly reflect the scene seen by the human eye, the electronic device generally converts the HDR image into a low dynamic range (LDR) image through multiple series of neural networks and then presents it to the user.

[0003] However, the final LDR image suffers from severe color cast and information loss, and cannot guarantee that the image can truly reflect the scene seen by the human eye. Summary of the invention

[0004] Based on this, the present application provides an image processing method and electronic device, which solves the problems of texture detail loss and color deviation in the process of converting high dynamic range images into low dynamic range images, and improves the accuracy of the obtained low dynamic range images.

[0005] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0006] In a first aspect, the present application provides an image processing method. In the method, an electronic device obtains a first image with a high dynamic range, and then obtains a global mapping relationship and a local mapping relationship of the first image to a corresponding low dynamic range image in terms of brightness and color, that is, a global mapping matrix and a local mapping matrix. After determining the global mapping matrix and the local mapping matrix, the brightness of the first image is corrected based on the global mapping matrix to obtain a second image; based on the local mapping matrix, the color of the first image is corrected to obtain a third image with a low dynamic range corresponding to the first image.

[0007] Among them, the above-mentioned image processing method is implemented based on a neural network, that is, a neural network can be used to implement tone mapping, that is, to convert a high dynamic range image into a low dynamic range image, avoid error accumulation, and ensure the accuracy of the color of the low dynamic range image finally output. In addition, the neural network includes two independent channels (such as an upper channel for brightness correction of the first image based on a global mapping matrix and a lower channel for color correction of the first image based on a local mapping matrix), which respectively realize brightness and color restoration processing. It can be seen that partial decoupling of brightness and color is achieved through these two independent channels, avoiding the loss of detail texture of the restored low dynamic range image. In summary, the accuracy of the obtained low dynamic range image is improved.

[0008] In addition, by performing color correction processing inside the neural network, negative truncation and high-bit interception of the image are avoided, further ensuring the accurate restoration of texture details.

[0009] In an implementation manner of the first aspect, after the first image is obtained, the first image may be downsampled, that is, the first image is compressed to reduce the number of pixel values ​​in the image and reduce the amount of calculation.

[0010] In an implementation manner of the first aspect, obtaining a global mapping matrix and a local mapping matrix of the first image may include: obtaining image features of the first image, then determining global features and local features of the first image based on the image features, and determining corresponding global mapping matrices and local mapping matrices according to the obtained global features and local features and different weight coefficients. For example, the global mapping matrix is ​​determined based on the global features, the local features, and the first weight coefficient, and the local mapping matrix is ​​determined based on the global features, the local features, and the second weight coefficient.

[0011] The brightness and color are restored respectively by the global mapping matrix and the local mapping matrix of the first image. Since the global mapping matrix and the layout mapping matrix include not only global features but also local features, that is, when restoring brightness and color, not only the global features of the image but also the local features of the image are considered, that is, the interaction between brightness and color is realized through the parameter sharing mechanism, and the loss of texture details of the restored low dynamic range image is further avoided. In addition, because the global features focus more on the brightness effect, the weight value corresponding to the global features in the first weight coefficient is greater than the weight value corresponding to the local features, so as to ensure the main processing of brightness; similarly, for local features, the color effect is more focused, so the weight value corresponding to the local features in the second weight coefficient is greater than the weight value corresponding to the global features, so as to ensure the main processing of color. The probability of texture detail loss and color deviation of the image is further reduced, thereby ensuring that the details and colors of the third image finally output from the whole to the local are clearer and more accurate.

[0012] In an implementation manner of the first aspect, correcting the brightness of the first image may include: first acquiring texture information of the first image, performing interpolation processing on a global mapping matrix of the first image according to the texture information to obtain a processed mapping relationship graph, and performing brightness correction on the first image based on the mapping relationship graph to obtain a second image. For example, the second image may be obtained by the following formula.

[0013] Contrast(R,G,B)(x,y)=input(R,G,B)(x,y)*Gain_map(x,y)

[0014] Among them, Contrast(R, G, B) is the second image, input(R, G, B)(x, y) is the first image, and Gain_map(x, y) is the mapping relationship diagram.

[0015] In the above, the mapping relationship diagram is first determined, and then the brightness correction is performed on the first image based on the mapping relationship diagram, so as to obtain the corrected second image, thereby ensuring the effect of image brightness alignment.

[0016] In an implementable manner of the first aspect, correcting the color of the first image may include: first determining a color correction matrix based on the second image and a local mapping matrix, such as interpolating the local correction matrix using the second image to obtain a color correction matrix, and then correcting the color of the first image based on the color correction matrix, such as performing color correction on the RGB channels of the second image based on the color correction matrix, thereby determining a corrected image, i.e., a third image. That is, the third image is an image after both brightness and color are corrected, i.e., an image with a low dynamic range. For example, the third image may be obtained by the following formula.

[0017] R'(x, y)=a0(x, y)*R(x, y)+a1(x, y)*G(x, y)+a2(x, y)*B(x, y)

[0018] G'(x, y)=b0(x, y)*R(x, y)+b1(x, y)*G(x, y)+b2(x, y)*B(x, y)

[0019] B'(x, y)=c0(x, y)*R(x, y)+c1(x, y)*G(x, y)+c2(x, y)*B(x, y)

[0020] Among them, a0, a1, a2, b0, b1, b2, c0, c1, c2 are coefficients in the color correction matrix; R(x, y), G(x, y), B(x, y) are the second images of the R channel, G channel, and B channel; R'(x, y), G'(x, y), B'(x, y) are the images of the R channel, G channel, and B channel after being processed by the color correction matrix.

[0021] The above-mentioned separate correction processing of brightness and color ensures partial decoupling of color and brightness, and the parameter sharing processing ensures the interaction between color and brightness, thereby effectively ensuring that the color of the third image finally output is accurate and the details are clear, thereby improving the user experience.

[0022] In an implementable manner of the first aspect, for the above-mentioned neural network, brightness constraints and color constraints can be used as constraints for image output to train the neural network, thereby ensuring that the image obtained through the neural network processing meets the corresponding constraints, such as the image output by the upper channel meets the brightness constraint, and the image output by the lower channel meets the color constraint. Thereby ensuring the completion of converting the high dynamic range image to the low dynamic range image.

[0023] In a second aspect, the present application provides an image processing device having the function of implementing the electronic device behavior in the method of the first aspect. The function can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions, for example, an input unit or module, a display unit or module, and a processing unit or module.

[0024] In a third aspect, an electronic device is provided, comprising: a processor; a memory; a camera module; and a computer program, wherein the computer program is stored in the memory, and when the computer program is executed by the processor, the electronic device executes the image processing method as described in the first aspect and any of its implementations.

[0025] In a fourth aspect, a computer-readable storage medium is provided, the computer-readable storage medium comprising a computer program, and when the computer program runs on an electronic device, the electronic device can execute any one of the methods described in the first aspect.

[0026] In a fifth aspect, a computer program product comprising instructions is provided, which, when executed on an electronic device, enables the electronic device to execute the image processing method in the first aspect and any implementation manner thereof.

[0027] In a sixth aspect, an embodiment of the present application provides a chip, the chip including a processor, the processor being used to call a computer program in a memory to execute an image processing method as in the first aspect and any implementation thereof.

[0028] It can be understood that the beneficial effects that can be achieved by the device described in the second aspect, the electronic device described in the third aspect, the computer-readable storage medium described in the fourth aspect, the computer program product described in the fifth aspect, and the chip described in the sixth aspect provided above can refer to the beneficial effects in the first aspect and any possible implementation method thereof, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1A A schematic diagram of an image processing method provided for related technologies;

[0030] Figure 1B A color-biased image provided for related art;

[0031] Figure 1C An image with lost texture details provided for related art;

[0032] Figure 2 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0033] Figure 3 A schematic diagram of a processing flow of an image method provided in an embodiment of the present application;

[0034] Figure 4 A flowchart of another image processing method is provided for an embodiment of the present application;

[0035] Figure 5 A schematic diagram of the structure of a chip system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0036] In the description of this application, unless otherwise specified, "and / or" in this application is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural.

[0037] In the description of this application, unless otherwise specified, "plurality" means two or more than two. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0038] In order to clearly describe the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish the same or similar items with substantially the same functions and effects. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit the difference.

[0039] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way for easy understanding.

[0040] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0041] Scenes under natural light viewed from the perspective of the human eye are often more exciting than scenes viewed through electronic devices. This is because the brightness relationship in the scene seen by the human eye is nonlinear, while the brightness relationship in the scene recorded by the electronic device is linear, so the dynamic range of the scene seen by the human eye is greater than the dynamic range that the electronic device can record. The so-called dynamic range (DR) is one of the most important parameters of the image sensor in the electronic device. It determines the intensity distribution range from the darkest part (or shadow part) to the brightest part (or highlight part) that the image sensor can receive, that is, it determines the details and levels of the image taken by the electronic device. Among them, HDR images are images with a very high dynamic range, which means that the difference between the brightest and darkest parts in the HDR image is very large. In other words, the purpose of high dynamic range imaging is to correctly represent the brightness of the real world, from direct sunlight to the darkest shadow.

[0042] However, the images currently captured by electronic devices are often affected by natural light and the electronic devices themselves, and the brightness of the resulting HDR images is very uneven, that is, the bright parts are overexposed and the dark parts are too dark. In addition, the dynamic range of most output devices such as monitors and printers is much smaller than the ordinary high dynamic range, so the output image cannot better reflect the real scene. In order to solve this problem, tone mapping technology came into being. Tone mapping is the process of converting HDR images to LDR images, so that the images presented to users can reflect the real scene seen by the human eye. In other words, tone mapping is a key step in reconstructing natural images.

[0043] For example, taking a mobile phone as an electronic device, what is normally seen by human eyes is Figure 1A Take the real scene shown in Figure 11 as an example. When a user uses the camera module in the mobile phone (such as Figure 1A When the rear camera 10) captures the picture shown in the real scene 11, the brightness range of the captured image may be large due to factors such as light. Figure 1A As shown in image 12, it can be seen that some of the images have lost details due to overexposure, that is, the final output image cannot reflect the image of the real scene. The captured image can be converted into Figure 1A The image 12 in the image is converted into an LDR image and then presented to the user, so that the output image can better reflect the real scene.

[0044] In the related art, tone mapping technology mainly realizes tone mapping through two or more neural networks connected in series. This technology generally includes a dynamic range compression (DRC) network and a local contrast enhancement (LCE) network. Among them, the DRC network is mainly responsible for processing the brightness of the image. The LCE network is mainly responsible for processing the color of the image. For example, the HDR image can be firstly improved in brightness through the DRC network, and then the color details can be adjusted through the LCE network to finally output the LDR image.

[0045] However, the shortcomings of this technology are mainly reflected in the following aspects:

[0046] On the one hand, the processing method of two or more neural networks in series may cause the error of the previous network to be transmitted to the next network, which is easy to cause error accumulation. For example, in the above technology, the output of the DRC network is the input of the LCE network. In this architecture, if the DRC network has an error during the processing, the error will be transmitted to the next LCE network, resulting in error accumulation, thereby affecting the effect of the final output LDR image, such as color deviation. Figure 1B As shown, Figure 1B A color-biased image provided for related art.

[0047] On the other hand, the LCE network mainly uses the color correction matrix (CCM) for color detail processing. However, because the LCE network is a separate network, the image needs to be negatively truncated and high-bit intercepted before the color correction matrix is ​​used for color processing on the entire image. Such operations will cause the loss of texture details in the highly saturated areas of the image, resulting in the loss of texture details in the final output LDR image. Figure 1C As shown, Figure 1C An image with lost texture details is provided for related technologies. The pixel value range of the image is generally 0-255, but the actual value of the image pixel may be outside this range, so as to ensure the wonderfulness of the image (for example, an image with a pixel value range of 0-625). Negative truncation means that the image input to the LCE network needs to remove the pixels with pixel values ​​before 0. High-bit truncation means that the image input to the LCE network needs to remove the pixels with pixel values ​​after 255.

[0048] On the other hand, the above technology uses two independent neural networks to process the color and brightness of the image separately, but in actual processing, it is impossible to completely decouple color and brightness. In other words, when processing color, the brightness will often change, causing the color of the image to deviate; when processing brightness, the color will also be affected, resulting in information loss. Therefore, it is impossible to guarantee that the image can truly restore the actual scene.

[0049] In summary, the second type of technology mentioned above will cause problems such as loss of texture details and color cast.

[0050] To solve the above problems, the embodiment of the present application provides an image processing method, which processes the image color and brightness through two independent paths, thereby realizing partial decoupling of brightness and color. In addition, through the parameter sharing mechanism, the interaction between brightness and color is realized. The problem of color deviation and loss of texture details in the process of converting HDR images to LDR images is solved, the wonderfulness of the output image is guaranteed, and the user experience is improved.

[0051] Exemplarily, the device performing image processing in the embodiments of the present application may be an electronic device, such as a mobile phone, a tablet computer, a smart watch, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) device, a virtual reality (VR) device, etc. The embodiments of the present application do not impose any special restrictions on the specific form of the electronic device.

[0052] In addition, it should be noted that the image processing method provided in this embodiment can be applied to the process of tone mapping the captured HDR image to obtain an LDR image. It can also be applied to the preprocessing of target (e.g., object) detection, the preprocessing of portrait recognition, and other scenes requiring image enhancement. The application does not limit the application scenario of the solution of this application.

[0053] For example, taking the mobile phone 200 as an example of the above electronic device, Figure 2 FIG. 2 shows a schematic diagram of the structure of the mobile phone 200. Figure 2As shown, the mobile phone 200 may include a processor 210, an external memory interface 220, an internal memory 221, a mobile communication module 230, a wireless communication module 240, a charging management module 250, a power management module 260, a battery 270, an antenna 1, an antenna 2, an audio module 280, a speaker 280A, a receiver 280B, a microphone 280C, an earphone interface 280D, a camera 290, a display screen 291, and a sensor module 292, etc. The sensor module 292 may include a distance sensor 292A and an ambient light sensor 292B, etc.

[0054] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the mobile phone 200. In other embodiments of the present application, the mobile phone 200 may include more or fewer components than those shown in the figure, or combine certain components, or separate certain components, or arrange the components differently. For example, the mobile phone 200 may also include: a subscriber identification module (SIM) card interface, etc. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0055] The processor 210 may include one or more processing units, for example, the processor 210 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0056] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0057] A memory may also be provided in the processor 210 for storing instructions and data. In some embodiments, the memory in the processor 210 is a cache memory. The memory may store instructions or data that the processor 210 has just used or circulated. If the processor 210 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 210, and thus improves the efficiency of the system. In some embodiments, the processor 210 may include one or more interfaces.

[0058] The charging management module 250 is used to receive charging input from a charger. The charger can be a wireless charger or a wired charger. While the charging management module 250 charges the battery 270 , the power management module 250 can also power the mobile phone 200 .

[0059] The power management module 260 is used to connect the battery 270, the charging management module 260 and the processor 210. The power management module 260 receives input from the battery 270 and / or the charging management module 250, and supplies power to the processor 210, the internal memory 221, the display screen 291, the camera 290, and the wireless communication module 240. The power management module 260 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 260 can also be set in the processor 210. In other embodiments, the power management module 260 and the charging management module 250 can also be set in the same device.

[0060] The wireless communication function of the mobile phone 200 can be realized through the antenna 1, the antenna 2, the mobile communication module 230, the wireless communication module 240, the modem processor and the baseband processor.

[0061] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in mobile phone 200 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0062] The mobile communication module 230 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the mobile phone 200. The mobile communication module 230 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 230 may receive electromagnetic waves from the antenna 1, and perform filtering, amplification, etc. on the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation.

[0063] The wireless communication module 240 can provide wireless communication solutions for application on the mobile phone 200, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (blue tooth, BT), global navigation satellite system (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), near field communication technology (near field communication, NFC), infrared technology (infrared, IR), etc.

[0064] In some embodiments, the antenna 1 of the mobile phone 200 is coupled to the mobile communication module 230, and the antenna 2 is coupled to the wireless communication module 240, so that the mobile phone 200 can communicate with the network and other devices through wireless communication technology.

[0065] The mobile phone 200 implements the display function through a GPU, a display screen 291, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 291 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 210 may include one or more GPUs, which execute program instructions to generate or change display information.

[0066] The display screen 291 is used to display images, videos, etc. The display screen 291 includes a display panel. For example, the display screen 291 can be a touch screen. In some embodiments of the present application, after the user opens the camera application, the display screen 291 can be used to display the preview stream collected by the camera 290.

[0067] The camera 290 is used to capture static images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the mobile phone 200 may include 1 or N cameras 290, where N is a positive integer greater than 1. In some embodiments of the present application, after the mobile phone 200 receives an operation from the user to open the camera application, the mobile phone 200 may control the camera 290 to turn on. After the camera 290 is turned on, the camera 290 can be used to collect a preview image of the current scene. After receiving an operation from the user to click to take a photo, the mobile phone 200 can use the camera 290 to capture an image. In the case where the image is an HDR image, the solution of the present application can be used for processing to obtain a corresponding LDR image. The mobile phone 200 may save the obtained LDR image into a gallery.

[0068] The internal memory 221 can be used to store computer executable program codes, which include instructions. The internal memory 221 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the mobile phone 200 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 221 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 210 executes various functions and data processing of the mobile phone 200 by running instructions stored in the internal memory 221, and / or instructions stored in a memory provided in the processor 210. The audio module 280 is used to convert digital audio information into an analog audio signal output, and is also used to convert analog audio input into a digital audio signal. The audio module 280 can also be used to encode and decode audio signals. In some embodiments, the audio module 280 may be disposed in the processor 210 , or some functional modules of the audio module 280 may be disposed in the processor 210 .

[0069] The speaker 280A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The mobile phone 200 can listen to music or listen to a hands-free call through the speaker 280A.

[0070] The receiver 280B, also called a "handset", is used to convert audio electrical signals into sound signals. When the mobile phone 200 receives a call or voice message, the voice can be received by placing the receiver 280B close to the ear.

[0071] Microphone 280C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to the microphone 280C to input the sound signal into the microphone 280C. The mobile phone 200 can be provided with at least one microphone 280C. In other embodiments, the mobile phone 200 can be provided with two microphones 280C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the mobile phone 200 can also be provided with three, four or more microphones 280C to realize the collection of sound signals, noise reduction, identification of sound sources, and realization of directional recording function, etc.

[0072] The earphone interface 280D is used to connect a wired earphone and can be a USB interface 210, or a 3.5 mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0073] The distance sensor 292A is used to measure the distance. The mobile phone 200 can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the mobile phone 200 can use the distance sensor 292A to measure the distance to achieve fast focusing.

[0074] The ambient light sensor 292B is used to sense the ambient light brightness. The mobile phone 200 can adaptively adjust the brightness of the display screen 291 according to the perceived ambient light brightness. The ambient light sensor 292B can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 292B can also cooperate with the proximity light sensor 292B to detect whether the mobile phone 200 is in a pocket to prevent accidental touches.

[0075] The following combination Figure 3 and Figure 4 This paper introduces the specific process of tone mapping in electronic devices. Figure 4 As shown, Figure 4 A flowchart of an image processing method is provided for an embodiment of the present application. The method includes S401-S406.

[0076] S401. Acquire a first image; the first image is a high dynamic range image.

[0077] The first image may refer to a high dynamic range image that needs tone mapping. In other words, the first image may refer to an image with a large difference in brightness due to a large dynamic range, or an image in which the bright part is overexposed or the dark part is too dark. That is, the content displayed by the first image is unclear.

[0078] In some examples, the difference between brightness and darkness of the first image is relatively large, which may be caused by the angle of natural light during shooting. For example, in the first image obtained by backlighting, the subject is relatively dark and the background is relatively bright, resulting in unclear details of the subject in the first image, only shadows and color deviations. In other examples, it may also be caused by the narrow dynamic range of the electronic device itself, which is what people often call the situation where electronic devices reduce image quality. For example, the sunset scenery seen by the naked eye is very shocking, but the sunset scenery recorded by the electronic device is not as shocking as that seen by the naked eye.

[0079] In some embodiments, the first image obtained above may specifically be an image captured by a camera of an electronic device. For example, after receiving an operation from a user to open a camera application, the electronic device may open a camera application. Afterwards, in response to the user's shooting operation, the electronic device may capture an image through the camera of the electronic device to obtain the first image. For another example, in a scene where portrait recognition is required, the electronic device may respond to the user's operation and use a camera to capture an image of the user to obtain the first image. In some other embodiments, the first image may also be received from other devices, downloaded from a network, or read from a memory of the electronic device.

[0080] Considering that it is impossible to completely decouple the color and brightness of an image, the processing of the two may affect each other. That is, when processing the brightness of the image, it may be possible to ensure the restoration of the brightness of the image, but it affects the color; similarly, when processing the color of the image, it may be possible to ensure the restoration of the color, but the brightness of the image is changed accordingly. Therefore, in an embodiment of the present application, two paths, such as the upper path and the lower path, are used to restore the color and brightness respectively. Before the color and brightness are restored respectively through the two paths, parameters for color and brightness restoration processing can be obtained, such as a global mapping matrix and a local mapping matrix. That is, after obtaining the first image, the global mapping relationship and local mapping relationship of the first image to the corresponding low dynamic range image in brightness and color can be obtained, that is, the global mapping matrix and the local mapping matrix of the first image are obtained for subsequent processing of color and brightness. Among them, the process of obtaining the global mapping matrix and the local mapping matrix of the first image can specifically include: S402-S406.

[0081] S402: Perform downsampling processing on the first image.

[0082] Here, downsampling means sampling a sample value sequence once every several sample values, so that the new sequence obtained is the result of downsampling the original sequence. Downsampling an image can be simply understood as reducing the image. In an embodiment of the present application, a first image can be downsampled to obtain a downsampled first image. For example, the pixel values ​​in the first image can be decimated to obtain a downsampled first image. For example, combined with Figure 3 , the first image is represented by the input image (full input), and the full input can be downsampled to obtain the downsampled first image, such as Figure 3 As shown, a low input preview image (low input_ccm preview) is used. By downsampling the image, the amount of calculation in the subsequent image processing process can be reduced, thereby improving the processing efficiency.

[0083] For example, assuming that the first image is an image of 1024 pixels (Pixel, px)*1024px, the first image obtained after downsampling processing may be an image of 512px*512px. It should be noted that the first image after downsampling processing in the above embodiment is exemplified by a size of 512px*512px, and may also be 64px*64px, which is not specifically limited.

[0084] In some embodiments, S402 is an optional step.

[0085] S403. Obtain image features of the first image, where the image features include one or more of color features, texture features, shape features, and spatial relationship features.

[0086] In some embodiments of the present application, the image features of the first image can be extracted using a processing unit in a neural network. A neural network is a nonlinear, adaptive information processing system composed of a large number of interconnected processing units. As an example, a neural network can include a feature processing unit. For example, Figure 3 After obtaining the first image (such as fullinput) and downsampling the first image low input_ccm preview, the downsampled first image can be input into the feature processing unit of the neural network so that it can complete the feature extraction of the downsampled first image and output the image feature of the first image, represented by (low level feature). Among them, the image feature of the first image is a two-dimensional data, which can be simply understood as a plane data. In the embodiment of the present application, the image feature can be understood as a feature map of the first image. Image features may include one or more of color features, texture features, shape features, and spatial relationship features.

[0087] Among them, color features are used to describe the surface properties of the scene corresponding to an image or an image region. Texture features are used to describe the surface properties of the scene corresponding to an image or an image region. Unlike color features, texture features are not based on pixel values, and they need to be statistically calculated in a region containing multiple pixel values. There are two types of representation methods for shape features, one is contour features and the other is regional features. The contour features of an image are mainly aimed at the outer boundaries of an object, while the regional features of an image are related to the entire shape region. Spatial relationship features are used to describe the spatial position or relative direction relationship between multiple targets segmented from an image. These relationships can also be divided into connection / adjacency relationships, overlap / overlapping relationships, and inclusion / inclusion relationships.

[0088] It should be noted that when the first image is processed by the feature processing unit of the neural network, feature extraction is achieved through multi-layer convolution. In this process, the obtained image features are also compressed. In other words, the image features obtained for the first image are an image feature map that is smaller than the first image (or than the first image after downsampling). Continuing with the above example, the first image is an image of 1024px*1024px size, and the first image obtained after downsampling is an image of 512px*512px size. Then the image features of the first image obtained after processing by the feature processing unit can be multiple images of 64px*64px size.

[0089] S404. Determine global features and local features of the first image based on the image features.

[0090] Specifically, after acquiring the image features of the first image, the electronic device may determine the global features and local features corresponding to the first image based on the image features.

[0091] Among them, the global feature is used to characterize the global mapping relationship of the high dynamic range image to the low dynamic range image in terms of color and brightness. The so-called global mapping relationship means that the brightness and color values ​​in the global area of ​​the high dynamic range image and the global area of ​​the low dynamic range image are in a one-to-one correspondence. That is to say, the number of pixel values ​​in the global area of ​​the high dynamic range image and the number of pixel values ​​in the global area of ​​the low dynamic range image are the same. For example, the general value range of the pixel value of the high dynamic range image is [0-255], and the general value range of the pixel value of the low dynamic range image is also [0-255]. The local feature is used to characterize the local mapping relationship of the high dynamic range image to the low dynamic range image in terms of color and brightness. The so-called local mapping relationship means that the brightness and color pixel values ​​in the partial area of ​​the high dynamic range image and the local area corresponding to the low dynamic range image are in a one-to-one correspondence. That is to say, in the local area, the number of pixel values ​​of the high dynamic range image and the number of pixel values ​​of the low dynamic range image are the same. For example, if the high dynamic range image is [0-10], the low dynamic range image is also [0-10], or if the high dynamic range image is [0-10], the low dynamic range image is also [5-15]. In simple terms, the local features can reflect the features of the local area of ​​the first image, such as the features of the particularly bright or dark areas in the first image, so as to facilitate subsequent targeted processing, such as adaptively brightening the dark areas in the image and adaptively dimming the bright areas in the image. In this embodiment, the local features of the first image may include multiple ones. For example, it can be simply understood that the image features of the first image are divided into multiple small areas, such as 4 3*3 area maps.

[0092] In some embodiments of the present application, the processing unit in the neural network may be used to extract the global features and local features of the first image. As an example, the neural network may also include a global feature processing unit and a local feature processing unit. Figure 3 After the image features of the first image are acquired, or after the feature processing unit outputs the low-level feature, the low-level feature can be input into the global feature processing unit and the local feature processing unit. Afterwards, the global feature processing unit can extract the global features of the first image based on the image features of the first image to obtain the global features of the first image, such as Figure 3The local feature processing unit can extract the local features of the first image based on the image features of the first image to obtain the local features of the first image, such as Figure 3 It is represented by local features (Local_Feature).

[0093] As an example, the extraction of the global features of the first image may be implemented through a fully connected layer, and the extraction of the local features of the first image may be implemented through convolution and downsampling.

[0094] It can be understood that both the global feature processing unit and the local feature processing unit are pre-trained, and the core of the processing unit is fixed. For example, the output obtained by inputting the image feature of the first image into the global feature processing unit can be a global feature of 64px*64px size. This is because the core of the global feature processing unit is 64px*64px, so the global feature finally output is an image of 64px*64px size. For local features, the output obtained by inputting the image feature of the first image into the local feature processing unit is a local feature of multiple local area sizes. Among them, multiple local area sizes refer to splitting the image feature of the first image into N*M local areas, such as, N and M are both set to 3, then the local area size is 3px*3px. Assuming that the convolution kernel of the local feature processing unit is 3px*3px, then after processing by the local feature processing unit, the image feature of the first image is divided into multiple images of 3px*3px size, such as the local features finally output are four 3px*3px size [0-10] images.

[0095] It should be noted that, whether it is a global feature processing unit or a local feature processing unit, the number of cores is set according to actual needs for training and is not limited to the size of the core in the above example. It can also be a core of other sizes, such as the core of the global feature processing unit is 512px*512px, and the core of the local feature processing unit is 24px*24px.

[0096] Taking into account the different manifestations of brightness and color in image features, brightness focuses on global features, and color focuses on local features. After obtaining the global features and local features of the first image, in some embodiments, the brightness of the first image can be restored based on the global features, and the color can be restored based on the local features. In some other embodiments, considering that brightness and color cannot be completely decoupled, or that the two influence each other, therefore, when restoring the brightness, not only the global features of the image can be considered, but also the local features of the image can be considered. Similarly, when restoring the color, not only the local features of the image can be considered, but also the local features can be taken into account. Specifically, the global mapping matrix corresponding to the global features and the local mapping matrix corresponding to the local features can be obtained, which are used for the subsequent restoration of brightness and color, respectively. That is, the following S405 and S406 are executed.

[0097] S405. Determine a global mapping matrix based on the global features, the local features and the first weight coefficient.

[0098] Among them, the global features and local features can be multiplied by different weight coefficients to obtain the global mapping matrix.

[0099] In some embodiments of the present application, the global mapping matrix can be obtained by a gate fire mechanism (GFM). Figure 4 After obtaining the global feature (Global_Feature) and the local feature (Local_Feature) of the first image, the global feature (Global_Feature) and the local feature (Local_Feature) can be input into the GFM. Then, the GFM can output the global mapping matrix corresponding to the global feature based on the input global feature, the local feature and the first weight coefficient, such as Figure 4 In other words, the global mapping matrix can be determined based on the global features, local features and the first weight coefficient in the GFM. For example, the global mapping matrix is ​​determined by multiplying the global features and local features by different weight values. The global mapping matrix is ​​mainly used for subsequent brightness restoration processing, that is, more emphasis is placed on global feature processing, so the first weight coefficient should focus on global features.

[0100] As an example, the first weight coefficient may include multiple weight values ​​for global features and local features. In addition, because there may be multiple local features, there are also multiple weight values ​​for local features, and they correspond one to one with multiple local features. Among the multiple weight values, the weight value for global features is different from the weight value for local features. For example, the weight value for global features is greater than the weight value for local features. The weight values ​​for different local features may be the same or different.

[0101] For example, taking the global feature represented by G and two local features represented by L1 and L2, the first weight coefficient may include three weight values, namely, the weight value for the global feature G, such as A, and the weight values ​​for the two local features L1 and L2, such as B and C. After the global feature G and the two local features L1 and L2 are input into GFM, the output global mapping matrix coeff1 can be simply expressed as: coeff1 = G*A+L1*B+L2*C. Among them, A>B, A>C.

[0102] S406. Determine a local mapping matrix based on the global features, the local features and the second weight coefficient.

[0103] Similar to S405 , the global features and the local features may be multiplied by different weight coefficients to obtain a local mapping matrix.

[0104] In some embodiments, the local mapping matrix can be obtained by GFM. For example, combining Figure 3 , the global feature (Global_Feature) and the local feature (Local_Feature) can be input into GFM. Then, GFM can output the local mapping matrix corresponding to the local feature based on the input global feature, local feature and the second weight coefficient, such as Figure 3 In other words, the local mapping matrix can be determined based on the global features, local features and the second weight coefficient in the GFM. The local mapping matrix is ​​mainly used for subsequent color restoration processing, that is, it focuses more on local feature processing, so the second weight coefficient should focus on local features.

[0105] Similarly, the second weight coefficient may include multiple weight values ​​for global features and local features. Among the multiple weight values, the weight value for global features is different from the weight value for local features. For example, the weight value for local features is greater than the weight value for global features.

[0106] For example, continuing to use the example that the global feature is represented by G, and there are two local features, represented by L1 and L2 respectively, the second weight coefficient can include three weight values, namely, the weight value for the global feature G, such as X, and the weight values ​​for the two local features L1 and L2, such as Y and Z respectively. After the global feature G and the two local features L1 and L2 are input into GFM, it can also output the local mapping matrix coeff2. The local mapping matrix coeff2 can be simply expressed as: coeff2 = G*X+L1*Y and G*X+L2*Z. Among them, Y>X, Z>X.

[0107] It should be noted that this embodiment does not limit the execution order of the above S405 and S406. For example, S405 can be executed first and then S406, or S406 can be executed first and then S405, or S405 and S406 can be executed at the same time.

[0108] Based on the above, it can be understood that in this embodiment, the global mapping matrix and the global features are both used to indicate the global mapping relationship of the first image to the corresponding low dynamic range image in terms of brightness and color, and the difference between the two is that the global mapping matrix not only considers the global features, but also adds local features. The local mapping matrix and the local features are both used to indicate the local mapping relationship of the first image to the corresponding low dynamic range image in terms of brightness and color, and the difference between the two is that the local mapping matrix not only considers the local features, but also adds the global features.

[0109] After obtaining the global mapping matrix and the local mapping matrix, the color and brightness restoration processing can be performed respectively through different channels (such as upper channel and lower channel) based on the global mapping matrix and the local mapping matrix. For example, the brightness of the first image can be corrected based on the global mapping matrix to obtain the second image. Afterwards, the color of the first image can be corrected based on the local mapping matrix and the second image to obtain the third image. Finally, the third image can be output, and the third image is a low dynamic range image corresponding to the first image. Specifically, it can include the following: S407-S411.

[0110] S407. Obtain a texture image of the first image.

[0111] In some embodiments of the present application, a processing unit in a neural network may be used to extract a texture image of the first image. As an example, the neural network may also include a guiding processing unit. For example, continue to combine Figure 3 , the first image can be input to the guidance processing unit of the neural network so that it can complete the acquisition of the texture image of the first image, that is, so that it can output the texture image of the first image (also called Guidance_map). Specifically, the acquisition of the texture image of the first image avoids the loss of details of the first image to a certain extent.

[0112] S408. Determine a mapping relationship graph of the first image based on the texture image and the global mapping matrix.

[0113] Specifically, after acquiring the texture image of the first image, the electronic device may determine the mapping relationship diagram of the first image through the upper channel based on the texture image and the global mapping matrix.

[0114] In some embodiments of the present application, a processing unit in a neural network may be used to determine a mapping relationship diagram of the first image according to the texture image and the global mapping matrix. As an example, the neural network may also include a decoding processing unit 1. For example, continue to combine Figure 3 After the texture image of the first image is obtained, or after the guidance processing unit outputs Guidance_map, Guidance_map and coeff1 can be input into the decoding processing unit 1 (also called Decoder_DRC). Afterwards, the decoding processing unit 1 performs decoding processing based on Guidance_map and coeff1 to obtain a mapping relationship diagram of the first image (such as Gain_map).

[0115] Specifically, the decoding processing unit 1 performs guided interpolation processing on coeff1 through Guidance_map to obtain a mapping relationship map of the first image. For example, the texture image and the global mapping matrix are input to Decoder_DRC to obtain an output Gain_map.

[0116] So-called interpolation, sometimes called "resampling," is a method of increasing the size of an image's pixels without generating them. The color of the missing pixels is calculated mathematically based on the colors of the surrounding pixels, artificially increasing the image's resolution.

[0117] It should be understood that the mapping relationship diagram obtained in step S408 is mainly determined by interpolating the global mapping matrix, and interpolation will increase the size of the image. In an embodiment of the present application, the size of the specific interpolation can be determined based on the size of the first image. That is, this step can be understood as restoring the size of the first image to obtain the mapping relationship diagram of the first image. In the above embodiment, the image size is first compressed (such as performing downsampling, feature extraction, etc.) and then the image is restored, which can reduce the amount of calculation in the neural network processing and reduce the probability of errors. For example, the decoding processing unit 1 outputs a global full-resolution mapping image of 1024px*1024px, that is, the above-mentioned mapping relationship diagram.

[0118] Based on the above, it can be understood that, in this embodiment, the mapping relationship diagram and the global mapping matrix are both used to indicate the global mapping relationship of the first image to the corresponding low dynamic range image in terms of brightness and color. The difference between the two is that the mapping relationship diagram is the image after upsampling by the global mapping matrix, that is, the image after the image size is restored.

[0119] In an example, in the above steps S401-S407, the size of the first image is 1024px*1024px, and the first image is downsampled to obtain a first image of size 512px*512px, and then the image features of the first image are obtained to obtain multiple 64px*64px feature maps. The size of the global mapping matrix determined by the global features is 64px*64px, and based on the texture image and the global mapping matrix, the size of the mapping relationship map of the first image is determined to be 1024px*1024px.

[0120] S409. Based on the mapping relationship diagram, correct the brightness of the first image to obtain a second image.

[0121] Specifically, after determining the mapping relationship diagram, the electronic device may perform brightness restoration processing on the first image according to the mapping relationship diagram to obtain a second image after the brightness restoration processing.

[0122] In some embodiments of the present application, the three channels (R channel, G channel, B channel) of the first image can be subjected to brightness restoration processing based on the mapping relationship diagram obtained in the upper channel to obtain an intermediate image. Figure 3 , the intermediate image can be determined according to the mapping relationship diagram and the first image, such as calculating the product of the mapping relationship diagram and the first image to obtain the intermediate image, such as Figure 3 The intermediate image may be the second image in the present application.

[0123] The second image can be determined by the following formula:

[0124] Contrast(R,G,B)(x,y)=input(R,G,B)(x,y)*Gain_map(x,y)

[0125] Among them, Contrast(R,G,B) is the second image, i.e., img1; input(R,G,B)(x,y) is the first image, i.e., fullinput; Gain_map(x,y) is the mapping relationship diagram.

[0126] The above-mentioned S407-S409 are the restoration processing of brightness in the upper channel. The following describes the process of restoring color in the lower channel, such as including S410-S411.

[0127] S410. Determine a color correction matrix for the second image based on the second image and the local mapping matrix.

[0128] Specifically, after determining the second image, the electronic device may determine the color correction matrix of the second image through the lower channel based on the second image and the local mapping matrix. The color correction matrix may be a 3*3 matrix, that is, the color correction matrix may include 9 coefficients. The 9 coefficients of the color correction matrix may be used to perform color correction on the second image.

[0129] In some embodiments of the present application, a processing unit in a neural network may be used to determine a color correction matrix of the second image. As an example, the neural network may also include a decoding processing unit 2. For example, continue to combine Figure 3 After obtaining img1, img1 and coeff2 may be decoded to obtain a color correction matrix of the second image, such as CC(x,y).

[0130] Similarly, the decoding processing unit 2 interpolates coeff2 through img1 to obtain the color correction matrix of the first image. For example, the second image and the local mapping matrix are input to Decoder_CC to obtain the output CC(x,y). That is, the decoding neural network processing unit 2 outputs a 1024px*1024px global full-resolution mapping image.

[0131] In an example, in the above steps S401-S407, the first image is 1024px*1024px, and the first image is downsampled to obtain a first image of size 512px*512px. Then, the image features of the first image are obtained to obtain multiple feature maps of size 64px*64px. The size of the local mapping matrix determined by the local features is 64px*64px. Based on the second image and the local mapping matrix, the size of the color correction matrix of the first image is determined to be 1024px*1024px.

[0132] S411. Based on the color correction matrix, correct the color of the second image to obtain a third image.

[0133] Specifically, after determining the color correction matrix, the electronic device may perform color restoration processing on the second image using the color correction matrix to obtain a third image after the color restoration processing.

[0134] In some embodiments of the present application, color correction can be performed on the RGB channels of the second image based on the color correction matrix obtained in the lower channel to obtain a corrected third image, such as Figure 3 For example, color correction is performed based on the product of CC(x, y) and the RGB channels of img2, and a third image is output.

[0135] The third image can be determined by the following formula:

[0136] R'(x, y)=a0(x, y)*R(x, y)+a1(x, y)*G(x, y)+a2(x, y)*B(x, y)

[0137] G'(x, y)=b0(x, y)*R(x, y)+b1(x, y)*G(x, y)+b2(x, y)*B(x, y)

[0138] B'(x, y)=c0(x, y)*R(x, y)+c1(x, y)*G(x, y)+c2(x, y)*B(x, y)

[0139] Among them, a0, a1, a2, b0, b1, b2, c0, c1, c2 are coefficients in the color correction matrix; R(x, y), G(x, y), B(x, y) are the second images of the R channel, G channel, and B channel; R'(x, y), G'(x, y), B'(x, y) are the images of the R channel, G channel, and B channel after being processed by the color correction matrix.

[0140] It can be understood from the above that, in the embodiment of the present application, a neural network can be used to achieve tone mapping, that is, to convert a high dynamic range image into a low dynamic range image, avoid error accumulation, and ensure the accuracy of the color of the low dynamic range image finally output. In addition, the neural network includes two independent channels (such as an upper channel and a lower channel) to respectively realize the restoration processing of brightness and color. It can be seen that partial decoupling of brightness and color is achieved through these two independent channels. In addition, when performing brightness and color restoration, not only the global features of the image are considered, but also the local features of the image are considered, that is, the interaction between brightness and color is achieved through a parameter sharing mechanism. The loss of texture details of the restored low dynamic range image is avoided. In addition, by performing color correction processing inside the neural network, negative truncation and high-bit interception of the image are avoided, further ensuring the accurate restoration of texture details.

[0141] It should be noted that before applying the neural network provided in this application for tone mapping, the neural network needs to be trained. In the process of training the neural network, after the image output by the upper channel, such as img1, and the image output by the lower channel, such as img2, both meet the constraints of the neural network, the iterative training of the neural network is stopped, and the neural network that can be finally used for tone application is obtained.

[0142] For example, img1 output from the upper channel needs to comply with the brightness constraints in the neural network; img2 output from the lower channel needs to comply with the color constraints in the neural network.

[0143] As an example, the brightness constraint function can be determined based on the average value of the difference between the brightness value of img1 and the brightness value of the full input. For example, the brightness constraint function can be the following formula:

[0144] Light_LOSS=Avg(F(Img1)-F(Label)).

[0145] Among them, Light_LOSS is used to represent the constraint function of brightness, Avg is the mean, Label is the first image, and F is the formula for calculating brightness.

[0146] As another example, the color constraint function is determined according to the difference between the RGB image and the LAB image. For example, the color constraint function may be the following formula:

[0147] O_L, O_A, O_B=rgb2lab(Img2)

[0148] L_L, L_A, L_B=rgb2lab(Label)

[0149]

[0150]

[0151] SC=1+0.0225*(C1+C2)

[0152] SH=1+0.0075*(C1+C2)

[0153] delta_a=O_A-L_A

[0154] delta_b=O_B-L_B

[0155] delta_C=C1-C2

[0156] delta_L=O_L-L_L

[0157] delta_H=delta_a**2+delta_b**2-delta_C**2

[0158]

[0159]

[0160] Among them, Color_LOSS is used to represent the color constraint function, Label image is the standard image, O_L is the brightness of img2 image, O_A is the chroma of img2 image, O_B is the hue of img2 image, L_L is the brightness of Label image, O_A, is the chroma of Label image, O_B, is the hue of Label image, C1 is the total value of hue and chroma of img2 image, C2 is the total value of hue and chroma of Label image, SC is the total value of img2 image and lab under the coefficient of 0.0225 The weight coefficient of the el image, SH is the weight coefficient of the img2 image and the label image under the coefficient of 0.0075; delta_A is the color difference between the red / green color of the img2 image and the Label image, delta_B is the color difference between the yellow / blue color of the img2 image and the Label image, delta_C is the color difference between the img2 image and the Label image, delta_L is the brightness difference between the img2 image and the Label image, and delta_H is the color loss value of the img2 image and the Label image. It should be noted that the coefficients of 0.0225 and 0.0075 are the preferred values ​​under the current constraints, and can also be 0.045 and 0.015, or other values, which are not limited here.

[0161] Some other embodiments of the present application provide an electronic device, which may include: the above-mentioned display screen, camera, memory and one or more processors. The display screen, camera, memory and processor are coupled. The memory is used to store computer program code, and the computer program code includes computer instructions. When the processor executes the computer instructions, the electronic device can execute the various functions or steps executed by the mobile phone in the above-mentioned method embodiment. The structure of the electronic device can refer to Figure 2 The structure of the mobile phone shown.

[0162] The present application also provides a chip system, such as Figure 5As shown, the chip system 500 includes at least one processor 501 and at least one interface circuit 502. The processor 501 and the interface circuit 502 can be interconnected through lines. For example, the interface circuit 502 can be used to receive signals from other devices (such as a memory of an electronic device). For another example, the interface circuit 502 can be used to send signals to other devices (such as processor 501). Exemplarily, the interface circuit 502 can read instructions stored in the memory and send the instructions to the processor 501. When the instructions are executed by the processor 501, the electronic device can execute the various steps in the above embodiments. Of course, the chip system can also include other discrete devices, which are not specifically limited in the embodiments of the present application.

[0163] An embodiment of the present application also provides a computer storage medium, which includes computer instructions. When the computer instructions are executed on the above-mentioned electronic device, the electronic device executes each function or step executed by the mobile phone in the above-mentioned method embodiment.

[0164] The embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute each function or step executed by the mobile phone in the above method embodiment.

[0165] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0166] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0167] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0168] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0169] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0170] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application shall be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An image processing method, characterized in that: include: Acquire image features of a first image; the first image is a high dynamic range image, and the image features include one or more of color features, texture features, shape features, and spatial relationship features; Determining global features and local features of the first image based on the image features; Determine a global mapping matrix based on the global feature, the local feature and the first weight coefficient; the global mapping matrix is ​​used to indicate a global mapping relationship between the first image and the corresponding low dynamic range image in terms of brightness and color; Determine a local mapping matrix based on the global feature, the local feature and the second weight coefficient; the local mapping matrix is ​​used to indicate a local mapping relationship between the first image and the corresponding low dynamic range image in terms of brightness and color; Based on the global mapping matrix, correcting the brightness of the first image to obtain a second image; Based on the local mapping matrix and the second image, the color of the first image is corrected to obtain a third image, and the third image is output, where the third image is a low dynamic range image corresponding to the first image.

2. The method according to claim 1, characterized in that: Before acquiring the image features of the first image, the method further includes: Down-sampling is performed on the first image.

3. The method according to claim 1 or 2, characterized in that: The step of correcting the brightness of the first image based on the global mapping matrix to obtain a second image includes: Acquire a texture image of the first image; Determining a mapping relationship graph of the first image according to the texture image and the global mapping matrix; According to the mapping relationship diagram, the brightness of the first image is corrected to obtain the second image.

4. The method according to claim 3, characterized in that The step of determining a mapping relationship diagram of the first image according to the texture image and the global mapping matrix comprises: The global mapping matrix is ​​interpolated based on the texture image to obtain the mapping relationship graph.

5. The method according to any one of claims 1, 2 or 4, characterized in that: The method of correcting the color of the first image based on the local mapping matrix and the second image to obtain a third image includes: determining a color correction matrix for the first image based on the second image and the local mapping matrix; Based on the color correction matrix, the color of the first image is corrected to obtain the third image.

6. The method according to claim 5, characterized in that The step of determining the color correction matrix of the first image based on the second image and the local mapping matrix comprises: The local mapping matrix is ​​interpolated based on the second image to obtain the color correction matrix.

7. The method according to claim 6, characterized in that The step of correcting the color of the first image based on the color correction matrix to obtain the third image includes: Based on the color correction matrix, color correction is performed on the RGB channels of the second image to obtain the third image.

8. An electronic device, characterized in that: The electronic device comprises: a memory and one or more processors; the memory is coupled to the processor; The memory is used to store computer program codes, and the computer program codes include computer instructions; when the computer instructions are executed by the processor, the electronic device executes the method as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that: including computer instructions; When the computer instructions are executed on an electronic device, the electronic device is caused to execute the method as claimed in any one of claims 1 to 7.

Images