Image processing method and electronic equipment

Abstract

The embodiment of the invention relates to the technical field of image processing, in particular to an image processing method and electronic equipment, which can avoid supersaturation, detail loss and display blurring of a high-saturation shooting object in an image after color correction. The method comprises the following steps: acquiring a first image acquired by a camera; and under the condition that color overflow exists in a second image obtained by performing color correction on the first image by adopting the first color correction matrix, performing color correction on the first image by adopting a second color correction matrix. Wherein the second color correction matrix is determined according to the first color card and the second color card. The saturation of the color block corresponding to the target color block in the first color card is lower than that of the target color block in the standard color card. The target color block is a color block, of which the similarity with the color overflowing from the second image meets the condition, in the standard color card. The second color card is used for representing the RAW map acquired by the camera aiming at the standard color card. The first color correction matrix is determined according to a standard color card.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and more particularly to an image processing method and an electronic device. Background Technology

[0002] Modern electronic devices generally have photo and video recording functions, allowing users to capture images or videos and record their lives anytime, anywhere.

[0003] A Color Correction Matrix (CCM) is typically used to correct the colors of the original image captured by the image sensor, making the corrected image closer to natural colors or human perception. However, when images include highly saturated subjects such as sunsets, evening glows, red flowers, green leaves, or pure-color light sources like neon lights at night, color correction may result in oversaturation, loss of detail, and blurring of the subject, leading to display defects. Summary of the Invention

[0004] This application provides an image processing method and an electronic device that can avoid oversaturation, loss of detail, and blurring of highly saturated subjects in images after color correction.

[0005] To achieve the above objectives, the embodiments of this application adopt the following technical solutions:

[0006] Firstly, an image processing method is provided, applied to an electronic device, including a camera. The method includes: acquiring a first image captured by the camera; and, if a second image obtained by color correction of the first image using a first color correction matrix exhibits color overflow, performing color correction on the first image using a second color correction matrix. The second color correction matrix is ​​determined based on a first color chart and a second color chart. The saturation of the color patch corresponding to the target color patch in the first color chart is lower than the saturation of the target color patch in the standard color chart. The target color patch is a color patch in the standard color chart whose similarity to the overflowing color in the second image meets a certain condition. The second color chart is used to represent the RAW image captured by the camera against the standard color chart. The first color correction matrix is ​​determined based on the standard color chart. The first color chart is obtained by reducing the saturation of the target color patch in the standard color chart, and the color patch corresponding to the target color patch in the first color chart is a color patch with reduced saturation of the target color patch.

[0007] The second color chart serves as a RAW image captured by an electronic device against a standard color chart, reflecting the camera's response to the red, green, and blue spectra. Since the second color correction matrix can correct the second color chart to match the first color chart, and compared to the standard color chart, the color patches on the first color chart that are close to the overflowing colors in the second image have lower saturation, using the second color correction matrix to correct the first image can reduce the saturation of colors that would otherwise overflow in the first image (i.e., overflowing colors in the second image). Due to the lower saturation, the first image corrected by the second color matrix has less or no data overflow, thus avoiding oversaturation, loss of detail, and blurring caused by data overflow.

[0008] In one possible implementation of the first aspect, the electronic device can first perform color correction on the first image using a first color correction matrix to obtain a second image. Then, it identifies whether color overflow exists in the second image based on the number of target pixels in the second image. Specifically, the electronic device counts the target pixels in the second image. Target pixels are pixels where the value of a color channel in each color channel is greater than a first value; and / or, the value of a color channel in each color channel is less than a second value. If the number of target pixels meets a first preset condition, it is determined that color overflow exists in the second image. The first preset condition can be, for example, that the number of target pixels is greater than a first preset value. The first value can be a maximum value that the electronic device can represent, such as 255, and the second value can be a minimum value that the electronic device can represent, such as 0. Since the color channel values ​​of the target pixels have data overflow, the electronic device determines whether the second image has color overflow based on the number of target pixels with data overflow. This implementation provides a convenient way to determine whether color overflow exists in a second image.

[0009] In one possible implementation of the first aspect, before performing color correction on the first image using the second color correction matrix, the electronic device can acquire the second color correction matrix, which is determined by the first color chart and the second color chart. The electronic device can determine the target color patch in the standard color chart based on the target pixel in the second image; reduce the saturation of the target color patch in the standard color chart to obtain the first color chart. The second color correction matrix is ​​obtained based on the first and second color charts. Due to data overflow in the target pixel, the saturation of the target color patch corresponding to the target pixel in the standard color chart is reduced to obtain the first color chart. The second color correction matrix obtained based on the first and second color charts can reduce the saturation of the color (the color corresponding to the target pixel) that would originally overflow in the first image. The electronic device can multiply the standard color chart by the inverse of the first color correction matrix to obtain the second color chart. Optionally, the second color chart can be the original image captured by the electronic device against the standard color chart, which can be pre-saved in the mobile phone.

[0010] In one possible implementation of the first aspect, the electronic device determines the overflow color in the second image based on the target pixel. The electronic device then determines the target color patch in the standard color chart based on the similarity between the overflow color in the second image and a color patch in the standard color chart. This implementation provides a feasible way to determine a target color patch from a standard color chart.

[0011] In one possible implementation of the first aspect, the color channels of the target pixel include an R channel, a G channel, and a B channel. The first image includes at least one target object, and the saturation of the target object satisfies a second preset condition. The second preset condition may be that the saturation of the target object is greater than a second preset value.

[0012] When the number of target objects is equal to 1, the R, G, and B channel values ​​of the target pixels are calculated separately to obtain the R, G, and B channel values ​​of the overflow color corresponding to the target object. This calculation can be a mean, mode, or average, etc.

[0013] When the number of target objects is greater than one, the target pixels are classified to obtain target pixels that correspond one-to-one with multiple target objects. For each target object, the R, G, and B channels of the corresponding target pixel are calculated separately to obtain the R, G, and B channel values ​​of the overflow color for each target object. This calculation can be the mean, mode, or average. Distinguishing between these values ​​makes the calculated overflow color closer to the actual overflow color in the second image. For example, if the overflow colors in the second image are red and blue, without this distinction, the calculated overflow color in the second image might be purple, which does not match the actual overflow situation.

[0014] In one possible implementation of the first aspect, the electronic device adjusts the saturation of the target color patch in the standard color chart according to a coefficient less than 1, the coefficient being proportional to the similarity between the overflow color in the second image and the target color patch. This implementation provides a feasible way to adjust the saturation of the target color patch.

[0015] In one possible implementation of the first aspect, the electronic device pre-stores multiple coefficients and selects a coefficient for the target color patch based on the similarity between the overflowing color in the second image and the target color patch. Specifically, the higher the similarity between the target color patch and the overflowing color in the second image, the smaller the coefficient selected by the electronic device.

[0016] In one possible implementation of the first aspect, the electronic device can calculate a coefficient for the target color patch. The coefficient corresponding to the target color patch is the proportion of the similarity between the target color patch and the overflowing color in the second image to the sum of the similarities between multiple target color patches and the overflowing colors in the second image.

[0017] In one possible implementation of the first aspect, the electronic device determines the distance between the overflowing color in the second image and a color patch in a standard color chart. The distance is used to indicate the similarity between the overflowing color and the color patch, and the distance is inversely proportional to the similarity. The electronic device selects the color patches corresponding to the first few preset distances in ascending order from the smallest distance as target color patches.

[0018] In a second aspect, an electronic device is provided, comprising: a memory, a camera, and one or more processors; the camera, the memory, and the processors are coupled; wherein the memory is used to store computer program code, the computer program code including computer instructions; when the computer instructions are executed by the processor, the electronic device performs the method as described in any of the first aspects.

[0019] Thirdly, a chip system is provided that can be applied to an electronic device including memory. The chip system includes one or more interface circuits and one or more processors. The interface circuits and processors are interconnected via lines. The interface circuits are used to receive signals from the aforementioned memory and send the signals to the processors, the signals including computer instructions stored in the memory. When the processor executes the computer instructions, the electronic device performs a method as described in the first aspect and any of its possible design embodiments.

[0020] Fourthly, a computer-readable storage medium is provided, including computer instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any of the first aspects.

[0021] Fifthly, a computer program product comprising a computer program / instructions that, when executed by a processor, implement the steps of any of the methods in the first aspect.

[0022] In understanding, the beneficial effects that can be achieved by the electronic device of any possible design of the second aspect, the chip system of the third aspect, the computer-readable storage medium of the fourth aspect, and the computer program product of the fifth aspect can be referred to as the beneficial effects of the first aspect and any possible design, which will not be repeated here. Attached Figure Description

[0023] Figure 1 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application;

[0024] Figure 2A schematic diagram of the software architecture of an electronic device provided in an embodiment of this application;

[0025] Figure 3 A schematic flowchart of an image processing method provided in an embodiment of this application;

[0026] Figure 4 A schematic diagram of a second image provided for an embodiment of this application;

[0027] Figure 5 A schematic diagram of a standard color chart provided for an embodiment of this application;

[0028] Figure 6 A schematic diagram of a second and third image provided for an embodiment of this application;

[0029] Figure 7 A flowchart illustrating another image processing method provided in an embodiment of this application;

[0030] Figure 8 This is a schematic diagram illustrating an image format conversion method provided in an embodiment of this application. Detailed Implementation

[0031] Modern electronic devices generally have photo and video recording functions, allowing users to capture images or videos and record their lives anytime, anywhere.

[0032] Electronic devices, such as mobile phones, may include at least one camera. Taking image capture as an example, in response to a user tapping the shutter button on the camera app's preview screen, the camera can capture a raw image (RAW image). Then, the image signal processor (ISP) in the phone can process the RAW image to obtain a processed image. The phone can save this processed image for the user to view.

[0033] A camera may include components such as a lens, holder, image sensor, voice coil motor (VCM), and flexible printed circuit (FPC). An image sensor, also known as a photosensitive device, is a device that converts optical images into electrical signals. For example, the surface of an image sensor contains hundreds of thousands to millions of photodiodes. When these photodiodes are illuminated, they generate light signals. The image sensor can then convert these light signals into electrical signals and output a RAW image.

[0034] An Image Processor (ISP) is a processor that processes image signals. An ISP can process RAW images to obtain images in YUV format (Luma, Chrominance, Chroma) or Red, Green, Blue, and RGB formats. The ISP can then transmit the processed image to a central processing unit (CPU). The CPU can store the image and / or control the display screen to show it.

[0035] The image processing described above includes, but is not limited to, black level correction (BLC), lens shading correction (LSC), RAW noise reduction, automatic white balance (AWB), dynamic range control (DRC), color correction, gamma correction, and RGB2YUV (RGB to YUV conversion). Color correction, in particular, is used to correct the colors of the RAW images acquired by the image sensor, making the corrected image closer to natural colors or human visual perception.

[0036] Color correction matrix (CCM) is typically used to correct the colors of the raw images captured by the image sensor. A CCM can be a 3x3 matrix. For example, a CCM could be... The image to be corrected may include, for example, m pixels, such as pixels [R1, G1, B1], pixels [R2, G2, B2], ..., pixels [R... m G m B m The pixel value of each pixel in the image to be corrected is multiplied by the CCM to obtain the corrected pixel value. The pixels with the corrected pixel values ​​make up the corrected image.

[0037] Taking pixel values ​​[R1, G1, B1] as an example, Where [R`1,G`1,B`1] are the corrected pixel values ​​corresponding to the pixel values ​​[R1,G1,B1]. Specifically, R`1 = R1*M11 + G1*M21 + B1*M31, G`1 = R1*M12 + G1*M22 + B1*M32, and B`1 = R1*M13 + G1*M23 + B1*M33.

[0038] A color correction matrix is ​​typically obtained by comparing the RAW image of a standard color chart captured by an electronic device with the actual colors of the standard color chart. For example, the electronic device can photograph each color patch in the standard color chart to obtain a RAW image of the standard color chart. The product of this RAW image and the color correction matrix can be the actual color of the standard color chart, and the mobile phone can then derive the color correction matrix based on this RAW image and the actual colors of the standard color chart. Here, a color patch is a color in the color chart. Optionally, the RAW image of the standard color chart is an image that has undergone DRC processing.

[0039] When an image includes a highly saturated subject, after color correction, the highly saturated subject in the image may exhibit display defects such as oversaturation, loss of detail, and blurring. Saturation indicates the vividness of a color; higher saturation indicates a more vivid color. A highly saturated subject can be, for example, a subject whose average saturation exceeds a preset value. A highly saturated subject can also be a subject where the number of pixels with saturation exceeding a preset value exceeds a preset number. Furthermore, since the saturation of a pure color is higher than that of a mixed color (such as a pure color after adding black, white, or gray), a highly saturated subject in the image can be a pure color object. A pure color object can be, for example, a subject with colors such as red, orange, yellow, green, blue, or purple. In some embodiments, a highly saturated subject can be, for example, a sunset, evening glow, red flowers, green leaves, or a pure light source such as neon lights at night.

[0040] The reason for the aforementioned display defect is that electronic devices typically use one byte to store and transmit the pixel values ​​of each pixel in an RGB format image, such as the values ​​of the color channels (R, G, and B channels). One byte consists of 8 binary bits and can represent a number between 0 and 255. Because some of the R, G, and B channel values ​​of the pixels corresponding to a highly saturated subject are close to 255, and some are close to 0, after multiplying the pixel by the color correction matrix, the value of at least one of the R, G, or B channels will overflow. Overflow means that the channel value is greater than the maximum value that one byte can represent, such as 255, or less than the minimum value that one byte can represent, such as 0; that is, the channel value exceeds the data range that the electronic device can represent, i.e., [0, 255]. Taking a highly saturated subject like a red flower as an example, the R channel values ​​of multiple pixels corresponding to the red flower are between 200 and 255, the G channel values ​​are between 0 and 50, and the B channel values ​​are between 0 and 50. After multiplying with the color correction matrix, some or all of the R channel values ​​of multiple pixels are greater than 255, while some or all of the G or B channel values ​​are less than 0. Since this exceeds the data range that the electronic device can represent ([0, 255]), the device will truncate the overflow values. Truncation means setting values ​​greater than 255 to 255 and values ​​less than 0 to 0. After truncation, corresponding image details will be lost, resulting in image blurring. Furthermore, because the channel values ​​are truncated to 255 or 0, oversaturation of highly saturated subjects can also occur.

[0041] Therefore, embodiments of this application provide an image processing method and an electronic device. The electronic device first processes a first image based on a preset first color correction matrix. If the processed first image (referred to as the second image) experiences color overflow, the electronic device reduces the saturation of color patches in a standard color chart that are close to the overflowing color in the second image, obtaining a first color chart. The electronic device then obtains a second color correction matrix based on the first and second color charts. Subsequently, the electronic device can process the first image based on the second color correction matrix, and the processed first image (referred to as the third image) is then processed.

[0042] The second color chart can be viewed as a RAW image captured by the electronic device against the standard color chart, reflecting the camera's response to the red, green, and blue spectra. In other words, the second color correction matrix can correct the second color chart to resemble the first color chart. Compared to the standard color chart, the color patches in the first color chart that are close to the overflowing colors in the second image have lower saturation. Therefore, using the second color correction matrix to correct the first image can reduce the saturation of colors that would otherwise overflow in the first image, i.e., the overflowing colors in the second image. Due to the lower saturation, correspondingly, there is less or no data overflow in the third image, thus avoiding oversaturation, loss of detail, and blurring caused by data truncation due to overflow.

[0043] The method provided in this application can be applied to electronic devices with data processing capabilities. These electronic devices may include mobile phones, tablets, laptops, netbooks, personal digital assistants (PDAs), wearable devices (e.g., smartwatches, smart bracelets), in-vehicle devices, virtual reality devices, etc., and this application does not impose any limitations on them. In this application, the aforementioned electronic devices are those capable of running an operating system and installing applications. Optionally, the operating system running on the electronic device may be... system, system, Systems, etc.

[0044] Take a mobile phone as an example. Figure 1 As shown, the electronic device 100 may include: a processor 110, a memory 120, a universal serial bus (USB) interface 130, a power management module 140, antennas such as antenna 1 and antenna 2, a communication module 150, a display screen 160, an audio module 170, a camera 180, a sensor module 190, etc.

[0045] Processor 110 may include one or more processing units, such as: application processor (AP), central processing unit, modem processor, graphics processing unit (GPU), ISP, controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors. The controller may be the nerve center and command center of electronic device 100. The controller can generate operation control signals based on instruction opcodes and timing signals to control instruction fetching and execution.

[0046] In this embodiment, the processor 110, such as an ISP, can perform image processing on the RAW image captured by the camera 180 to obtain an image in YUV or RGB format. Then, the processor 110, such as the ISP, can transmit the processed image to the central processing unit (CPU). The CPU can store the image and / or control the display screen 160 to display the image. This image processing includes color correction. Specifically, the processor 110, such as the ISP, can use a first color correction matrix to correct the first image to obtain a second image. If the second image has color overflow, the ISP can also recalculate the second color correction matrix and use it to correct the first image to obtain a third image. Then, the third image can be transmitted to the CPU for storage or display to the user.

[0047] The memory 120 can be used to store computer executable program code, which includes instructions. The processor 110 executes various functional applications and data processing of the electronic device by running the instructions stored in the memory 120. The memory 120 may include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function (such as sound playback, interface display, etc.). The data storage area may store data created during the use of the electronic device (such as notification messages). Furthermore, the memory 120 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0048] In this embodiment, the memory 120 stores computer-executable program code, which includes instructions. The processor 110 executes an image processing method provided in this embodiment by running the instructions stored in the memory 120.

[0049] The power management module 140 is used to connect the battery to the processor 110. The power management module 140 receives battery and / or power input to power the processor 110, memory 120, communication module 150, display screen 160, and camera 180, etc. The power management module 140 can also be used to monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 140 may also be located within the processor 110.

[0050] The communication module 150 can provide solutions for wireless communication applications on the electronic device 100, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR). The communication module 150 can be one or more devices integrating at least one communication processing module. The communication module 150 receives electromagnetic waves via an antenna, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to the processor 110. The communication module 150 can also receive signals to be transmitted from the processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via the antenna.

[0051] In some embodiments, the antenna of the electronic device 100 is coupled to the communication module 150, enabling the electronic device 100 to communicate with networks and other devices via wireless communication technologies. The wireless communication technologies may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BitTorrent, Global Navigation Satellite System (GNSS), WLAN, NFC, FM, and / or IR technologies. The GNSS may include Global Positioning System (GPS), BeiDou Navigation Satellite System (BDS), GLONASS, and / or Galileo.

[0052] Electronic device 100 implements display functions through a GPU, a display screen 160, and an application processor. The GPU is a microprocessor for image processing, connecting the display screen 160 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0053] The display screen 160 is used to display images, videos, etc. The display screen 160 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Mini-LED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc.

[0054] Electronic device 100 can achieve shooting and recording functions through ISP, camera 180, video codec, GPU, display screen 160 and application processor.

[0055] The audio module 170 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 170 may be located in the processor 110, or some functional modules of the audio module 170 may be located in the processor 110.

[0056] The camera 180 is used to capture still images or videos. An object passes through the lens, generating an optical image that is projected onto a photosensitive element. This photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP (Image Signal Processor) for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP (Digital Signal Processor) for further processing. The DSP converts the digital image signal into standard image signals in formats such as RGB and YUV.

[0057] The sensor module 190 may include pressure sensors, gyroscope sensors, barometric pressure sensors, magnetic sensors, accelerometers, distance sensors, proximity sensors, fingerprint sensors, temperature sensors, touch sensors, ambient light sensors, and bone conduction sensors, etc.

[0058] It is understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device 100. In other embodiments, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0059] Taking the aforementioned electronic device 100 as a mobile phone as an example, the software system of the electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment uses a layered architecture... Taking the system as an example, the software structure of electronic device 100 is illustrated.

[0060] Figure 2 This is a software structure block diagram of the electronic device 100 according to an embodiment of this application.

[0061] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the system libraries, and the kernel layer.

[0062] The application layer can include a series of application packages.

[0063] like Figure 2 As shown, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, SMS, and social networking.

[0064] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0065] like Figure 2 As shown, the application framework layer may include a content provider, a view system, a resource manager, a notification manager, an input system, an image processing module, etc.

[0066] The input system is used to monitor the phone's input modules (such as touchscreen drivers) and convert the parameters input by the input modules into usable events, which are then passed to the relevant upper-layer modules. For example, the input system is used to monitor the phone's touchscreen through the touchscreen driver and convert the touch parameters generated by the touchscreen input into usable events, which are then passed to the upper-layer APP.

[0067] Content providers store and retrieve data, making that data accessible to applications. This data can include videos, images, audio, phone calls made and received, browsing history and bookmarks, phone books, and more.

[0068] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build the display interface of an application.

[0069] The notification manager allows applications to display notification information in the status bar. It can be used to convey informational messages and can disappear automatically after a short time without user interaction.

[0070] The image processing module receives RAW images acquired from the camera 180 by the camera driver, performs image processing on the RAW images, and stores the processed images in the memory 120 of the electronic device 100. This image processing includes color correction.

[0071] System libraries can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), etc.

[0072] The Surface Manager is used to manage the display subsystem and provides the blending of 2D and 3D layers for multiple applications.

[0073] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0074] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0075] A 2D graphics engine is a graphics engine for 2D drawing.

[0076] The kernel layer is the layer between hardware and software. The kernel layer can contain touchscreen drivers, display drivers, camera drivers, sensor drivers, etc.

[0077] The following uses a mobile phone as an example to introduce an image processing method and an electronic device provided in this application. The mobile phone includes a camera, which can be a front-facing camera or a rear-facing camera. The number of cameras can be one or more. Figure 3 An image processing method provided in this application embodiment may include:

[0078] S301, the phone acquires the first image.

[0079] This solution applies to scenarios involving taking photos or recording videos. The first image can be a photograph or a video frame. The first image can be captured by the phone's camera, and the phone can acquire the first image captured by the camera. The phone can respond to user actions and use its camera to capture the first image. For example, this action could be a "take a picture" operation on the camera app's photo preview interface, such as a tap operation. Another example is a "take a picture" operation on the camera app's video preview interface, such as a tap operation. Yet another example is that the user speaks a voice control command near the phone's microphone, such as "take a picture" or "start recording." This application does not limit the specific implementation method of triggering the phone to capture the first image.

[0080] The first image includes at least one target object, wherein the saturation of the target object satisfies a second preset condition. The second preset condition may be, for example, that the saturation of the target object exceeds a second preset value, or that, among the multiple pixels corresponding to the target object, the number of pixels with saturation exceeding the second preset value exceeds a preset number. The saturation of the target object may be the average saturation of the multiple pixels corresponding to the target object. The target object may, for example, be a high-saturation target as described above.

[0081] The following section uses a high-saturation subject as an example to introduce this solution.

[0082] After acquiring the first image, the phone uses a first color correction matrix to perform color correction on the first image to obtain the second image. If color overflow exists in the second image, the second color correction matrix is ​​used to perform color correction on the first image to obtain the third image. Specifically, the phone can execute S302-S309.

[0083] S302, the mobile phone uses a preset first color correction matrix to perform color correction on the first image to obtain the second image.

[0084] The second image is a color-corrected version of the first image using the first color correction matrix. If the first image includes a highly saturated subject, the highly saturated subject in the second image may exhibit color clipping. For example... Figure 4 The second image shown includes highly saturated red flowers, where details are lost and the flowers appear blurry.

[0085] The first color correction matrix can be pre-configured in the phone by the developers. The phone can retrieve this first color correction matrix from a preset storage location. The first color correction matrix can be determined based on a standard color chart. For example, the phone can photograph each color patch in the standard color chart to obtain a RAW image of the standard color chart. The product of this RAW image and the first color correction matrix can be the true color of the standard color chart, and the phone can obtain the first color correction matrix based on this RAW image and the true color of the standard color chart.

[0086] The mobile phone multiplies the first image with a first color correction matrix to correct the color of the first image, thus obtaining the second image. For example, the first image includes m pixels such as pixel [R1, G1, B1], pixel [R2, G2, B2], ..., pixel [R...]. m G m B m The second image includes m pixels, such as pixels [R`1, G`1, B`1], pixels [R`2, G`2, B`2], ..., pixels [R`1, G`2, B`2], ..., pixels [R`1, G`1, B`1], ..., pixels [R`2, G`2, B`2], ..., pixels [R`1, G`1 ... m ,G` m ,B` m ].

[0087] So,

[0088] in, This is the first color correction matrix.

[0089] Optionally, the phone may preprocess the first image before executing S302. Preprocessing includes, but is not limited to, BLC correction, LSC, Denoise, AWB, DRC, etc.

[0090] After obtaining the second image, the phone can determine whether the first color correction matrix needs to be corrected. If the second image exhibits color overflow, the phone determines that the first color correction matrix needs correction. If the second image does not exhibit color overflow, the phone determines that the first color correction matrix does not need correction.

[0091] The mobile phone can determine whether the second image has color overflow based on the pixel values ​​of each pixel in the second image. For example, if the second image includes a large number of overflow pixels, the mobile phone determines that the second image has color overflow. If the second image includes few or no overflow pixels, the mobile phone determines that the second image does not have color overflow. As another example, if the number of overflow pixels meets a first preset condition, the mobile phone determines that the second image has color overflow. The first preset condition may be that the number of overflow pixels is greater than a first preset value.

[0092] In some embodiments, the overflow pixel may be referred to as the target pixel. Among the color channels of the overflow pixel, there are color channel values ​​greater than a first value; and / or, there are color channel values ​​less than a second value. The first value can be the maximum value that can be represented by one byte, such as 255, and the second value can be the minimum value that can be represented by one byte, such as 0. In some embodiments, the color channels of the overflow pixel may be, for example, the R channel, G channel, and B channel. In other embodiments, the color channels of the overflow pixel may be, for example, the Y channel, U channel, and V channel.

[0093] In some embodiments, a pixel is an overflow pixel if the value of at least one of its R, G, and B channels is greater than a first value, such as 255, and / or the value of at least one of its R, G, and B channels is less than a second value, such as 0. That is, an overflow pixel includes channels with values ​​greater than 255 in its R, G, and B channels, and / or includes channels with values ​​less than 0 in its R, G, and B channels. For example, pixel a [260,0,0] is an overflow pixel, pixel b [270,20,50] is an overflow pixel, and pixel c [255,0,-23] is an overflow pixel. In conventional techniques, mobile phones truncate pixel a to [255,0,0], pixel b to [255,20,50], and pixel c to [255,0,0], which can lead to loss of detail, blurring, and oversaturation in the second image.

[0094] The mobile phone can first count the number of overflowing pixels in the second image, and then determine whether the second image has color overflow based on the number of overflowing pixels. For example, the mobile phone can count 'a' pixels out of m pixels in the second image where at least one of the R, G, and B channels has a value greater than 255. If this number 'a' is greater than a preset value of 1, the mobile phone determines that the second image has color overflow; if this number 'a' is less than the preset value of 1, the mobile phone determines that the second image does not have color overflow. As another example, the mobile phone can count 'b' pixels out of m pixels in the second image where at least one of the R, G, and B channels has a value less than 0. If this number 'b' is greater than a preset value of 2, the mobile phone determines that the second image has color overflow; if this number 'b' is less than the preset value of 2, the mobile phone determines that the second image does not have color overflow. As yet another example, the mobile phone can count the sum of the above numbers 'a' and 'b', such as 'c'. If 'c' is greater than or equal to a preset value of 3, the mobile phone determines that the second image has color overflow; if 'c' is less than the preset value of 3, the mobile phone determines that the second image does not have color overflow. Here, preset values ​​1, 2, and 3 can be integers greater than 1.

[0095] In some embodiments, overflow pixels can be pixels in the R, G, and B channels that include values ​​greater than or equal to a first value, such as 255, and / or pixels in the R, G, and B channels that include values ​​less than or equal to a second value, such as 0. Although the phone does not truncate data equal to 255 or equal to 0, the saturation of a pixel is relatively high when the channel equals 255 or equal to 0. As an example, saturation can be calculated using the following formula: Saturation = (MAX(R, G, B) - MIN(R, G, B)) / MAX(R, G, B). Where MAX() is used to find the maximum value and MIN() is used to find the minimum value. For example, the saturation of pixel d[240, 25, 0] is (240 - 0) / 240. The saturation of pixel e[240, 25, 23] is (240 - 23) / 240. The saturation of pixel d is higher than that of pixel e. To reduce the saturation of the color-corrected image, the overflow pixels in the second image can be pixels in the R, G, and B channels that include values ​​greater than or equal to a first value, such as 255, and / or pixels in the R, G, and B channels that include values ​​less than or equal to a second value, such as 0. In this embodiment, the specific implementation of the mobile phone determining whether the second image has color overflow is similar to the implementation in the aforementioned embodiments, and will not be described in detail here.

[0096] The following explanation uses the example of overflow pixels being pixels whose R, G, and B channels include channels with values ​​greater than 255, and / or whose R, G, and B channels include channels with values ​​less than 0.

[0097] If the phone determines that the first color correction matrix needs to be corrected, it can execute S303. If the phone determines that the first color correction matrix does not need to be corrected, it can perform subsequent processing on the second image, such as Gamma correction and RGB2YUV (RGB format to YUV format) conversion. After processing is complete, the phone can save or display the second image.

[0098] In the event of color overflow in the second image, the mobile phone can determine the overflowing color in the second image. The mobile phone can calculate the overflowing color in the second image based on the pixel values ​​of the overflowing pixels as previously calculated. For example, the mobile phone can calculate a first pixel value based on the pixel values ​​of the overflowing pixels as previously calculated, and this first pixel value is used to represent the overflowing color in the second image. Specifically, the mobile phone can execute S303.

[0099] S303: The mobile phone counts the pixel values ​​of overflowing pixels and obtains the first pixel value based on the pixel values ​​of the overflowing pixels.

[0100] The mobile phone can first count the overflow pixels in the second image. For example, in digital imaging technology, the second image can be a matrix, such as a matrix... Each column of the matrix represents the pixel value of a single pixel. These pixel values ​​include the values ​​from the R, G, and B channels. The phone iterates through the matrix column by column, recording each column containing values ​​greater than 255 and / or less than 0. These columns represent the overflow pixel values. For example, the phone can record values ​​according to [R` i ,G` i ,B` i The pixel value of the overflowing pixel is recorded in the manner shown in the image.

[0101] When the first image includes a highly saturated, solid-color subject such as a red flower, the overflow pixels in the second image are typically some or all of the pixels corresponding to that highly saturated, solid-color subject. These overflow pixels generally have their R channel overflowing upwards (greater than 255) and their G and / or B channels overflowing downwards (less than 0). The phone can calculate the first pixel value using the following averaging method: R channel value of the first pixel = Sum of R channel values ​​of the overflow pixels / Total number of overflow pixels. G channel value of the first pixel = Sum of G channel values ​​of the overflow pixels / Total number of overflow pixels. B channel value of the first pixel = Sum of B channel values ​​of the overflow pixels / Total number of overflow pixels. For example, if the first image includes a highly saturated red flower, and the overflow pixels in the second image are pixel a[260,0,0], pixel b[270,20,50], and pixel c[255,0,-23], then the R channel value of the first pixel = (260+270+255) / 3 = 261. The G channel value of the first pixel is (0 + 20 + 0) / 3 = 7. The B channel value of the first pixel is (0 + 50 + (-23)) / 3 = 9. Thus, the first pixel value is [261, 7, 9]. Optionally, during the calculation process, if the result is a decimal, the phone can round the result up. Alternatively, the phone can retain a preset number of decimal places, such as one or two decimal places.

[0102] Optionally, the phone can calculate the first pixel value by taking the median. For example, the R channel of the first pixel value can be the median of the R channels of multiple overflow pixels, the G channel of the first pixel value can be the median of the G channels of multiple overflow pixels, and the B channel of the first pixel value can be the median of the B channels of multiple overflow pixels.

[0103] Optionally, the phone can calculate the first pixel value by taking the mode. For example, the R channel of the first pixel value can be the mode among the R channels of multiple overflow pixels, the G channel of the first pixel value can be the mode among the G channels of multiple overflow pixels, and the B channel of the first pixel value can be the mode among the B channels of multiple overflow pixels.

[0104] In other words, when the first image includes a highly saturated subject, the phone calculates the R, G, and B channel values ​​of the overflow pixels to obtain the R, G, and B channel values ​​of the overflow color corresponding to the highly saturated subject. This calculation may include calculating the mean, median, and mode.

[0105] When the first image includes two or more highly saturated objects of a solid color (hereinafter referred to as "objects"), the mobile phone can separately count the overflow pixels corresponding to each object in the second image and calculate the first pixel value corresponding to each object. Distinguishing between these overflow pixels makes the calculated first pixel value closer to the overflow color in the second image. For example, if the overflow colors in the second image are red and blue, without distinguishing them, the calculated overflow color in the second image might be purple, which does not match the actual overflow situation.

[0106] Taking a first image containing three subjects, such as a red flower, green leaves, and a blue light, as an example, the phone can first count all overflow pixels in the second image. Overflow pixels can include some or all pixels corresponding to the first subject, such as the red flower. These overflow pixels are generally greater than 255 for the R channel and less than 0 for the G and / or B channels. Overflow pixels can also include some or all pixels corresponding to the second subject, such as the green leaves. These overflow pixels are generally greater than 255 for the G channel and less than 0 for the R and / or B channels. Overflow pixels can also include some or all pixels corresponding to the third subject, such as the blue light. These overflow pixels are generally greater than 255 for the B channel and less than 0 for the R and / or G channels.

[0107] Next, the phone categorizes the overflow pixels. The phone can use a clustering algorithm to categorize the overflow pixels. This clustering algorithm could be, for example, K-Nearest Neighbor (KNN) or k-means clustering. As an example, the phone uses red [255,0,0], green [0,255,0], and blue [0,0,255] as base colors. The phone calculates the distance 1 between the overflow pixel and red, the distance 2 with green, and the distance 3 with blue. The phone identifies overflow pixels with a distance 1 greater than a threshold as overflow pixels corresponding to the red subject, overflow pixels with a distance 2 greater than a threshold as overflow pixels corresponding to the green subject, and overflow pixels with a distance 3 greater than a threshold as overflow pixels corresponding to the blue subject.

[0108] After classification, the phone can obtain the overflow pixels corresponding to the first, second, and third shooting objects. Then, the phone can obtain the first pixel value corresponding to the first shooting object, the first pixel value corresponding to the second shooting object, and the first pixel value corresponding to the third shooting object using the method described above. Optionally, if multiple shooting objects, including those with overflow pixels less than a threshold, are included, the phone may not adjust the saturation of those objects. For example, if the number of overflow pixels corresponding to a shooting object is less than a third preset value, the phone does not generate the first pixel value for that object, and correspondingly, does not adjust the saturation of the color block corresponding to that first pixel value in the standard color chart. Specifically, if the total number of overflow pixels is greater than the first preset value, the phone recognizes the need to correct the first color correction matrix. Next, the phone classifies the overflow pixels to obtain a group of overflow pixels corresponding to a shooting object. Then, when the overflow pixel group includes a group of overflow pixels with a number of pixels greater than the third preset value, the phone only calculates the first pixel value for the overflow pixel group with a number of pixels greater than the third preset value, and then adjusts the saturation of the color block in the standard color chart based on the first pixel value. When the overflow pixel group does not include an overflow pixel group with a number greater than the third preset value, the phone further recognizes that there is no need to correct the first color correction matrix. The phone can then perform subsequent preprocessing on the second image and display or save the preprocessed second image. The first preset value is greater than the third preset value.

[0109] In other words, when the first image includes two or more highly saturated subjects, the phone categorizes the overflow pixels, obtaining overflow pixels that correspond one-to-one with each of the highly saturated subjects. For each overflow pixel corresponding to a highly saturated subject, the R, G, and B channels are calculated separately to obtain the R, G, and B channel values ​​of the overflow color corresponding to each highly saturated subject. This calculation may include calculating the mean, median, and mode.

[0110] S304, the mobile phone acquires a target color patch in the standard color chart that is close to the color overflowing in the second image.

[0111] A color chart is a commonly used color reference tool. A standard color chart can be, for example, a color chart with 24 color patches, a color chart with 96 color patches, a color chart with 140 color patches, etc. Of course, a standard color chart can also include other numbers of color patches; this application does not specifically limit this. One color patch on a standard color chart represents a real color in nature.

[0112] The target color patch is a color patch in the standard color chart whose similarity to the overflowing color in the second image meets the condition. The target color patch is one that is close to the overflowing color in the second image. This condition can be that the target color patch has a high similarity to the overflowing color in the second image.

[0113] In some embodiments, the mobile phone can acquire a preset number of color patches in a standard color chart that are close to the color of the first pixel value, as target color patches. Specifically, the mobile phone can first acquire the similarity between the first pixel value and each color patch in the standard color chart, and sort the similarity in descending order. Then, the mobile phone can acquire the color patches corresponding to the first preset number of similarities after sorting, as target color patches that are close to the color of the first pixel value.

[0114] For example, the phone includes a mapping relationship between the number of color blocks and a preset number. For instance, if the standard color chart includes 24 color blocks, the preset number could be 2 or 3. If the standard color chart includes 96 color blocks, the preset number could be 3-5. If the standard color chart includes 140 color blocks, the preset number could be 4-7. This is because as the number of color blocks increases, the number of colors the color chart can represent increases, and the number of color blocks in the standard color chart that are close to the color of the first pixel value also increases; therefore, the preset number also increases. As another example, the phone can calculate the preset number corresponding to color charts with different numbers of color blocks. For instance, the phone can use the hues of red [255, 0, 0], blue [0, 0, 255], and green [0, 255, 0] as reference values ​​and set a hue distance threshold, which could be, for example, 15. Then, the phone calculates the hue distance between each color block in the color chart and red. It counts the number of color blocks with a hue distance greater than the threshold, such as 15, and uses this number as the preset number corresponding to the red subject. The phone calculates the hue distance between each color block in the color chart and green. It counts the number of color blocks with a hue distance greater than a threshold (e.g., 15) and uses this number as the preset number of green subjects. Similarly, the phone calculates the hue distance between each color block in the color chart and blue. It counts the number of color blocks with a hue distance greater than a threshold (e.g., 15) and uses this number as the preset number of blue subjects. Thus, after obtaining the first pixel value, the preset number of target color blocks corresponding to the first pixel value can be determined based on the color represented by that first pixel value.

[0115] The mobile phone can calculate the similarity between the first pixel value and the color block in multiple ways.

[0116] As an example, a mobile phone can calculate the hue distance between a first pixel value and each color patch in a standard color chart. This hue distance indicates the similarity between the first pixel value and each color patch. A smaller hue distance indicates higher similarity, meaning the color represented by that patch is closer to the color represented by the first pixel value. Conversely, a larger hue distance indicates lower similarity, meaning the color represented by that patch is further away from the color represented by the first pixel value. For instance, a mobile phone can convert the first pixel value from RGB format to LCH format, and convert the pixel values ​​of each color patch in the standard color chart into lightness, saturation, and hue (LCH) formats. The LCH format pixel value includes L channels, C channels, and H channels. The L channel represents lightness, the C channel represents chroma (saturation), and the H channel represents hue. For example, a mobile phone can convert the pixel value of the first pixel from RGB format to Hue, Saturation, Value (HSV) format, and convert the pixel values ​​of each color block in the standard color chart to HSV format. The HSV format pixel value includes H channel, S channel, and V channel. The H channel represents hue, the S channel represents saturation, and the V channel represents brightness. Then, the mobile phone can calculate the hue distance between the first pixel value and each color block. For example, hue distance = H channel of color block k - H channel of the first pixel value. Taking a standard color chart with 140 color blocks as an example, k takes values ​​sequentially from [1, 2, ..., 140]. Optionally, the pixel values ​​of the color blocks in the standard color chart are stored in the mobile phone in a format where 'B' represents the color range from magenta, red to green, and 'B' represents the color range from yellow to blue (Luminosity, a, b, Lab). Therefore, the mobile phone can convert the pixel values ​​of the standard color chart from Lab format to LCH or HSV format. Optionally, the pixel values ​​of the color blocks in the standard color chart can be stored in the mobile phone in LCH or HSV format, in which case there is no need to convert the color format of each color block in the color chart.

[0117] As another example, the mobile phone can calculate ΔE (Delta-E) between the first pixel value and each color patch in the color chart. ΔE indicates the similarity between the first pixel value and each color patch in the color chart. The smaller the ΔE, the higher the similarity; the larger the ΔE, the lower the similarity. The mobile phone can calculate ΔE using conventional methods, which will not be discussed further in this application.

[0118] In other embodiments, the mobile phone can obtain the similarity between each color block in the standard color chart and the first pixel value, and the mobile phone will select the color block with a similarity greater than a preset threshold as the target color block whose color is close to the first pixel value. The method by which the mobile phone obtains the similarity between each color block in the standard color chart and the first pixel value is described in the previous embodiment and will not be repeated here.

[0119] Optionally, to reduce the computational load on the phone, improve computational performance, and reduce power consumption, low-saturation color blocks in the standard color chart can be excluded before calculating similarity. Low-saturation color blocks include black, white, gray, and other darker colors. The phone can exclude low-saturation color blocks from the standard color chart using the following two methods: That is, the phone can calculate the similarity between the first pixel value and color blocks in the standard color chart other than low-saturation color blocks, but does not calculate the similarity between low-saturation color blocks in the standard color chart and the first pixel value.

[0120] Method 1: The positions of low-saturation color blocks in the standard color chart are fixed, and the mobile phone has pre-stored the position information of low-saturation color blocks in the standard color chart. For example, taking a standard color chart that includes 140 color blocks as an example... Figure 5 A standard color chart is shown. The outermost color blocks of this standard color chart include: white, black, and gray blocks. The second column of the second row contains a dark red block, and the ninth column of the second row contains a dark blue block. The seventh to tenth columns of the sixth row contain white, gray, black, and gray blocks, respectively. The position information of the color blocks can be represented as [i,j], where i represents the i-th row and j represents the j-th column. When the mobile phone reads the pixel values ​​of the color blocks from the standard color chart to calculate similarity, the mobile phone does not read the pixel values ​​of the color blocks from the positions of low-saturation color blocks. That is, the mobile phone calculates the similarity between the color blocks at positions other than those specified in the pre-stored position information and the first pixel value. For example, continuing to combine... Figure 5 When calculating similarity by reading the pixel values ​​of color blocks, the phone does not read the pixel values ​​of color blocks in columns 1, 14, 1, and 10. It also does not read the pixel values ​​of the color block in the second column of the second row, the ninth column of the second row, or the seventh to tenth columns of the sixth row. The phone reads the pixel values ​​of color blocks in the remaining positions.

[0121] Alternatively, the phone can pre-store the location information of high-saturation color blocks in a standard color chart. A high-saturation color chart refers to color blocks other than the low-saturation color blocks mentioned above. In this way, the phone can calculate the similarity between the color block at the location indicated by that location information in the standard color chart and the first pixel value, based on the pre-stored location information.

[0122] Method 2: The phone pre-stores the saturation levels of each color block. The phone does not use color blocks with saturation levels lower than a preset value for similarity calculation. The phone can pre-calculate the saturation of each color block and establish a mapping relationship between the color blocks and their corresponding saturation levels. In this way, before reading the pixel values ​​of a color block from the standard color chart to calculate similarity, the phone can first obtain the saturation level of that color block. Based on the saturation level, it determines whether to read the pixel values ​​of that color block for similarity calculation. That is, the phone can calculate the similarity between the first pixel value and a color block with a saturation level higher than the preset value in the standard color chart, based on the pre-stored saturation levels of each color block in the standard color chart. For the method of calculating the saturation of each color block, please refer to existing technologies; it will not be elaborated here.

[0123] Based on the preceding description, when the first image includes two or more highly saturated objects of a solid color, the first pixel value corresponding to each object can be calculated separately, meaning the number of first pixel values ​​is greater than or equal to 1. For each first pixel value, the phone can execute the method shown in S304 to obtain target color patches that are close to that first pixel value. Continuing with the example of the first image including three objects such as a red flower, green leaves, and a blue light, the phone can execute the method shown in S303 to find the first pixel value 'a' corresponding to the red flower, the first pixel value 'b' corresponding to the green leaves, and the first pixel value 'c' corresponding to the blue light. Then, the phone can execute the method shown in S304 to find x target color patches corresponding to the first pixel value 'a', y target color patches corresponding to the first pixel value 'b', and z target color patches corresponding to the first pixel value 'b'. Here, x, y, and z can be the same or different.

[0124] After obtaining the target color patch in the standard color chart that is close to the color of the first pixel value, the mobile phone can reduce the saturation of the target color patch that is close to the color of the first pixel value, and calculate the second color correction matrix based on the color chart including the target color patch with reduced saturation. Specifically, the mobile phone can execute S305-S306 to reduce the saturation of the target color patch in the standard color chart to obtain the first color chart.

[0125] S305, the mobile phone determines the saturation reduction coefficient of each target color block in the target color block that is close to the first pixel value.

[0126] As mentioned earlier, the number of first pixel values ​​is greater than or equal to 1. For each first pixel value, the mobile phone can execute the method shown in S305 to determine the saturation reduction coefficient (which can be simply referred to as the coefficient) of each target color block in the target color block that is close to each first pixel value.

[0127] Taking a first pixel value as an example, if there are N target color patches with colors close to the first pixel value, the phone can determine N saturation reduction coefficients corresponding to these N target color patches. These saturation reduction coefficients are greater than 0 and less than 1.

[0128] The saturation reduction coefficient is directly proportional to the similarity between the overflowing color in the second image and the target color patch. The higher the similarity of the target color patch to the overflowing color in the second image, the greater the saturation reduction, and the smaller its corresponding saturation reduction coefficient. Conversely, the lower the similarity of the target color patch to the overflowing color in the second image, the less the saturation reduction, and the larger its corresponding saturation reduction coefficient. Since the first pixel value can represent the overflowing color in the second image, the higher the similarity of the target color patch to the first pixel value, the greater the saturation reduction, and the smaller its corresponding saturation reduction coefficient. The lower the similarity of the target color patch to the first pixel value, the less the saturation reduction, and the larger its corresponding saturation reduction coefficient. Therefore, the mobile phone can determine the saturation reduction coefficient corresponding to N target color patches based on the similarity between N target color patches and the first pixel value.

[0129] For example, the saturation reduction coefficient Wp of target color block p is the ratio of the similarity between target color block p and the overflowing color in the second image to the sum of the similarities between N target color blocks and the overflowing colors in the second image. In other words, the saturation reduction coefficient Wp of target color block p is the proportion of the similarity between target color block p and the overflowing color in the second image to the sum of the similarities between N target color blocks and the overflowing colors in the second image. For example, the saturation reduction coefficient Wp of target color block p = similarity between target color block p and the first pixel value / sum of the similarities between N target color blocks and the first pixel value, where p takes values ​​sequentially in [1, 2, ..., N]. Since hue distance can be used to indicate similarity, the saturation reduction coefficient of target color block p = hue distance between target color block p and the first pixel value / sum of the hue distances between N target color blocks and the first pixel value. Taking a scenario where the first pixel value corresponds to 5 target color blocks as an example, the saturation reduction coefficient Wp of the target color block p is equal to the hue distance between the target color block p and the first pixel value, divided by the sum of the hue distances between the 5 target color blocks and the first pixel value. p takes values ​​sequentially from [1, 2, ..., 5]. The smaller the hue distance, the more similar the target color block is to the first pixel value; the larger the hue distance, the further away the target color block is from the first pixel value.

[0130] For example, a mobile phone may have multiple preset saturation reduction coefficients. The phone can select the appropriate saturation reduction coefficient for each target color patch based on its similarity to the first pixel value. The greater the similarity between the target color patch and the first pixel value, the smaller the selected saturation reduction coefficient. The number of saturation reduction coefficients equals the number of target color patches. High-similarity target color patches correspond to low saturation reduction coefficients. For instance, a mobile phone might have five preset saturation reduction coefficients: 0.05, 0.1, 0.2, 0.25, and 0.4. These five coefficients correspond to the top five target color patches with the most similar first pixel values.

[0131] The S306 mobile phone uses a saturation reduction factor to reduce the saturation of target color blocks in a standard color chart that are close to the color of the first pixel value.

[0132] As mentioned earlier, the number of first pixel values ​​is greater than or equal to 1. For each first pixel value, the mobile phone can execute the method shown in S306 to reduce the saturation of each target color block in the target color block that is close to the color of each first pixel value in the standard color chart.

[0133] Taking a single pixel value as an example, saturation is related to the values ​​of the pixel's R, G, and B channels. For easier calculation, the phone can convert the standard color chart to LCH or HSV format. In LCH format, the C channel represents chroma, also known as saturation. In HSV format, the S channel represents saturation. Then, the phone can reduce the saturation of each target color patch in the standard color chart based on the saturation reduction factor determined earlier.

[0134] Continuing with the example of N target color patches whose colors are close to the first pixel value, for instance, if the standard color chart is converted to LCH format, then the C channel of the first target color patch p multiplied by the saturation reduction factor Wp of the target color patch p equals the C channel of the second target color patch p. If the pixel value of the target color patch is converted to HSV format, then the S channel of the first target color patch p multiplied by the saturation reduction factor Wp of the target color patch p equals the C channel of the second target color patch S. Here, the first target color patch is the target color patch before saturation reduction, and the second target color patch is the color patch corresponding to the first target color patch after saturation reduction.

[0135] The mobile phone reduces the saturation of the target color block in the standard color chart to obtain the first color chart. For example, the mobile phone replaces the pixel values ​​of the first target color block in the standard color chart with the corresponding pixel values ​​of the second target color block to obtain the first color chart.

[0136] S307: The mobile phone multiplies the standard color chart with the inverse of the first color correction matrix to obtain the second color chart.

[0137] The pixel values ​​of each color patch in the second color chart can be the pixel values ​​of the standard color chart before correction by the first color matrix. In other words, the product of the second color chart and the first color correction matrix is ​​the standard color chart, and the product of the standard color chart and the inverse of the first color correction matrix is ​​the second color chart. The second color chart can represent the RAW image captured by an electronic device against the standard color chart, and can reflect the camera's response to the red, green, and blue spectra.

[0138] The purpose of this is to find the pixel values ​​of the standard color chart before it was corrected by the first color matrix, i.e., the second color chart.

[0139] S308: The mobile phone obtains a second color correction matrix based on the first and second color cards.

[0140] The phone uses the first color chart as the second color chart, after color correction. That is, second color chart * second color correction matrix = first color chart.

[0141] Take a color chart containing 140 color blocks as an example. The second color chart could be... The first color card can be The second color correction matrix can The following system of equations can be established:

[0142]

[0143] The following constraints can be added to this equation:

[0144] D11+D12+D13=1, D21+D22+D23=1, D31+D32+D33=1.

[0145] The mobile phone can solve the above system of equations to obtain the elements of the second color correction matrix. For example, the mobile phone can use the least squares method to solve the above system of equations.

[0146] S309: The phone uses a second color correction matrix to correct the colors of the first image to obtain the third image.

[0147] The phone multiplies the first image with the second color correction matrix to obtain the third image. For example,

[0148] It is the third image.

[0149] As can be seen, this scheme first processes the first image using a first color correction matrix. If the processed first image, like the second image, exhibits color overflow, another color correction matrix, such as a second color correction matrix, is needed to reprocess the first image to avoid overflow. The processed first image, like the third image, then exhibits little or no color overflow. To find a suitable second color correction matrix, this scheme first determines the overflowing color in the second image, such as the first pixel value. Then, it finds a first color patch similar to the overflowing color from a standard color chart. To avoid this color overflow, the phone reduces the saturation of the target color patch similar to this color, resulting in a first color chart. Finally, the second color correction matrix is ​​obtained based on the first and second color charts. Multiplying the second color chart by the first color correction matrix yields the standard color chart, and multiplying the second color chart by the second color correction matrix yields the first color chart. In other words, for the same input, such as the second color chart, the first color correction matrix can correct the color of the second color chart to match the color corresponding to the standard color chart, and the second color correction matrix can correct the color of the second color chart to match the color corresponding to the first color chart. Compared to the standard color chart, the target color patch on the first color chart, which is similar to the color represented by the first pixel value in the first image and is prone to overflow, has a lower saturation. Therefore, processing the first image using the second correction matrix can reduce the saturation of some pixels, avoid data overflow, and thus avoid display defects such as loss of detail, blurring, and oversaturation caused by data overflow.

[0150] like Figure 6 As shown, compared to the second image, the target object, such as the red flower, in the third image includes more detail and is clearer; for example, the dewdrops on the red flower and the outline of the petals can be seen. The saturation of red in the third image is lower than that in the second image. It can be seen that the red in the third image is paler than in the second image.

[0151] Furthermore, since this solution only reduces the saturation of target color blocks in the standard color chart that are similar to the overflowing color in the first image, without modifying other color blocks in the standard color chart, it will not affect other colors.

[0152] The following is combined Figure 7 This application introduces an image processing method provided by an embodiment. For example... Figure 7 As shown, the above method may include S1-S11. Figure 7 Let's take a mobile phone as an example, which includes two highly saturated subjects, such as a red flower and a blue light.

[0153] S1, the phone performs a first color correction on the first image to obtain a second image.

[0154] The mobile phone multiplies the first image with the first color correction matrix to obtain the second image. Both the first and second images can be in standard RGB format (standard Red Green Blue, sRGB).

[0155] S2, the mobile phone counts the overflow pixels in the second image to obtain the first pixel value a and the first pixel value b.

[0156] The mobile phone can count the overflow pixels in the second image and classify them. It obtains a first pixel value 'a' based on the first type of overflow pixels and a first pixel value 'b' based on the second type of overflow pixels. (See reference here.) Figure 3 The details of S303 will not be elaborated here.

[0157] S3, the phone converts the second image from sRGB format to LCH format.

[0158] The following section will introduce methods for converting image formats; these will not be repeated here. Please refer to [link / reference]. Figure 8 .

[0159] S4, the phone converts the standard color chart from Lab to LCH format.

[0160] The following section will introduce methods for converting image formats; these will not be repeated here. Please refer to [link / reference]. Figure 8 .

[0161] S5, the phone finds X color blocks that are close to the hue distance of the first pixel value a and Y color blocks that are close to the hue distance of the first pixel value b from the standard color chart.

[0162] S6, the phone determines the saturation reduction coefficient Wx for X color blocks and the saturation reduction coefficient Wy for Y color blocks.

[0163] The mobile phone can determine the saturation reduction coefficient Wx for each of the X color patches based on their similarity to the first pixel value a. Similarly, the mobile phone can determine the saturation reduction coefficient Wy for each of the Y color patches based on their similarity to the first pixel value b. (See reference here.) Figure 3 The S305 mentioned above will not be discussed further here.

[0164] S7: The phone reduces the saturation of X and Y color blocks in the standard color chart to obtain the first color chart.

[0165] Compared to the standard color chart, the color blocks corresponding to the overflowing colors in the first color chart and the second image have lower saturation. Specifically, the overflowing colors in the second image can be highly saturated reds and highly saturated blues.

[0166] The S8 phone converts the standard color chart from Lab format to sRGB format.

[0167] The following section will introduce methods for converting image formats; these will not be repeated here. Please refer to [link / reference]. Figure 8 .

[0168] S9, the phone multiplies the standard color card with the inverse of the first correction matrix to obtain the second color card.

[0169] If the second color chart * the first color correction matrix = the standard color chart, then the second color chart = the inverse of the standard color chart * the first color correction matrix.

[0170] S10: The mobile phone obtains the second color correction matrix based on the first and second color cards.

[0171] Second color chart * Second color correction matrix = First color chart, where the pixel values ​​of each color block in the second color chart and the pixel values ​​of each color block in the first color chart are known. The mobile phone can calculate the second color correction matrix based on the least squares method.

[0172] S11, the phone uses a second color correction matrix to perform second color correction on the first image to obtain the third image.

[0173] First image * second color correction matrix = third image. Compared to the second image, the third image has reduced saturation of highly saturated subjects, resulting in less data overflow and thus greater clarity. Specifically, compared to the second image, the third image shows reduced saturation of red flowers with more and clearer details, and reduced saturation of blue lights with more and clearer details.

[0174] The following is combined Figure 8 This section introduces methods for converting image formats.

[0175] like Figure 8 As shown, converting an sRGB image to an LCH image requires the following steps:

[0176] sRGB2XYZ (sRGB to XYZ): Converts an image from the sRGB color space to the XYZ color space;

[0177] XYZ2Lab (XYZ to Lab): Converts the color space from XYZ to Lab.

[0178] Lab2LCH (Lab to LCH): Converts the color space from Lab to LCH, ultimately resulting in an image in LCH format.

[0179] Specifically, the phone first converts the sRGB format image to an XYZ format image. That is, it converts the image from the sRGB color space to the XYZ color space. In the XYZ color space, each pixel includes an X channel, a Y channel, and a Z channel. The X channel represents the red primary color stimulus, the Y channel represents the green primary color stimulus, and the Z channel represents the blue primary color stimulus. For example, the conversion for each pixel in an sRGB format image can be based on the following formula.

[0180] X=0.412453*R+0.357580*G+0.180423*B.

[0181] Y=0.212671*R+0.715160*G+0.072169*B.

[0182] Z=0.019334*R+0.119193*G+0.950227*B.

[0183] For a given pixel, R is the value of the R channel, G is the value of the G channel, and B is the value of the B channel.

[0184] Next, the phone converts the XYZ format image to a Lab format image. In the Lab color space, each pixel includes an L channel, an a channel, and a b channel. The L channel represents luminance, the a channel represents red-green hue, and the b channel represents yellow-blue hue. For example, the conversion for each pixel in a Lab format image can be based on the following steps.

[0185] Step 1: Using the D65 light source standard white point as a reference, normalize the X, Y, and Z channels of each pixel to obtain the x, y, and z channels. Specifically, x = X / Xn1, y = Y / Yn1, and z = Z / Zn1. Where Xn1 is the normalization parameter for the X channel, Yn1 is the normalization parameter for the Y channel, and Zn1 is the normalization parameter for the Z channel. For example, the values ​​of Xn1, Yn1, and Zn1 are 0.95047, 1.0, and 1.08883, respectively.

[0186] Step 2: Apply a non-linear transformation to the normalized x, y, and z channels. For example, use the following formula for the non-linear transformation. Substitute the x, y, and z channels of each pixel into the following formula to calculate F(x), F(y), and F(z) respectively. For example, if the x channel value of a pixel is greater than... Then F(x) is (x channel) If the x channel is less than Then F(x) is aisle.

[0187]

[0188] Step 3: Complete the conversion of the Lab color space using the following method: L = 116 * F(y) - 16, a = 500 * (F(x) - F(y)), b = 200 * (F(y) - F(z)).

[0189] Finally, the phone converts the Lab format image into an LCH format image.

[0190] In the LCH color space, each pixel includes an L channel, a C channel, and an H channel. The L channel represents luminance, the C channel represents chrominance, and the H channel represents hue.

[0191] C = sqrt(0.2989*a² + 0.1368*b² + 0.2330*(ba)²). The sqrt() function returns the square root. For example, sqrt(16) = 4.

[0192] H = acos((0.57732*(ba)+1.42879*(b+a)) / (sqrt(0.2989*a²+0.1368*b²+0.2330*(ba)²))). Here, the acos() function calculates the arccosine of a given value. For example, acos(0.5) = 1.047198.

[0193] In this way, the phone can convert sRGB format images, such as the second image, to LCH format. The phone can also use the same method to convert Lab format standard color charts to LCH format standard color charts.

[0194] like Figure 8 As shown, converting an LCH format image to an sRGB format image requires the following steps:

[0195] LCH2Lab (LCH to Lab): Converts an image from the LCH color space to the Lab color space;

[0196] Lab2XYZ (Lab to XYZ): Converts from the Lab color space to the XYZ color space;

[0197] XYZ2sRGB (XYZ to sRGB): Converts the color space from XYZ to sRGB.

[0198] Specifically, the phone first converts the image from the LCH color space to the Lab color space. Specifically, the phone can use the following calculation method to substitute the L, C, and H channels of each pixel in the image into the following calculation formula to obtain the L, a, and b channels of each pixel.

[0199] a=(exp 10(L / 100-1)-1) / 0.0764.

[0200] b = sqrt(C 2 -(1-sqrt(1-(a / 298.9) 2 ))*C 2 )*sign(a). Among them, sign(a)={1if a>=0,-1if a<0}.

[0201] Where L is the L channel of each pixel in the image, C is the C channel of each pixel in the image, and H is the H channel of each pixel in the image. a is the a channel of each pixel in the image converted to Lab format. b is the b channel of each pixel in the image converted to Lab format.

[0202] Next, the phone converts the image from the Lab color space to the XYZ color space. Specifically, the phone can use the following calculation method to substitute the L, a, and b channels of each pixel in the image into the following calculation formula to obtain the X, Y, and Z channels of each pixel.

[0203] Step 1: The phone performs linear transformations on the L, a, and b channels of each pixel in the image. Specifically, y = (L + 16) / 116; x = a / 500 + y; z = yb / 200.

[0204] Step 2: The phone performs a nonlinear transformation on the x, y, and z values ​​obtained in Step 1.

[0205] Nonlinear transformation:

[0206] For example, if the x-channel of a pixel is greater than Then F(x) is (x channel) If the x channel is less than Then F(x) is aisle.

[0207] Step 3: The phone performs inverse normalization on F(x), F(y), and F(z) obtained in step 2. X = F(x) * Xn², Y = F(Y) * Yn², Z = F(Z) * Zn². Where Xn² is the inverse normalization parameter for the X channel, Yn² is the inverse normalization parameter for the Y channel, and Zn² is the inverse normalization parameter for the Z channel. For example, the values ​​of Xn², Yn², and Zn² are 0.95047, 1.0, and 1.08883, respectively.

[0208] Next, the phone converts the image from the XYZ color space to the RGB color space. Specifically, the phone can use the following calculation method to substitute the X, Y, and Z channels of each pixel in the image into the following calculation formula to obtain the R, G, and B channels of each pixel.

[0209] [RGB] = [XYZ](M T RGB2XYZ ) -1 Among them, M RGB2XYZ M is the first transformation matrix for converting RGB color space values ​​to XYZ color space. T RGB2XYZ This is the transpose of the first transformation matrix. (M) T RGB2XYZ ) -1 For M T RGB2XYZ The inverse matrix.

[0210] For example,

[0211] Then, the phone converts the image from the RGB color space to the srgb color space.

[0212] The first step is for the phone to perform gamma correction on the RGB format image. Specifically, the phone substitutes the R, G, and B channels of each pixel in the RGB format image into the following formula to obtain the corrected f(R), f(G), and f(B).

[0213]

[0214] For example, if the R channel of a pixel is greater than 0.0031308, then f(R) is 1.055*R. 1 / 2.4 -0.055. If the x channel is less than 0.0031308, then f(R) is 12.92*R.

[0215] The second step is to trim the f(R), f(G), and f(B) obtained in the first step.

[0216]

[0217] The third step is to denormalize f(R)`, f(G)`, and f(B)` obtained in the second step.

[0218] Specifically, r = f(R) * 255. g = f(G) * 255. b = f(B) * 255.

[0219] In this way, the phone can convert LCH format images to sRGB format. The phone can also use the same method to convert Lab format standard color charts to sRGB format.

[0220] Optionally, the first image and the color-corrected third image can be used to train a neural network model. The input to the trained neural network model can be an image including the target subject, and the output of the neural network model can be an image of the target subject without oversaturation, loss of detail, or blur.

[0221] Optionally, the second image and the color-corrected third image can be used to train a neural network model. The input to the trained neural network model can be an image of the target subject that is oversaturated, lacks detail, and is blurred, and the output of the neural network model can be an image of the target subject that is not oversaturated, lacks detail, or is blurred.

[0222] This application provides an electronic device including a memory, a display screen, and one or more processors. The display screen is coupled to the processors. The memory stores computer program code. The computer program code includes computer instructions. When the processor executes the computer instructions, the electronic device can perform various functions or steps performed by the mobile phone in the above method embodiments. The structure of the electronic device can be referred to... Figure 1 The structure of the electronic device 100 shown.

[0223] This application embodiment also provides a computer storage medium, which includes computer instructions, when the computer instructions are executed in the aforementioned electronic device (such as...). Figure 1 When the electronic device 100 shown is run, it causes the electronic device to perform the various functions or steps in the above method embodiments.

[0224] This application also provides a computer program product that, when run on a computer, causes the computer to perform the various functions or steps described in the above method embodiments.

[0225] This application also provides a chip system including at least one processor and at least one interface circuit. The processor and the interface circuit are interconnected via lines. For example, the interface circuit can be used to receive signals from other devices (e.g., the memory of an electronic device). As another example, the interface circuit can be used to send signals to other devices (e.g., the processor). Exemplarily, the interface circuit can read instructions stored in the memory and send the instructions to the processor. When the instructions are executed by the processor, the electronic device can perform the steps in the above embodiments. Of course, the chip system may also include other discrete devices, and this application does not specifically limit this.

[0226] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0227] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0228] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0229] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0230] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0231] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image processing method, characterized in that, Applied to an electronic device, the electronic device including a camera, the method includes: Acquire the first image captured by the camera; If the second image obtained by color correction of the first image using the first color correction matrix has color overflow, the first image is color corrected using the second color correction matrix. The second color correction matrix is ​​determined based on the first color chart and the second color chart; the saturation of the color block corresponding to the target color block in the first color chart is lower than the saturation of the target color block in the standard color chart; the target color block is a color block in the standard color chart whose similarity to the overflow color in the second image meets the condition; the second color chart is used to represent the RAW image captured by the camera for the standard color chart; the first color correction matrix is ​​determined based on the standard color chart.

2. The method according to claim 1, characterized in that, The method further includes: The first image is color-corrected using the first color correction matrix to obtain the second image; The target pixels in the second image are counted; the target pixels are those in each color channel whose color channel value is greater than a first value; and / or those in each color channel whose color channel value is less than a second value. If the number of target pixels meets the first preset condition, it is determined that the second image has color overflow.

3. The method according to claim 1 or 2, characterized in that, Before performing color correction on the first image using the second color correction matrix, the method further includes: Based on the target pixels in the second image, the target color block in the standard color chart is determined; Reduce the saturation of the target color block in the standard color chart to obtain the first color chart; The second color correction matrix is ​​determined based on the first color chart and the second color chart.

4. The method according to claim 3, characterized in that, The step of determining the target color block in the standard color chart based on the target pixels in the second image includes: Based on the target pixel, determine the overflow color in the second image; The target color block in the standard color chart is determined based on the similarity between the overflow color in the second image and the color block in the standard color chart.

5. The method according to claim 4, characterized in that, The target pixel's color channels include an R channel, a G channel, and a B channel; the first image includes at least one target object, and the saturation of the target object satisfies a second preset condition; Determining the overflow color in the second image based on the target pixel includes: When the number of target objects is equal to 1, the values ​​of the R channel, G channel and B channel of the target pixel are calculated respectively to obtain the values ​​of the R channel, G channel and B channel of the overflow color corresponding to the target object; When the number of target objects is greater than 1, the target pixels are classified to obtain target pixels that correspond one-to-one with the multiple target objects; the R channel, G channel and B channel of the target pixel corresponding to each target object are calculated separately to obtain the R channel, G channel and B channel values ​​of the overflow color corresponding to each target object.

6. The method according to any one of claims 3-5, characterized in that, The step of reducing the saturation of the target color patch in the standard color chart to obtain the first color chart includes: The saturation of the target color block in the standard color chart is adjusted according to a coefficient less than 1. The coefficient is proportional to the similarity between the overflow color in the second image and the target color block.

7. The method according to claim 6, characterized in that, The coefficient corresponding to the target color block is the proportion of the similarity between the target color block and the overflowing color in the second image to the sum of the similarities between multiple target color blocks and the overflowing colors in the second image.

8. The method according to claim 4 or 5, characterized in that, The step of determining the target color patch in the standard color chart based on the similarity between the overflow color in the second image and the color patch in the standard color chart includes: Determine the distance between the overflowing color in the second image and the color patch in the standard color chart, the distance being used to indicate the similarity between the overflowing color and the color patch, and the distance being inversely proportional to the similarity; The color blocks corresponding to the first few preset distances in the order of increasing distances are taken as the target color blocks.

9. An electronic device, characterized in that, The electronic device includes: a memory, a camera, and one or more processors; the camera, the memory, and the processors are coupled; wherein the memory is used to store computer program code, the computer program code including computer instructions; when the computer instructions are executed by the processor, the electronic device performs the method as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, Includes computer instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1-8.

11. A computer program product, comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method as described in any one of claims 1-8.

Images