Image processing method, device, electronic device and readable storage medium

Abstract

This application discloses an image processing method, apparatus, electronic device, and readable storage medium, belonging to the field of image technology. The image processing method includes: after performing target processing on a first image, determining pixel overflow positions in the first image; obtaining a first pixel value at the pixel overflow position, a second pixel value truncated at the overflow position, and a third pixel value of a non-overflow boundary region; based on the first pixel value, the second pixel value, and the third pixel value, obtaining a first fusion coefficient corresponding to the target brightness difference and a second fusion coefficient corresponding to the overflow degree; wherein the target brightness difference indicates the brightness difference between the pixel overflow position and the non-overflow boundary region; updating the first pixel value based on the first fusion coefficient and the second fusion coefficient to obtain a second image.

Description

Technical Field

[0001] This application belongs to the field of image technology, specifically relating to an image processing method, apparatus, electronic device, and readable storage medium. Background Technology

[0002] Electronic devices are being used more and more frequently in everyday photography, and their applications are becoming increasingly widespread.

[0003] For example, users' demand for photography in scenes such as bars and stages is increasing. However, some stage scenes, such as concerts, plays, dramas, and small theaters for crosstalk, restrict the use of professional equipment such as cameras. Therefore, mobile phones, due to their convenience, have gradually become an important device for photography in these scenes. Because the lighting environment in scenes such as bars and stages differs significantly from that in regular scenes, the image signal processing (ISP) process of electronic devices often results in severe color overflow in these scenes. This leads to a loss of image detail, a decrease in the sense of depth, and an inability to effectively present the atmosphere of the scene.

[0004] It is evident that in related technologies, color overflow occurs during image processing, leading to a loss of detail in the image content. Summary of the Invention

[0005] The purpose of this application is to provide an image processing method that can solve the problem of loss of image content details caused by color overflow during image processing.

[0006] In a first aspect, embodiments of this application provide an image processing method, the method comprising: after performing target processing on a first image, determining a pixel overflow position in the first image; obtaining a first pixel value at the pixel overflow position, a second pixel value at the truncated overflow position, and a third pixel value of a non-overflow boundary region; obtaining a first fusion coefficient corresponding to a target brightness difference and a second fusion coefficient corresponding to the degree of overflow based on the first pixel value, the second pixel value, and the third pixel value; wherein the target brightness difference indicates the brightness difference between the pixel overflow position and the non-overflow boundary region; and updating the first pixel value based on the first fusion coefficient and the second fusion coefficient to obtain a second image.

[0007] Secondly, embodiments of this application provide an image processing apparatus, comprising: a determining module, configured to determine a pixel overflow position in the first image after target processing; an acquiring module, configured to acquire a first pixel value at the pixel overflow position, a second pixel value at the truncated overflow position, and a third pixel value of a non-overflow boundary region; a processing module, configured to obtain a first fusion coefficient corresponding to a target brightness difference and a second fusion coefficient corresponding to the degree of overflow based on the first pixel value, the second pixel value, and the third pixel value; wherein the target brightness difference indicates the brightness difference between the pixel overflow position and the non-overflow boundary region; and a first updating module, configured to update the first pixel value based on the first fusion coefficient and the second fusion coefficient to obtain a second image.

[0008] Thirdly, embodiments of this application provide an electronic device including a processor and a memory, wherein the memory stores a program or instructions that can run on the processor, and the program or instructions, when executed by the processor, implement the steps described in the first aspect.

[0009] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0010] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect.

[0011] In a sixth aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method described in the first aspect.

[0012] In the embodiments of this application, after target processing of the first image, the pixel overflow position in the first image is determined, and the initial pixel value of the pixel overflow position is retained to obtain the corresponding first pixel value. After truncating the initial pixel value of the pixel overflow position, the corresponding second pixel value can be obtained. After truncation, the corresponding third pixel value is determined in the non-overflow boundary area. Then, based on the first pixel value, the second pixel value, and the third pixel value, a first fusion coefficient corresponding to the brightness difference between the pixel overflow position and the non-overflow boundary area, and a second fusion coefficient corresponding to the degree of overflow are obtained. Thus, the first pixel value is updated based on the first fusion coefficient and the second fusion coefficient, and the updated first pixel value corresponds to the second image. It can be seen that, based on the embodiments of this application, the brightness in the overflow state and the truncated state are fused, thereby restoring the detail loss caused by overflow while maintaining the hue, and also preventing artifacts and improving image quality. Attached Figure Description

[0013] Figure 1 This is one of the flowcharts of image processing methods according to some embodiments of this application;

[0014] Figure 2 This is an illustrative diagram illustrating some embodiments of the image processing methods of this application;

[0015] Figure 3 This is a second flowchart of an image processing method according to some embodiments of this application;

[0016] Figure 4 This is a block diagram of an image processing apparatus according to some embodiments of this application;

[0017] Figure 5 These are schematic diagrams of the structure of electronic devices according to some embodiments of this application;

[0018] Figure 6 These are schematic diagrams of the hardware structure of electronic devices according to some embodiments of this application. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0020] The terms "first," "second," etc., used in this specification are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class, without limiting the number of objects; for example, there may be one or more target objects. Furthermore, in the specification, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0021] The image processing method provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0022] like Figure 1 The diagram illustrates a flowchart of an image processing method according to an embodiment of this application. Taking the application of this method to an electronic device as an example, the method includes:

[0023] Step 110: After performing target processing on the first image, determine the pixel overflow position in the first image.

[0024] For example, in the first image after target processing, the actual pixel value at each position is divided by the maximum pixel value corresponding to that bit depth to obtain a normalized result. For ease of distinction, the normalized value here is defined as the initial pixel value. Taking an 8-bit bit depth as an example, the corresponding maximum pixel value is: 2. 8 -1 = 255. If the initial pixel value is greater than 1 or less than 0, it corresponds to the pixel overflow position.

[0025] Step 120: Obtain the first pixel value at the pixel overflow position, the second pixel value at the overflow position, and the third pixel value of the non-overflow boundary area.

[0026] First, retain the first pixel value at the pixel overflow position. This first pixel value is obtained based on the initial pixel value at the pixel overflow position.

[0027] Then, the initial pixel value at the pixel overflow position is truncated, and the second pixel value is obtained based on the truncated initial pixel value. For reference, the truncation method is as shown in formula (1):

[0028] I c =max (min(I o ,1),0) (1)

[0029] In formula (1), I o Indicates the initial pixel value, I c This represents the initial pixel value at which the cutoff point was made.

[0030] As can be seen, based on formula (1), pixel values ​​greater than 1 or less than 0 are truncated.

[0031] It should be noted that for the pixel overflow position in the first image, the value before truncation corresponds to the first pixel value, and the value after truncation corresponds to the second pixel value.

[0032] Furthermore, referring to formula (2):

[0033]

[0034] In formula (2), R represents the red channel value, G represents the green channel value, B represents the blue channel value, and I represents the green channel value. b This represents the pixel value of the truncated region that has not overflowed the boundary. This pixel value is also a normalized value. After truncating each of the three channels, the three color channel values ​​of the truncated region that has not overflowed the boundary are either 0 or 1, thus obtaining the third pixel value based on the pixel value of the region that has not overflowed the boundary.

[0035] Step 130: Based on the first pixel value, the second pixel value, and the third pixel value, obtain the first fusion coefficient corresponding to the target brightness difference and the second fusion coefficient corresponding to the overflow degree. The target brightness difference indicates the brightness difference between the overflow position and the non-overflow boundary area.

[0036] In this step, firstly, a first fusion coefficient corresponding to the target brightness difference is calculated, whereby the target brightness difference indicates the brightness difference between the pixel overflow position and the non-overflow boundary area; secondly, a second fusion coefficient corresponding to the overflow degree is calculated.

[0037] In some embodiments, the smaller the brightness difference between the pixel overflow location and the non-overflow boundary area, the closer the brightness of the pixel overflow location and the non-overflow boundary area are, and the greater the weight of the brightness of the pixel overflow location in the subsequent fusion algorithm.

[0038] In some embodiments, the degree of overflow is measured by the difference between the first pixel value and the second pixel value. The greater the degree of overflow, the greater the weight of the brightness of the overflowing pixel location in the subsequent fusion algorithm.

[0039] Step 140: Update the first pixel value based on the first fusion coefficient and the second fusion coefficient to obtain the second image.

[0040] In this step, a weighted fusion is calculated based on the degree of overflow and the brightness distribution at the overflow position of the pixel to fuse the brightness in the overflow state and the truncated state. Then, the first pixel value is updated according to the fusion result, so as to restore the detail loss caused by overflow while preventing artifacts.

[0041] In this process, the second image is obtained when the first pixel value is updated.

[0042] In the embodiments of this application, after target processing of the first image, the pixel overflow position in the first image is determined, and the initial pixel value of the pixel overflow position is retained to obtain the corresponding first pixel value. After truncating the initial pixel value of the pixel overflow position, the corresponding second pixel value can be obtained. After truncation, the corresponding third pixel value is determined in the non-overflow boundary area. Then, based on the first pixel value, the second pixel value, and the third pixel value, a first fusion coefficient corresponding to the brightness difference between the pixel overflow position and the non-overflow boundary area, and a second fusion coefficient corresponding to the degree of overflow are obtained. Thus, the first pixel value is updated based on the first fusion coefficient and the second fusion coefficient, and the updated first pixel value corresponds to the second image. It can be seen that, based on the embodiments of this application, the brightness in the overflow state and the truncated state are fused, thereby restoring the detail loss caused by overflow while maintaining the hue, and also preventing artifacts and improving image quality.

[0043] In another embodiment of the image processing method of this application, step 120 includes:

[0044] Sub-step A1: Based on the scene brightness, scale the pixel value at the pixel overflow position to obtain the first pixel value.

[0045] Sub-step A2: Based on the scene brightness, scale the pixel value at the truncated overflow position to obtain the second pixel value.

[0046] Sub-step A3: Based on the scene brightness, scale the pixel values ​​of the area that has not overflowed the boundary to obtain the third pixel value.

[0047] In some embodiments, an ambient light sensor is used to collect the scene brightness L.

[0048] It should be noted that the process of converting linear RGB space to Jzazbz uniform color space is as follows: RGB→XYZ→LMS→Jzazbz. The RGB→XYZ transformation is completed through linear transformation. Therefore, by multiplying the normalized RGB value by the scene brightness, after conversion to XYZ space, Y represents the average brightness of the scene, which corresponds to its physical meaning, rather than the normalized value. This allows for a more realistic reproduction of the scene's color distribution in Jzazbz space.

[0049] In some embodiments, the scaling method is described in formula (3):

[0050] I = L × I (3)

[0051] In formula (3), I on the right side of the formula represents the pixel value before scaling, which is a normalized value, and L represents the scene brightness. I on the left side of the formula represents the pixel value after scaling, which can represent the first pixel value, the second pixel value, and the third pixel value, respectively.

[0052] For example, multiply the initial pixel value at the pixel overflow position by the scene brightness to obtain the first pixel value; truncate the initial pixel value and multiply it by the scene brightness to obtain the second pixel value; finally, multiply the pixel value of the region that has not overflowed the boundary by the scene brightness to obtain the third pixel value.

[0053] In this embodiment, the normalized value is scaled based on the scene brightness measured by the ambient light sensor so that the scaling result can be converted to a uniform color space to accurately restore the color distribution of the scene and ensure that the brightness is the same as that perceived by the human eye.

[0054] In another embodiment of the image processing method of this application, after the step of updating the first pixel value based on the first fusion coefficient and the second fusion coefficient, the method further includes:

[0055] Step B1: Update the pixel value at the pixel overflow position based on the scene brightness and the updated first pixel value.

[0056] In this embodiment, the inverse operation of formula (3) is used to restore the result to the normalized form before scaling. The inverse operation of formula (3) is referenced in formula (4):

[0057] I m =L a ×I m (4)

[0058] In formula (4), I on the right side of the formula m L represents the updated first pixel value. a The value I represents the reciprocal of the scene brightness, i.e., 1 / L. m This represents the update result in normalized form corresponding to the pixel overflow position.

[0059] Furthermore, the updated pixel value at the pixel overflow position is a normalized value, which, when multiplied by 255, yields the color channel values ​​at the pixel overflow position.

[0060] In this embodiment, after scaling the normalized pixel value based on the scene brightness, it is necessary to use an inverse operation to eliminate this scaling effect so that it can be finally restored to the values ​​of each color channel.

[0061] In another embodiment of the image processing method of this application, step 130 includes:

[0062] Sub-step C1: Based on the first pixel value, obtain the brightness, red-green color components, and yellow-blue color components corresponding to the first pixel value; based on the second pixel value, obtain the brightness, red-green color components, and yellow-blue color components corresponding to the second pixel value; based on the third pixel value, obtain the brightness, red-green color components, and yellow-blue color components corresponding to the third pixel value.

[0063] In this step, mapping the first pixel value to the Jzazbz uniform color space yields the brightness J corresponding to the first pixel value. o Red and green color components a o and yellow and blue color components b o Mapping the second pixel value to the Jzazbz uniform color space yields the second brightness J corresponding to that pixel value. c Red and green color components a c and yellow and blue color components b c Mapping the third pixel value to the Jzazbz uniform color space yields the third brightness J corresponding to the third pixel value. b Red and green color components a b and yellow and blue color components b b .

[0064] Furthermore, based on the red-green color component a o and yellow and blue color components b o Calculate the chroma C corresponding to the first pixel value. o and hue H o According to the red-green color component a c and yellow and blue color components b c Calculate the chroma C corresponding to the second pixel value. c and hue H c According to the red-green color component a c and yellow and blue color components b c Calculate the chroma C corresponding to the third pixel value. b and hue H b .

[0065] Among them, the Jzazbz uniform color space, proposed in 2017, is suitable for illuminance ranging from 0.001 to 10000 cd / m². 2 High dynamic range scenes with 3-component brightness J z Red and green color components a z Yellow and blue color components b z Based on this, the lightness J, chroma C, and hue H can be calculated.

[0066] In some embodiments, the Jzazbz uniform color space offers better hue preservation and color uniformity, but the mapping between RGB and Jzazbz spaces is relatively complex. Other uniform color spaces with simpler computations, such as Oklab, can be used instead.

[0067] Sub-step C2: Determine the first brightness based on the hue corresponding to the third pixel value and the preset correspondence of the non-overflowing boundary area. The hue corresponding to the third pixel value is determined based on the red-green and yellow-blue color components of the third pixel value.

[0068] In the region that does not overflow the boundary, the lightness J, hue H, and chroma C corresponding to the third pixel value of this region are represented by a three-dimensional look-up table (LUT) with J and H equally spaced, using bilinear interpolation. b LUT b The preset correspondence used to describe the non-overflow boundary area is the correspondence between lightness J, hue H, and chroma C.

[0069] It should be noted that since the un-overflowed boundary region is a two-dimensional surface in a uniform color space, converting it into a three-dimensional (3D) LUT can be achieved by taking equally spaced values ​​for two of its dimensions, namely lightness (J) and hue (H), and then calculating the value of the third dimension, chroma (C), at the corresponding position. In RGB space, the color gamut boundary is represented as a set of equally spaced points, but after mapping it to a uniform color space, these points are no longer equally spaced. Two-dimensional interpolation is needed to obtain equally spaced points. Here, the commonly used two-dimensional interpolation method, bilinear interpolation, is chosen.

[0070] In this step, based on the hue corresponding to the third pixel value, in the LUT b The first brightness level is determined in the middle.

[0071] Sub-step C3: According to Calculate the first fusion coefficient corresponding to the difference in target brightness. Where R1 represents the first fusion coefficient, J... o J represents the brightness corresponding to the first pixel value. cusp Indicates the first brightness level.

[0072] In some embodiments, refer to formula (5):

[0073]

[0074] In formula (5), R1 represents the first fusion coefficient, J o J represents the brightness corresponding to the first pixel value. cusp Indicates the first brightness level.

[0075] The purpose of formula (5) is to calculate the first fusion coefficient based on the difference between the brightness corresponding to the first pixel value and the first brightness, so as to ensure that the first fusion coefficient is larger only when the brightness corresponding to the first pixel value is close to the first brightness. Correspondingly, in the fusion algorithm, the brightness corresponding to the first pixel value has a larger weight.

[0076] Sub-step C4: According to Calculate the second fusion coefficient corresponding to the degree of overflow. Where R² represents the second fusion coefficient, J... o J represents the brightness corresponding to the first pixel value. c a represents the brightness corresponding to the second pixel value. o a represents the red and green color components corresponding to the first pixel value. c b represents the red and green color components corresponding to the second pixel value. o b represents the yellow and blue color components corresponding to the first pixel value. c J represents the yellow and blue color components corresponding to the second pixel value. b a represents the brightness corresponding to the third pixel value. b b represents the red and green color components corresponding to the third pixel value. b This represents the yellow and blue color components corresponding to the third pixel value.

[0077] In some embodiments, refer to formula (6):

[0078]

[0079] In formula (6), R2 represents the second fusion coefficient, J o J represents the brightness corresponding to the first pixel value. c a represents the brightness corresponding to the second pixel value. o a represents the red and green color components corresponding to the first pixel value. c b represents the red and green color components corresponding to the second pixel value. o b represents the yellow and blue color components corresponding to the first pixel value. c J represents the yellow and blue color components corresponding to the second pixel value. b a represents the brightness corresponding to the third pixel value. b b represents the red and green color components corresponding to the third pixel value. b This represents the yellow and blue color components corresponding to the third pixel value.

[0080] The purpose of formula (6) is to calculate the normalized (J) o a o b o ) and (J c ,ac,b cThe Euclidean distance between the two pixels is used to measure the degree of overflow. Based on this, the second fusion coefficient is calculated. In the fusion algorithm, the weight of the brightness corresponding to the first pixel value is positively correlated with the degree of overflow.

[0081] In this embodiment, within the preset correspondence of the non-overflowing boundary region, a larger brightness corresponds to a smaller chroma. If the first pixel value corresponds to a larger brightness, it is mapped to the preset correspondence, i.e., mapped to the LUT. b [int(J o )][int(H o This will cause a loss of chroma, resulting in artifacts. Therefore, in this embodiment, the luminance of pixels with severe overflow near the first luminance is updated. Specifically, a second fusion coefficient is calculated based on the overflow degree of the pixel overflow position, and a first fusion coefficient is calculated based on the luminance distribution of the pixel overflow position. The first and second fusion coefficients are used in subsequent luminance fusion, and the fused luminance is used to update the luminance corresponding to the first pixel value.

[0082] In the flow of the image processing method according to another embodiment of this application, step C2 includes:

[0083] Sub-step D1: Determine the maximum chroma corresponding to the third pixel value based on the preset correspondence between the hue corresponding to the third pixel value and the non-overflow boundary area.

[0084] In this step, the brightness J is based on the value of the third pixel. b The chroma C corresponding to the third pixel value b For the hue Hb corresponding to the third pixel value, bilinear interpolation is used to characterize the region that has not overflowed the boundary as a three-dimensional lookup table (LUT) with equally spaced lightness J and hue H. b For three-dimensional lookup table (LUT) b For each hue H in the spectrum, the corresponding chroma C is counted to find the maximum chroma.

[0085] Taking a hue=70 hue page as an example, as the brightness J changes from small to large, the actual possible color boundary C increases from small to large and then decreases again. Therefore, a larger J corresponds to a smaller chroma C.

[0086] Sub-step D2: Determine the first brightness based on the preset correspondence between the maximum chroma and the unoverflowed boundary area.

[0087] In this step, for the three-dimensional lookup table (LUT) b Find the lightness J corresponding to the maximum chroma, and denote it as J. cusp That is, the first brightness.

[0088] In this embodiment, the first brightness is determined based on the maximum chroma, so that the first brightness can be used in the calculation of the first fusion coefficient.

[0089] In another embodiment of the image processing method of this application, step 140 includes:

[0090] Sub-step E1: Calculate the second brightness according to formula (7).

[0091] Formula (7):

[0092]

[0093] In formula (7), R represents the fusion coefficient, R1 represents the first fusion coefficient, R2 represents the second fusion coefficient, and J... o J represents the brightness corresponding to the first pixel value. c J represents the brightness corresponding to the second pixel value. m Indicates the second brightness level.

[0094] Sub-step E2: Determine the chroma corresponding to the first pixel value based on the second brightness, the hue corresponding to the first pixel value, and the preset correspondence of the non-overflow boundary area. The hue corresponding to the first pixel value is determined based on the red-green and yellow-blue color components corresponding to the first pixel value.

[0095] In this step, for the pixel overflow position, the hue of that position is retained, that is, the hue corresponding to the first pixel value is used, and the calculated second brightness and the hue corresponding to the first pixel value are mapped to the preset correspondence of the non-overflow boundary area to determine the chroma corresponding to the first pixel value. Here, the chroma corresponding to the first pixel value is determined, that is, the updated chroma.

[0096] Please refer to formula (8):

[0097] C m =LUT b [int(J m )][int(H o (8)

[0098] In formula (8), C m J represents the chroma corresponding to the updated first pixel value. m H represents the second brightness. o This represents the hue corresponding to the first pixel value.

[0099] Sub-step E3: Update the first pixel value based on the second brightness, the hue corresponding to the first pixel value, and the chroma corresponding to the first pixel value.

[0100] In this step, (J) m C m H o) as the color component representation of the overflow image processing result, based on (J m C m H o Get the updated first pixel value.

[0101] In this embodiment, the hue corresponding to the first pixel value is retained, the brightness corresponding to the first pixel value is updated to the second brightness, and the chroma corresponding to the first pixel value is updated. Specifically, the chroma at the overflow position is mapped to the color gamut boundary. While preserving the hue, the overflow portion is mapped to the non-overflow portion, thus restoring the image content loss caused by color overflow.

[0102] In another embodiment of the image processing method of this application, the target processing includes any one of color correction processing, white balance processing, and three-dimensional lookup table processing.

[0103] The color overflow prevention processing in this application can be performed by the color overflow prevention module. This processing can be added after color correction, white balance, 3DLUT and other processing, because color correction, white balance, 3DLUT and other processing may cause color overflow.

[0104] The application scenarios of this application are as follows: Figure 2 The image on the left is an image that has not undergone the color overflow prevention processing described in this application. Figure 2 The image on the right is an image processed by this application to prevent color overflow, which effectively avoids the loss of image details.

[0105] like Figure 3 As shown, in one embodiment of this application, the color correction (CC) processing result is used as input, specifically including retaining overflow I. o ,Truncation of overflow c and overflow boundary I b The input is scaled based on the illuminance measured by the sensor and then converted to the Jzazbz space to calculate the color attributes lightness (J), chroma (C), and hue (H). Among these, the overflow boundary I... b Color attributes, generate LUT b Used for first brightness Jcus p In the calculation, the second brightness J is obtained through fusion. m Then, the second brightness J m Mapping to LUT b Calculate chroma C m Output fusion processing result (J) m C m H o Finally, it is converted to RGB space to eliminate illuminance-based scaling.

[0106] In summary, this application provides a method for preventing color overflow based on a uniform color space. It updates the brightness of the overflowing pixel location in the uniform color space and then maps it to the color gamut boundary corresponding to the brightness and hue. This allows for rapid and efficient recovery of details in the overflowed area while maintaining hue and overall brightness, restoring image content loss caused by color overflow, enhancing image depth, and better recreating the atmosphere of stage and bar scenes. Furthermore, this application can also retain the initial hue of the overflowing location and, combined with the fused brightness, map the chroma of the overflowing location to the color gamut boundary. This allows for mapping the overflowing part to the non-overflowing part while maintaining hue, thus restoring image content loss caused by color overflow. Additionally, this application treats the overflowing and non-overflowing parts as the original and target color gamuts, respectively. A color gamut mapping algorithm maps the overflowing part to the non-overflowing part, thereby restoring image details and enhancing image depth to achieve the effect of preventing color overflow. In this process, this application characterizes the boundaries of the original and target color gamuts to define the boundary of the overflowing part and proposes a computationally efficient color gamut mapping algorithm that can be implemented in real-time on platforms such as mobile phones.

[0107] The image processing method provided in this application can be executed by an image processing device. This application uses an image processing device executing the image processing method as an example to illustrate the image processing device provided in this application.

[0108] Figure 4 A block diagram of an image processing apparatus according to an embodiment of this application is shown. The apparatus includes:

[0109] The determination module 10 is used to determine the pixel overflow position in the first image after target processing of the first image;

[0110] The acquisition module 20 is used to acquire the first pixel value at the pixel overflow position, the second pixel value at the truncated overflow position, and the third pixel value of the non-overflow boundary region;

[0111] The processing module 30 is used to obtain a first fusion coefficient corresponding to the target brightness difference and a second fusion coefficient corresponding to the overflow degree based on the first pixel value, the second pixel value and the third pixel value; wherein, the target brightness difference indicates the brightness difference between the pixel overflow position and the non-overflow boundary area;

[0112] The first update module 40 is used to update the first pixel value based on the first fusion coefficient and the second fusion coefficient to obtain the second image.

[0113] In the embodiments of this application, after target processing of the first image, the pixel overflow position in the first image is determined, and the initial pixel value of the pixel overflow position is retained to obtain the corresponding first pixel value. After truncating the initial pixel value of the pixel overflow position, the corresponding second pixel value can be obtained. After truncation, the corresponding third pixel value is determined in the non-overflow boundary area. Then, based on the first pixel value, the second pixel value, and the third pixel value, a first fusion coefficient corresponding to the brightness difference between the pixel overflow position and the non-overflow boundary area, and a second fusion coefficient corresponding to the degree of overflow are obtained. Thus, the first pixel value is updated based on the first fusion coefficient and the second fusion coefficient, and the updated first pixel value corresponds to the second image. It can be seen that, based on the embodiments of this application, the brightness in the overflow state and the truncated state are fused, thereby restoring the detail loss caused by overflow while maintaining the hue, and also preventing artifacts and improving image quality.

[0114] In some embodiments, the acquisition module 20 includes:

[0115] The first scaling unit is used to scale the pixel value at the pixel overflow position according to the scene brightness to obtain the first pixel value;

[0116] The second scaling unit is used to scale the pixel value at the truncated overflow position according to the scene brightness to obtain the second pixel value;

[0117] The third scaling unit is used to scale the pixel values ​​of the area that has not overflowed the boundary according to the scene brightness, so as to obtain the third pixel value.

[0118] In some embodiments, the device further includes:

[0119] The second update module is used to update the pixel value at the pixel overflow position based on the scene brightness and the updated first pixel value.

[0120] In some embodiments, the processing module 30 includes:

[0121] The acquisition unit is used to acquire the brightness, red-green color components, and yellow-blue color components corresponding to the first pixel value based on the first pixel value; acquire the brightness, red-green color components, and yellow-blue color components corresponding to the second pixel value based on the second pixel value; and acquire the brightness, red-green color components, and yellow-blue color components corresponding to the third pixel value based on the third pixel value.

[0122] The first determining unit is used to determine the first brightness based on the hue corresponding to the third pixel value and a preset correspondence relationship of the non-overflow boundary area; wherein the hue corresponding to the third pixel value is determined based on the red-green color component and the yellow-blue color component corresponding to the third pixel value.

[0123] The first calculation unit is used to calculate based on Calculate the first fusion coefficient corresponding to the difference in target brightness; where R1 represents the first fusion coefficient, J o J represents the brightness corresponding to the first pixel value. cusp Indicates the first value;

[0124] The second calculation unit is used to calculate based on Calculate the second fusion coefficient corresponding to the degree of overflow; where R2 represents the second fusion coefficient, J o J represents the brightness corresponding to the first pixel value. c a represents the brightness corresponding to the second pixel value. o a represents the red and green color components corresponding to the first pixel value. c b represents the red and green color components corresponding to the second pixel value. o b represents the yellow and blue color components corresponding to the first pixel value. c J represents the yellow and blue color components corresponding to the second pixel value. b a represents the brightness corresponding to the third pixel value. b b represents the red and green color components corresponding to the third pixel value. b This represents the yellow and blue color components corresponding to the third pixel value.

[0125] In some embodiments, the first determining unit includes:

[0126] The first determining subunit is used to determine the maximum chroma corresponding to the third pixel value based on the preset correspondence between the hue corresponding to the third pixel value and the non-overflow boundary area;

[0127] The second determining sub-unit is used to determine the first brightness based on the preset correspondence between the maximum chroma and the unoverflowed boundary area.

[0128] In some embodiments, the first update module 40 includes:

[0129] The third calculation unit is used to calculate based on Calculate the second brightness; where R represents the blending coefficient, R1 represents the first blending coefficient, R2 represents the second blending coefficient, and J... o J represents the brightness corresponding to the first pixel value. c J represents the brightness corresponding to the second pixel value. m Indicates the second lightness;

[0130] The second determining unit is used to determine the chroma corresponding to the first pixel value based on the second brightness, the hue corresponding to the first pixel value, and a preset correspondence between the non-overflow boundary area; wherein, the hue corresponding to the first pixel value is determined based on the red-green color component and the yellow-blue color component corresponding to the first pixel value;

[0131] The update unit is used to update the first pixel value based on the second brightness, the hue corresponding to the first pixel value, and the chroma corresponding to the first pixel value.

[0132] In some embodiments, target processing includes any one of color correction processing, white balance processing, and three-dimensional lookup table processing.

[0133] The device in the embodiments of this application can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television set (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the scope of the device.

[0134] The apparatus in this application embodiment can be an apparatus with an action system. The action system can be an Android action system, an iOS action system, or other possible action systems, and this application embodiment does not specifically limit it.

[0135] The apparatus provided in this application embodiment can implement the various processes implemented in the above method embodiments and achieve the same technical effects. To avoid repetition, it will not be described again here.

[0136] In some embodiments, such as Figure 5 As shown, this application embodiment also provides an electronic device 100, including a processor 101, a memory 102, and a program or instructions stored in the memory 102 and executable on the processor 101. When the program or instructions are executed by the processor 101, they implement the various steps of any of the above-described image processing method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0137] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0138] Figure 6 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0139] The electronic device 1000 includes, but is not limited to, the following components: radio frequency unit 1001, network module 1002, audio output unit 1003, input unit 1004, sensor 1005, display unit 1006, user input unit 1007, interface unit 1008, memory 1009, processor 1010, camera 1011, etc.

[0140] Those skilled in the art will understand that the electronic device 1000 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 1010 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 6 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0141] The processor 1010 is configured to, after performing target processing on the first image, determine the pixel overflow position in the first image; obtain a first pixel value at the pixel overflow position, a second pixel value at the truncated overflow position, and a third pixel value in the non-overflow boundary region; based on the first pixel value, the second pixel value, and the third pixel value, obtain a first fusion coefficient corresponding to the target brightness difference and a second fusion coefficient corresponding to the overflow degree; wherein the target brightness difference indicates the brightness difference between the pixel overflow position and the non-overflow boundary region; and update the first pixel value based on the first fusion coefficient and the second fusion coefficient to obtain a second image.

[0142] In the embodiments of this application, after target processing of the first image, the pixel overflow position in the first image is determined, and the initial pixel value of the pixel overflow position is retained to obtain the corresponding first pixel value. After truncating the initial pixel value of the pixel overflow position, the corresponding second pixel value can be obtained. After truncation, the corresponding third pixel value is determined in the non-overflow boundary area. Then, based on the first pixel value, the second pixel value, and the third pixel value, a first fusion coefficient corresponding to the brightness difference between the pixel overflow position and the non-overflow boundary area, and a second fusion coefficient corresponding to the degree of overflow are obtained. Thus, the first pixel value is updated based on the first fusion coefficient and the second fusion coefficient, and the updated first pixel value corresponds to the second image. It can be seen that, based on the embodiments of this application, the brightness in the overflow state and the truncated state are fused, thereby restoring the detail loss caused by overflow while maintaining the hue, and also preventing artifacts and improving image quality.

[0143] In some embodiments, the processor 1010 is further configured to scale the pixel value at the pixel overflow position according to the scene brightness to obtain the first pixel value; scale the pixel value at the truncated overflow position according to the scene brightness to obtain the second pixel value; and scale the pixel value of the non-overflow boundary region according to the scene brightness to obtain the third pixel value.

[0144] In some embodiments, the processor 1010 is further configured to update the pixel value at the pixel overflow position based on the scene brightness and the updated first pixel value.

[0145] In some embodiments, the processor 1010 is further configured to: obtain, based on the first pixel value, the brightness, red-green color components, and yellow-blue color components corresponding to the first pixel value; obtain, based on the second pixel value, the brightness, red-green color components, and yellow-blue color components corresponding to the second pixel value; obtain, based on the third pixel value, the brightness, red-green color components, and yellow-blue color components corresponding to the third pixel value; determine, based on the hue corresponding to the third pixel value and in conjunction with a preset correspondence relationship of the non-overflowing boundary area, the first brightness; wherein, the hue corresponding to the third pixel value is determined based on the red-green color components and yellow-blue color components corresponding to the third pixel value; based on... Calculate the first fusion coefficient corresponding to the target brightness difference; where R1 represents the first fusion coefficient, J o J represents the brightness corresponding to the first pixel value. cusp Indicates the first brightness; according to Calculate the second fusion coefficient corresponding to the overflow level; where R2 represents the second fusion coefficient, J o J represents the brightness corresponding to the first pixel value. c a represents the brightness corresponding to the second pixel value. o a represents the red-green color component corresponding to the first pixel value. c b represents the red and green color components corresponding to the second pixel value. o b represents the yellow and blue color components corresponding to the first pixel value. c J represents the yellow-blue color component corresponding to the second pixel value. b a represents the brightness corresponding to the third pixel value. b b represents the red-green color component corresponding to the third pixel value. b This represents the yellow-blue color component corresponding to the third pixel value.

[0146] In some embodiments, the processor 1010 is further configured to determine the maximum chroma corresponding to the third pixel value based on a preset correspondence between the hue corresponding to the third pixel value and the non-overflow boundary area; and to determine the first brightness based on the preset correspondence between the maximum chroma and the non-overflow boundary area.

[0147] In some embodiments, the processor 1010 is further configured to, according to Calculate the second brightness; where R represents the blending coefficient, R1 represents the first blending coefficient, R2 represents the second blending coefficient, and J o J represents the brightness corresponding to the first pixel value. c J represents the brightness corresponding to the second pixel value. m The second brightness is represented; the chroma corresponding to the first pixel value is determined according to the second brightness, the hue corresponding to the first pixel value, and the preset correspondence of the non-overflow boundary area; wherein, the hue corresponding to the first pixel value is determined based on the red-green color component and the yellow-blue color component corresponding to the first pixel value; the first pixel value is updated according to the second brightness, the hue corresponding to the first pixel value, and the chroma corresponding to the first pixel value.

[0148] In some embodiments, the target processing includes any one of color correction processing, white balance processing, and three-dimensional lookup table processing.

[0149] In summary, this application provides a method for preventing color overflow based on a uniform color space. It updates the brightness of the overflowing pixel location in the uniform color space and then maps it to the color gamut boundary corresponding to the brightness and hue. This allows for rapid and efficient recovery of details in the overflowed area while maintaining hue and overall brightness, restoring image content loss caused by color overflow, enhancing image depth, and better recreating the atmosphere of stage and bar scenes. Furthermore, this application can also retain the initial hue of the overflowing location and, combined with the fused brightness, map the chroma of the overflowing location to the color gamut boundary. This allows for mapping the overflowing part to the non-overflowing part while maintaining hue, thus restoring image content loss caused by color overflow. Additionally, this application treats the overflowing and non-overflowing parts as the original and target color gamuts, respectively. A color gamut mapping algorithm maps the overflowing part to the non-overflowing part, thereby restoring image details and enhancing image depth to achieve the effect of preventing color overflow. In this process, this application characterizes the boundaries of the original and target color gamuts to define the boundary of the overflowing part and proposes a computationally efficient color gamut mapping algorithm that can be implemented in real-time on platforms such as mobile phones.

[0150] It should be understood that, in this embodiment, the input unit 1004 may include a graphics processing unit (GPU) 10041 and a microphone 10042. The GPU 10041 processes image data of still images or video images obtained by an image capture device (such as a camera) in video image capture mode or image capture mode. The display unit 1006 may include a display panel 10061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, etc. The user input unit 1007 includes at least one of a touch panel 10071 and other input devices 10072. The touch panel 10071 is also called a touch screen. The touch panel 10071 may include a touch detection device and a touch controller. Other input devices 10072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here. The memory 1009 can be used to store software programs and various data, including but not limited to applications and motion systems. Processor 1010 may integrate an application processor and a modem processor. The application processor mainly handles the action system, user page, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into processor 1010.

[0151] The memory 1009 can be used to store software programs and various data. The memory 1009 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 1009 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 1009 in this embodiment includes, but is not limited to, these and any other suitable types of memory.

[0152] Processor 1010 may include one or more processing units; in some embodiments, processor 1010 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the modem processor may also not be integrated into processor 1010.

[0153] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described image processing method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0154] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0155] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled. The processor is used to run programs or instructions to implement the various processes of the above-described image processing method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0156] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0157] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above-described image processing method embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0158] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0159] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of this application.

[0160] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An image processing method, characterized in that, The method includes: After performing target processing on the first image, determine the pixel overflow position in the first image; Obtain the first pixel value at the pixel overflow position, the second pixel value at the truncated overflow position, and the third pixel value of the non-overflow boundary region; Based on the first pixel value, the second pixel value, and the third pixel value, a first fusion coefficient corresponding to the target brightness difference and a second fusion coefficient corresponding to the overflow degree are obtained; wherein, the target brightness difference indicates the brightness difference between the pixel overflow position and the non-overflow boundary area; The first pixel value is updated based on the first fusion coefficient and the second fusion coefficient to obtain the second image.

2. The method according to claim 1, characterized in that, The step of obtaining the first pixel value at the pixel overflow position, the second pixel value at the truncated overflow position, and the third pixel value of the non-overflow boundary region includes: Based on the scene brightness, the pixel value at the pixel overflow position is scaled to obtain the first pixel value; Based on the scene brightness, the pixel value at the truncated overflow position is scaled to obtain the second pixel value; Based on the scene brightness, the pixel values ​​of the non-overflow boundary area are scaled to obtain the third pixel value.

3. The method according to claim 2, characterized in that, After updating the first pixel value based on the first fusion coefficient and the second fusion coefficient, the method further includes: The pixel value at the pixel overflow position is updated based on the scene brightness and the updated first pixel value.

4. The method according to claim 1, characterized in that, The step of obtaining a first fusion coefficient corresponding to the target brightness difference and a second fusion coefficient corresponding to the overflow degree based on the first pixel value, the second pixel value, and the third pixel value includes: Based on the first pixel value, obtain the brightness, red-green color components, and yellow-blue color components corresponding to the first pixel value; based on the second pixel value, obtain the brightness, red-green color components, and yellow-blue color components corresponding to the second pixel value; based on the third pixel value, obtain the brightness, red-green color components, and yellow-blue color components corresponding to the third pixel value. The first brightness is determined based on the hue corresponding to the third pixel value and the preset correspondence of the non-overflow boundary area; wherein the hue corresponding to the third pixel value is determined based on the red-green color component and the yellow-blue color component corresponding to the third pixel value. according to Calculate the first fusion coefficient corresponding to the target brightness difference; where R1 represents the first fusion coefficient, J o J represents the brightness corresponding to the first pixel value. cusp Indicates the first brightness; according to Calculate the second fusion coefficient corresponding to the overflow level; where R2 represents the second fusion coefficient, J o J represents the brightness corresponding to the first pixel value. c a represents the brightness corresponding to the second pixel value. o a represents the red-green color component corresponding to the first pixel value. c b represents the red and green color components corresponding to the second pixel value. o b represents the yellow and blue color components corresponding to the first pixel value. c J represents the yellow-blue color component corresponding to the second pixel value. b a represents the brightness corresponding to the third pixel value. b b represents the red-green color component corresponding to the third pixel value. b This represents the yellow-blue color component corresponding to the third pixel value.

5. The method according to claim 4, characterized in that, The step of determining the first brightness based on the hue corresponding to the third pixel value and in conjunction with the preset correspondence relationship of the non-overflowing boundary area includes: The maximum chroma corresponding to the third pixel value is determined based on the preset correspondence between the hue corresponding to the third pixel value and the non-overflow boundary area; The first brightness is determined based on the preset correspondence between the maximum chroma and the unoverflowed boundary area.

6. The method according to claim 4, characterized in that, Updating the first pixel value based on the first fusion coefficient and the second fusion coefficient includes: according to Calculate the second brightness; where R represents the blending coefficient, R1 represents the first blending coefficient, R2 represents the second blending coefficient, and J o J represents the brightness corresponding to the first pixel value. c J represents the brightness corresponding to the second pixel value. m Indicates the second brightness; The chroma corresponding to the first pixel value is determined based on the second brightness, the hue corresponding to the first pixel value, and the preset correspondence of the non-overflow boundary area; wherein, the hue corresponding to the first pixel value is determined based on the red-green color component and the yellow-blue color component corresponding to the first pixel value; The first pixel value is updated based on the second brightness, the hue corresponding to the first pixel value, and the chroma corresponding to the first pixel value.

7. The method according to any one of claims 1 to 6, characterized in that, The target processing includes color correction processing.

8. An image processing apparatus, characterized in that, The device includes: The determination module is used to determine the pixel overflow position in the first image after target processing is performed on the first image; The acquisition module is used to acquire the first pixel value at the pixel overflow position, the second pixel value at the truncated overflow position, and the third pixel value of the non-overflow boundary region; The processing module is configured to obtain a first fusion coefficient corresponding to the target brightness difference and a second fusion coefficient corresponding to the overflow degree based on the first pixel value, the second pixel value and the third pixel value; wherein, the target brightness difference indicates the brightness difference between the pixel overflow position and the non-overflow boundary area; The first update module is used to update the first pixel value based on the first fusion coefficient and the second fusion coefficient to obtain the second image.

9. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a program or instructions that can run on the processor, the program or instructions being executed by the processor to implement the steps of the image processing method as described in any one of claims 1 to 7.

10. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the image processing method as described in any one of claims 1 to 7.

Images