Image processing method and device, electronic equipment and storage medium

Abstract

The present disclosure provides an image processing method and device, electronic equipment and storage medium, relates to the technical field of image processing, and particularly relates to the field of computer vision. The specific implementation scheme is: for a pixel in a first image, determining a saturation adjustment weight according to at least one of a hue value, a saturation value and a lightness value of the pixel; determining a target lightness value according to a numerical range in which the lightness value of the pixel is located; adjusting the saturation value of the pixel according to the saturation adjustment weight, and adjusting the lightness value of the pixel according to the target lightness value, to obtain a color-enhanced image.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and more particularly to the field of computer vision. More specifically, this disclosure provides an image processing method, apparatus, electronic device, storage medium, and computer program product. Background Technology

[0002] In practical applications, images and videos played by electronic devices often suffer from insufficient clarity, low contrast, color distortion, or uneven brightness, especially in low-resolution resources, compressed videos, or videos shot under limited conditions. These problems cause the image to appear blurry, dim, or unnatural, negatively impacting the user's viewing experience.

[0003] Enhancement technology refers to technical means used to improve the visual quality of images or videos. Its goal is to enhance the visual effect of images or videos without changing the original content. By processing resources such as images and videos played on electronic devices, optimizations can be achieved in areas such as image sharpness, contrast, color saturation, and brightness, thereby improving the user's viewing experience. Summary of the Invention

[0004] This disclosure provides an image processing method, apparatus, electronic device, storage medium, and computer program product.

[0005] According to one aspect of this disclosure, an image processing method is provided, comprising: for a pixel in a first image, determining a saturation adjustment weight based on at least one of a pixel's hue value, saturation value, and brightness value; determining a target brightness value based on the numerical range of the pixel's brightness value; adjusting the pixel's saturation value according to the saturation adjustment weight; and adjusting the pixel's brightness value according to the target brightness value to obtain a color-enhanced image.

[0006] According to another aspect of this disclosure, an image processing apparatus is provided, comprising: a saturation adjustment weight determination module, a target brightness value determination module, and a color adjustment module. The saturation adjustment weight determination module is used to determine a saturation adjustment weight for a pixel in a first image based on at least one of the pixel's hue value, saturation value, and brightness value. The target brightness value determination module is used to determine a target brightness value based on the numerical range of the pixel's brightness value. The color adjustment module is used to adjust the saturation value of the pixel according to the saturation adjustment weight, and to adjust the brightness value of the pixel according to the target brightness value, to obtain a color-enhanced image.

[0007] According to another aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the methods provided in this disclosure.

[0008] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause a computer to perform the methods provided in this disclosure.

[0009] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the methods provided in this disclosure.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0011] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0012] Figure 1 This is a schematic diagram illustrating an application scenario of the image processing method and apparatus according to embodiments of the present disclosure;

[0013] Figure 2 This is a schematic flowchart of an image processing method according to an embodiment of the present disclosure;

[0014] Figure 3 This is a schematic diagram illustrating the saturation strategy in color enhancement processing according to an embodiment of the present disclosure;

[0015] Figure 4 This is a schematic diagram illustrating the contrast strategy in color enhancement processing according to an embodiment of the present disclosure.

[0016] Figure 5 This is a schematic diagram illustrating the principle of an image processing method according to an embodiment of the present disclosure;

[0017] Figure 6 This is a schematic diagram of an image processing method according to another embodiment of the present disclosure;

[0018] Figure 7 This is a schematic structural block diagram of an image processing apparatus according to embodiments of the present disclosure; and

[0019] Figure 8 This is a structural block diagram of an electronic device used to implement the image processing method of the embodiments of this disclosure. Detailed Implementation

[0020] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0021] The collection, storage, use, processing, transmission, provision, and disclosure of any type of information, such as user personal information, in this technical solution comply with relevant laws and regulations and do not violate public order and good morals. In this technical solution, user authorization or consent is obtained before acquiring or collecting user personal information.

[0022] In some technical solutions, large models can be used on the server side to implement enhancement techniques. However, due to limitations in model size and performance, when deploying these models on mobile devices, it is usually impossible to achieve large-scale support for various models while maintaining high playback quality. Therefore, electronic devices typically employ a single enhancement solution. For example, a super-resolution model might be deployed on a high-end device to achieve enhancement, but this model is not suitable for mid- to low-end devices. Furthermore, the quality improvement effect of a single enhancement solution is limited.

[0023] In other technical solutions, image processing algorithms can be used to adjust parameters such as image saturation and contrast. However, the adjustment process lacks adaptability. For example, applying the same adjustment amount to all pixels in the image can cause problems such as oversaturation, skin tone shift, and loss of highlight details, resulting in uncontrollable enhancement effects and a poor subjective experience.

[0024] This disclosure aims to provide an image processing method that adaptively adjusts pixel parameters to achieve a more natural display and improve user experience. This solution is also applicable to deployment in electronic devices, achieving high-quality, stable enhancements that can be widely applied across multiple device models while reducing computational costs.

[0025] The embodiments disclosed herein can be applied to applications such as cloud storage applications and video applications. These applications can be deployed on electronic devices such as mobile phones, tablets, and smart TVs. The electronic devices can perform image enhancement processing during the display of images, GIFs, and videos to improve the picture quality and thus enhance the user experience.

[0026] The technical solutions provided in this disclosure will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0027] Figure 1This is a schematic diagram illustrating an application scenario of the image processing method and apparatus according to embodiments of the present disclosure.

[0028] It is important to note that Figure 1 The examples shown are merely examples of system architectures that can be applied to the embodiments of this disclosure, in order to help those skilled in the art understand the technical content of this disclosure, but do not mean that the embodiments of this disclosure cannot be used in other devices, systems, environments or scenarios.

[0029] like Figure 1 As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, and 103, a network 104, and a server 105. The network 104 serves as a medium for providing a communication link between the terminal devices 101, 102, and 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0030] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Terminal devices 101, 102, and 103 can be various electronic devices with displays and web browsing capabilities, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0031] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using terminal devices 101, 102, and 103 (for example only). The backend management server can analyze and process data such as received user requests and feed the processing results back to the terminal devices.

[0032] For example, a user sends a video playback request to server 105 using terminal device 101. Server 105 returns a resource to be played to terminal device 101. The resource can be a video or an image; taking a video as an example, terminal device 101 can execute image processing methods to adjust each frame of the video. For example, terminal device 101 can adjust the saturation and contrast of the image to make the picture more vibrant and enhance the sense of depth, thereby achieving color enhancement. Terminal device 101 can also adjust the brightness of the image by simulating the gamma curve of the display, thereby achieving brightness enhancement. Terminal device 101 can also improve the image resolution by performing pixel interpolation through stretching algorithms, and can also perform sharpening processing through sharpening algorithms to improve the outline details of the image. Then, terminal device 101 displays the processed image.

[0033] It should be noted that the image processing method provided in the embodiments of this disclosure can generally be executed by terminal devices 101, 102, and 103. Accordingly, the image processing apparatus provided in the embodiments of this disclosure can generally be disposed in terminal devices 101, 102, and 103.

[0034] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0035] Figure 2 This is a schematic flowchart of an image processing method according to an embodiment of the present disclosure.

[0036] like Figure 2 As shown, the image processing method 200 may include operations S210 to S230.

[0037] In operation S210, for a pixel in the first image, a saturation adjustment weight is determined based on at least one of the pixel's hue value, saturation value, and brightness value.

[0038] In operation S220, the target brightness value is determined based on the range of brightness values ​​of the pixels.

[0039] In operation S230, the saturation value of the pixel is adjusted according to the saturation adjustment weight, and the brightness value of the pixel is adjusted according to the target brightness value, so as to obtain the color-enhanced image.

[0040] Image processing methods are mainly used to process images, which can be a single image, an image frame in a video, or an image frame in an animation. This embodiment does not limit this.

[0041] For example, this embodiment mainly performs color enhancement processing on the first image. The color enhancement processing is performed in the HSV (Hue, Saturation, Value) space. If the first image is data in the RGB (Red, Green, Blue) space, the pixels in the first image can be converted from the RGB space to the HSV space to obtain the hue value (H, Hue), saturation value (S, Saturation), and lightness value (V / L, Value / Lightness). The first image includes multiple pixels, and the color enhancement processing process for each pixel can be the same. This embodiment uses the color enhancement processing process of one pixel as an example for explanation. By performing color enhancement processing on each pixel in the image, the color-enhanced image can be obtained.

[0042] Color enhancement processing includes saturation enhancement. A first correspondence between hue values ​​and a first sub-weight, a second correspondence between saturation values ​​and a second sub-weight, and a third correspondence between lightness values ​​and a third sub-weight can be pre-configured. Based on these three correspondences, three sub-weights are determined. These three sub-weights are then multiplied or weighted, and the result is used as the saturation adjustment weight. After obtaining the saturation adjustment weight, the target saturation adjustment amount is determined by multiplying the saturation adjustment weight by a predetermined saturation adjustment amount. Then, the target saturation value is determined by summing the pixel's saturation value and the target saturation adjustment amount. Finally, the pixel's saturation value can be adjusted to the target saturation value.

[0043] Color enhancement processing also includes contrast enhancement, which is achieved by changing the brightness values ​​of each pixel. For example, a correspondence between the original brightness values ​​and the target brightness values ​​is pre-configured, and after determining the target brightness value based on this correspondence, the adjusted brightness value is set to the target brightness value.

[0044] This embodiment does not limit the order in which the saturation and contrast are adjusted. After adjusting the saturation and contrast of the first image as described above, a color-adjusted image is obtained. If an RGB space image is needed subsequently, the color-adjusted image can be converted from HSV space to RGB space, and then the converted image can be used as the color-enhanced image.

[0045] According to the image processing method provided in this disclosure, the method adaptively adjusts the parameters of each pixel. For example, instead of using a uniform adjustment amount for the saturation and brightness of each pixel, a saturation adjustment weight is determined based on a comprehensive analysis of the hue, saturation, and brightness values. This saturation adjustment weight is then used to adjust the pixel's saturation value, resulting in differences in the adjustment amount for each pixel. Similarly, the target brightness value is determined and adjusted based on the range of brightness values ​​within each pixel, again leading to differences in the adjustment amount for each pixel's brightness value. This image processing method allows for adaptive adjustment based on the pixel's own parameters, resulting in a more natural image display effect. For example, it alleviates problems such as oversaturation, skin tone shift, and loss of highlight details, thus improving the user experience. Furthermore, the technical solution provided in this disclosure does not require a large-scale model, making it a lightweight algorithm suitable for deployment in electronic devices. It can achieve high-quality, stable enhancement effects with broad coverage across multiple device models while reducing computational costs.

[0046] Figure 3 This is a schematic diagram illustrating the saturation strategy in color enhancement processing according to an embodiment of the present disclosure.

[0047] like Figure 3As shown, in this embodiment, the color enhancement processing includes saturation enhancement. During the saturation enhancement process, for any pixel in the first image, the brightness value 301, saturation value 302, and hue value 303 of the pixel can be determined. Then, a high brightness protection weight 304 is determined based on the pixel's brightness value 301, a low saturation protection weight 305 and a high saturation protection weight 306 are determined based on the pixel's saturation value 302, and a first skin tone protection weight 307 is determined based on the pixel's hue value 303. Then, a saturation adjustment weight 308 is determined based on the high brightness protection weight 304, low saturation protection weight 305, high saturation protection weight 306, and first skin tone protection weight 307. Afterward, a target saturation adjustment amount 310 can be determined based on the product of the saturation adjustment weight 308 and a predetermined saturation adjustment amount 309. Finally, a target saturation value 311 is determined based on the sum of the pixel's saturation value 302 and the target saturation adjustment amount 310. The pixel's saturation value 302 can then be adjusted to the target saturation value 311. The aforementioned predetermined saturation adjustment amount 309 can be pre-configured based on experience or offline test results.

[0048] This embodiment describes the process of determining the high brightness protection weight based on the brightness value of a pixel.

[0049] In this embodiment, the high brightness protection weight is positively correlated with the saturation adjustment weight. When the brightness value of a pixel is greater than a predetermined upper limit of brightness, the high brightness protection weight is negatively correlated with the brightness value of the pixel, or the high brightness protection weight is the predetermined minimum brightness protection weight.

[0050] For example, when the pixel's brightness value is less than a predetermined lower brightness limit, the high brightness protection weight is the predetermined maximum brightness protection weight, such as 1. When the pixel's brightness value is between the predetermined lower and upper brightness limits, the high brightness protection weight is negatively correlated with the pixel's brightness value. When the pixel's brightness value is greater than the predetermined upper brightness limit, the high brightness protection weight is the predetermined minimum brightness protection weight, such as 0.3.

[0051] For example, the high brightness protection weight can be calculated using the following formula:

[0052]

[0053] in, Indicates the high brightness protection weight. Represents the brightness value of a pixel. . It is a smooth transition function. In the above smooth transition function, when L is less than or equal to 0.8, the function value is 0; when L ≥ 0.9, the function value is 1; and it smoothly transitions between 0.8 and 0.9.

[0054] This embodiment uses a high brightness protection weight to prevent bright areas in the image, such as white walls, lights, and clouds, from being incorrectly colored or intensified during color enhancement, thus making the highlight areas more natural and preventing color distortion.

[0055] This embodiment describes the process of determining the low saturation protection weight based on the saturation value of a pixel.

[0056] In this embodiment, the low saturation protection weight is positively correlated with the saturation adjustment weight. When the saturation value of a pixel is less than a predetermined lower limit of saturation, the low saturation protection weight is positively correlated with the pixel's saturation value, or the low saturation protection weight is a predetermined minimum low saturation protection weight.

[0057] For example, when the pixel's saturation value is less than a predetermined lower saturation limit, the low saturation protection weight is a predetermined minimum low saturation protection weight, such as 0. When the pixel's saturation value is between the predetermined lower and upper saturation limits, the low saturation protection weight is positively correlated with the pixel's saturation value. When the pixel's saturation value is greater than the predetermined upper saturation limit, the low saturation protection weight is a predetermined maximum low saturation protection weight, such as 1.

[0058] For example, the low saturation protection weight can be calculated using the following formula:

[0059]

[0060] in, This indicates a low saturation protection weight, where S represents the pixel's saturation value. . It is a smooth transition function. In the above smooth transition function, when S is less than or equal to 0.05, the function value is 0; when S ≥ 0.3, the function value is 1; and it smoothly transitions between 0.05 and 0.3.

[0061] Taking the presence of slight gray noise in a white wall as an example, if a conventional saturation adjustment strategy is used, the gray noise will be amplified, making the image look dirty. This embodiment uses a low saturation protection weight to suppress color enhancement in near-gray areas of the image, preventing noise and slight gray areas from being magnified into obvious color spots or dirt, thereby keeping the image clean.

[0062] This embodiment describes the process of determining the high saturation protection weight based on the saturation value of a pixel.

[0063] In this embodiment, the high saturation protection weight is positively correlated with the saturation adjustment weight. When the saturation value of a pixel is greater than the upper limit threshold of saturation, the high saturation protection weight is negatively correlated with the pixel's saturation value, or the high saturation protection weight is a predetermined minimum high saturation protection weight.

[0064] For example, when the pixel's saturation value is less than the lower saturation threshold, the high saturation protection weight is a predetermined maximum high saturation protection weight, such as 1. When the pixel's saturation value is between the lower and upper saturation thresholds, the high saturation protection weight is negatively correlated with the pixel's saturation value. When the pixel's saturation value is greater than the upper saturation threshold, the high saturation protection weight is a predetermined minimum high saturation protection weight, such as 0.

[0065] For example, the high saturation protection weight is calculated using the following formula:

[0066]

[0067] in, This indicates a high saturation protection weight, where S represents the pixel's saturation value. . It is a smooth transition function. In the above smooth transition function, when S is less than or equal to 0.7, the function value is 0; when S ≥ 0.97, the function value is 1; and it smoothly transitions between 0.7 and 0.97.

[0068] It should be noted that when the saturation value of a pixel is already very high, further increasing the saturation value will result in a loss of detail. This embodiment uses a high saturation protection weight to prevent over-enhancing of areas in the image that are already very vibrant in color, avoiding the loss of detail caused by over-saturation, and ensuring that the color-enhanced image retains its original details.

[0069] This embodiment describes the process of determining the first skin color protection weight based on the hue value of the pixel.

[0070] In this embodiment, at least one reference hue value can be pre-configured. For example, the first reference hue value, representing yellow skin tone, can be 0.05. The second reference hue value, representing white skin tone with a reddish tint, can be 0.95.

[0071] A first target distance can be determined between a pixel's hue value and at least one reference hue value. For example, a first distance can be determined between a pixel's hue value and a first reference hue value, and a second distance can be determined between a pixel's hue value and a second reference hue value. Then, based on the first and second distances, a first target distance is determined, for example, by using the minimum of the first and second distances as the first target distance.

[0072] Next, the first skin color protection weight can be determined based on the first target distance. The first skin color protection weight is negatively correlated with the first target distance and negatively correlated with the saturation adjustment weight.

[0073] For example, the first skin color protection weight is calculated using the following formula:

[0074]

[0075] in, Indicates the first skin color protection weight. Indicates the distance to the first target.

[0076] This embodiment, through a first skin tone protection weight, can automatically identify and protect facial skin areas in an image during global color enhancement, preventing excessive saturation that could cause faces to appear overly yellow, red, or unnatural, thus ensuring the authenticity of the subject's skin tone. Furthermore, by setting two representative reference hue values, it can more comprehensively cover different skin tone ranges, making the skin tone protection mechanism more universal and accurate. Regardless of the skin tone of the faces appearing in the image, this method can effectively identify and protect them.

[0077] According to another embodiment of this disclosure, after obtaining the high brightness protection weight, low saturation protection weight, high saturation protection weight, and first skin tone protection weight, a saturation adjustment weight can be determined based on at least one of the above four weights. For example, one or more of the above four weights can be selected to calculate the saturation adjustment weight.

[0078] For example, the saturation adjustment weight can be calculated using the following formula:

[0079]

[0080]

[0081] in, Indicates the saturation adjustment weight. Indicates the first skin color protection weight. Indicates low saturation protection weight. Indicates high saturation protection weight. This represents the high brightness protection weight. It is understood that the above formula can also omit any one or more of the high brightness protection weight, low saturation protection weight, high saturation protection weight, and first skin tone protection weight; that is, the influence of some factors among high brightness, low saturation, high saturation, and skin tone on saturation can be ignored.

[0082] Figure 4 This is a schematic diagram illustrating the contrast strategy in color enhancement processing according to an embodiment of the present disclosure.

[0083] like Figure 4 As shown, in this embodiment, the color enhancement process also includes contrast enhancement, which is achieved by changing the brightness value of each pixel. During contrast enhancement, a target brightness value needs to be determined based on the range of brightness values ​​of the pixels. For example, a reference brightness value can be determined based on the pixel's brightness value and a predetermined contrast coefficient. Then, the target brightness value is determined based on the range of brightness values ​​of the pixels and the reference brightness value. If the pixel's brightness value is greater than or equal to a first brightness value and less than or equal to a second brightness value, the target brightness value is the same as the reference brightness value; if the pixel's brightness value is less than the first brightness value, the target brightness value is greater than the reference brightness value; if the pixel's brightness value is greater than the second brightness value, the target brightness value is less than the reference brightness value.

[0084] In this embodiment, contrast enhancement can make darker pixels darker and brighter pixels brighter. When a pixel is neither too bright nor too dark, the contrast coefficient is adjusted to ensure the adjusted target brightness value matches the reference brightness value. When a pixel is too dark, the adjustment amount is reduced to prevent the adjusted target brightness value from being too low, thus protecting dark areas and preventing completely black pixels. When a pixel is too bright, the adjustment amount is reduced to prevent the adjusted target brightness value from being too high, thus protecting highlights and preventing clipping.

[0085] This embodiment enhances the overall contrast of the image by introducing a reference brightness value and differentiating the brightness values, while protecting the dark and bright areas, avoiding problems such as completely black dark areas and overexposed highlights, so that the adjusted image retains details.

[0086] According to another embodiment of this disclosure, when the brightness value of a pixel is less than a reference brightness value, the reference brightness value is less than the brightness value of the pixel. When the brightness value of a pixel is greater than the reference brightness value, the reference brightness value is greater than the brightness value of the pixel. The reference brightness value is greater than a first brightness value and less than a second brightness value; for example, the reference brightness value is 0.5, the first brightness value is 0.2, and the second brightness value is 0.8.

[0087] For example, the reference lightness value can be calculated using the following formula:

[0088]

[0089] in, This indicates a reference brightness value. Represents the brightness value of a pixel. Indicates the reference brightness value. The default value can be configured to 0.5. It is the contrast ratio. This enhances contrast, and the contrast ratio can improve the contrast of dark areas (i.e., The brightness value of the pixels (i.e., the pixels) decreases, thus becoming darker, and the bright areas (i.e., the pixels) become darker. The brightness value of the pixel is increased, thus making it brighter.

[0090] This embodiment centers on a baseline brightness value, making dark areas darker and bright areas brighter, thereby effectively widening the grayscale levels of the image.

[0091] According to another embodiment of this disclosure, the process of determining the target lightness value is described. This embodiment uses a first lightness value of 0.2 and a second lightness value of 0.8 as an example. It is understood that the first and second lightness values ​​can be configured to other values ​​according to actual needs. This embodiment does not limit the values ​​of the first and second lightness values.

[0092] If the pixel's brightness value is greater than or equal to 0.2 and less than or equal to 0.8, the reference brightness value is determined as the target brightness value.

[0093] For example, the target brightness value can be determined using the following formula:

[0094]

[0095] in, Indicates the target brightness value. . This indicates a reference brightness value. Represents the brightness value of a pixel. . Indicates the reference brightness value. It can be configured to 0.5. It is the contrast ratio. .

[0096] In this embodiment, when the brightness value of a pixel is neither too high nor too low, a reference brightness value is used as the target brightness value. This makes the dark areas in the image darker and the bright areas brighter, thereby highlighting the sense of depth in the highlights and shadows.

[0097] When the pixel's brightness value is less than 0.2, the target brightness value can be determined based on the pixel's brightness value, a first weight for that pixel's brightness value, a reference brightness value, and a second weight for that reference brightness value. The first weight is negatively correlated with the pixel's brightness value, while the second weight is positively correlated with the pixel's brightness value.

[0098] For example, the target brightness value can be determined using the following formula:

[0099]

[0100] in, Indicates the target brightness value. This indicates a reference brightness value. Representing the brightness value of a pixel, the first weight in the above formula is... The second weight is The first and second weights can also be calculated using other formulas.

[0101] In this embodiment, when the pixel's brightness value is too low, the reference brightness value is even lower relative to the pixel's brightness value. At this point, the target brightness value determined by the first and second weights lies between the pixel's brightness value and the reference brightness value. Furthermore, the lower the pixel's brightness value, the larger the first weight, and the closer the target brightness value is to the pixel's brightness value, thus reducing the adjustment amount. This achieves shadow protection and prevents completely black areas. In addition, since the first and second weights are dynamically adjusted based on the pixel's brightness value, a smooth transition can be achieved, making the adjustment effect more natural and continuous.

[0102] When the pixel's brightness value is greater than 0.8, the target brightness value can be determined based on the pixel's brightness value, a first weight for that pixel's brightness value, a reference brightness value, and a second weight for that reference brightness value. The first weight is positively correlated with the pixel's brightness value, and the second weight is negatively correlated with the pixel's brightness value.

[0103] For example, the target brightness value can be determined using the following formula:

[0104]

[0105] in, Indicates the target brightness value. This indicates a reference brightness value. This represents the brightness value of a pixel. In the formula above, the first weight is... The second weight is The first and second weights can also be calculated using other formulas.

[0106] In this embodiment, when the pixel's brightness value is too high, the reference brightness value is even higher relative to the pixel's brightness value. At this point, the target brightness value determined by the first and second weights falls between the pixel's brightness value and the reference brightness value. Furthermore, the higher the pixel's brightness value, the larger the first weight, and the closer the target brightness value is to the pixel's brightness value, thus reducing the adjustment amount. This achieves highlight protection and prevents overexposure. In addition, since the first and second weights are dynamically adjusted based on the pixel's brightness value, a smooth transition can be achieved, making the adjustment effect more natural and continuous.

[0107] According to another embodiment of this disclosure, after color enhancement of the first image to obtain a color-enhanced image, the gamma correction index can be used to adjust the red channel value, green channel value, and blue channel value of the pixels in the color-enhanced image to obtain a brightness-enhanced image.

[0108] For example, the following formula can be used to adjust image brightness:

[0109]

[0110] in, For input color, each component . To output color, This is the gamma value, which can be less than 1, for example, a gamma value of 0.95.

[0111] This embodiment uses a power function to non-linearly stretch pixels, simulating the response curve of a real-world display device to naturally adjust image brightness. This non-linear adjustment method allows for more delicate control of midtone brightness, avoiding localized overexposure or loss of detail in dark areas caused by simple addition. Furthermore, in practical applications, the saturation and contrast of the first image can be adjusted first, followed by brightness adjustment of the color-enhanced image. This approach, performing color enhancement before brightness adjustment, alters the overall non-linear response of the RGB values ​​during brightness adjustment, effectively performing gamma correction on uniform brightness and reducing disturbance to the preceding color protection logic.

[0112] Figure 5 This is a schematic diagram illustrating the principle of an image processing method according to an embodiment of the present disclosure.

[0113] like Figure 5 As shown, in this embodiment, after performing color enhancement processing 501 or brightness enhancement processing 502, sharpness adjustment, sharpening, and other processing can also be performed. In this embodiment, the color-enhanced image or the brightness-enhanced image can be used as the second image. Then, based on at least one of the image resolution of the second image and the device information of the device used to process the second image, a target processing algorithm is determined from multiple candidate processing algorithms. The second image is then processed according to the target processing algorithm to obtain a sharpness-enhanced image.

[0114] Image resolutions include, for example, 280P, 360P, 720P, 1080P, 4K, etc. Device information may include device model, processor model, etc.

[0115] For example, a target processing algorithm may include at least one of the following: original value mapping, face denoising algorithm, stretching algorithm, and sharpening algorithm. One or more algorithms can be selected and combined sequentially to form the target processing algorithm. For instance, one target processing algorithm is called super-resolution. One super-resolution algorithm 503 may include a face denoising algorithm, a stretching algorithm, and a sharpening algorithm executed sequentially. Another super-resolution algorithm may omit the face denoising algorithm, in which case it includes a stretching algorithm and a sharpening algorithm executed sequentially. The stretching algorithm is used to perform pixel interpolation to increase image resolution, and the sharpening algorithm is used to sharpen the image, improving the clarity of details in the image, such as making the contours in the image clearer. Another target processing algorithm is called sharpening 504, which only includes a sharpening algorithm. Another target processing algorithm is called original value mapping 505, which means neither super-resolution nor sharpening is performed; instead, the second image is directly used as the image with enhanced clarity.

[0116] This embodiment provides an adaptive resolution enhancement strategy that selects an appropriate enhancement algorithm based on the original resolution of the resource and the hardware capabilities of the playback device. This avoids stuttering, overheating, and power consumption caused by running high-load algorithms on low-performance devices, and also avoids unnecessary interpolation processing on high-quality video sources, achieving a balance between performance and clarity.

[0117] Figure 6 This is a schematic diagram of an image processing method according to another embodiment of the present disclosure.

[0118] like Figure 6 In this embodiment, the first image is first subjected to color enhancement processing 601, for example, adjusting the saturation and brightness values ​​of pixels. Then, brightness enhancement processing 602 is performed, for example, adjusting the brightness in the RGB space using the gamma index. Next, the target processing algorithm can be determined based on the image resolution and device information.

[0119] If the image resolution is greater than or equal to the first predetermined resolution, the first predetermined resolution image can be 4K. Since such an image has a large number of pixels, the burden of the stretching algorithm on the electronic device is too heavy. Furthermore, since the image resolution is already large enough, there is no need to further increase the resolution. Therefore, sharpening 605 is determined as the target processing algorithm. Sharpening 605 does not include the stretching algorithm, but only includes the sharpening algorithm used for image sharpening.

[0120] If the image resolution is less than or equal to the second predetermined resolution, the second predetermined resolution image can be 360P. Since the clarity of such resources is too poor and the image is blurry and has many mosaics, stretching or sharpening algorithms will further amplify the mosaic effect. Therefore, the original value mapping 604 is determined as the target processing algorithm. The original value mapping 604 is used to treat the second image as an image with enhanced clarity.

[0121] If the image resolution is greater than a second predetermined resolution but less than a first predetermined resolution, and the device information is within a predetermined range, then Super-Resolution 603 is selected as the target processing algorithm. Super-Resolution 603 may include stretching and sharpening algorithms. If the image resolution is greater than a second predetermined resolution but less than a first predetermined resolution, and the device information is outside the predetermined range, then sharpening is selected as the target processing algorithm. In practical applications, high-performance high-end models can be selected through offline evaluation and added to a whitelist, which is used as the predetermined range. If the image resolution is neither too high nor too low, and the device model is in the whitelist, then Super-Resolution 603 is used; otherwise, sharpening 605 is used.

[0122] It should be noted that related technologies cannot dynamically adjust the image processing algorithm used based on device information and image quality, resulting in limited coverage of different device models and difficulty in accommodating various devices. In this embodiment, considering that the stretching algorithm is time-consuming and consumes a lot of computing resources, a whitelist is established through offline evaluation. The target processing algorithm is selected based on device information and image quality. For example, the stretching algorithm is executed on high-end models, while it is not executed on mid-to-low-end models, thereby ensuring compatibility with more models and improving the applicability of the solution.

[0123] According to another embodiment of this disclosure, a target processing algorithm is called super-resolution, which may include a face denoising algorithm, a stretching algorithm, and a sharpening algorithm executed sequentially. The face denoising algorithm may employ a bilateral filtering algorithm to denoise the face region and avoid amplifying noise. In practical applications, a color-enhanced image or a brightness-enhanced image can be used as input to the face denoising algorithm. The stretching algorithm and the sharpening algorithm will be described below.

[0124] According to another embodiment of this disclosure, a stretching algorithm is used to perform pixel interpolation, thereby improving image resolution. The stretching algorithm is described below.

[0125] The image with enhanced color or brightness can be used as the second image, or the second image can be used as the image before scaling. Alternatively, the image obtained by performing other image processing on the second image can be used as the image before scaling, for example, by processing the second image using a face denoising algorithm, and then using the processed image as the image before scaling.

[0126] The stretching algorithm can process pixels in YUV space. Therefore, if the image before scaling is an RGB image, it can be converted from RGB space to YUV space before processing. During the stretching process, the algorithm maps the pixel coordinates in the scaled image to the original pixel coordinates in the original image according to a predetermined scaling ratio. In the original image, the algorithm expands outwards from the pixel indicated by the integer value in the original pixel coordinates, resulting in an expanded region containing multiple pixels. Based on the decimal values ​​in the original pixel coordinates and the pre-configured correspondence between decimal values ​​and weight sets, the pixel weights of each pixel in the expanded region are determined. Based on the brightness values ​​of each pixel in the expanded region and their respective pixel weights, the target brightness value for the pixels in the scaled image is determined. Based on the target brightness value, the brightness values ​​of the pixels in the scaled image are set, resulting in the scaled image.

[0127] For example, the original pixel coordinates can be determined using the following formula:

[0128]

[0129]

[0130]

[0131]

[0132] Wherein, the pixel coordinates in the scaled image are The original pixel coordinates in the image before scaling are The resolution of the image before scaling is The resolution of the scaled image is , Indicates the predetermined scaling ratio in the horizontal direction. This represents the predetermined scaling ratio in the vertical direction. The 0.5 in the formula is for aligning the pixel center.

[0133] Taking the pixel coordinates in the scaled image as (320, 240) and the predetermined scaling ratios in both the horizontal and vertical directions as an example, the original pixel coordinates in the image before scaling can be calculated using the above formula as (159.75, 119.75).

[0134] The x-axis is then split into an integer value of 159 and a decimal value of 0.75, and the y-axis is split into an integer value of 119 and a decimal value of 0.75. The integer value is used as the center reference pixel to generate the extended region, and the decimal value is used to determine the pixel weight of each pixel in the extended region.

[0135] For integer values, the coordinates (159, 119) can be determined. Starting from these coordinates, the area can be expanded outwards, for example, by 2 pixels upwards and to the left, and 3 pixels downwards and to the right. This results in a 6x6 rectangular expansion area with the top-left corner at (157, 117) and the bottom-right corner at (162, 122). Each pixel in this expansion area has a luminance value, which is the Y value in the YUV space. It is understood that in other embodiments, the expansion in the four directions (upwards, downwards, left, and right) can be of other numbers of pixels; this embodiment does not limit the expansion distance in each direction.

[0136] For small numerical values, a phase table can be pre-configured. The phase table includes the correspondence between phase and small numerical value, as well as the correspondence between phase and pixel weight. Different phases correspond to different weights, which are used to guide the importance of each pixel in the extended region to the brightness value of the pixel at (320, 240) in the scaled image.

[0137] For example, a phase table with four pre-configured phases (0, 16, 48, and 63) is used. Taking the horizontal direction as an example, a phase of 0 corresponds to a value of 0, indicating that the interpolation position is roughly aligned to an integer pixel, and the interpolation behavior is almost no interpolation. A phase of 16 corresponds to a value of 0.25, indicating that the interpolation position is roughly aligned to 1 / 4 of a pixel, and the interpolation behavior is biased towards the left pixel. A phase of 48 corresponds to a value of 0.75, indicating that the interpolation position is roughly aligned to 3 / 4 of a pixel, and the interpolation behavior is biased towards the right pixel. A phase of 63 corresponds to a value of 0.984, indicating that the interpolation position is almost to the next pixel, and the interpolation behavior almost completely uses the right pixel.

[0138] The phase table above determines that the phase corresponding to the horizontal direction is 48, and the phase corresponding to the vertical direction is also 48. It should be noted that this embodiment only uses 0, 16, 48, and 63 as examples. In other embodiments, other phases can be configured, for example, configuring pixel weights for 64 phases from 0 to 63. Then, the pixel weights are read based on the horizontal and vertical phases. It should be noted that if the pre-configured correspondence does not include small values ​​from the original pixel coordinates in the pre-scaled image, a small value close to the calculated result (i.e., a small value from the original pixel coordinates in the pre-scaled image) can be selected from the pre-configured small values, and the pixel weights are determined based on the correspondence of this small value.

[0139] In one example, multiple pre-configured phases correspond to multiple pixel weights, and each pixel weight corresponds one-to-one with a pixel in the extended region. For example, taking the aforementioned 6*6 rectangular extended region as an example, the same phase corresponds to 64 pixel weights. Then, the brightness values ​​of each pixel can be weighted and summed according to the pixel weights to obtain the target brightness value of the pixel at (320, 240) in the scaled image.

[0140] In another example, among the pre-configured multiple phases, each phase corresponds to a horizontal set of pixel weights and a vertical matrix of pixels. For example, taking the aforementioned 6x6 rectangular extended region as an example, the horizontal and vertical directions each correspond to 6 pixel weights. Filtering can be performed first along the vertical direction; that is, for any column in the extended region, the weighted sum of the pixels in that column with the 6 vertical pixel weights is calculated to obtain an intermediate result. This calculation is performed on each of the 6 columns to obtain 6 intermediate results. Then, horizontal filtering is performed, that is, the 6 intermediate results are weighted summed with the 6 horizontal pixel weights, and the result is used as the target brightness value of the pixel at (320, 240) in the scaled image. In other embodiments, horizontal filtering can also be performed first, followed by vertical filtering.

[0141] After obtaining the target brightness value of the pixel at (320, 240) in the scaled image, set the brightness value of that pixel (i.e., the Y value in YUV space) as the target brightness value to complete the interpolation processing of the pixel at (320, 240). Repeat the above process for each pixel coordinate in the scaled image to obtain the scaled image. Next, if an RGB image is needed later, the image can be converted from YUV space to RGB space. If further processing is needed in YUV space, no color space conversion is required.

[0142] The stretching algorithm in this embodiment first maps the pixel coordinates in the scaled image to the original pixel coordinates in the image before scaling. Then, it uses the pixel coordinates indicated by the original pixel coordinates in the image before scaling, as well as the brightness values ​​of multiple surrounding pixels, to comprehensively determine the target brightness value at the pixel coordinates in the scaled image. This can improve the accuracy of the target brightness value, thereby improving the interpolation accuracy.

[0143] According to another embodiment of this disclosure, a sharpening algorithm is used to sharpen an image, thereby improving the clarity of details in the image. The sharpening algorithm is described below.

[0144] During the sharpening algorithm execution, the image to be sharpened is first determined. For example, the image to be sharpened can be a second image, i.e., a color-enhanced image or a brightness-enhanced image. Alternatively, the image to be sharpened can be an image obtained by performing other image processing on the second image, for example, using the image processed by a stretching algorithm as the image to be sharpened. The sharpening algorithm only processes the Y value of pixels in the YUV space, that is, it only processes the luminance channel. This embodiment does not limit the processing methods for the U and V channels.

[0145] For any pixel in the image to be sharpened, determine the pixel sharpening weight and the second skin tone protection weight. The pixel sharpening weight characterizes the degree of pixel sharpening and is negatively correlated with the target sharpening weight. The second skin tone protection weight is negatively correlated with the second target distance between the hue value of the pixel (i.e., the pixel in the image to be sharpened) and at least one reference hue value, and is negatively correlated with the target sharpening weight. The calculation method for the second skin tone protection weight can refer to the first skin tone protection weight described above.

[0146] Next, the target sharpening weight is determined based on at least one of the pixel sharpening weight and the second skin color protection weight.

[0147] For example, the target sharpening weights can be determined using the following formula:

[0148]

[0149] in, Indicates the target sharpening weight. This indicates the second skin color protection weight. This represents the pixel sharpening weight.

[0150] As can be seen from the above formula, the closer the pixel's hue value is to the reference hue value, the lower the sharpening enhancement, thus preventing facial noise and blemishes from being magnified. The sharper the pixel (the stronger the contrast), the lower the sharpening enhancement, thus avoiding unnecessary sharpening of already sharp areas.

[0151] Next, the target adjustment amount is determined based on the target sharpening weight and the base adjustment amount, where the target sharpening weight and the target adjustment amount are positively correlated. Then, the brightness value of the pixel (i.e., the pixel in the image to be sharpened) is adjusted using the target adjustment amount.

[0152] For example, the product of the target sharpening weight and the base adjustment amount is calculated as the target adjustment amount. Then, the sum of the target adjustment amount and the original brightness value of the pixel is used as the adjusted brightness value, which is the Y value in the YUV space.

[0153] In this embodiment, the sharpening algorithm does not use a globally uniform target adjustment amount. Instead, it adaptively determines the target adjustment amount based on whether the pixel's hue value is close to the reference hue value and the degree of sharpening in the pixel region. This avoids unnecessary sharpening of already sharpened edges and avoids over-sharpening of faces, thus making faces appear more natural.

[0154] According to another embodiment of this disclosure, pixel sharpening weights can be determined as follows: For any pixel in the image to be sharpened, starting from or centered on that pixel, search for multiple surrounding pixels along at least one predetermined direction in the image to be sharpened, obtaining at least one pixel set corresponding to each of the at least one predetermined direction. Then, for each pixel set in the at least one pixel set, based on the relative positional relationship of each pixel in the pixel set and the brightness value of each pixel in the pixel set, determine the sharpening intensity characterizing the degree of difference between the brightness values ​​of each pixel in the pixel set.

[0155] For example, the predetermined direction can be horizontal, vertical, 45°, or 135°. Taking the horizontal direction as an example, if we take one pixel as the center and search for two pixels to the left and two pixels to the right along the horizontal direction, we can obtain a pixel set. This pixel set includes the center pixel and two pixels to its left and two pixels to its right, for a total of 5 pixels. The vertical direction is similar; taking one pixel as the center and searching for two pixels upwards and two pixels downwards, we obtain a pixel set including 5 consecutive pixels. A sharpening intensity is calculated for each pixel set.

[0156] After obtaining the sharpening intensity for each predetermined direction, the pixel sharpening weight can be determined based on each sharpening intensity. For example, the sharpening intensity for each predetermined direction can be normalized to obtain a normalized value, which is positively correlated with the sharpening intensity. Then, the average of the four normalized values ​​is used as the pixel sharpening weight.

[0157] This embodiment calculates the sharpening intensity in multiple predetermined directions separately, and then comprehensively calculates the pixel sharpening weight. This allows for accurate identification of details and contours in various directions within the image. By analyzing the sharpening intensity in different directions and then fusing them into a pixel sharpening weight, the bias caused by judging from a single direction is avoided, thereby accurately assessing the sharpness of the region where the pixel is located.

[0158] According to another embodiment of this disclosure, the sharpening intensity for each predetermined direction can be determined as follows: Multiple pixels in a pixel set are divided into two subsets based on their relative positional relationships. For each subset, the maximum brightness difference between the brightness values ​​of each pixel in the subset is determined. The sharpening intensity is then determined based on the two maximum brightness differences between the two subsets.

[0159] The predetermined direction can be horizontal, vertical, 45°, or 135°. Taking the horizontal direction as an example, if we take one pixel as the center and search for two pixels to the left and two pixels to the right along the horizontal direction, we can obtain a pixel set including five consecutive pixels. Then, the pixel set is divided into two subsets, for example, the three pixels on the left are the first subset, and the three pixels on the right are the second subset. The intersection of the two subsets is the pixel at the center position. For the first subset, the highest and lowest brightness values ​​are determined from the three pixels, and the difference between them is calculated as the maximum brightness difference. This maximum brightness difference reflects the contrast level within the first subset. A maximum brightness difference is calculated for the second subset in the same way, resulting in two maximum brightness differences. Then, the ratio between the larger and smaller maximum brightness differences can be calculated as the sharpening intensity in the horizontal direction.

[0160] This embodiment uses the contrast ratio between subsets consisting of three pixels on each side to measure the "structural asymmetry" of the pixel set. A more uneven contrast between the two subsets indicates the presence of strong edges, jagged edges, or other image content, thus requiring a higher sharpening intensity. Conversely, a more uniform contrast between the two subsets indicates a relatively flat image content, resulting in a lower sharpening intensity.

[0161] This embodiment compares the contrast difference within local areas on both sides of the pixel cluster, which can accurately distinguish whether a pixel is in an edge area or a flat area, thus improving the accuracy of sharpening intensity.

[0162] This embodiment uses a smartphone video player app (Application) as an example to illustrate the application process of the image processing algorithm in practice. The image processing algorithm can be executed by a mobile phone with the video player app installed.

[0163] First, the video signal is decoded. For example, when a user selects a video through a mobile phone video player, the player decodes the video data into multiple frames of images and converts them into textures, which are then used as input for image processing algorithms.

[0164] Then, based on the device information and resource resolution, the target processing algorithm used in the resolution enhancement process can be selected. For example, one of super-resolution, sharpening, or original value mapping can be selected as the target processing algorithm.

[0165] The pixel information is then converted from RGB space to HSV space. Color enhancement is then performed in HSV space, for example, by calculating saturation adjustment weights based on reference hue, saturation, and brightness values, and then adjusting the pixel's saturation value to make the image more vibrant. Next, contrast processing is performed, filtering out brightness values ​​and adjusting them to change the contrast. Simultaneously, bright and dark areas are processed to create a sense of depth. After processing, the pixels can be converted back from HSV space to RGB space.

[0166] Next, brightness enhancement processing is performed in the RGB space, which improves the brightness of the screen display by simulating the adjustment mode of the monitor.

[0167] Next, sharpness enhancement is performed. For example, pixel information is first converted from RGB space to YUV space, and then the Y value in YUV space is processed through stretching and sharpening algorithms. Taking super-resolution as an example, face noise reduction can be performed first, then resolution can be increased through interpolation, such as from 720P to 1080P, and then details can be enhanced through sharpening algorithms. After processing, it can be converted back from YUV space to RGB space.

[0168] The processed texture is then output to the screen for rendering, allowing users to see a more vibrant, layered, and clearer image.

[0169] Figure 7 This is a schematic structural block diagram of an image processing apparatus according to an embodiment of the present disclosure.

[0170] like Figure 7 As shown, the image processing device 700 may include a saturation adjustment weight determination module 710, a target brightness value determination module 720, and a color adjustment module 730.

[0171] The saturation adjustment weight determination module 710 is used to determine the saturation adjustment weight for a pixel in the first image based on at least one of the pixel's hue value, saturation value, and brightness value.

[0172] The target brightness value determination module 720 is used to determine the target brightness value based on the value range of the pixel's brightness value.

[0173] The color adjustment module 730 is used to adjust the saturation value of a pixel according to the saturation adjustment weight, and to adjust the brightness value of a pixel according to the target brightness value, so as to obtain a color-enhanced image.

[0174] According to another embodiment of this disclosure, the target brightness value determination module includes a reference brightness value determination submodule and a target brightness value determination submodule. The reference brightness value determination submodule is used to determine a reference brightness value based on the brightness value of a pixel and a predetermined contrast coefficient. The target brightness value determination submodule is used to determine a target brightness value based on the numerical range of the brightness value of a pixel and the reference brightness value. When the brightness value of a pixel is greater than or equal to a first brightness value and less than or equal to a second brightness value, the target brightness value is the same as the reference brightness value; when the brightness value of a pixel is less than the first brightness value, the target brightness value is greater than the reference brightness value; when the brightness value of a pixel is greater than the second brightness value, the target brightness value is less than the reference brightness value.

[0175] According to another embodiment of this disclosure, the target brightness value determination submodule includes: a target brightness value determination unit, configured to determine a target brightness value based on a pixel's brightness value, a first weight for the pixel's brightness value, a reference brightness value, and a second weight for the reference brightness value. When the pixel's brightness value is less than the first brightness value, the first weight is negatively correlated with the pixel's brightness value, and the second weight is positively correlated with the pixel's brightness value. When the pixel's brightness value is greater than the second brightness value, the first weight is positively correlated with the pixel's brightness value, and the second weight is negatively correlated with the pixel's brightness value.

[0176] According to another embodiment of this disclosure, when the brightness value of a pixel is less than a reference brightness value, the reference brightness value is less than the brightness value of the pixel. When the brightness value of a pixel is greater than the reference brightness value, the reference brightness value is greater than the brightness value of the pixel. The reference brightness value is greater than a first brightness value and less than a second brightness value.

[0177] According to another embodiment of this disclosure, the saturation adjustment weight determination module includes a high brightness protection weight submodule and a first determination submodule. The high brightness protection weight submodule is used to determine a high brightness protection weight for any pixel in the first image based on the pixel's brightness value. The first determination submodule is used to determine a saturation adjustment weight based on the high brightness protection weight. The high brightness protection weight is positively correlated with the saturation adjustment weight; when the pixel's brightness value is greater than a predetermined upper limit value, the high brightness protection weight is negatively correlated with the pixel's brightness value, or the high brightness protection weight is a predetermined minimum brightness protection weight.

[0178] According to another embodiment of this disclosure, the saturation adjustment weight determination module includes: a low saturation protection weight determination submodule and a second determination submodule. The low saturation protection weight determination submodule is used to determine a low saturation protection weight for any pixel in the first image based on the pixel's saturation value. The second determination submodule is used to determine a saturation adjustment weight based on the low saturation protection weight. The low saturation protection weight is positively correlated with the saturation adjustment weight; when the pixel's saturation value is less than a predetermined lower saturation limit, the low saturation protection weight is positively correlated with the pixel's saturation value, or the low saturation protection weight is a predetermined minimum low saturation protection weight.

[0179] According to another embodiment of this disclosure, the saturation adjustment weight determination module includes a high saturation protection weight determination submodule and a third determination submodule. The high saturation protection weight determination submodule is used to determine a high saturation protection weight for any pixel in the first image based on the pixel's saturation value. The third determination submodule is used to determine a saturation adjustment weight based on the high saturation protection weight. The high saturation protection weight is positively correlated with the saturation adjustment weight; when the pixel's saturation value is greater than a saturation upper limit threshold, the high saturation protection weight is negatively correlated with the pixel's saturation value, or the high saturation protection weight is a predetermined minimum high saturation protection weight.

[0180] According to another embodiment of this disclosure, the saturation adjustment weight determination module includes: a first skin tone protection weight determination submodule and a fourth determination submodule. The first skin tone protection weight determination submodule is used to determine a first skin tone protection weight for any pixel in the first image based on a first target distance between the pixel's hue value and at least one reference hue value. The fourth determination submodule is used to determine a saturation adjustment weight based on the first skin tone protection weight. The first skin tone protection weight is negatively correlated with the saturation adjustment weight; the first skin tone protection weight is also negatively correlated with the first target distance.

[0181] According to another embodiment of this disclosure, at least one reference hue value includes a first reference hue value and a second reference hue value; the first skin tone protection weight determination submodule includes: a distance determination unit, a first target distance determination unit, and a weight determination unit. The distance determination unit is used to determine, for any pixel in the first image, a first distance between the hue value of the pixel and the first reference hue value, and a second distance between the hue value of the pixel and the second reference hue value. The first target distance determination unit is used to determine a first target distance based on the first distance and the second distance. The weight determination unit is used to determine a first skin tone protection weight based on the first target distance.

[0182] According to another embodiment of this disclosure, it further includes: a brightness enhancement module, used to adjust the red channel value, green channel value, and blue channel value of pixels in the color-enhanced image using a gamma correction index, so as to obtain a brightness-enhanced image.

[0183] According to another embodiment of this disclosure, the system further includes a target processing algorithm determination module and a sharpness enhancement module. The target processing algorithm determination module is used to determine a target processing algorithm from a plurality of candidate processing algorithms based on at least one of the image resolution of the second image and device information of the device used to process the second image; the second image is a color-enhanced image or a brightness-enhanced image. The sharpness enhancement module is used to process the second image according to the target processing algorithm to obtain a sharpness-enhanced image.

[0184] According to another embodiment of this disclosure, the target processing algorithm determination module includes: a first algorithm determination submodule, a second algorithm determination submodule, a third algorithm determination submodule, and a fourth algorithm determination submodule. The first algorithm determination submodule is used to determine a sharpening algorithm for image sharpening as the target processing algorithm in response to detecting that the image resolution is greater than or equal to a first predetermined resolution. The second algorithm determination submodule is used to determine a pre-value mapping algorithm as the target processing algorithm in response to detecting that the image resolution is less than or equal to a second predetermined resolution, wherein the pre-value mapping algorithm is used to treat the second image as an image with enhanced sharpness. The third algorithm determination submodule is used to determine a stretching algorithm for pixel interpolation and a sharpening algorithm for image sharpening as target processing algorithms in response to detecting that the image resolution is greater than the second predetermined resolution and less than the first predetermined resolution, and that the device information is within a predetermined range. The fourth algorithm determination submodule is used to determine a sharpening algorithm for image sharpening as the target processing algorithm in response to detecting that the image resolution is greater than the second predetermined resolution and less than the first predetermined resolution, and that the device information is outside a predetermined range.

[0185] According to another embodiment of this disclosure, the candidate processing algorithm includes a sharpening algorithm for image sharpening. The sharpening algorithm includes a target sharpening weight determination module, a target adjustment amount determination module, and a brightness adjustment module. The target sharpening weight determination module determines a target sharpening weight for any pixel in the image to be sharpened, based on at least one of the pixel sharpening weight and a second skin tone protection weight. The image to be sharpened is a second image or an image obtained by performing other image processing on a second image. The pixel sharpening weight characterizes the degree of pixel sharpening, and the pixel sharpening weight is negatively correlated with the target sharpening weight. The second skin tone protection weight is negatively correlated with a second target distance between the hue value of the pixel and at least one reference hue value, and the second skin tone protection weight is negatively correlated with the target sharpening weight. The target adjustment amount determination module determines a target adjustment amount based on the target sharpening weight and a base adjustment amount. The target sharpening weight is negatively correlated with the target adjustment amount. The brightness adjustment module adjusts the brightness value of the pixel (i.e., the pixel in the image to be sharpened) using the target adjustment amount.

[0186] According to another embodiment of this disclosure, the sharpening algorithm further includes: a search module, a sharpening intensity determination module, and a pixel sharpening weight module. The search module is used to search for multiple surrounding pixels in the image to be sharpened, starting from or centered on that pixel, along at least one predetermined direction, to obtain at least one pixel set corresponding to each of the at least one predetermined direction. The sharpening intensity determination module is used to determine, for each pixel set in the at least one pixel set, a sharpening intensity characterizing the degree of difference between the brightness values ​​of the pixels in the pixel set, based on the relative positional relationship of the pixels in the pixel set and the brightness values ​​of the pixels in the pixel set. The pixel sharpening weight module is used to determine a pixel sharpening weight based on at least one sharpening intensity of the at least one pixel set.

[0187] According to another embodiment of this disclosure, the sharpening intensity determination module includes: a partitioning submodule, a brightness difference determination submodule, and a sharpening intensity determination submodule. The partitioning submodule is used to divide multiple pixels in a pixel set into two subsets based on their relative positional relationships. The brightness difference determination submodule is used to determine the maximum brightness difference between the brightness values ​​of each pixel in each subset. The sharpening intensity determination submodule is used to determine the sharpening intensity based on the two maximum brightness differences between the two subsets.

[0188] According to another embodiment of this disclosure, the candidate processing algorithm includes a stretching algorithm for pixel interpolation. The stretching algorithm includes a mapping module, an expansion module, a pixel weight determination module, a target brightness value determination module, and a setting module. The mapping module maps pixel coordinates in the scaled image to the original pixel coordinates in the image before scaling, according to a predetermined scaling ratio; the image before scaling is a second image or an image obtained by performing other image processing on the second image. The expansion module expands the image before scaling from the pixel indicated by the integer value in the original pixel coordinates to surrounding pixels, obtaining an expanded region including multiple pixels. The pixel weight determination module determines the pixel weights of the multiple pixels in the expanded region based on the small values ​​in the original pixel coordinates and a pre-configured correspondence between small values ​​and a weight set. The target brightness value determination module determines the target brightness value for the pixels in the scaled image based on the brightness values ​​of each pixel in the expanded region and the pixel weights of the multiple pixels in the expanded region. The setting module sets the brightness values ​​of the pixels in the scaled image according to the target brightness value, obtaining the scaled image.

[0189] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0190] Figure 8This is a structural block diagram of an electronic device used to implement the image processing method of the embodiments of this disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the disclosure described and / or claimed herein.

[0191] like Figure 8 As shown, device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 802 or a computer program loaded from storage unit 808 into random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0192] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0193] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as image processing methods. For example, in some embodiments, the image processing method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the image processing method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform image processing methods by any other suitable means (e.g., by means of firmware).

[0194] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0195] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0196] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0197] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0198] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0199] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other.

[0200] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0201] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. An image processing method, comprising: For each pixel in the first image, a saturation adjustment weight is determined based on at least one of the pixel's hue value, saturation value, and brightness value. The target brightness value is determined based on the range of brightness values ​​of the pixel. The saturation value of the pixel is adjusted according to the saturation adjustment weight, and the brightness value of the pixel is adjusted according to the target brightness value to obtain a color-enhanced image.

2. The method according to claim 1, wherein, Determining the target brightness value based on the range of brightness values ​​of the pixel includes: A reference brightness value is determined based on the brightness value of the pixel and a predetermined contrast coefficient; The target brightness value is determined based on the range of brightness values ​​of the pixel and the reference brightness value. Specifically, when the brightness value of a pixel is greater than or equal to a first brightness value and less than or equal to a second brightness value, the target brightness value is the same as the reference brightness value; when the brightness value of a pixel is less than the first brightness value, the target brightness value is greater than the reference brightness value; and when the brightness value of a pixel is greater than the second brightness value, the target brightness value is less than the reference brightness value.

3. The method according to claim 2, wherein, Determining the target brightness value based on the range of brightness values ​​of the pixel and the reference brightness value includes: The target brightness value is determined based on the brightness value of the pixel, a first weight for the brightness value of the pixel, the reference brightness value, and a second weight for the reference brightness value. Wherein, when the brightness value of the pixel is less than the first brightness value, the first weight is negatively correlated with the brightness value of the pixel, and the second weight is positively correlated with the brightness value of the pixel; When the brightness value of the pixel is greater than the second brightness value, the first weight is positively correlated with the brightness value of the pixel, and the second weight is negatively correlated with the brightness value of the pixel.

4. The method according to claim 2, wherein, If the brightness value of a pixel is less than the reference brightness value, the reference brightness value is less than the brightness value of the pixel. When the brightness value of the pixel is greater than the reference brightness value, the reference brightness value is greater than the brightness value of the pixel. Wherein, the reference lightness value is greater than the first lightness value, and the reference lightness value is less than the second lightness value.

5. The method according to claim 1, wherein, The step of determining the saturation adjustment weight for a pixel in the first image based on at least one of the pixel's hue value, saturation value, and brightness value includes: For any pixel in the first image, a high brightness protection weight is determined based on the brightness value of the pixel; The saturation adjustment weight is determined based on the high brightness protection weight; The high brightness protection weight is positively correlated with the saturation adjustment weight; when the brightness value of a pixel is greater than a predetermined upper limit of brightness, the high brightness protection weight is negatively correlated with the brightness value of the pixel, or the high brightness protection weight is a predetermined minimum brightness protection weight.

6. The method according to claim 1, wherein, The step of determining the saturation adjustment weight for a pixel in the first image based on at least one of the pixel's hue value, saturation value, and brightness value includes: For any pixel in the first image, a low saturation protection weight is determined based on the pixel's saturation value; The saturation adjustment weight is determined based on the low saturation protection weight; Wherein, the low saturation protection weight is positively correlated with the saturation adjustment weight; when the saturation value of the pixel is less than a predetermined saturation lower limit, the low saturation protection weight is positively correlated with the saturation value of the pixel, or the low saturation protection weight is a predetermined minimum low saturation protection weight.

7. The method according to claim 1, wherein, The step of determining the saturation adjustment weight for a pixel in the first image based on at least one of the pixel's hue value, saturation value, and brightness value includes: For any pixel in the first image, a high saturation protection weight is determined based on the pixel's saturation value; The saturation adjustment weight is determined based on the high saturation protection weight; The high saturation protection weight is positively correlated with the saturation adjustment weight; when the saturation value of a pixel is greater than the upper limit threshold of saturation, the high saturation protection weight is negatively correlated with the saturation value of the pixel, or the high saturation protection weight is a predetermined minimum high saturation protection weight.

8. The method according to claim 1, wherein, The step of determining the saturation adjustment weight for a pixel in the first image based on at least one of the pixel's hue value, saturation value, and brightness value includes: For any pixel in the first image, a first skin color protection weight is determined based on a first target distance between the pixel's hue value and at least one reference hue value. The saturation adjustment weight is determined based on the first skin color protection weight; The first skin color protection weight is negatively correlated with the saturation adjustment weight; the first skin color protection weight is negatively correlated with the first target distance.

9. The method according to claim 8, wherein, The at least one reference hue value includes a first reference hue value and a second reference hue value; determining the first skin tone protection weight for any pixel in the first image based on a first target distance between the pixel's hue value and at least one reference hue value includes: For any pixel in the first image, a first distance is determined between the hue value of the pixel and the first reference hue value, and a second distance is determined between the hue value of the pixel and the second reference hue value; The first target distance is determined based on the first distance and the second distance; The first skin color protection weight is determined based on the first target distance.

10. The method according to claim 1, wherein, Also includes: By using the gamma correction index, the red, green, and blue channel values ​​of pixels in the color-enhanced image are adjusted respectively to obtain a brightness-enhanced image.

11. The method according to claim 1, wherein, Also includes: The target processing algorithm is determined from a plurality of candidate processing algorithms based on at least one of the image resolution of the second image and the device information of the device used to process the second image; The second image is either a color-enhanced image or a brightness-enhanced image; The second image is processed according to the target processing algorithm to obtain an image with enhanced sharpness.

12. The method according to claim 11, wherein, The step of determining the target processing algorithm from multiple candidate processing algorithms based on at least one of the image resolution of the second image and the device information of the device used to process the second image includes: In response to detecting that the image resolution is greater than or equal to a first predetermined resolution, the sharpening algorithm used for image sharpening is determined as the target processing algorithm; In response to detecting that the image resolution is less than or equal to a second predetermined resolution, the original value mapping algorithm is determined as the target processing algorithm, wherein the original value mapping algorithm is used to treat the second image as a sharpness-enhanced image; In response to detecting that the image resolution is greater than the second predetermined resolution and less than the first predetermined resolution, and that the device information is within a predetermined range, the stretching algorithm for pixel interpolation and the sharpening algorithm for image sharpening are determined as the target processing algorithm. In response to detecting that the image resolution is greater than the second predetermined resolution and less than the first predetermined resolution, and that the device information is outside the predetermined range, the sharpening algorithm used for image sharpening is determined as the target processing algorithm.

13. The method according to claim 11, wherein, The candidate processing algorithm includes a sharpening algorithm for image sharpening, the sharpening algorithm comprising: For any pixel in the image to be sharpened, a target sharpening weight is determined based on at least one of the pixel sharpening weight and the second skin tone protection weight; wherein, the image to be sharpened is the second image or an image obtained by performing other image processing on the second image; the pixel sharpening weight characterizes the degree of pixel sharpening, and the pixel sharpening weight is negatively correlated with the target sharpening weight; the second skin tone protection weight is negatively correlated with a second target distance between the hue value of a pixel in the image to be sharpened and at least one reference hue value, and the second skin tone protection weight is negatively correlated with the target sharpening weight; The target adjustment amount is determined based on the target sharpening weight and the base adjustment amount; wherein the target sharpening weight and the target adjustment amount are positively correlated. The brightness values ​​of pixels in the image to be sharpened are adjusted using the target adjustment amount.

14. The method according to claim 13, wherein, The sharpening algorithm also includes: For any pixel in the image to be sharpened, starting from or centering on that pixel, search for multiple surrounding pixels along at least one predetermined direction in the image to be sharpened, thereby obtaining at least one set of pixels corresponding to each of the at least one predetermined direction. For each pixel set in at least one pixel set, a sharpening intensity characterizing the degree of difference between the brightness values ​​of the pixels in the pixel set is determined based on the relative positional relationship of the pixels in the pixel set and the brightness value of the pixels in the pixel set. The pixel sharpening weight is determined based on at least one sharpening intensity of at least one set of pixels.

15. The method according to claim 14, wherein, The step of determining the sharpening intensity, which characterizes the degree of difference between the brightness values ​​of the pixels in at least one pixel set, based on the relative positional relationship of the pixels in the pixel set and the brightness values ​​of the pixels in the pixel set, includes: Based on the relative positional relationship, the pixels in the pixel set are divided into two subsets; For each of the two subsets, determine the maximum brightness difference of the brightness values ​​of each pixel in the subset; The sharpening intensity is determined based on the two maximum brightness differences between the two subsets.

16. The method according to claim 11, wherein, The candidate processing algorithm includes a stretching algorithm for pixel interpolation, the stretching algorithm comprising: According to a predetermined scaling ratio, the pixel coordinates in the scaled image are mapped to the original pixel coordinate values ​​in the image before scaling; the image before scaling is the second image or an image obtained by performing other image processing on the second image; In the image before scaling, the region is expanded outward from the pixel indicated by the integer value in the original pixel coordinates to obtain an expanded region including multiple pixels. Based on the small values ​​in the original pixel coordinates and the pre-configured correspondence between the small values ​​and the weight set, the pixel weights of multiple pixels in the extended region are determined. Based on the brightness values ​​of each pixel in the extended region and the pixel weights of each pixel in the extended region, the target brightness value for the pixels in the scaled image is determined. Based on the target brightness value, the brightness values ​​of the pixels in the scaled image are set to obtain the scaled image.

17. An image processing apparatus, comprising: The saturation adjustment weight determination module is used to determine the saturation adjustment weight for a pixel in the first image based on at least one of the pixel's hue value, saturation value, and brightness value. The target brightness value determination module is used to determine the target brightness value based on the value range of the brightness value of the pixel; The color adjustment module is used to adjust the saturation value of the pixel according to the saturation adjustment weight, and to adjust the brightness value of the pixel according to the target brightness value, so as to obtain a color-enhanced image.

18. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 16.

19. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1 to 16.

20. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 16.

Images