Cloud game client image enhancement optimization method and device

Abstract

The application provides a cloud game client image enhancement optimization method and device, the method comprising: for each pixel point of a to-be-processed client image, obtaining pixel color values of the current pixel point and a plurality of adjacent pixel points around the current pixel point; calculating a gradient of the pixel color values according to the pixel color values of the plurality of adjacent pixel points, and calculating an adaptive sharpening threshold unsharpThreshold; calculating a maximum pixel color value mxVal and a minimum pixel color value mnVal of the current pixel point and the plurality of adjacent pixel points, and calculating an adaptive sharpening intensity unsharpAmount based on the mxVal and the mnVal; performing enhancement processing on the current pixel point by using the unsharpThreshold and the unsharpAmount, performing amplitude limiting protection on the enhancement result, and obtaining an enhanced pixel color value. The application effectively solves the problem of poor enhancement effect caused by fixed cloud game client image enhancement parameters, the sharpening threshold and the sharpening intensity can both be dynamically changed according to the image scene, and the GPU calculation amount is significantly reduced, thereby improving the image quality and smoothness of the cloud game on a mobile terminal.

Description

Technical Field

[0001] This invention relates to the field of cloud computing, and in particular to a method and apparatus for enhancing and optimizing images in a cloud gaming client. Background Technology

[0002] With the rapid development of cloud gaming technology, users' demands for cloud game image quality are increasing. The image processing chain for cloud gaming is as follows: the server renders the game screen, which is then captured, encoded, compressed, and transmitted over the network to the client. The client then decodes the image and renders it for display. In this chain, encoding, compression, and network transmission inevitably lead to a decrease in image quality. To compensate for this loss of image quality, image enhancement processing is typically performed on the client side to improve the final display quality by highlighting image details.

[0003] The existing cloud gaming client image enhancement scheme uses the Unsharp Mask algorithm, which has the following steps: Gaussian blurring is applied to the current pixel in a 7×7 pixel neighborhood to obtain the blurred pixel color value blurColor; the difference between the current pixel color value and blurColor is used to obtain the detail mask mask; when the mask brightness is greater than the fixed sharpening threshold unsharpThreshold, the pixel color value of the current pixel is added to the mask multiplied by the fixed sharpening intensity unsharpAmount.

[0004] However, the above solution has the following drawbacks: Defect 1: The unsharpAmount sharpening intensity is a fixed parameter, which cannot adapt to different game scenarios. When the value is too small, the enhancement effect is not obvious; when it is too large, white edges, black edges, or noise amplification problems are likely to occur in bright or dark edge areas.

[0005] Defect 2: The unsharpThreshold sharpening threshold is a fixed parameter and cannot adapt to different image content. When the value is too small, flat areas of the image are mistakenly enhanced, thus introducing noise; when the value is too large, the enhancement effect on effective detail areas is suppressed.

[0006] Defect 3: Gaussian blur uses a 7×7 pixel matrix, requiring weighted calculations on 49 pixels, resulting in a large computational load. On low-end mobile devices, this operation leads to excessive GPU usage, causing increased image rendering latency and screen stuttering, severely impacting the cloud gaming experience.

[0007] There is currently no effective solution that can simultaneously address all three types of problems. Summary of the Invention

[0008] The purpose of this invention is to provide an image enhancement and optimization method for cloud gaming clients, so as to solve the problems of fixed sharpening parameters that cannot adapt to image scenes and excessive computational cost of Gaussian blur in existing solutions.

[0009] In a first aspect, embodiments of this application provide a cloud gaming client image enhancement and optimization method, including: In the image color space, for each pixel of the client image to be processed, the pixel color value of the current pixel and the pixel color values ​​of multiple neighboring pixels around the current pixel are obtained, wherein the number of multiple neighboring pixels is less than 48. The gradient of pixel color value is calculated based on the pixel color values ​​of multiple adjacent pixels, and the adaptive sharpening threshold unsharpThreshold is calculated based on the gradient of pixel color value. Calculate the maximum pixel color value mxVal and the minimum pixel color value mnVal of the current pixel and multiple neighboring pixels, and calculate the adaptive sharpening intensity unsharpAmount based on mxVal and mnVal; The current pixel is enhanced using unsharpThreshold and unsharpAmount, and the enhancement result is protected by limiting to obtain the enhanced pixel color value.

[0010] Secondly, embodiments of this application also provide a cloud gaming client image enhancement and optimization device, comprising: The acquisition unit is used to acquire the pixel color value of the current pixel and the pixel color values ​​of multiple neighboring pixels around the current pixel in the image color space for each pixel of the client image to be processed, wherein the number of multiple neighboring pixels is less than 48. Computational unit, used for: Calculate the gradient of pixel color values ​​based on the pixel color values ​​of multiple adjacent pixels, and calculate the adaptive sharpening threshold unsharpThreshold based on the gradient of pixel color values; and Calculate the maximum and minimum pixel color values ​​(mxVal) of the current pixel and its multiple neighboring pixels, and then calculate the adaptive sharpening intensity (unsharpAmount) based on mxVal and mnVal; and The current pixel is enhanced using unsharpThreshold and unsharpAmount, and the enhancement result is protected by limiting to obtain the enhanced pixel color value.

[0011] The beneficial effects of this invention are: 1. Adaptive Sharpening Threshold: The sharpening threshold is dynamically calculated based on the gradient of neighboring pixel color values. Regions with large gradients (i.e., large differences in pixel color values ​​between adjacent pixels) have a high threshold, enhancing only significant details; flat regions with small gradients have a low threshold, enhancing minor details while avoiding noise amplification. No manual adjustment of the threshold parameters is required, achieving good results in various game scenarios.

[0012] 2. Adaptive Sharpening Intensity: The sharpening intensity is dynamically calculated based on the extreme values ​​of neighboring pixel color values. When a pixel color value is close to the extreme value of the color space, the intensity is automatically reduced to avoid artifacts such as white edges and black edges in bright or dark areas. The intensity is higher when it is in the mid-tones to fully enhance the details.

[0013] 3. Significantly reduced computational load: The number of adjacent pixels is limited to less than 48, which is less than the number of adjacent pixels in a traditional 7×7 matrix. In practical applications, even fewer adjacent pixels (such as 4) can be selected, reducing the computational load by about 90% while retaining sufficient accuracy, significantly reducing the GPU load on low-end devices.

[0014] 4. Amplitude Limiting Protection: Amplitude limiting protection is applied to the enhancement results to fundamentally eliminate artifacts such as white edges and black edges caused by over-enhancement.

[0015] 5. Color space universality: The solution does not restrict the color space. RGB (red, green, blue), YUV (luminance-chrominance) or other color spaces can be directly applied, which has wide compatibility. Attached Figure Description

[0016] Figure 1 A flowchart illustrating an embodiment of a cloud gaming client image enhancement and optimization method provided in this application; Figure 2 This is a schematic diagram of an embodiment of a cloud gaming client image enhancement and optimization device provided in this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0018] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0019] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0020] Reference Figure 1 The diagram shown is a flowchart illustrating a cloud gaming client image enhancement and optimization method provided in an embodiment of this application. The method includes: S10. In the image color space, for each pixel of the client image to be processed, obtain the pixel color value of the current pixel and the pixel color values ​​of multiple neighboring pixels around the current pixel, wherein the number of multiple neighboring pixels is less than 48. The color space can be RGB, YUV, etc. Multiple adjacent pixels are symmetrically distributed around the current pixel, for example, by taking the four adjacent pixels above, below, left, and right of the current pixel. In this paper, the pixel color value can be the original value. To simplify computation, the pixel color value can also be a preprocessed value, which includes normalizing the components of the original value. The magnitude of the pixel color value is measured by the magnitude of the vector corresponding to the pixel color value. The number of multiple adjacent pixels is less than 48, which reduces the GPU load compared to the 48 adjacent pixels in the existing 7×7 matrix.

[0021] S11. Calculate the gradient of pixel color values ​​based on the pixel color values ​​of multiple adjacent pixels, and calculate the adaptive sharpening threshold unsharpThreshold based on the gradient of pixel color values. The adaptive sharpening threshold, unsharpThreshold, is calculated based on the gradient of pixel color values. Essentially, it determines the sharpening threshold based on the degree of change in the pixel color value at the current pixel's location. Areas with more drastic color value changes receive a larger sharpening threshold, while areas with less change receive a smaller threshold. This ensures that for strong edge areas (such as outlines and text), only the most prominent edges are enhanced, while for flat areas (such as the sky and walls), subtle details are also enhanced, while noise amplification is avoided. With adaptive sharpening thresholding, within the same frame, flat areas retain subtle textures, and edge areas only have clean outlines enhanced. The overall image is detailed yet smooth, avoiding the problems of overly smooth flat areas or excessively noisy edge areas.

[0022] S12. Calculate the maximum pixel color value mxVal and the minimum pixel color value mnVal of the current pixel and multiple adjacent pixels, and calculate the adaptive sharpening intensity unsharpAmount based on mxVal and mnVal. The adaptive sharpening intensity `unsharpAmount`, calculated using `mxVal` and `mnVal`, essentially determines the sharpening intensity based on the distance of the pixel color value in the current pixel's neighborhood (containing the current pixel and multiple adjacent pixels) from the color space boundary. The closer the pixel is to the color space boundary, the more sharpening it receives, potentially hitting the boundary, thus requiring a reduction in sharpening intensity to avoid white / black edges. Conversely, the farther the pixel is from the color space boundary, the less likely it is to overflow the boundary after sharpening, allowing for a safe increase in sharpening intensity. The final effect is: automatically adjusting the enhancement level in different brightness areas, ensuring detail enhancement while avoiding artifacts. For bright areas (such as sky highlights, white explosions, etc.), the sharpening intensity is low, preventing white edge overflow; for mid-brightness areas (such as grass, skin, etc.), the sharpening intensity is high, resulting in clear image details and rich textures; for dark areas (such as shadows, vignetting in night scenes, etc.), the sharpening intensity is low, preventing the deepening of black edges. With adaptive sharpening intensity, different brightness areas within the same frame receive appropriate sharpening intensity, resulting in a clear and clean overall image without artifacts caused by excessive enhancement in certain areas.

[0023] S13. Use unsharpThreshold and unsharpAmount to enhance the current pixel and limit the enhancement result to obtain the enhanced pixel color value.

[0024] The purpose of limiting the amplitude of the enhancement result is to ensure that the enhanced pixel color value does not exceed the reasonable range of the neighborhood.

[0025] Based on the aforementioned method embodiments, the formula for calculating the adaptive sharpening threshold unsharpThreshold can be: unsharpThreshold=mix(a,b,smoothstep(Glow,Ghigh,grad)), Where mix(a, b, t) = a × (1-t) + b × t, a and b are constant coefficients, t is the independent variable, smoothstep() is a function that limits the value of grad minus the lower bound of the interval to the interval length and then performs smooth interpolation, grad represents the degree of change of the color value of the pixels around the current pixel, Glow and Ghigh are the lower bound and upper bound of the interval, respectively, and the interval length is the difference between Ghigh and Glow.

[0026] In this formula, grad represents the degree of drastic change in the color values ​​of pixels surrounding the current pixel: grad close to 0 indicates that the color values ​​of pixels surrounding the current pixel are relatively similar, belonging to a flat region (such as a sky or a wall); grad larger indicates that the color values ​​of pixels surrounding the current pixel change drastically, belonging to an edge region (such as a person's outline or an object's boundary). In the scheme, grad is used to control the sharpening threshold: a low threshold for flat regions (enhancing details), and a high threshold for edge regions (enhancing only significant edges, avoiding noise amplification). Specifically, the calculation can be expressed as the square root of the sum of the squares of the magnitudes corresponding to the gradients of pixel color values ​​in two directions. The gradients of pixel color values ​​in the two directions can include the gradient gradX in the horizontal direction and the gradient gradY in the vertical direction, where gradX = (L_right - L_left) × 0.5 and gradY = (L_bottom - L_top) × 0.5, where L_right, L_left, L_bottom, and L_top are the adjacent pixels to the right, left, bottom, and top of the current pixel, respectively.

[0027] Where mix(a, b, t) is a linear interpolation function. The calculation process of smoothstep(Glow, Ghigh, grad) can be as follows: calculate P = clamp((grad-Glow) / (Ghigh-Glow), L, H), where clamp means subtracting the lower bound of the interval Glow from grad and dividing by the interval length (Ghigh-Glow), and limiting the amplitude to the preset interval [L, H], and the return value is P×P×(3-2×P).

[0028] In this embodiment, the preferred values ​​for each parameter are as follows: a=0.02, b=0.14, Glow=0.02, Ghigh=0.2, L=0, H=1. In flat regions (grad close to 0), the unsharpThreshold is close to 0.02, which is a low threshold and can enhance even slight details. In strong edge regions (grad close to 0.20 and above), the unsharpThreshold is close to 0.14, which is a high threshold and only enhances significant edges while suppressing noise amplification.

[0029] Based on the aforementioned method embodiments, the formula for calculating the adaptive sharpening intensity unsharpAmount can be: unsharpAmount=unsharpAmountBase×(α×(1-amp)+β×amp), Where unsharpAmountBase is the preset base sharpening intensity, α and β are constant coefficients, amp=clamp(magnitude of the vector corresponding to ampVal, 0, 1), clamp means that the magnitude of the vector corresponding to ampVal is limited to the interval [0, 1], ampVal is a comprehensive margin that measures the distance of the color value of the neighboring pixel from the extreme values ​​(pure white and pure black) in the color space, ampVal=min(mnVal, Vmax-mxVal) / mxVal, and Vmax is the maximum value of the pixel color value.

[0030] The subtraction in the formula on the right side of ampVal can be calculated by subtracting the corresponding color channel value of mxVal from each color channel value of Vmax. The division in the formula on the right side can be calculated by dividing each color channel value of the numerator by the corresponding color channel value of the denominator. Vmax represents the maximum value of a pixel's color value in the color space.

[0031] In this embodiment, the preferred values ​​for each parameter are as follows: unsharpAmountBase=1.5, α=0.35, β=1.25. When the color value of the neighboring pixel is close to the extreme value (too bright or too dark), amp is relatively small, and unsharpAmount approaches 1.5×0.35=0.525, resulting in low sharpening intensity and avoiding white or black edge artifacts in the extreme value area. When the color value of the neighboring pixel is in the mid-tone, amp is relatively large, and unsharpAmount approaches 1.5×1.25=1.875, resulting in high sharpening intensity and fully enhancing the image details in the mid-tone area.

[0032] Based on the aforementioned method embodiments, the enhancement processing of the current pixel using unsharpThreshold and unsharpAmount may include: Gaussian blur is applied to the current pixel using multiple adjacent pixels to obtain the blurred pixel color value blurVal; The detail mask is obtained by subtracting the pixel color value of the current pixel from blurVal, and the mask luminance is calculated. Calculate the smooth transition coefficient threshold based on maskLuminance and unsharpThreshold; The enhanced pixel color value outVal is synthesized based on the current pixel color value, mask, unsharpAmount, and threshold.

[0033] Based on the aforementioned method embodiments, the formula for calculating the blurred pixel color value blurVal can be: blurVal=centralVal×γ+neighborAvg×(1-γ), Where centralVal is the pixel color value of the current pixel, γ is the weighting coefficient, and neighborAvg is the average pixel color value of multiple neighboring pixels.

[0034] In this embodiment, the preferred value of γ is 0.52. Then the weight of the center pixel γ=0.52 is greater than the average weight of multiple adjacent pixels (1-γ)=0.48, which ensures that the blurring result is biased towards the center pixel and does not become overly smooth.

[0035] Based on the aforementioned method embodiments, the formula for calculating the smooth transition coefficient threshold can be: threshold=smoothstep(Tlow,Thigh,maskLuminance), Among them, smoothstep() is a function that limits the value of maskLuminance after subtracting the lower bound of the interval and dividing it by the length of the interval to a preset interval range before performing smooth interpolation. Tlow and Thigh are the lower bound and upper bound of the interval, respectively. Tlow = unsharpThreshold × ζ, Thigh = unsharpThreshold + η, where ζ and η are constant coefficients.

[0036] The form of the smoothstep function in this formula is the same as in the previous embodiment, and will not be repeated here. ζ and η are preferably set to 0.3 and 0.04, respectively. By setting the threshold, the enhancement level of the current pixel can be determined: a small maskLuminance (the pixel has almost no detail) → threshold close to 0 → no enhancement; a large maskLuminance (the pixel has significant detail) → threshold close to 1 → full enhancement; a maskLuminance in the middle → threshold smoothly transitions between 0 and 1, avoiding abrupt visual changes caused by hard switching between enhanced and non-enhanced areas.

[0037] Based on the aforementioned method embodiments, the formula for calculating the enhanced pixel color value outVal can be: outVal=mix(centralVal,T,threshold), Where mix(a, b, t) = a × (1-t) + b × t, T is the result after limiting centralVal + mask × unsharpAmount to the preset range, and centralVal is the pixel color value of the current pixel.

[0038] In this embodiment, the lower and upper limits of the preset interval can be taken from the extreme points in the color space. During the clipping process, each color channel value of centralVal + mask × unsharpAmount needs to be clipped to the channel value range corresponding to the preset interval.

[0039] Based on the aforementioned method embodiments, the step of limiting the enhancement result to obtain the enhanced pixel color value may include: The limiting margin is calculated based on the brightness gradient and unsharpThreshold, and the enhancement result is limited to the range of [mnVal-margin, mxVal+margin] to obtain the enhanced pixel color value after limiting protection.

[0040] The formula for calculating the margin is as follows: margin=clamp(λ+μ×gate,0,1), Here, `clamp()` represents the function of limiting λ + μ × gate to the interval [0, 1], where λ and μ are constant coefficients. `gate = smoothstep(Qlow, Qhigh, grad)` is a function that limits the value of grad minus the lower bound of the interval to the interval length and then performs smooth interpolation. `grad` represents the degree of change in the color value of the pixels around the current pixel. `Qlow` and `Qhigh` are the lower and upper bounds of the interval, respectively. `Qlow = unsharpThreshold` and `Qhigh = unsharpThreshold + τ` are constant coefficients.

[0041] The form of the function smoothstep in this formula is the same as in the previous example, and will not be repeated here.

[0042] In this embodiment, the preferred values ​​for each parameter are as follows: λ=0.01, μ=0.06, τ=0.04, and the gate is calculated based on grad using smoothstep: at strong edges (grad>Qhigh), the gate approaches 1, and the margin approaches 0.01+0.06=0.07, leaving sufficient space for edge contrast; in flat areas (grad<Qlow), the gate approaches 0, and the margin approaches 0.01, almost not allowing it to exceed the range of neighboring colors.

[0043] Ultimately, `outVal` is limited to the range [mnVal - margin, mxVal + margin], using the extreme values ​​of the actual pixel color values ​​in the neighborhood as a benchmark, plus an adaptive margin as upper and lower bounds. This ensures that the enhancement result does not deviate excessively from the reasonable color range of the neighborhood, fundamentally eliminating artifacts such as white edges and black edges caused by over-enhancement. It should be noted that limiting `outVal` to the range [mnVal - margin, mxVal + margin] means: if a color channel value of `outVal` is less than the color channel value corresponding to `mnVal - margin`, then the value of that color channel in `outVal` is adjusted to the value of the color channel corresponding to `mnVal - margin`; if a color channel value of `outVal` is greater than the color channel value corresponding to the color channel corresponding to `mxVal + margin`, then the value of that color channel in `outVal` is adjusted to the value of the color channel corresponding to the color channel corresponding to `mxVal + margin`; if a color channel value of `outVal` is neither less than the color channel value corresponding to `mnVal - margin` nor greater than the color channel value corresponding to the color channel corresponding to `mxVal + margin`, then the value of that color channel in `outVal` remains unchanged. When calculating mnVal-margin and mxVal+margin, the margin needs to be converted into pixel color values. During the conversion, if the pixel color value involved in the calculation is the original pixel color value in the color space, the margin needs to be converted into a pixel color value according to the ratio. For example, for RGB and YUV spaces, the ratio is margin:1, and the value of each color channel of the converted pixel color value is margin×255. If the pixel color value involved in the calculation is the normalized value of the original pixel color value in the color space, the margin needs to be converted into a pixel color value, and the value of each color channel of the converted pixel color value is the margin.

[0044] Reference Figure 2 The diagram shown is a structural schematic of a cloud gaming client image enhancement and optimization device provided in an embodiment of this application. The device includes: The acquisition unit 20 is used to acquire the pixel color value of the current pixel and the pixel color values ​​of multiple adjacent pixels around the current pixel for each pixel of the client image to be processed in the image color space, wherein the number of multiple adjacent pixels is less than 48. Calculation unit 21, used for: Calculate the gradient of pixel color values ​​based on the pixel color values ​​of multiple adjacent pixels, and calculate the adaptive sharpening threshold unsharpThreshold based on the gradient of pixel color values; and Calculate the maximum and minimum pixel color values ​​(mxVal) of the current pixel and its multiple neighboring pixels, and then calculate the adaptive sharpening intensity (unsharpAmount) based on mxVal and mnVal; and The current pixel is enhanced using unsharpThreshold and unsharpAmount, and the enhancement result is protected by limiting to obtain the enhanced pixel color value.

[0045] The cloud gaming client image enhancement and optimization device provided in this application embodiment is implemented in the same way as the cloud gaming client image enhancement and optimization method provided in this application embodiment, and the effect it can achieve is also the same as the cloud gaming client image enhancement and optimization method provided in this application embodiment, so it will not be described again here.

[0046] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for enhancing and optimizing images in a cloud gaming client, characterized in that, include: In the image color space, for each pixel of the client image to be processed, the pixel color value of the current pixel and the pixel color values ​​of multiple neighboring pixels around the current pixel are obtained, wherein the number of multiple neighboring pixels is less than 48. The gradient of pixel color value is calculated based on the pixel color values ​​of multiple adjacent pixels, and the adaptive sharpening threshold unsharpThreshold is calculated based on the gradient of pixel color value. Calculate the maximum pixel color value mxVal and the minimum pixel color value mnVal of the current pixel and multiple neighboring pixels, and calculate the adaptive sharpening intensity unsharpAmount based on mxVal and mnVal; The current pixel is enhanced using unsharpThreshold and unsharpAmount, and the enhancement result is protected by limiting to obtain the enhanced pixel color value.

2. The method as described in claim 1, characterized in that, The formula for calculating the adaptive sharpening threshold unsharpThreshold is: unsharpThreshold=mix(a,b,smoothstep(Glow,Ghigh,grad)), Where mix(a, b, t) = a × (1-t) + b × t, a and b are constant coefficients, t is the independent variable, smoothstep() is a function that limits the value of grad minus the lower bound of the interval to the interval length and then performs smooth interpolation, grad represents the degree of change of the color value of the pixels around the current pixel, Glow and Ghigh are the lower bound and upper bound of the interval, respectively, and the interval length is the difference between Ghigh and Glow.

3. The method as described in claim 1 or 2, characterized in that, The formula for calculating the adaptive sharpening intensity unsharpAmount is: unsharpAmount=unsharpAmountBase×(α×(1-amp)+β×amp), Where unsharpAmountBase is the preset base sharpening intensity, α and β are constant coefficients, amp=clamp(magnitude of the vector corresponding to ampVal, 0, 1), clamp means to limit the magnitude of the vector corresponding to ampVal to the interval [0, 1], ampVal=min(mnVal, Vmax-mxVal) / mxVal, Vmax is the maximum value of the pixel color value.

4. The method as described in claim 1, characterized in that, The enhancement process for the current pixel using unsharpThreshold and unsharpAmount includes: Gaussian blur is applied to the current pixel using multiple adjacent pixels to obtain the blurred pixel color value blurVal; The detail mask is obtained by subtracting the pixel color value of the current pixel from blurVal, and the mask luminance is calculated. Calculate the smooth transition coefficient threshold based on maskLuminance and unsharpThreshold; The enhanced pixel color value outVal is synthesized based on the current pixel color value, mask, unsharpAmount, and threshold.

5. The method as described in claim 4, characterized in that, The formula for calculating the blur pixel color value (blurVal) is: blurVal=centralVal×γ+neighborAvg×(1-γ), Where centralVal is the pixel color value of the current pixel, γ is the weighting coefficient, and neighborAvg is the average pixel color value of multiple neighboring pixels.

6. The method as described in claim 4, characterized in that, The formula for calculating the smooth transition coefficient threshold is: threshold=smoothstep(Tlow,Thigh,maskLuminance), Among them, smoothstep() is a function that limits the value of maskLuminance after subtracting the lower bound of the interval and dividing it by the length of the interval to the preset interval range before performing smooth interpolation. Tlow and Thigh are the lower bound and upper bound of the interval, respectively. Tlow=unsharpThreshold×ζ, Thigh=unsharpThreshold+η, where ζ and η are constant coefficients.

7. The method as described in claim 4, characterized in that, The formula for calculating the enhanced pixel color value outVal is: outVal=mix(centralVal,T,threshold), Where mix(a, b, t) = a × (1-t) + b × t, a and b are constant coefficients, t is the independent variable, T is the result after limiting centralVal + mask × unsharpAmount to the preset range, and centralVal is the pixel color value of the current pixel.

8. The method as described in claim 1, characterized in that, The process of limiting and protecting the enhancement result to obtain the enhanced pixel color value includes: The limiting margin is calculated based on the brightness gradient and unsharpThreshold, and the enhancement result is limited to the range of [mnVal-margin, mxVal+margin] to obtain the enhanced pixel color value after limiting protection.

9. The method as described in claim 8, characterized in that, The formula for calculating the margin is: margin=clamp(λ+μ×gate,0,1), Here, `clamp()` represents the function of limiting λ + μ × gate to the interval [0, 1], where λ and μ are constant coefficients. `gate = smoothstep(Qlow, Qhigh, grad)` is a function that limits the value of grad minus the lower bound of the interval to the interval length and then performs smooth interpolation. `grad` represents the degree of change in the color value of the pixels around the current pixel. `Qlow` and `Qhigh` are the lower and upper bounds of the interval, respectively. `Qlow = unsharpThreshold` and `Qhigh = unsharpThreshold + τ` are constant coefficients.

10. A cloud gaming client image enhancement and optimization device, characterized in that, include: The acquisition unit is used to acquire the pixel color value of the current pixel and the pixel color values ​​of multiple neighboring pixels around the current pixel in the image color space for each pixel of the client image to be processed, wherein the number of multiple neighboring pixels is less than 48. Computational unit, used for: Calculate the gradient of pixel color values ​​based on the pixel color values ​​of multiple adjacent pixels, and calculate the adaptive sharpening threshold unsharpThreshold based on the gradient of pixel color values; and Calculate the maximum and minimum pixel color values ​​(mxVal) of the current pixel and its multiple neighboring pixels, and then calculate the adaptive sharpening intensity (unsharpAmount) based on mxVal and mnVal; and The current pixel is enhanced using unsharpThreshold and unsharpAmount, and the enhancement result is protected by limiting to obtain the enhanced pixel color value.

Images