Image processing method and device, electronic equipment and storage medium

Abstract

The invention discloses an image processing method and device, electronic equipment and a storage medium, and relates to the technical field of image processing. The method comprises the following steps: acquiring an input image, performing up-sampling processing on a first to-be-processed pixel position in the input image to obtain an up-sampled image, performing interpolation processing on a first adjacent pixel position of the first to-be-processed pixel position in the up-sampled image to obtain an interpolated image, performing sharpening processing on the interpolated image to obtain a sharpened image, and outputting the sharpened image. A first output image is obtained. According to the method, the information of the adjacent pixel positions is obtained by performing interpolation processing on the adjacent pixel positions, so that sharpening operation can be performed in the same stage as up-sampling, additional input and output operation does not need to be introduced, and power consumption in the image super-division processing process is effectively reduced.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and more specifically, to an image processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] Super-resolution (SR) image processing aims to transform low-resolution images into high-resolution images with rich details. However, current methods for super-resolution image processing often suffer from excessive power consumption. Summary of the Invention

[0003] In view of the above problems, this application proposes an image processing method, apparatus, electronic device, and storage medium to solve the above problems.

[0004] In a first aspect, embodiments of this application provide an image processing method, the method comprising: acquiring an input image; performing upsampling processing on a first pixel position to be processed in the input image to obtain an upsampled image; performing interpolation processing on a first adjacent pixel position of the first pixel position to be processed in the upsampled image to obtain an interpolated image; and performing sharpening processing on the interpolated image to obtain a first output image.

[0005] Secondly, embodiments of this application provide an image processing apparatus, the apparatus comprising: an input image acquisition module for acquiring an input image; an upsampled image acquisition module for upsampling a first pixel position to be processed in the input image to obtain an upsampled image; an interpolated image acquisition module for interpolating a first adjacent pixel position of the first pixel position to be processed in the upsampled image to obtain an interpolated image; and a first output image acquisition module for sharpening the interpolated image to obtain a first output image.

[0006] Thirdly, embodiments of this application provide an electronic device, including a memory and a processor, wherein the memory is coupled to the processor, the memory stores instructions, and when the instructions are executed by the processor, the processor performs the above-described method.

[0007] Fourthly, embodiments of this application provide a computer-readable storage medium storing program code, which can be invoked by a processor to execute the above-described method.

[0008] The image processing method, apparatus, electronic device, and storage medium provided in this application embodiment acquire an input image, upsample the first pixel position to be processed in the input image to obtain an upsampled image, interpolate the first adjacent pixel position of the first pixel position to be processed in the upsampled image to obtain an interpolated image, and sharpen the interpolated image to obtain a first output image. By interpolating adjacent pixel positions, information about adjacent pixel positions is obtained, allowing sharpening to be performed at the same stage as upsampling, eliminating the need for additional input / output operations and effectively reducing power consumption during image super-resolution processing. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 A schematic flowchart of an image processing method provided in an embodiment of this application is shown; Figure 2 A schematic flowchart of an image processing method provided in an embodiment of this application is shown; Figure 3 A schematic flowchart of an image processing method provided in an embodiment of this application is shown; Figure 4 This application shows Figure 3 The flowchart of step S330 of the image processing method shown is illustrated. Figure 5 This application shows Figure 3 The flowchart of step S340 of the image processing method shown is illustrated. Figure 6 This application shows Figure 3 The flowchart of step S350 of the image processing method shown is illustrated. Figure 7 A schematic flowchart of an image processing method provided in an embodiment of this application is shown; Figure 8 A schematic flowchart of an image processing method provided in an embodiment of this application is shown; Figure 9 A flowchart illustrating the single-step super-resolution algorithm provided in an embodiment of this application is shown. Figure 10 This illustration shows a flowchart of super-resolution processing in the color space provided in an embodiment of this application; Figure 11A block diagram of an image processing apparatus provided in one embodiment of this application is shown; Figure 12 A block diagram of an electronic device for performing an image processing method according to an embodiment of this application is shown; Figure 13 An embodiment of the present application shows a storage unit for storing or carrying program code that implements the image processing method according to the embodiment of the present application. Detailed Implementation

[0011] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0012] FidelityFX Super Resolution (FSR) 1 is a super-resolution method developed by AMD. Super-resolution technology aims to convert low-resolution input images or video frames into higher-resolution output images, improving image clarity and detail. FSR1 achieves this goal through two key steps: upsampling and sharpening. Upsampling is the first step of FSR1, which uses a specific algorithm to enlarge the size of the low-resolution image, increasing the number of pixels and initially improving the image resolution to meet the target resolution requirements in terms of size. However, simple upsampling can make the image blurry and lose details. After upsampling, FSR1 enters the second step of sharpening. Sharpening enhances the contrast of image edges and details, making the image that became blurry after upsampling clear again, restoring lost details, and thus making the final high-resolution output image visually clearer and sharper.

[0013] AMD FidelityFX Adaptive Contrast Sharpening (CAS) is a graphics processing technology that simultaneously performs image upsampling and sharpening in a single process, with the sharpening level adaptively adjusted based on the local contrast of the image. Traditionally, image resolution enhancement and sharpening are separate steps, but AMD FidelityFX CAS integrates these two operations into a single workflow. This simplifies the process and reduces processing time and resource consumption. Instead of applying a uniform sharpening parameter to the entire image, this technology flexibly adjusts the sharpening level based on the local contrast of different areas. In high-contrast areas, such as object edges, the sharpening intensity is increased to make edges clearer and more prominent; in low-contrast, smooth areas, the sharpening intensity is appropriately reduced to avoid over-sharpening, which can produce noise or unnatural textures. Taking a landscape game scene as an example, the outline of the mountain peaks (high contrast area) will be sharpened more strongly to make the mountain peaks look more angular; while the sky (low contrast area) will be sharpened less strongly to maintain the smooth texture of the sky.

[0014] Snapdragon™ Game Super Resolution (Snapdragon™ GSR or SGSR) is a relatively efficient super-resolution method that requires only a single processing step. In game graphics processing, super-resolution technology is used to upscale low-resolution game visuals to high resolution, enhancing image quality. Most super-resolution algorithms may require multiple processing steps, consuming more computing resources and time. SGSR, however, achieves super-resolution with a single processing step, making it highly efficient. When playing games on electronic devices, the computing resources and battery power are limited. SGSR, with its single-process advantage, can quickly upscale the game's resolution without consuming excessive system resources, reducing jagged edges and blurring, allowing players to enjoy a clearer and smoother gaming experience on mobile devices.

[0015] However, the inventors discovered through research that the above-mentioned super-resolution method has at least the following drawbacks: FSR1 is a two-pass algorithm whose processing consists of two main steps. Due to the caching and retrieval operations of intermediate results, this algorithm has relatively high power consumption. Details are as follows: First pass: Edge Adaptive Spatial Upsampling (EASU): In the first pass of FSR1 processing, Edge Adaptive Spatial Upsampling (EASU) is used. The main purpose of this step is to upsample the input low-resolution image, that is, to increase the number of pixels in the image through a specific algorithm, thereby improving the image resolution. This upsampling method adaptively processes the edge information of the image, preserving edge details as much as possible and avoiding excessive blurring or jagged edges when the image is enlarged. After the EASU operation is completed, the resulting upsampled image is saved in the framebuffer.

[0016] Second pass: Robust Contrast Adaptive Sharpening (RCAS): In the second pass, the image stored in the frame buffer after the first pass of EASU processing is used as input, and Robust Contrast Adaptive Sharpening (RCAS) is applied to it. The role of RCAS is to sharpen the already upsampled image, making the image appear clearer and sharper by enhancing the contrast of the edges and details, thus further improving the visual effect of the image.

[0017] Because the FSR1 needs to save the result of the first EASU pass to the frame buffer during processing (this is a write operation), and then read the result from the frame buffer again during the second RCAS pass (this is a read operation), this caching and reading of intermediate results introduces additional input / output (I / O) overhead. I / O operations typically consume certain resources and energy, so the additional I / O operations make the FSR1's power consumption relatively high. In some scenarios with strict power consumption constraints, this may have a certain impact on the device's battery life and performance.

[0018] AMD FidelityFX Adaptive Contrast Sharpening (CAS) technology can sometimes result in jagged edges in images. In graphics processing, jagged edges manifest as uneven, stepped, or jagged edges on the edges of objects in an image. This is because while CAS can perform upsampling and sharpening simultaneously in a single process, adaptive sharpening, based on local contrast, can over-enhance the contrast of edge pixels. This is especially problematic when processing images with complex edges, such as slanted or curved lines. CAS's pixel adjustments may not precisely match the actual edge contours, easily causing abrupt pixel changes at the edges, resulting in a jagged appearance.

[0019] Snapdragon Super-Resolution for Gaming (SGSR) can produce jagged edges that appear "coarse" and unnatural. While SGSR is an efficient super-resolution method, it may not accurately reconstruct edge pixels when converting low-resolution images to high-resolution ones. When dealing with slanted or curved edges, the algorithm may simply fill or stretch pixels without adequately considering the smooth transition of real edges, resulting in a stepped, jagged effect. SGSR may overemphasize edge pixels when enhancing edges, leading to an increased number of pixels or abnormal contrast in edge areas, giving the visual impression of "thicker" edges. Furthermore, because this over-enhancement does not reflect the real-world appearance of object edges, the edges of objects in the image or scene look unnatural.

[0020] To address the aforementioned problems, the inventors, through long-term research, discovered and proposed the image processing method, apparatus, electronic device, and storage medium provided in the embodiments of this application. By interpolating the positions of adjacent pixels, information about the positions of adjacent pixels is obtained, allowing sharpening operations to be performed at the same stage as upsampling. This eliminates the need for additional input / output operations and effectively reduces power consumption during image super-resolution processing. The specific image processing method will be described in detail in subsequent embodiments.

[0021] Please see Figure 1 , Figure 1 A schematic flowchart of an image processing method according to an embodiment of this application is shown. This method obtains information about the positions of adjacent pixels by interpolating their positions, thereby enabling sharpening operations to be performed at the same stage as upsampling. This eliminates the need for additional input / output operations and effectively reduces power consumption during image super-resolution processing. In a specific embodiment, this image processing method is applied to, for example... Figure 11 The image processing device 200 and the electronic device 100 equipped with the image processing device 200 are shown. Figure 12 It is understood that the electronic devices used in this embodiment may include smartphones, tablets, in-vehicle systems, wearable devices, etc., and are not limited thereto. The following will focus on... Figure 1 The process shown will be described in detail. The image processing method may specifically include the following steps: Step S110: Obtain the input image.

[0022] In this embodiment, an input image can be acquired.

[0023] Optionally, the input image can be an image, a video frame, a rendered frame, etc. That is, an image can be directly obtained as the input image, or an image frame can be extracted from a video as the input image; there is no limitation here.

[0024] In some implementations, the image that the user expects to undergo super-resolution processing can be determined and obtained as the input image. For example, if the user expects to perform super-resolution processing on a certain image, that image can be obtained as the input image; if the user expects to perform super-resolution processing on a certain video, each video frame in the video can be extracted as the input image, without limitation.

[0025] This application's embodiments can be applied to both the HSV (Hue Saturation Value) color space and the RGB (Red Green Blue) color space. In the field of image processing, the RGB color space represents colors through different intensities of the three color channels: red, green, and blue, and is the most common color model used for displaying and storing images. The HSV color space, on the other hand, describes colors from three dimensions: hue, saturation, and value, which is more in line with how humans perceive color.

[0026] This application breaks the limitations of color spaces, enabling it to function effectively in two different color spaces. When applied to the RGB color space, it directly performs super-resolution processing on pixels in the red, green, and blue channels, improving the overall resolution and detail of the image. In the HSV color space, this method can leverage the characteristics of the HSV color space to perform more complex super-resolution algorithms on the luminance channel, while employing simpler algorithms, such as nearest-neighbor interpolation, for the hue and saturation channels, thereby reducing computational costs while maintaining visual quality. This characteristic allows the single-channel super-resolution method to demonstrate adaptability and flexibility in various image processing scenarios. Whether it's conventional RGB-based image processing or HSV-based scenarios that prioritize color perception and processing efficiency, it can effectively improve image resolution and optimize image quality.

[0027] Therefore, in this embodiment, when an input image is obtained, the first pixel position to be processed in the input image can be upsampled in the HSV color space and / or RGB color space to obtain an upsampled image.

[0028] Step S120: Upsample the first pixel position to be processed in the input image to obtain the upsampled image.

[0029] In this embodiment, when an input image is obtained, the first pixel position to be processed in the input image can be upsampled to increase the number of pixels in the input image, improve the resolution, and obtain an upsampled image.

[0030] Optionally, the input image may consist of multiple pixels, and the first pixel to be processed may be the position of any pixel among the multiple pixels that needs to be processed.

[0031] In some implementations, an upsampling method can be preset and stored. When an input image is acquired, the first pixel position to be processed in the input image can be upsampled using the preset and stored upsampling method to obtain an upsampled image.

[0032] As an feasible approach, upsampling methods can include bilinear interpolation. Given an input image, bilinear interpolation can be used to upsample the first pixel location to be processed, resulting in an upsampled image. Bilinear interpolation determines the value of the new pixel to be inserted by linearly calculating the values ​​of the four neighboring pixels of the pixel to be processed. After upsampling the selected first pixel location, the number of pixels in the image increases, the resolution is improved, and the upsampled image is obtained. Compared to the original input image, the upsampled image has richer details and a clearer picture.

[0033] As another feasible approach, upsampling methods can include bicubic interpolation. Therefore, given an input image, bicubic interpolation can be used to upsample the first pixel position in the input image to obtain an upsampled image.

[0034] Of course, other upsampling methods can also be used in this embodiment, which will not be elaborated here.

[0035] Step S130: Perform interpolation processing on the first neighboring pixel position of the first pixel position to be processed in the upsampled image to obtain the interpolated image.

[0036] In this embodiment, after obtaining the upsampled image, interpolation processing can be performed on the first adjacent pixel positions of the first pixel position to be processed in the upsampled image to further optimize the pixel values, smooth pixel transitions, and make the image more natural, thus obtaining the interpolated image. Specifically, after the interpolation processing is completed on the first adjacent pixel position, its pixel value is updated. When all adjacent pixel positions requiring processing in the entire upsampled image are processed sequentially using the above interpolation method, the transitions between image pixels become more natural and smooth, and the details become more coherent, ultimately resulting in the interpolated image with optimized visual effects.

[0037] Optionally, the position of the first pixel to be processed in the upsampled image can be the same as the position of the first pixel to be processed in the input image, and the position of the first adjacent pixel can be the position of the pixel directly adjacent to the pixel corresponding to the position of the first pixel to be processed. As an example, in the pixel grid of a two-dimensional image, adjacent pixels can be pixels that are adjacent to each other in the horizontal, vertical, or diagonal directions. For example, for a pixel with coordinates (x, y), its horizontal adjacent pixel coordinates may be (x-1, y) and (x+1, y), its vertical adjacent pixel coordinates may be (x, y-1) and (x, y+1), and its diagonal adjacent pixel coordinates may be (x-1, y-1), (x-1, y+1), (x+1, y-1), and (x+1, y+1).

[0038] In some implementations, an interpolation method can be preset and stored. When a sampled image is obtained, the first adjacent pixel position of the first pixel position to be processed in the upsampled image can be interpolated using the preset and stored interpolation method to obtain the interpolated image.

[0039] As an feasible approach, interpolation methods can include linear interpolation. Therefore, given an upsampled image, linear interpolation can be used to interpolate the first neighboring pixel position of the first pixel to be processed in the upsampled image, obtaining the interpolated image. For example, let the pixel value of the first neighboring pixel position be P1, its adjacent pixel value be P2, and the interpolation coefficient be t (0 ≤ t ≤ 1). The formula for calculating the new pixel value P is P = P1 × (1 - t) + P2 × t. For instance, if P1 = 100, P2 = 120, and t = 0.5, then P = 100 × (1 - 0.5) + 120 × 0.5 = 110.

[0040] As another feasible approach, interpolation methods can include bilinear interpolation. Given an upsampled image, bilinear interpolation can be used to interpolate the first neighboring pixel position of the first pixel to be processed in the upsampled image, obtaining the interpolated image. For the first neighboring pixel position, bilinear interpolation first performs linear interpolation on the adjacent pixels in the horizontal direction to obtain a temporary value; then, in the vertical direction, it combines the temporary values ​​from the corresponding positions in the upper and lower rows for linear interpolation to obtain the final interpolated pixel value. This method considers more information from neighboring pixels, resulting in a smoother interpolation effect.

[0041] As another feasible approach, the interpolation method can include bicubic interpolation. Given an upsampled image, bicubic interpolation can be used to interpolate the position of the first neighboring pixel of the first pixel to be processed in the upsampled image, obtaining the interpolated image. Bicubic interpolation constructs a bicubic polynomial function using 16 pixels (4×4 neighborhood) surrounding the pixel to be interpolated. By solving for the polynomial coefficients and substituting the coordinates of the first neighboring pixel into the function, the interpolated pixel value is calculated. This method is computationally intensive but better preserves image details.

[0042] Step S140: Sharpen the interpolated image to obtain the first output image.

[0043] In this embodiment, after obtaining the interpolated image, the interpolated image can be sharpened to increase the contrast of image edges and details, making the image clearer and obtaining the first output image. It is understood that the first output image is obtained from the input image through super-resolution processing, and compared to the input image, it is optimized in terms of resolution, clarity, etc.

[0044] After sharpening, the image edges are clearer, details are more obvious, and previously blurry object outlines become sharper, with textures becoming more distinct. The sharpened image (the first output image) shows a significant improvement in clarity and visual effect compared to the interpolated image, meeting subsequent usage requirements such as displaying it on a screen for user viewing or using it as input for other image processing algorithms.

[0045] In some implementations, a sharpening method can be preset and stored. Then, when an interpolated image is obtained, the interpolated image can be sharpened using the preset and stored sharpening method to obtain a first output image.

[0046] As one feasible approach, this sharpening method can include convolution-based sharpening. Thus, given an interpolated image, the interpolated image can be sharpened using convolution-based sharpening to obtain a first output image. This involves using a convolution kernel, such as a Laplacian kernel, to perform a weighted summation operation on each pixel in the image and its surrounding pixels, thereby highlighting edges and details in the image.

[0047] As another feasible approach, this sharpening method can include an anti-sharpening mask. Then, given the interpolated image, the anti-sharpening mask can be used to sharpen the interpolated image to obtain a first output image. Specifically, the anti-sharpening mask first applies a Gaussian blur to the original image, resulting in a blurred image. Then, the blurred image is subtracted from the original image to obtain a difference image containing image detail information. In this difference image, the pixel values ​​of edges and details differ significantly. Next, the difference image is multiplied by a coefficient (usually called a gain coefficient), and the result is added to the original image to enhance the image's edges and details, achieving a sharpening effect.

[0048] Therefore, this embodiment can obtain information about neighboring pixels without performing an upsampling step. In this way, the sharpening operation can be completed in the same step as the upsampling. That is, only the upsampling result of the target position is used, and the information of neighboring pixels is estimated from nearby input pixels through a relatively simple algorithm, avoiding the time-consuming operation of upsampling neighboring pixels. This allows sharpening and upsampling to be completed in the same step, improving processing efficiency.

[0049] An embodiment of this application provides an image processing method, apparatus, electronic device, and storage medium. The method acquires an input image, upsamples a first pixel position to be processed in the input image to obtain an upsampled image, interpolates the first adjacent pixel positions of the first pixel position to be processed in the upsampled image to obtain an interpolated image, and sharpens the interpolated image to obtain a first output image. By interpolating adjacent pixel positions, information about adjacent pixel positions is obtained, allowing sharpening to be performed at the same stage as upsampling. This eliminates the need for additional input / output operations and effectively reduces power consumption during image super-resolution processing.

[0050] Please see Figure 2 , Figure 2 A schematic flowchart of an image processing method according to an embodiment of this application is shown. The following will focus on... Figure 2 The process shown will be described in detail. The image processing method may specifically include the following steps: Step S210: Obtain the input image.

[0051] Step S220: Upsample the first pixel position to be processed in the input image to obtain the upsampled image.

[0052] Step S230: Perform interpolation processing on the first neighboring pixel position of the first pixel position to be processed in the upsampled image to obtain the interpolated image.

[0053] For a detailed description of steps S210-S230, please refer to steps S110-S130, which will not be repeated here.

[0054] Step S240: Based on the upsampling result of the first pixel position to be processed and the interpolation result of the first adjacent pixel position, sharpen the interpolated image to obtain the first output image.

[0055] Specifically, by upsampling the first pixel location to be processed in the input image, an upsampled result for that pixel location can be obtained. That is, a complex algorithm is used to upsample the first pixel location to generate a high-resolution pixel value. This value contains the detailed information preserved during the upsampling process; that is, the upsampled result for the first pixel location to be processed.

[0056] Specifically, by interpolating the first neighboring pixel position of the first pixel to be processed in the upsampled image, the interpolation result of the first neighboring pixel position can be obtained. That is, by using a simplified algorithm (such as bilinear interpolation) to estimate the pixel value of the neighboring positions (such as 3×3 or 5×5 neighborhoods) around the first pixel to be processed, these values ​​are used to provide local context information, avoid repeatedly calculating the complete upsampled result, and thus reduce the computational cost. That is, the interpolation result of the first neighboring pixel position.

[0057] In this process, after obtaining the upsampling result of the first pixel position to be processed and the interpolation result of the first adjacent pixel position, the upsampling result of the first pixel position to be processed can be used as the center and the interpolation result of the first adjacent pixel position can be used as the neighborhood to form a local pixel matrix. Subsequently, the contrast between pixels is enhanced by convolution operation to highlight edges and details, so as to sharpen the interpolated image and obtain the first output image.

[0058] An embodiment of this application provides an image processing method that acquires an input image, upsamples a first pixel position to be processed in the input image to obtain an upsampled image, interpolates the first neighboring pixel positions of the first pixel position to be processed in the upsampled image to obtain an interpolated image, and sharpens the interpolated image based on the upsampling result of the first pixel position to be processed and the interpolation result of the first neighboring pixel positions to obtain a first output image. Compared to Figure 1 The image processing method shown in this embodiment also sharpens the interpolated image based on the upsampling result of the pixel position to be processed and the interpolation result of the adjacent pixel positions. This can achieve sharpening operation at the same stage as upsampling, effectively reducing the power consumption in the image super-resolution process.

[0059] Please see Figure 3 , Figure 3 A schematic flowchart of an image processing method according to an embodiment of this application is shown. The following will focus on... Figure 3 The process shown will be described in detail. The image processing method may specifically include the following steps: Step S310: Obtain the input image.

[0060] For a detailed description of step S310, please refer to step S110, which will not be repeated here.

[0061] Step S320: Calculate the brightness information of the second neighboring pixel position of the second pixel position to be processed in the input image.

[0062] In this embodiment, given an input image, the brightness information of the second neighboring pixel positions of the second pixel position to be processed in the input image can be calculated. This obtains brightness information of a local region of the input image, facilitating a better inference of the appropriate brightness value for the second pixel position at high resolution, thereby improving the overall clarity and quality of the image.

[0063] Optionally, the input image may consist of multiple pixels, the second pixel position to be processed may be the position corresponding to any pixel among the multiple pixels that needs to be processed, and the second adjacent pixel position may be the position of the pixel directly adjacent to the pixel corresponding to the second pixel position to be processed. The second pixel position to be processed may be the same as or different from the first pixel position to be processed, and the second adjacent pixel position may be the same as or different from the first adjacent pixel position; no limitation is imposed here.

[0064] In some implementations, a brightness calculation method can be pre-set and stored. Then, given an input image, this brightness calculation method can be used to calculate the brightness information of the second neighboring pixel position of the second pixel position to be processed in the input image.

[0065] As an feasible approach, brightness calculation methods can include the averaging method. Given an input image, this averaging method can be used to calculate the brightness information of the second neighboring pixel positions of the second pixel to be processed in the input image. For example, for a pixel, considering its four directly adjacent pixels (up, down, left, and right in a two-dimensional image plane), the brightness values ​​of these four pixels are added together and divided by 4; the resulting average can be used as a brightness reference value for that region. If a wider neighborhood is considered, such as an 8-neighborhood (including diagonally opposite pixels), the brightness values ​​of the eight adjacent pixels are summed and divided by 8. This method is simple and intuitive, quickly obtaining an approximate brightness representation of the region, but may lose some detailed information.

[0066] As another feasible approach, brightness calculation methods can include weighted averaging. Given an input image, this weighted averaging method can be used to calculate the brightness information of the second neighboring pixels of the second pixel to be processed. For example, pixels closer to the second pixel to be processed are assigned higher weights, while those farther away are assigned lower weights. Using Gaussian weighting as an example, the weight of each second neighboring pixel is calculated based on a Gaussian function; pixels closer to the second pixel to be processed have a higher weight and are more dominant in the final brightness calculation result. This can more reasonably reflect local brightness characteristics and performs better in preserving image details.

[0067] As another feasible approach, brightness calculation methods can include gradient-based methods. Given an input image, this gradient-based method can be used to calculate the brightness information of the second neighboring pixel positions of the second pixel position to be processed in the input image. For example, the brightness difference between the second neighboring pixel positions in the horizontal and vertical directions is first calculated to obtain the brightness gradient value. If the gradient change is large in a certain direction, it indicates a significant brightness change in that direction. This gradient information can be comprehensively considered to better capture brightness changes in details such as image edges. For example, in image edge regions, the gradient is larger. This method can more accurately calculate brightness values ​​that conform to edge features, avoiding problems such as edge blurring after super-resolution processing.

[0068] Step S330: Perform super-resolution calculation in the brightness channel using the brightness information to obtain the estimated brightness of the second pixel position to be processed.

[0069] In this embodiment, when the brightness information of the second adjacent pixel position of the second pixel position to be processed in the input image is obtained by calculation, the brightness information can be used to perform super-resolution calculation in the brightness channel to obtain the estimated brightness of the second pixel position to be processed.

[0070] Understandably, in the HSV (Hue, Saturation, Value) color space of an image, the luminance channel reflects the brightness of the image, while hue and saturation are mainly related to the variety and vividness of colors. Super-resolution algorithms focus on improving image sharpness and detail, which are largely determined by luminance information. For example, details such as edges and contours in an image are primarily revealed through changes in luminance. In contrast, hue and saturation contribute less directly to improving image resolution. Therefore, performing super-resolution algorithms only on the luminance channel reduces computation and allows for more targeted enhancement of resolution-related parts of the image, concentrating algorithmic resources on critical luminance information processing and improving processing efficiency and effectiveness.

[0071] In some implementations, when the brightness information of the second neighboring pixel position of the second pixel position to be processed in the input image is obtained by calculation, the brightness value of the second pixel position to be processed can be estimated by super-resolution algorithm based on the known brightness information of the second neighboring pixel position, so as to obtain the estimated brightness of the second pixel position to be processed.

[0072] As one feasible approach, the super-resolution algorithm may include an interpolation algorithm. Therefore, after calculating the brightness information of the second neighboring pixel positions of the second pixel position to be processed in the input image, an interpolation algorithm can be used to perform super-resolution calculations in the brightness channel using this brightness information to obtain an estimated brightness of the second pixel position to be processed. Specifically, using an interpolation algorithm, the estimated brightness of the second pixel position can be calculated based on the brightness information of the second neighboring pixel positions using a specific interpolation formula. Optionally, the interpolation algorithm may include nearest-neighbor interpolation, bilinear interpolation, bicubic interpolation, etc., and is not limited thereto.

[0073] As another feasible approach, super-resolution algorithms can include machine learning-based methods. Thus, after calculating the brightness information of the second neighboring pixel position of the second pixel position to be processed in the input image, a machine learning-based method can be used to perform super-resolution calculations on the brightness channel using this brightness information to obtain an estimated brightness of the second pixel position. Specifically, using machine learning allows the network to learn mapping relationships during training, converting low-resolution brightness information into an estimated brightness of the second pixel position at a high resolution. Optionally, machine learning-based methods can include convolutional neural networks, generative adversarial networks, etc., without limitation.

[0074] As another feasible approach, the super-resolution algorithm may include frequency domain processing methods. Then, having calculated the brightness information of the second neighboring pixel position of the second pixel position to be processed in the input image, the frequency domain processing method can be used to perform super-resolution calculations in the brightness channel to obtain the estimated brightness of the second pixel position.

[0075] Please see Figure 4 , Figure 4 This application shows Figure 3 The diagram shows a flowchart of step S330 of the image processing method. The following will focus on... Figure 4 The process shown will be described in detail, and the method may specifically include the following steps: Step S331: Determine available computing resources.

[0076] In this embodiment, the platform (electronic device) on which the image processing method runs can be determined, and the idle computing resources of the platform can be determined.

[0077] As an feasible approach, operating system monitoring tools can be used to determine the available computing resources on a given platform. For example, Windows Task Manager can display real-time CPU, memory, disk I / O, and network usage; based on this information, available computing resources can be identified.

[0078] As another feasible approach, hardware monitoring software can be used to determine the available computing resources of the corresponding platform.

[0079] As another feasible approach, the idle computing resources of the corresponding platform can be determined through third-party monitoring applications.

[0080] Step S332: Determine the target super-resolution algorithm corresponding to the idle computing resources from multiple super-resolution algorithms.

[0081] This includes mobile devices (phones, tablets) and some low-configuration embedded devices. These devices have relatively small processor performance, memory capacity, and limited battery life. In situations with limited idle computing resources, using computationally intensive and complex super-resolution algorithms can lead to slow device operation or even lag, severely impacting the user experience. Therefore, choosing simple, single-pass algorithms with lower computational requirements is more suitable. Such algorithms include simple interpolation algorithms, like nearest neighbor interpolation. Its calculation process is simple, requiring only the finding of the nearest pixel and copying its value. It has low computational cost and fast processing speed, enabling rapid super-resolution processing on resource-constrained platforms, improving image resolution to a certain extent while ensuring smooth device operation.

[0082] These include high-performance computers and professional graphics workstations. These platforms are equipped with powerful processors, large amounts of memory, and high-performance graphics cards, providing ample computing resources. In this situation, with relatively high levels of available computing resources, choosing more complex algorithms with greater computational demands can fully leverage the hardware's advantages, achieving better super-resolution results. This results in higher quality and clearer images after super-resolution, making it ideal for scenarios with extremely high image quality requirements, such as professional image processing and high-definition video production.

[0083] Therefore, in this embodiment, multiple super-resolution algorithms can be pre-set, each requiring different amounts of idle computing resources. Based on this, given a fixed amount of idle computing resources, the super-resolution algorithm corresponding to that available resource can be selected as the target super-resolution algorithm from among the multiple algorithms.

[0084] As an example, suppose we pre-set multiple super-resolution algorithms, including a first super-resolution algorithm, a second super-resolution algorithm, and a third super-resolution algorithm, and multiple idle computing resources, including a first idle computing resource, a second idle computing resource, and a third idle computing resource, where the first super-resolution algorithm corresponds to the first idle computing resource, the second super-resolution algorithm corresponds to the second idle computing resource, and the third super-resolution algorithm corresponds to the third idle computing resource. Then, if the idle computing resource of the applied screen is determined to be the second idle computing resource, then the second super-resolution algorithm can be identified as the target super-resolution algorithm.

[0085] Step S333: Using the luminance information, perform super-resolution calculation in the luminance channel using the target super-resolution algorithm to obtain the estimated luminance.

[0086] In this embodiment, after calculating and obtaining the brightness information of the second adjacent pixel position of the second pixel position to be processed in the input image, and determining the target super-resolution algorithm from multiple super-resolution algorithms, the target super-resolution algorithm can use the brightness information to perform super-resolution calculation in the brightness channel to obtain the estimated brightness of the second pixel position to be processed.

[0087] Step S340: Obtain the estimated hue and estimated saturation of the second pixel position to be processed.

[0088] In this embodiment, given an input image, the estimated hue and estimated saturation of the second neighboring pixel positions of the second pixel position to be processed in the input image can be obtained. Since hue and saturation have a relatively small direct contribution to image resolution improvement in the HSV (Hue, Saturation, Value) color space of the image, a relatively simple algorithm can be used to obtain the estimated hue and estimated saturation of the second pixel position to be processed, thereby reducing power consumption during the super-resolution processing.

[0089] In some implementations, estimation methods for hue and saturation can be preset and stored. Then, given an input image, the estimated hue and saturation of the second pixel location to be processed can be obtained using the preset and stored estimation methods.

[0090] As an feasible approach, the estimation method can include nearest neighbor interpolation. Given an input image, nearest neighbor interpolation can be used to obtain the estimated hue and saturation of the second pixel location to be processed. Nearest neighbor interpolation involves finding the pixel closest to the second pixel location in the low-resolution image and directly assigning its hue and saturation values ​​to that location. For example, when enlarging a low-resolution image to generate a high-resolution image, for newly added pixels, the hue and saturation information of the nearest known pixel is directly used.

[0091] As another feasible approach, the estimation method can include bilinear interpolation. Given an input image, bilinear interpolation can be used to obtain the estimated hue and saturation of the second pixel location to be processed. Specifically, bilinear interpolation considers the hue and saturation information of the four neighboring pixels surrounding the second pixel location. It performs linear interpolation calculations in both the horizontal and vertical directions, first calculating two intermediate values ​​in the horizontal direction, and then interpolating these two intermediate values ​​in the vertical direction to obtain the estimated hue and saturation values ​​for the second pixel location.

[0092] Of course, in the process of obtaining the estimated hue and saturation of the second pixel location to be processed, image context information can be incorporated to improve the accuracy of the estimation. For example, in a landscape image, the hue and saturation of the sky region have a certain continuity and similarity. When estimating the hue and saturation of a second pixel location located in the sky region, the average hue and saturation characteristics of the surrounding large sky region can be referenced to make the estimation result more consistent with the visual effect of the entire scene. This can avoid the problem of the estimated hue and saturation being inconsistent with the overall scene due to the limitations of local information.

[0093] Furthermore, during the process of obtaining the estimated hue and saturation of the second pixel location to be processed, fine-tuning can be made by combining the estimated brightness. For example, in some brighter areas, appropriately reducing the saturation can make the colors look more natural and avoid being too vibrant and glaring; while in darker areas, moderately increasing the saturation can enhance the color expressiveness and prevent the image from appearing too dull. In this way, the overall visual effect of the image can be further optimized, achieving a better balance between hue, saturation, and brightness.

[0094] Please see Figure 5 , Figure 5 This application shows Figure 3 The diagram shows a flowchart of step S340 of the image processing method. The following will focus on... Figure 5 The process shown will be described in detail, and the method may specifically include the following steps: Step S341: Interpolate the position of the second adjacent pixel using the nearest neighbor interpolation method to obtain the pixel value of the second adjacent pixel position.

[0095] In some implementations, when an input image is obtained, the pixel value of the second neighboring pixel position of the second pixel position to be processed in the input image can be obtained by interpolating the second neighboring pixel position using the nearest neighbor interpolation method.

[0096] Nearest neighbor interpolation is a simple and direct interpolation algorithm. During image resolution upscaling, when it's necessary to determine the pixel value of the second neighboring pixel (a newly generated pixel position when constructing a high-resolution image), the nearest pixel to that position can be found in the original low-resolution image. Then, the color channel value of this nearest pixel is directly assigned to the second neighboring pixel position, thus obtaining the pixel value of the second neighboring pixel position through nearest neighbor interpolation.

[0097] Step S342: Based on the pixel value at the second adjacent pixel position, obtain the estimated hue and the estimated saturation.

[0098] In some implementations, when the pixel value at the second adjacent pixel position is obtained, the estimated hue and estimated saturation can be obtained based on the pixel value at the second adjacent pixel position.

[0099] In this process, when the pixel value at the second adjacent pixel position is obtained, since this pixel value contains hue and saturation information (in the HSV color space), the hue value can be directly extracted as the estimated hue, and the saturation value can be extracted as the estimated saturation. This allows for a simple way to obtain the estimated hue and saturation, reducing power consumption during processing.

[0100] Step S350: Based on the estimated brightness, the estimated hue, and the estimated saturation, obtain a second output image in the HSV color space.

[0101] In this embodiment, given the estimated brightness, estimated hue, and estimated saturation of the second pixel location to be processed, a second output image in the HSV color space can be obtained based on these estimated brightness, estimated hue, and estimated saturation. It is understood that the second output image is obtained by performing super-resolution processing on the brightness channel of the input image, and compared to the input image, it is optimized in terms of resolution, sharpness, etc.

[0102] Please see Figure 6 , Figure 6 This application shows Figure 3 The diagram shows a flowchart of step S350 of the image processing method. The following will focus on... Figure 6The process shown will be described in detail, and the method may specifically include the following steps: Step S351: Adjust the estimated brightness to obtain the adjusted brightness, and adjust the estimated saturation to obtain the adjusted saturation.

[0103] In some implementations, given the estimated brightness, estimated hue, and estimated saturation of the second pixel location to be processed, the estimated brightness can be adjusted to obtain the adjusted brightness, and the estimated saturation can be adjusted to obtain the adjusted saturation. Adjusting saturation can alter the vibrancy of colors; for example, in a landscape photograph, appropriately increasing saturation can make the blue sky appear bluer and the flowers more vibrant, enhancing visual impact; while decreasing saturation can create a retro, soft atmosphere. Adjusting brightness can change the lightness and darkness of an image; increasing brightness makes the image brighter and clearer, suitable for depicting sunny scenes; decreasing brightness can highlight shadows, enhancing the image's depth and mystery. This makes the image's color and brightness more consistent with the desired effect.

[0104] In some implementations, the display device corresponding to the second output image can be determined, and the estimated brightness can be adjusted based on the display device to obtain the adjusted brightness, and the estimated saturation can be adjusted to obtain the adjusted saturation. Different display devices exhibit differences in color and brightness representation; to ensure that the second output image presents a good effect on various devices, the saturation and brightness can be adjusted according to the display device.

[0105] Step S352: Based on the adjusted brightness, the estimated hue, and the adjusted saturation, obtain the second output image in the HSV color space.

[0106] In this embodiment, given the adjusted brightness, estimated hue, and adjusted saturation of the second pixel position to be processed, a second output image in the HSV color space can be obtained based on the adjusted brightness, estimated hue, and adjusted saturation. It is understood that the second output image is obtained by performing super-resolution processing on the luminance channel of the input image, and compared to the input image, it is optimized in terms of resolution, sharpness, etc.

[0107] An embodiment of this application provides an image processing method that acquires an input image, calculates the brightness information of the second neighboring pixel positions of the second pixel position to be processed in the input image, performs super-resolution calculation in the brightness channel using the brightness information to obtain the estimated brightness of the second pixel position to be processed, obtains the estimated hue and estimated saturation of the second pixel position to be processed, and obtains a second output image in the HSV color space based on the estimated brightness, estimated hue, and estimated saturation. Compared to Figure 1The image processing method shown in this embodiment performs super-resolution calculation only on the luminance channel, which can effectively reduce the amount of computation and reduce the power consumption in the image super-resolution process while improving the resolution.

[0108] Please see Figure 7 , Figure 7 A schematic flowchart of an image processing method according to an embodiment of this application is shown. The following will focus on... Figure 7 The process shown will be described in detail. The image processing method may specifically include the following steps: Step S410: Obtain the input image.

[0109] For a detailed description of step S410, please refer to step S110, which will not be repeated here.

[0110] Step S420: Calculate the brightness information of the second neighboring pixel position of the second pixel position to be processed in the input image.

[0111] Step S430: Perform super-resolution calculation in the brightness channel using the brightness information to obtain the estimated brightness of the second pixel position to be processed.

[0112] Step S440: Obtain the estimated hue and estimated saturation of the second pixel position to be processed.

[0113] Step S450: Based on the estimated brightness, the estimated hue, and the estimated saturation, obtain a second output image in the HSV color space.

[0114] For a detailed description of steps S420-S450, please refer to steps S320-S350, which will not be repeated here.

[0115] Step S460: Convert the second output image in the HSV color space to a second output image in the RGB color space.

[0116] Since the RGB color space is the more commonly used display color space, when a second output image in the HSV color space is obtained, it can be converted into a second output image in the RGB color space to meet display requirements.

[0117] In some implementations, the corresponding RGB values ​​can be calculated based on the hue (H), saturation (S), and lightness (V) values ​​in the HSV color space, thus achieving the conversion from HSV to RGB color space. Specifically, firstly, the position of the color on the color wheel is determined based on the hue value, and then the two channels in the RGB color space that participate in the main composition of the color are determined. Then, the values ​​of these two main channels are adjusted based on saturation and lightness, finally obtaining the value of each channel in the RGB color space. For example, when the hue H is in a certain range, the red (R) channel may be the main channel. The value of the R channel is determined by calculating the saturation (S) and lightness (V), while the values ​​of the green (G) and blue (B) channels are calculated according to certain rules.

[0118] An embodiment of this application provides an image processing method that acquires an input image, calculates the brightness information of the second neighboring pixel positions of the second pixel position to be processed in the input image, performs super-resolution calculation in the brightness channel using the brightness information to obtain an estimated brightness of the second pixel position to be processed, obtains an estimated hue and estimated saturation of the second pixel position to be processed, and obtains a second output image in HSV color space based on the estimated brightness, estimated hue, and estimated saturation, and converts the second output image in HSV color space to a second output image in RGB color space. Compared to Figure 1 The image processing method shown in this embodiment only performs super-resolution calculations on the luminance channel, and then converts the obtained image in the HSV color space to an image in the RGB color space. This can effectively reduce the amount of computation and reduce the power consumption in the image super-resolution process while improving the resolution.

[0119] Please see Figure 8 , Figure 8 A schematic flowchart of an image processing method according to an embodiment of this application is shown. The following will focus on... Figure 8 The process shown will be described in detail. The image processing method may specifically include the following steps: Step S510: Obtain the input image.

[0120] For a detailed description of step S510, please refer to step S110, which will not be repeated here.

[0121] Step S520: Determine the application type corresponding to the application to which the input image belongs.

[0122] In this embodiment, upon acquiring an input image, the application type corresponding to the application to which the input image belongs can be determined. Optionally, the application type may include game type, video playback type, instant messaging type, query type, navigation type, shopping type, etc., and is not limited here.

[0123] In some implementations, when an input image is obtained, its metadata can be parsed, and the application to which the input image belongs can be determined based on the parsing results.

[0124] In some implementations, mapping relationships can be pre-set and stored. These mapping relationships can include multiple applications, multiple application types, and correspondences between multiple applications and multiple application types. The correspondence between multiple applications and multiple application types can include: one application corresponding to one application type, multiple applications corresponding to one application type, one application corresponding to multiple application types, etc., and is not limited here. In this embodiment, when the application to which the input image belongs is determined, the application type corresponding to the application to which the input image belongs can be determined based on this mapping relationship, and this application type is taken as the application type corresponding to the application to which the input image belongs.

[0125] Step S530: If the application type is a game type, then the first pixel position to be processed is upsampled to obtain the upsampled image.

[0126] In this embodiment, after determining the application type corresponding to the application to which the input image belongs, it can be determined whether the application type is a game. If the application type is determined to be a game, it can be concluded that the user has high requirements for image display and needs to maintain a high-resolution display. Therefore, upsampling and subsequent sharpening processing can be performed on the first pixel position to be processed in the input image to improve the image resolution. If the application type is determined not to be a game, it can be concluded that the user has low requirements for image display and can maintain a low-resolution display. Therefore, upsampling and subsequent sharpening processing can be omitted on the first pixel position to be processed in the input image to reduce processing power consumption.

[0127] Step S540: Perform interpolation processing on the first neighboring pixel position of the first pixel position to be processed in the upsampled image to obtain the interpolated image.

[0128] Step S550: Sharpen the interpolated image to obtain the first output image.

[0129] For a detailed description of steps S540-S550, please refer to steps S130-S140, which will not be repeated here.

[0130] An embodiment of this application provides an image processing method that acquires an input image, determines the application type corresponding to the application to which the input image belongs, and if the application type is a game, upsamples the first pixel position to be processed to obtain an upsampled image, interpolates the first adjacent pixel positions of the first pixel position to be processed in the upsampled image to obtain an interpolated image, and sharpens the interpolated image to obtain a first output image. Compared to Figure 1 The image processing method shown in this embodiment also performs super-resolution processing on the input image when it belongs to a game application, in order to improve the display effect of the game screen.

[0131] Please see Figure 9 , Figure 9 A flowchart illustrating the single-step super-resolution algorithm provided in an embodiment of this application is shown. Figure 9 As shown, this single-step super-resolution algorithm includes the following steps: 1. Input image: This represents the original input image, which is usually a low-resolution image.

[0132] 2. Upsampling: This involves using algorithms to increase the number of pixels in an image, thereby improving its resolution and preparing it for subsequent processing.

[0133] 3. Interpolate at neighboring locations: This method performs interpolation calculations at neighboring locations on the upsampled image. Interpolation estimates the new pixel value based on the values ​​of surrounding pixels, filling in the positions of newly added pixels after upsampling and making the image smoother.

[0134] 4. Sharpen: Sharpens an interpolated image. Sharpen enhances the contrast of image edges and details, making the image appear sharper.

[0135] 5. Output image: After the above processing, the final super-resolution output image is obtained, which has improved resolution and visual effect compared with the original image.

[0136] Please see Figure 10 , Figure 10 A schematic diagram illustrating the super-resolution processing flow in the color space provided in an embodiment of this application is shown. Figure 10 As shown, the super-resolution processing in this color space includes the following steps: 1. Input image: This represents the original input image, which is usually a low-resolution image.

[0137] 2. Calculate brightness of neighboring pixels and perform super-resolution processing: Calculate the brightness of neighboring pixels and then perform super-resolution processing to obtain high-resolution brightness information.

[0138] 3. Estimate hue and saturation of the target location: Estimate the hue and saturation of the target location.

[0139] 4. Adjust saturation and brightness: Combine the results of the two branches above, adjust saturation and brightness, and further optimize image color parameters.

[0140] 5. Color space conversion (HSV to RGB): Convert an image from the HSV color space to the RGB color space, because the RGB color space is more commonly used in display devices.

[0141] 6. Output image: After the above processing, the final super-resolution output image is obtained, which has improved resolution and visual effect compared with the original image.

[0142] Please see Figure 11 , Figure 11 A block diagram of an image processing apparatus according to an embodiment of this application is shown. The following will focus on... Figure 11 The image processing device 200, illustrated in the block diagram, includes: an input image acquisition module 210, an upsampled image acquisition module 220, an interpolated image acquisition module 230, and a first output image acquisition module 240, wherein: The input image acquisition module 210 is used to acquire the input image.

[0143] The upsampled image acquisition module 220 is used to perform upsampling processing on the first pixel position to be processed in the input image to obtain the upsampled image.

[0144] Further, the upsampled image acquisition module 220 includes: a first upsampled image acquisition submodule, wherein: The first upsampled image acquisition submodule is used to perform upsampling processing on the first pixel position to be processed in the HSV color space and / or RGB color space to obtain the upsampled image.

[0145] Further, the upsampled image acquisition module 220 includes: an application type determination submodule and a second upsampled image acquisition submodule, wherein: The application type determination submodule is used to determine the application type corresponding to the application to which the input image belongs.

[0146] The second upsampled image acquisition submodule is used to perform upsampling processing on the first pixel position to be processed to obtain the upsampled image if the application type is a game type.

[0147] The interpolated image acquisition module 230 is used to perform interpolation processing on the first adjacent pixel position of the first pixel position to be processed in the upsampled image to obtain the interpolated image.

[0148] Further, the interpolated image acquisition module 230 includes: an interpolated image acquisition submodule, wherein: The interpolated image acquisition submodule is used to interpolate the first adjacent pixel position using bilinear interpolation to obtain the interpolated image.

[0149] The first output image acquisition module 240 is used to sharpen the interpolated image to obtain the first output image.

[0150] Further, the first output image acquisition module 240 includes: a first output image acquisition submodule, wherein: The submodule for obtaining the first output image is used to sharpen the interpolated image based on the upsampling result of the first pixel position to be processed and the interpolation result of the first adjacent pixel position, so as to obtain the first output image.

[0151] Furthermore, the image processing device 200 further includes: a brightness information calculation module, a brightness estimation acquisition module, an estimation information acquisition module, and a second output image acquisition module, wherein: The brightness information calculation module is used to calculate the brightness information of the second adjacent pixel position of the second pixel position to be processed in the input image.

[0152] The brightness estimation module is used to perform super-resolution calculations in the brightness channel using the brightness information to obtain the estimated brightness of the second pixel position to be processed.

[0153] Furthermore, the brightness estimation acquisition module includes: an idle computing resource determination submodule, a super-resolution algorithm determination submodule, and a brightness estimation acquisition submodule, wherein: The idle computing resource determination submodule is used to determine idle computing resources.

[0154] The super-resolution algorithm determination submodule is used to determine the target super-resolution algorithm corresponding to the idle computing resources from multiple super-resolution algorithms.

[0155] The estimated brightness submodule is used to perform super-resolution calculations in the brightness channel using the brightness information through the target super-resolution algorithm to obtain the estimated brightness.

[0156] The estimation information acquisition module is used to acquire the estimated hue and estimated saturation of the second pixel to be processed.

[0157] Furthermore, the estimation information acquisition module includes: a pixel value acquisition submodule and an estimation information acquisition submodule, wherein: The pixel value acquisition submodule is used to perform interpolation processing on the second adjacent pixel position using the nearest neighbor interpolation method to obtain the pixel value of the second adjacent pixel position.

[0158] The estimation information acquisition submodule is used to obtain the estimated hue and the estimated saturation based on the pixel value at the second adjacent pixel position.

[0159] The second output image acquisition module is used to obtain a second output image in the HSV color space based on the estimated brightness, the estimated hue, and the estimated saturation.

[0160] Furthermore, the second output image acquisition module includes: an information adjustment submodule and a second output image acquisition submodule, wherein: The information adjustment submodule is used to adjust the estimated brightness to obtain the adjusted brightness, and to adjust the estimated saturation to obtain the adjusted saturation.

[0161] The second output image acquisition submodule is used to obtain a second output image in the HSV color space based on the adjusted brightness, the estimated hue, and the adjusted saturation.

[0162] Furthermore, the image processing device 200 further includes: a second output image conversion module, wherein: The second output image conversion module is used to convert the second output image in the HSV color space into a second output image in the RGB color space.

[0163] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described device and module can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0164] In the several embodiments provided in this application, the coupling between modules can be electrical, mechanical, or other forms of coupling.

[0165] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0166] Please see Figure 12 This document illustrates a structural block diagram of an electronic device 100 provided in an embodiment of this application. The electronic device 100 can be a smartphone, tablet computer, e-reader, or other electronic device capable of running applications. The electronic device 100 in this application may include one or more of the following components: a processor 110, a memory 120, and one or more applications, wherein the one or more applications can be stored in the memory 120 and configured to be executed by one or more processors 110, and the one or more applications are configured to perform the methods described in the foregoing method embodiments.

[0167] The processor 110 may include one or more processing cores. The processor 110 connects to various parts of the electronic device 100 using various interfaces and lines, and performs various functions and processes data of the electronic device 100 by running or executing instructions, programs, code sets, or instruction sets stored in the memory 120, and by calling data stored in the memory 120. Optionally, the processor 110 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 110 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content to be displayed; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 110 and may be implemented separately using a communication chip.

[0168] The memory 120 may include random access memory (RAM) or read-only memory (ROM). The memory 120 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 120 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing functions (such as touch functionality, sound playback functionality, image playback functionality, etc.), and instructions for implementing the various method embodiments described below. The data storage area may also store data created by the terminal 100 during use (such as phonebook data, audio and video data, chat log data, etc.).

[0169] Please see Figure 13 This diagram illustrates a structural block diagram of a computer-readable storage medium provided in an embodiment of this application. The computer-readable medium 300 stores program code that can be called by a processor to execute the methods described in the above method embodiments.

[0170] The computer-readable storage medium 300 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 300 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 300 has storage space for program code 310 that performs any of the method steps described above. This program code can be read from or written to one or more computer program products. The program code 310 may be compressed, for example, in a suitable form.

[0171] In summary, the image processing method, apparatus, electronic device, and storage medium provided in this application acquire an input image, upsample the first pixel position to be processed in the input image to obtain an upsampled image, interpolate the first adjacent pixel position of the first pixel position to be processed in the upsampled image to obtain an interpolated image, and sharpen the interpolated image to obtain a first output image. By interpolating adjacent pixel positions, information about adjacent pixel positions is obtained, allowing sharpening to be performed at the same stage as upsampling, eliminating the need for additional input / output operations and effectively reducing power consumption during image super-resolution processing.

[0172] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. An image processing method, characterized in that, The method includes: Obtain the input image; The first pixel position to be processed in the input image is upsampled to obtain the upsampled image; Interpolation processing is performed on the first adjacent pixel position of the first pixel position to be processed in the upsampled image to obtain the interpolated image; The interpolated image is then sharpened to obtain the first output image.

2. The method according to claim 1, characterized in that, The step of sharpening the interpolated image to obtain a first output image includes: Based on the upsampling result of the first pixel position to be processed and the interpolation result of the first adjacent pixel position, the interpolated image is sharpened to obtain the first output image.

3. The method according to claim 1, characterized in that, The step of interpolating the first neighboring pixel positions of the first pixel position to be processed in the upsampled image to obtain the interpolated image includes: The first adjacent pixel position is interpolated using bilinear interpolation to obtain the interpolated image.

4. The method according to claim 1, characterized in that, The step of upsampling the first pixel position to be processed in the input image to obtain the upsampled image includes: The first pixel to be processed is upsampled in the HSV color space and / or RGB color space to obtain the upsampled image.

5. The method according to any one of claims 1-4, characterized in that, After acquiring the input image, the process also includes: Calculate the brightness information of the second neighboring pixel position of the second pixel position to be processed in the input image; Using the brightness information, perform super-resolution calculations in the brightness channel to obtain the estimated brightness of the second pixel to be processed. Obtain the estimated hue and estimated saturation of the second pixel to be processed; Based on the estimated brightness, the estimated hue, and the estimated saturation, a second output image in the HSV color space is obtained.

6. The method according to claim 5, characterized in that, After obtaining the second output image in the HSV color space based on the estimated brightness, the estimated hue, and the estimated saturation, the method further includes: The second output image in the HSV color space is converted into a second output image in the RGB color space.

7. The method according to claim 5, characterized in that, The step of obtaining the estimated hue and estimated saturation of the second pixel position to be processed includes: The pixel value at the second adjacent pixel position is obtained by interpolating the position using the nearest neighbor interpolation method. The estimated hue and the estimated saturation are obtained based on the pixel values ​​at the second adjacent pixel positions.

8. The method according to claim 5, characterized in that, The step of obtaining a second output image in the HSV color space based on the estimated brightness, the estimated hue, and the estimated saturation includes: Adjust the estimated brightness to obtain the adjusted brightness, and adjust the estimated saturation to obtain the adjusted saturation; Based on the adjusted brightness, the estimated hue, and the adjusted saturation, a second output image in the HSV color space is obtained.

9. The method according to claim 5, characterized in that, The step of performing super-resolution calculations in the luminance channel using the luminance information to obtain the estimated luminance at the position of the second pixel to be processed includes: Identify available computing resources; From multiple super-resolution algorithms, determine the target super-resolution algorithm corresponding to the idle computing resources; The estimated brightness is obtained by performing super-resolution calculations in the brightness channel using the brightness information through the target super-resolution algorithm.

10. The method according to any one of claims 1-4, characterized in that, The step of upsampling the first pixel position to be processed in the input image to obtain the upsampled image includes: Determine the application type corresponding to the application to which the input image belongs; If the application type is a game, then the first pixel position to be processed is upsampled to obtain the upsampled image.

11. An image processing apparatus, characterized in that, The device includes: The input image acquisition module is used to acquire the input image; An upsampled image acquisition module is used to perform upsampling processing on the first pixel position to be processed in the input image to obtain an upsampled image; An interpolated image acquisition module is used to perform interpolation processing on the first adjacent pixel position of the first pixel position to be processed in the upsampled image to obtain an interpolated image; The first output image acquisition module is used to sharpen the interpolated image to obtain the first output image.

12. An electronic device, characterized in that, The method includes a memory and a processor, the memory being coupled to the processor, the memory storing instructions, and when the instructions are executed by the processor, the processor performing the method as described in any one of claims 1-10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1-10.

Images