Image enhancement method and electronic device

Abstract

This application discloses an image enhancement method and an electronic device, belonging to the field of image processing technology. The method includes: acquiring color information and a brightness matrix of a sampling location in an image to be processed; wherein the brightness matrix is ​​obtained based on first neighboring pixels of the sampling location; performing low-pass filtering on the brightness matrix to obtain a first brightness value of the sampling location; and performing high-pass filtering on second neighboring pixels of the sampling location to obtain a second brightness value of the sampling location; performing a mixing process on the first brightness value and the second brightness value based on a preset sharpness coefficient, and calculating the difference between the mixing result and the original brightness of the sampling location; and performing a reduction process on the color information of the sampling location based on the difference to obtain the enhanced color of the sampling location.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, specifically relating to an image enhancement method and an electronic device. Background Technology

[0002] When a mobile device's graphics processing unit (GPU) renders complex scenes in real time, it typically uses low-resolution rendering to obtain an initial image; then it uses resampling techniques to obtain a high-resolution image, where the resampling technique usually employs a spatial domain super-resolution algorithm.

[0003] Most spatial domain super-resolution algorithms require two rendering processes: a sampling calculation process and an enhancement calculation process. Only images after these two processes are usable, requiring two rendering pipelines to be started. However, the rendering compositor module of mobile devices (such as XR devices) needs to perform distortion pre-correction, dispersion correction, and color space correction calculations on each image passed by the application sequentially. Traditional two-step spatial domain super-resolution algorithms lead to increased computational latency, making them increasingly unsuitable for the rendering compositor working mode of mobile devices. Some single-rendering-flow spatial domain super-resolution algorithms combine the sampling and enhancement calculation processes. In one calculation flow, the difference between the color at the sampling location and the nearby colors is filtered to obtain enhanced brightness, which is then added to the sampled color. However, this algorithm can exacerbate the jagged edges in the input image. Summary of the Invention

[0004] The purpose of this application is to provide an image enhancement method and electronic device that can solve the problems of large computational delay or poor anti-aliasing effect in image enhancement algorithms.

[0005] In a first aspect, embodiments of this application provide an image enhancement method, comprising: acquiring color information and a brightness matrix of a sampling location in an image to be processed; wherein the brightness matrix is ​​obtained based on first neighboring pixels of the sampling location; performing low-pass filtering on the brightness matrix to obtain a first brightness value of the sampling location; and performing high-pass filtering on second neighboring pixels of the sampling location to obtain a second brightness value of the sampling location; performing a mixing process on the first brightness value and the second brightness value based on a preset sharpness coefficient, and calculating the difference between the mixing result and the original brightness of the sampling location; and performing a reduction process on the color information of the sampling location based on the difference to obtain the enhanced color of the sampling location.

[0006] Secondly, embodiments of this application provide an image enhancement device, comprising: an acquisition module, configured to acquire color information and a brightness matrix of a sampling location in an image to be processed; wherein the brightness matrix is ​​obtained based on first neighboring pixels of the sampling location; a filtering module, configured to perform low-pass filtering on the brightness matrix to obtain a first brightness value of the sampling location; and perform high-pass filtering on the second neighboring pixels of the sampling location to obtain a second brightness value of the sampling location; a calculation module, configured to perform mixing processing on the first brightness value and the second brightness value based on a preset sharpness coefficient, and calculate the difference between the mixing result and the original brightness of the sampling location; and an enhanced display module, configured to perform subtraction processing on the color information of the sampling location based on the difference to obtain the enhanced color of the sampling location.

[0007] Thirdly, embodiments of this application provide an electronic device including a processor and a memory, the memory storing programs or instructions executable on the processor, the programs or instructions, when executed by the processor, implementing the steps of the method described in the first aspect.

[0008] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0009] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect.

[0010] In a sixth aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method described in the first aspect.

[0011] In this embodiment, low-pass filtering is performed on the brightness matrix of the sampling location, and high-pass filtering is performed on the sampling location and its neighboring pixels. This achieves the separation of background and details in the image to be processed, filters out directional jagged edges, and reduces jagged edges and jagged edge jitter caused by super-resolution algorithms. The mixing ratio of the first and second brightness values ​​is controlled by a sharpness coefficient, and a reduction processing is performed in the color space based on the difference between the mixed result and the original brightness, achieving refined enhancement of image details. This method effectively overcomes the contradiction between detail enhancement and noise amplification in traditional methods, reducing jagged edges and jagged edge jitter. Furthermore, this algorithm does not require modification of the main flow of the upper-layer rendering program, has low computational load, helps reduce computational latency, and is easy to deploy and implement in XR devices. Attached Figure Description

[0012] Figure 1 A schematic flowchart of the image enhancement method provided in an embodiment of this application is shown; Figure 2 A schematic flowchart of the image enhancement method provided in an embodiment of this application is shown; Figure 3 A schematic flowchart of the image enhancement method provided in an embodiment of this application is shown; Figure 4 This diagram illustrates the compressed storage of pre-filtered brightness data provided in an embodiment of this application. Figure 5 This illustration shows a schematic diagram of the image enhancement region determined from the image to be processed, provided in an embodiment of this application. Figure 6 A schematic flowchart of the image enhancement method provided in an embodiment of this application is shown; Figure 7 A schematic flowchart of the image enhancement method provided in an embodiment of this application is shown; Figure 8 A schematic flowchart of the image enhancement method provided in an embodiment of this application is shown; Figure 9 This diagram illustrates the structure of the image enhancement device provided in an embodiment of this application. Figure 10 This invention provides a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 11 This invention provides a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Detailed Implementation The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0013] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0014] The image enhancement method provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0015] like Figure 1 As shown, this application embodiment provides an image enhancement method 100, which can be executed by an electronic device. In other words, the method can be executed by software or hardware installed on an electronic device, such as a mobile device, like an Extended Reality (XR) device, etc. The method includes the following steps.

[0016] S102: Obtain the color information and brightness matrix of the sampling position in the image to be processed; wherein, the brightness matrix is ​​obtained based on the first neighboring pixels of the sampling position.

[0017] In this embodiment, the image to be processed can be a digital image of any format, such as an RGB image, and can be the rendering result of a virtual scene. The sampling position can be any position in the space of the image to be processed, and can be calculated during rendering, i.e., the coordinates of the "pixel to be rendered" in the sampled texture space. The color information of the sampling position can be, for example, RGB information, where the color information of the sampling position may be the color of one or more pixels, or it may be a mixture of the colors of several pixels.

[0018] In some embodiments, the brightness matrix may be obtained based on the first neighboring pixels of the sampling position, which generally include the pixels where the sampling position is located.

[0019] For example, the brightness matrix of the sampling location can be obtained based on the following steps: take the sampling location as the center, select a first neighborhood window of a preset size, for example, the first neighborhood window can be a 4×4 square window; extract the brightness values ​​of all pixels in the first neighborhood window to form the brightness matrix corresponding to the sampling location. Optionally, the first neighborhood window can also include the sampling location.

[0020] In other embodiments, before S102, the brightness data of the image to be processed can be read, a blur filtering calculation can be performed, and the blurred and filtered brightness data can be compressed and stored. Thus, in S102, the brightness matrix of the sampling position can be obtained based on the compressed and stored brightness data.

[0021] The embodiments of this application can integrate the brightness information of the sampling position and its surrounding pixels by obtaining the brightness matrix, providing a data basis for subsequent filtering processing.

[0022] S104: Perform low-pass filtering on the brightness matrix to obtain a first brightness value at the sampling position; and perform high-pass filtering on the second neighboring pixels at the sampling position to obtain a second brightness value at the sampling position.

[0023] The first brightness value typically represents the low-frequency component of the sampling location in the image to be processed, i.e., the background, smooth areas, and other information at the sampling location in the image to be processed; the second brightness value typically represents the high-frequency component of the sampling location in the image to be processed, i.e., the edge, texture, and other detailed information at the sampling location in the image to be processed.

[0024] In some embodiments, this step can construct a kernel function for a low-pass filter, such as a mean filter kernel, a Gaussian filter kernel, or a Lanczos filter kernel function. Taking a 3x3 mean filter kernel function as an example, its kernel coefficients are all 1 / 9. Convolving the brightness matrix with the kernel function of the low-pass filter yields a first brightness value, which is actually a smoothed result of the pixel brightness within the first neighborhood window.

[0025] In some embodiments, this step may involve selecting a second neighborhood window of a preset size centered on the sampling location; constructing a kernel function for a high-pass filter, which may take various forms, such as the Laplacian operator or the Sobel operator; and using the kernel function of the high-pass filter to filter the pixel brightness within the second neighborhood window, extracting high-frequency components as the second brightness value.

[0026] In some embodiments, this step may perform high-pass filtering on the second neighboring pixels of the sampling location, which generally include the pixels where the sampling location is located.

[0027] S106: The first brightness value and the second brightness value are mixed based on a preset sharpness coefficient, and the difference between the mixed result and the original brightness at the sampling position is calculated.

[0028] In this embodiment, the sharpness factor is an adjustable parameter used to control the intensity of image enhancement for the image to be processed. This sharpness factor can be a fixed value set manually by the user; or it can be a value adaptively generated based on the content of the image to be processed.

[0029] In some embodiments, this step can be performed by weighted summation of the first luminance value and the second luminance value based on a sharpness coefficient to obtain a mixed luminance value, i.e., the mixed result. A typical mixing method is: L mix = (1 - t) * L low + t *L high L mix For the mixed brightness value, L low L is the first brightness value. hight is the second brightness value, and t is the sharpness coefficient (0≤t≤1). When t is larger, the image to be processed is enhanced more strongly, and the details are more prominent; when t is smaller, the image to be processed is enhanced less strongly, and the image to be processed is closer to the original smooth state.

[0030] In some embodiments, this step can be based on the following formula to calculate the brightness difference between the mixed brightness value and the original brightness at the sampling location: ΔL = L mix - L original L original This represents the original brightness value at the sampling location.

[0031] S108: Based on the difference, the color information of the sampling position is reduced to obtain the enhanced color of the sampling position.

[0032] In this embodiment, the calculated difference can be applied to each component of the original color information to enhance the image to be processed.

[0033] In some embodiments, this step may acquire the red component R, green component G, and blue component B from the RGB information of the sampling location. The enhanced red component R', green component G', and blue component B' are obtained by subtracting the aforementioned differences from the red component, green component, and blue component, respectively.

[0034] This application embodiment enhances the details of the image by subtracting the same brightness difference from the three RGB channels, which is equivalent to mapping the brightness adjustment inversely to the color space. This enhances the details of the image while maintaining the original hue and color balance, avoiding color cast.

[0035] The image enhancement method provided in this application separates the background and details of the image by performing low-pass filtering on the brightness matrix of the sampling location and high-pass filtering on the sampling location and its neighboring pixels. This effectively filters directional jagged edges, reducing jagged edges and jagged edge jitter caused by super-resolution algorithms. Furthermore, by controlling the mixing ratio of the first and second brightness values ​​through a sharpness coefficient and performing subtraction processing in the color space based on the difference between the mixed result and the original brightness, this method achieves refined enhancement of image details. This method effectively overcomes the contradiction between detail enhancement and noise amplification in traditional methods, reducing jagged edges and jagged edge jitter. Moreover, this algorithm does not require modification of the main flow of the upper-layer rendering program, has low computational load, helps reduce computational latency, and is easy to deploy and implement in XR devices.

[0036] The image enhancement methods provided in the various embodiments of this application can be applied to XR devices, specifically to the rendering compositor pipeline of an XR device, and can be directly integrated with the rendering link of the rendering compositor pipeline. The image to be processed can be the rendering result of a virtual scene; the enhanced image (i.e., the final output rendered image) can be used for display on the XR device.

[0037] In XR devices, due to the need to consider image distortion caused by the optical engine, the rendering compositor can perform pre-distortion processing on the input image to compensate for the effects of optical engine distortion. Most spatial domain super-resolution algorithms in related technologies do not consider pre-distortion calculations in XR rendering, which may exacerbate jagged edges and moiré patterns around the image caused by variations in sampling intervals. In an XR environment, even slight jagged edges or moiré patterns can be perceived by the user, leading to a decrease in immersion. Block pre-distortion processing algorithms in related technologies require rendering the virtual scene at different resolutions based on the distortion information of different regions. This step disrupts the core process of virtual scene rendering, adds multiple rendering steps, increases rendering latency, and makes the fragmented rendering results at different resolutions appear more pronounced. Existing super-resolution algorithms only perform upsampling enhancement calculations for low-resolution images. Furthermore, in the compositing layer of XR applications, the image buffer size passed to the compositor is often larger than the buffer size passed to the display driver, which may lead to moiré patterns due to downsampling.

[0038] To address the aforementioned technical problems, this embodiment provides an adaptive image enhancement method for determining the filtering scale. Based on embodiment 100, step S104 is further refined and optimized. Specifically, as follows... Figure 2 As shown, in S104, the process of performing low-pass filtering on the brightness matrix to obtain the first brightness value at the sampling position includes the following steps.

[0039] S202: Determine the first derivative value of the optomechanical pre-distortion curve of the pixel to be rendered based on the distortion radius data of the pixel to be rendered position.

[0040] A pixel to be rendered refers to the basic unit of GPU rendering. In GPU rendering, each GPU unit draws one pixel, and multiple GPU units draw simultaneously to achieve parallel computing. The position of the pixel to be rendered refers to the location of the pixel on the final output rendered image. The sampling position mentioned above can include the coordinates of the pixel to be rendered in the sampled texture space.

[0041] Distortion radius data typically refers to the distance from the pixel to be rendered to the distortion center (usually the center of the final output rendered image). The optomechanical distortion curve is a function describing the objective distortion characteristics of an optical system, usually expressed as the relationship between the distortion radius *r* and the original radius *r0*, or as the relationship between the distortion displacement and the radius. The optomechanical pre-distortion curve is the inverse function of the optomechanical distortion curve.

[0042] This application embodiment obtains the distortion rate of the pixel position to be rendered by differentiating the optomechanical pre-distortion curve. The physical meaning of this derivative is that it reflects the degree to which the image in the vicinity of the pixel position to be rendered is stretched or compressed. Specifically: when the first derivative value is greater than 1, it indicates that the region is stretched and the image information is relatively sparse; when the first derivative value is less than 1, it indicates that the region is compressed and the image information is relatively dense; when the first derivative value is equal to 1, it indicates that the region has no distortion or the distortion is completely compensated.

[0043] S204: Determine the spatial scale scaling factor of the low-pass filter kernel function based on the first derivative value.

[0044] This step adaptively adjusts the spatial scale scaling factor of the low-pass filter kernel function based on the degree of distortion (i.e., the magnitude of the first derivative) in each region of the final output rendered image. In one possible implementation, this step can scale the spatial scale of the low-pass filter kernel function by 1 / d.

[0045] S206: Based on the spatial scale scaling factor of the low-pass filter kernel function, perform low-pass filtering on the brightness matrix to obtain the first brightness value at the sampling position.

[0046] After determining the spatial scale of the filter kernel function that matches the distortion characteristics of the pixel to be rendered, a low-pass filter kernel function can be constructed, and the brightness matrix obtained in step S102 can be filtered to obtain the first brightness value.

[0047] It should be noted that, since the spatial scale of the filter function corresponding to different sampling positions may be different, the method in this embodiment can independently determine the spatial scale and perform filtering operations at each pixel position of the sampling position, thereby realizing adaptive image enhancement that is tightly coupled with image distortion characteristics.

[0048] This embodiment achieves adaptive adjustment of the spatial scale of the low-pass filter kernel function by introducing the first derivative value of the optomechanical pre-distortion curve. Using small-scale filtering in the stretched regions of the image better preserves the original texture details; using large-scale filtering in the compressed regions more effectively suppresses moiré noise caused by downsampling. This adaptive processing method deeply integrates image enhancement with the characteristics of the optical display system, significantly improving the overall quality of the final output rendered image. It avoids the detail loss that may occur in different distortion regions due to traditional fixed-scale filtering, or the nonlinear distributed moiré noise problem caused by image post-processing, which is relatively rare in traditional flat panel display rendering chains.

[0049] It should be noted that this embodiment is a further refinement of S104 in embodiment 100. Therefore, the technical solution of this embodiment can be combined with other steps in embodiment 100 to form a complete image enhancement method.

[0050] In image enhancement processing for XR devices, especially in real-time processing of high-resolution images or video streams, memory bandwidth and storage space often become performance bottlenecks. Traditional image enhancement methods require frequent access to the pixel data of the original image, resulting in numerous memory read / write operations, increasing system load and processing latency. Furthermore, operations such as low-pass filtering require reading neighboring pixel data, further exacerbating the demand for memory bandwidth.

[0051] To address the aforementioned technical problems, this embodiment provides an image enhancement method. Based on embodiment 100 or embodiment 200, a pre-filtering calculation and compression process is added before S102, and S102 is adaptively adjusted.

[0052] like Figure 3 As shown, before obtaining the brightness matrix of the sampling position in the image to be processed in S102, the embodiments of this application may further include the following preprocessing steps.

[0053] S302: Using preset blur convolution parameters, filter the 2x2 pixel block in the image to be processed to obtain the blurred brightness value.

[0054] This step can use a computation shader, where each computation thread collects the color of a 2x2 pixel block in the virtual rendering result (i.e., the image to be processed) and calculates its brightness; then it is filtered using preset blur convolution parameters to obtain the blurred brightness value.

[0055] In some embodiments, a convolution kernel optimized for computational speed can be used to process the brightness information of a 2x2 pixel block. Multiplying by the following matrix yields the blurred brightness value. ,

[0056] in,

[0057]

[0058]

[0059]

[0060] These represent the filtering intensities in the horizontal, vertical, and diagonal directions, respectively.

[0061] This step fuses the brightness information of the original four pixels into a single blurred brightness value. This process essentially involves a 2x downsampling and low-pass filtering, reducing the amount of data and providing initial noise suppression, resulting in better anti-aliasing in XR environments.

[0062] S304: Storing the blurred brightness value of each pixel block into multiple sub-pixel channels of a pixel to obtain a compressed and stored brightness value.

[0063] This step can Stored in the RGBA sub-pixel channel of a single pixel to achieve memory compression, its spatial location according to... Figure 4 The positions of XYZW in the middle are arranged as follows, among which, Figure 4 This is a schematic diagram of the compressed storage of pre-filtered brightness data. Figure 4 The left image shows a schematic diagram of the storage method before compression, and the right image shows a schematic diagram of the storage method after compression. Figure 4 Solid lines in the diagram represent pixels, and dashed lines represent sub-pixels.

[0064] Since a 2x2 pixel block is compressed into a single storage pixel, the amount of data in the image to be processed is theoretically reduced to 1 / 4 of the original, which can reduce memory read / write bandwidth consumption, GPU usage and computation time, and reduce subsequent computational load.

[0065] Based on this, the process of obtaining the brightness matrix of the sampling position in S102 is also adaptively adjusted accordingly, and the brightness matrix of the sampling position can be obtained based on the compressed and stored brightness values.

[0066] This embodiment significantly reduces the amount of original image data by introducing a 2x2 pixel block-based blur compression storage step before image enhancement processing, thereby reducing memory usage and bandwidth requirements. The blurred brightness values ​​are stored in multiple sub-pixel channels of a single pixel, cleverly utilizing the existing pixel storage structure without changing the underlying storage format, ensuring strong compatibility. In the subsequent brightness matrix acquisition step, the brightness matrix is ​​obtained from the compressed brightness data, enabling direct use of the compressed data. This approach allows the image enhancement method to process high-resolution images or video streams more efficiently, making it particularly suitable for resource-constrained XR devices or applications with high real-time requirements. Simultaneously, the blur filtering in S302 also serves as preliminary noise reduction, contributing to better anti-aliasing effects in XR environments.

[0067] It should be noted that this embodiment is a supplement to the steps before S102 in embodiment 100 or embodiment 200 and an adaptive adjustment to S102. Therefore, the technical solution of this embodiment can be combined with the technical solutions in embodiment 100 or embodiment 200 to jointly constitute a complete image enhancement method.

[0068] In applications such as XR, virtual reality, augmented reality, or near-eye displays, the human visual system exhibits a non-uniform resolution perception characteristic: the human eye has high-resolution perception capabilities only in the central region of the gaze point, while sensitivity to details decreases significantly in the peripheral visual field. However, existing image enhancement methods typically perform uniform processing on the entire image to be processed, failing to fully utilize the gaze point characteristic of human vision. This results in unnecessary fine-grained enhancement calculations in the peripheral areas, leading to a waste of computational resources.

[0069] To address the aforementioned technical problems, this embodiment provides an image enhancement method based on gaze point region adaptation. Before S102, the method further includes the following steps: determining the gaze point range based on the texture size, gaze point coordinates, and gaze point region ratio of the image to be processed. The gaze point region can also be referred to as the image enhancement region or the computational region of the preprocessed data. The image to be processed can be a virtual scene rendering result, specifically as follows: Figure 5 As shown.

[0070] Texture size refers to the full size of the image being processed, typically including width W and height H. Gaze coordinates refer to the user's gaze point position on the image, usually expressed as two-dimensional coordinates. Gaze region scale refers to the ratio of the gaze region to the overall image size.

[0071] In some embodiments, the gaze point range is defined as an elliptical region centered on the gaze point coordinates, with the major and minor axes of the ellipse corresponding to the width and height directions of the image, respectively, and its size is controlled by the gaze point region ratio.

[0072] In some embodiments, to simplify calculations, the gaze range can be defined as a rectangular region centered on the gaze coordinates, such as... Figure 5 As shown.

[0073] Thus, obtaining the color information and brightness matrix of the sampling location in the image to be processed in S102 includes: if the sampling location is within the gaze point range, obtaining the color information and brightness matrix of the sampling location in the image to be processed. It can be understood that if the sampling location is outside the gaze point range, i.e., in the peripheral area, a simplified processing strategy can be adopted. For example, the enhancement processing can be skipped directly, keeping the original pixel values ​​unchanged.

[0074] This embodiment introduces a region-adaptive processing mechanism based on foveated rendering, enabling the image enhancement method to dynamically allocate computational resources according to the characteristics of human vision. High-quality, fine-grained enhancement is performed in the foveated region to ensure optimal image quality in the core visual area perceived by the user; while processing is simplified or skipped in peripheral regions, significantly reducing the overall computational load.

[0075] It should be noted that this embodiment is a supplement to the steps before S102 and an adaptive adjustment to step S102. Therefore, the technical solution of this embodiment can be arbitrarily combined with the technical solutions of any of the previous embodiments to jointly constitute a complete image enhancement method.

[0076] In image enhancement processing, high-pass filtering is used to extract high-frequency detail information such as edges and textures. However, traditional high-pass filtering usually uses a fixed filter kernel and a fixed gain coefficient. This approach has inherent limitations when dealing with complex and varied image content: in weak edge regions, a fixed gain may not be sufficient to effectively enhance details; in strong edge regions, a fixed gain may lead to over-enhancement, producing visual artifacts and severely affecting image quality.

[0077] To address the aforementioned technical issues, this embodiment provides a method for adaptively adjusting the high-pass filter gain based on local brightness extrema. Based on any one or more of the above embodiments, S104 has been further refined and optimized.

[0078] Specifically, such as Figure 6 As shown, in S104, the high-pass filtering process is performed on the second neighboring pixels of the sampling position to obtain the second brightness value of the sampling position, which includes the following steps.

[0079] S602: Determine the extreme values ​​of the brightness of the second neighboring pixels at the sampling position in the four directions of up, down, left, and right.

[0080] This step, centered on the sampling location, searches for extreme brightness values ​​in four directions—up, down, left, and right—within a second neighborhood window (e.g., a 5×5 or 7×7 window). In one possible implementation, for each direction, the maximum and minimum brightness values ​​are recorded; these extreme values ​​in four directions reflect the severity and range of brightness variations in the area surrounding the sampling location. In another possible implementation, the information from the four directions can be further integrated to calculate the global maximum and minimum brightness values, which serve as a reference for subsequent gain adjustments.

[0081] For example, the sampling position is 1 pixel, the second neighborhood window is 7×7 pixels, and the sampling position is located at the center of the second neighborhood window. In this example, the second neighborhood pixels in the four directions of up, down, left, and right form a cross-shaped structure. In one possible implementation, for the upward direction of the sampling position, the maximum and minimum brightness values ​​are selected from the three pixels in that direction; similarly, for the downward, left, and right directions of the sampling position, the maximum and minimum brightness values ​​are selected from the three pixels in each direction.

[0082] S604: Adjust the gain of the high-pass filter kernel function based on the brightness extreme value.

[0083] This embodiment adaptively adjusts the high-pass filter gain based on the brightness variation (i.e., contrast) of a local area, appropriately increasing the gain in weak edge areas to enhance details and appropriately decreasing the gain in strong edge areas to avoid overshoot. One usable kernel function is:

[0084]

[0085] Where w is a coefficient. , These represent the minimum and maximum brightness values, respectively, and W1 and W2 are the horizontal pixel counts of the image to be processed and the final output rendered image, respectively.

[0086] S606: Using a high-pass filter with adjusted gain, perform high-pass filtering on the second neighboring pixels at the sampling position to obtain the second brightness value at the sampling position.

[0087] Once a gain matching the local brightness characteristics of the sampling location is determined, this gain can be applied to the high-pass filtering process. This embodiment achieves adaptive adjustment of the high-pass filtering gain based on the contrast information reflected by the local brightness extrema.

[0088] This embodiment achieves adaptive adjustment of the high-pass filter gain by introducing local contrast analysis based on the brightness extrema in four directions. In weak edge or flat areas, increasing the gain effectively enhances subtle details and avoids detail loss; in strong edge areas, decreasing the gain suppresses over-enhancement and prevents artifacts. This adaptive processing method makes detail enhancement more precise and natural, significantly improving the overall visual quality of the enhanced image and avoiding various artifact problems that may arise from traditional fixed-gain high-pass filtering. Furthermore, the calculation of this method is based on local window extremum search and simple gain adjustment, resulting in low computational overhead and ease of implementation in real-time image processing systems.

[0089] It should be noted that this embodiment is a further refinement of S104. Therefore, the technical solution of this embodiment can be combined with any one or more of the above embodiments to form a complete image enhancement method.

[0090] In image enhancement processing, the brightness difference obtained by mixing low-pass and high-pass filters can be directly used to reduce the original color information. However, in some extreme cases, such as when there are high-contrast edges, noise points, or over-enhanced areas in the image, the calculated brightness difference may be too large or too small, causing the enhanced pixel color to exceed the normal color range (e.g., RGB component overflow, less than 0 or greater than 255), or producing unnatural color shifts. This phenomenon can severely damage the visual quality of the image and even produce obvious artifacts.

[0091] To address the aforementioned technical issues, this embodiment provides a brightness difference limiting processing method, which optimizes the process in S106 to ensure that the enhanced pixel color remains within a reasonable range.

[0092] Calculating the difference between the blending result and the original brightness of the pixel at the sampling location in S106 may include the following steps: determining whether the difference between the blending result and the original brightness of the sampling location is within a preset threshold range; if the difference between the blending result and the original brightness of the sampling location exceeds the maximum value of the threshold range, then the maximum value of the threshold range is used as the difference; and / or, if the difference between the blending result and the original brightness of the sampling location is less than the minimum value of the threshold range, then the minimum value of the threshold range is used as the difference. It can be understood that if the difference between the blending result and the original brightness of the sampling location is within the threshold range, then the original difference remains unchanged.

[0093] In this embodiment, a reasonable threshold range [ΔL] can first be preset. min , ΔL max This threshold range is used to limit the allowable variation in brightness difference, thereby controlling the intensity of image enhancement and avoiding over-enhancement.

[0094] This embodiment effectively controls the magnitude of image enhancement by introducing a limiting mechanism before applying the brightness difference, preventing color distortion, color overflow, and image artifacts caused by excessively large or small differences. This protective measure ensures that the enhanced image always remains within a reasonable color space, maintaining visual naturalness and comfort. Simultaneously, the threshold range can be flexibly determined based on natural color space limitations, fixed empirical values, or image statistical characteristics, providing the system with excellent configuration capabilities and achieving an optimal balance between enhancement intensity and image fidelity.

[0095] It should be noted that this embodiment is a further limitation of S106. Therefore, the technical solution of this embodiment can be combined with any one or more of the above embodiments to form a complete image enhancement method.

[0096] In image enhancement processing, an image is divided into processing areas that require fine enhancement and non-processed areas that remain in their original state or undergo simplified processing. This division of regions can lead to noticeable visual abrupt changes at the boundaries between regions, manifested as sudden shifts in brightness, contrast, or sharpness. This discontinuity is easily perceived by users, affecting the comfort and naturalness of the visual experience.

[0097] To address the aforementioned technical problems, this embodiment provides an image fusion processing method based on a transition gradient mask. Based on any one or more of the above embodiments, the image to be processed is further optimized to achieve a smooth transition between enhanced and non-enhanced regions.

[0098] Specifically, the method further includes the following steps: acquiring or generating a gradient mask; processing the image to be processed based on the difference and the preset gradient mask to obtain the processed image.

[0099] In this embodiment, the gradient mask is a spatial variation mapping table used to control image fusion weights. The gradient mask has the same size as the image to be processed, and the value at each pixel position represents the fusion weight between the enhanced image and the original image (or other reference image) at that position.

[0100] This embodiment achieves a smooth blend between enhanced and non-enhanced areas by introducing a gradient mask. This processing method effectively eliminates visual boundaries that may be caused by region-selective enhancement, giving the entire image a natural and continuous visual experience.

[0101] It should be noted that the technical solution of this embodiment can be combined with any one or more of the above embodiments to form a complete image enhancement method.

[0102] To illustrate the image enhancement method provided in this application in detail, a specific embodiment will be described below.

[0103] This application provides an adaptive image enhancement method based on the needs of image enhancement processing, the specific rendering chain of XR devices, and computational limitations. This method can enhance image details, reduce jagged edges, adapt to the working logic of XR rendering compositors, and consider the image quality degradation caused by non-ideal effects such as optical-mechanical distortion in XR devices. Furthermore, the image enhancement algorithm has low performance overhead, meeting the low latency and low power consumption requirements of XR rendering compositors.

[0104] Compared with traditional super-resolution algorithms, the embodiments of this application are applicable to the underlying rendering link of the compositor in XR devices in terms of computational logic. They can be divided into two parts: the pre-filtering and compression calculation process and the core calculation process, of which the pre-filtering and compression calculation process is optional.

[0105] The core computational process of this application embodiment can be integrated into a normal XR rendering compositor without the need to start an additional rendering pipeline. In a rendering process, the core computational process performs filtering calculations (see core computational process step 3 below) and enhancement calculations (core computational process step 4) on the image to be processed separately, and then mixes the two according to the desired ratio to obtain the final result (core computational process step 5), thus avoiding jagged edges in the enhanced image.

[0106] To achieve better anti-aliasing in an XR environment, a pre-filtering calculation and compression process can be enabled during virtual scene rendering. This process quickly blurs the target area of ​​the rendered virtual scene (see step 2 of the pre-filtering calculation and compression process below) and then compresses and stores it, saving memory read / write performance and improving computational efficiency (step 3 of the pre-filtering calculation and compression process). This further reduces image jaggedness. Enabling the pre-filtering calculation and compression process simply requires adding this algorithm to the end of the original rendering pipeline; no modification to the main virtual scene rendering process is needed.

[0107] In the core calculation process of this application embodiment, the convolution kernel of the sampling part is continuously and adaptively scaled pixel-level according to the first derivative of the optical engine pre-distortion curve (which is an objective parameter determined by the XR optical design) (see step 3 of the core calculation process below). This can eliminate moiré patterns caused by nonlinear pre-distortion calculation in the XR system while avoiding excessive blurring, and does not require the virtual scene to be rendered in blocks according to pixel position.

[0108] The input data in this embodiment of the application are as follows: preset blur convolution parameters, sharpness coefficient t, gaze region ratio, gaze point coordinates, input texture size, optical-mechanical pre-distortion coefficients, preset low-pass filter parameters, preset high-pass filter parameters, and brightness enhancement clamping range. Transition gradient masking function (optional).

[0109] like Figure 7 and Figure 8 As shown, the pre-filtering calculation and compression process may include the following steps.

[0110] Step 1: Determine the gaze range based on texture size, gaze coordinates, and gaze region ratio, i.e., determine the computation area of ​​the preprocessed data.

[0111] Step 2: Using a computation shader, each computation thread collects the color of a 2x2 pixel block from the virtual rendering result, calculates its brightness value, and then filters it using preset blur convolution parameters to obtain the blurred brightness. This method uses a convolution kernel optimized for computation speed to extract the brightness information of the 2x2 pixel block. Multiplying by the following matrix yields the blurred brightness value. .

[0112]

[0113] in,

[0114]

[0115]

[0116]

[0117] These represent the filtering intensities in the horizontal, vertical, and diagonal directions, respectively.

[0118] Step 3: Put Stored in the RGBA subpixel channel of a single pixel to achieve memory compression.

[0119] like Figure 7 and Figure 8 As shown, the core computing process includes the following steps.

[0120] Step 1: During the fragment shader process in the normal rendering pipeline of the compositor, if the sampling position is within the foveation point range, the following image enhancement algorithm is performed; otherwise, the original color is output directly.

[0121] Step 2: If the pre-filtering calculation and compression process is enabled, read the compressed and stored 4x4 preprocessed luminance information into the luminance matrix. If the pre-filtering calculation and compression process is not enabled, the luminance information of 4x4 pixels is read from the original input into the luminance matrix. middle.

[0122] Step 3: Calculate the first derivative of the optical engine pre-distortion curve based on the distortion radius data of the current fragment to be rendered (corresponding to the pixel position to be rendered in other embodiments). The spatial scale of the pre-defined low-pass filter kernel function is enlarged. Then the brightness matrix Perform filtering calculations to obtain the first brightness value. .

[0123] Step 4: Calculate the extreme brightness values ​​at the sampling location (top, bottom, left, and right), control the gain of the high-pass filter convolution kernel function, and obtain the second brightness value after filtering the surrounding data. One available kernel function is:

[0124]

[0125] Where w is a coefficient , W1 and W2 are the minimum and maximum brightness values, respectively, and the horizontal pixel counts of the sampled image (i.e. the image to be processed) and the final output rendered image, respectively.

[0126] Step 5: According to the sharpness factor t and After linear mixing, calculate the difference between the result and the original brightness at the sampling location. restrain it Inside.

[0127] Step 6: Put Multiplying with a gradient mask smooths the transition between the inside and outside of the image enhancement area. This step is optional and can be performed or omitted.

[0128] Step 7: Simultaneously subtract the RGB channels at the sampling location The final enhanced color is obtained.

[0129] The embodiments of this application are designed for the calculation process of the XR device rendering compositor, which can be directly integrated with the rendering link of the compositor.

[0130] This application's embodiments filter directional aliasing, reducing aliasing and jitter caused by super-resolution algorithms; at the same time, the brightness data compression process in the computing pipeline can reduce memory read / write consumption, thereby reducing GPU usage and computing time.

[0131] The embodiments of this application perform pixel-level continuous compensation for the sampling interval changes caused by pre-distortion calculation, thereby reducing the intensity of moiré patterns and jagged edges appearing at the edge of the field of view.

[0132] The compression storage step in the pre-filtering calculation of this application embodiment can reduce memory read / write bandwidth consumption and reduce subsequent calculation load.

[0133] This application uses a fixation-based approach, applying the method only to the fixation-based region, further reducing the computational load.

[0134] The embodiments of this application do not have an edge detection process, which reduces the jaggedness caused by edge detection in the enhanced image.

[0135] The image enhancement method provided in this application can be executed by an image enhancement device. This application uses an image enhancement device executing the image enhancement method as an example to illustrate the image enhancement device provided in this application.

[0136] like Figure 9 As shown in the figure, this application provides an image enhancement device, which includes the following modules.

[0137] The acquisition module 902 is used to acquire color information and brightness matrix of the sampling position in the image to be processed; wherein the brightness matrix is ​​obtained based on the first neighboring pixels of the sampling position.

[0138] The filtering module 904 is used to perform low-pass filtering on the brightness matrix to obtain a first brightness value at the sampling position; and to perform high-pass filtering on the second neighboring pixels at the sampling position to obtain a second brightness value at the sampling position.

[0139] The calculation and processing module 906 is used to perform a mixing process on the first brightness value and the second brightness value based on a preset sharpness coefficient, and to calculate the difference between the mixing result and the original brightness at the sampling position.

[0140] The enhanced display module 908 is used to perform a reduction processing on the color information of the sampling position based on the difference, so as to obtain the enhanced color of the sampling position.

[0141] In this embodiment, low-pass filtering is performed on the brightness matrix of the sampling location, and high-pass filtering is performed on the sampling location and its neighboring pixels. This achieves the separation of background and details in the image to be processed, filters out directional jagged edges, and reduces jagged edges and jagged edge jitter caused by super-resolution algorithms. The mixing ratio of the first and second brightness values ​​is controlled by a sharpness coefficient, and a reduction processing is performed in the color space based on the difference between the mixed result and the original brightness, achieving refined enhancement of image details. This method effectively overcomes the contradiction between detail enhancement and noise amplification in traditional methods, reducing jagged edges and jagged edge jitter. Furthermore, this algorithm does not require modification of the main flow of the upper-layer rendering program, has low computational load, helps reduce computational latency, and is easy to deploy and implement in XR devices.

[0142] In some embodiments, the filtering module 904 is configured to: determine the first derivative value of the optomechanical pre-distortion curve of the pixel to be rendered based on the distortion radius data of the pixel to be rendered; determine the spatial scale scaling factor of the low-pass filter kernel function based on the first derivative value; and perform low-pass filtering on the luminance matrix based on the spatial scale scaling factor of the low-pass filter kernel function to obtain a first luminance value at the sampling position.

[0143] In some embodiments, the apparatus further includes a pre-filtering compression processing module, configured to: filter a 2x2 pixel block in the image to be processed using preset blur convolution parameters to obtain a blurred brightness value; store the blurred brightness value of each pixel block into multiple sub-pixel channels of a pixel to obtain a compressed and stored brightness value; wherein, the acquisition module 902 is configured to acquire a brightness matrix of the sampling position based on the compressed and stored brightness value.

[0144] In some embodiments, the pre-filter compression processing module is further configured to determine the gaze point range based on the texture size, gaze point coordinates, and gaze point region ratio of the image to be processed; wherein, the acquisition module 902 is configured to acquire the color information and brightness matrix of the sampling position in the image to be processed when the sampling position is located within the gaze point range.

[0145] In some embodiments, the filtering module 904 is configured to: determine the extreme brightness values ​​of the second neighboring pixels at the sampling position in the four directions of up, down, left, and right; adjust the gain of the high-pass filter kernel function based on the extreme brightness values; and perform high-pass filtering on the second neighboring pixels at the sampling position using the high-pass filter with adjusted gain to obtain the second brightness value at the sampling position.

[0146] In some embodiments, the calculation processing module 906 is configured to: if the difference between the mixing result and the original brightness of the sampling position exceeds the maximum value of a threshold range, then use the maximum value of the threshold range as the difference; and / or, if the difference between the mixing result and the original brightness of the sampling position is less than the minimum value of a threshold range, then use the minimum value of the threshold range as the difference.

[0147] In some embodiments, the enhanced display module 908 is further configured to: process the image to be processed based on the difference and a preset transition gradient mask to obtain a processed image.

[0148] The image enhancement device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the device.

[0149] The image enhancement device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit the specific operating system used.

[0150] The image enhancement device provided in this application embodiment can achieve... Figures 1 to 8 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0151] Optionally, such as Figure 10 As shown, this application embodiment also provides an electronic device 1000, including a processor 1001 and a memory 1002. The memory 1002 stores a program or instructions that can run on the processor 1001. When the program or instructions are executed by the processor 1001, they implement the various steps of the above-described image enhancement method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0152] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0153] Figure 11 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0154] The electronic device 1100 includes, but is not limited to, components such as: radio frequency unit 1101, network module 1102, audio output unit 1103, input unit 1104, sensor 1105, display unit 1106, user input unit 1107, interface unit 1108, memory 1109, and processor 1110.

[0155] Those skilled in the art will understand that the electronic device 1100 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 1110 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 11 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0156] The graphics processor 11041 is used to acquire color information and a brightness matrix of a sampling location in the image to be processed; wherein the brightness matrix is ​​obtained based on the first neighboring pixels of the sampling location; the brightness matrix is ​​subjected to low-pass filtering to obtain a first brightness value of the sampling location; and the second neighboring pixels of the sampling location are subjected to high-pass filtering to obtain a second brightness value of the sampling location; the first brightness value and the second brightness value are mixed based on a preset sharpness coefficient, and the difference between the mixed result and the original brightness of the sampling location is calculated; the color information of the sampling location is reduced based on the difference to obtain the enhanced color of the sampling location.

[0157] In this embodiment, low-pass filtering is performed on the brightness matrix of the sampling location, and high-pass filtering is performed on the sampling location and its neighboring pixels. This achieves the separation of background and details in the image to be processed, filters out directional jagged edges, and reduces jagged edges and jagged edge jitter caused by super-resolution algorithms. The mixing ratio of the first and second brightness values ​​is controlled by a sharpness coefficient, and a reduction processing is performed in the color space based on the difference between the mixed result and the original brightness, achieving refined enhancement of image details. This method effectively overcomes the contradiction between detail enhancement and noise amplification in traditional methods, reducing jagged edges and jagged edge jitter. Furthermore, this algorithm does not require modification of the main flow of the upper-layer rendering program, has low computational load, helps reduce computational latency, and is easy to deploy and implement in XR devices.

[0158] It should be understood that, in this embodiment, the input unit 1104 may include a graphics processing unit (GPU) 11041 and a microphone 11042. The GPU 11041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 1106 may include a display panel 11061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 1107 includes at least one of a touch panel 11071 and other input devices 11072. The touch panel 11071 is also called a touch screen. The touch panel 11071 may include a touch detection device and a touch controller. Other input devices 11072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0159] The memory 1109 can be used to store software programs and various data. The memory 1109 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 1109 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 1109 in this embodiment includes, but is not limited to, these and any other suitable types of memory.

[0160] Processor 1110 may include one or more processing units; optionally, processor 1110 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 1110.

[0161] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described image enhancement method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0162] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0163] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described image enhancement method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0164] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0165] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the image enhancement method embodiments described above, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0166] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0167] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0168] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An image enhancement method, characterized in that, include: Obtain the color information and brightness matrix of the sampling location in the image to be processed; wherein, the brightness matrix is ​​obtained based on the first neighboring pixels of the sampling location; The brightness matrix is ​​subjected to low-pass filtering to obtain a first brightness value at the sampling location; and the second neighboring pixels at the sampling location are subjected to high-pass filtering to obtain a second brightness value at the sampling location. The first brightness value and the second brightness value are mixed based on a preset sharpness coefficient, and the difference between the mixed result and the original brightness at the sampling position is calculated. Based on the difference, the color information at the sampling location is reduced to obtain the enhanced color at the sampling location.

2. The method according to claim 1, characterized in that, The step of performing low-pass filtering on the brightness matrix to obtain the first brightness value at the sampling position includes: The first derivative value of the optomechanical pre-distortion curve of the pixel to be rendered is determined based on the distortion radius data of the pixel to be rendered position; The spatial scale scaling factor of the low-pass filter kernel function is determined based on the first-order derivative value. Based on the spatial scale scaling factor of the low-pass filter kernel function, the brightness matrix is ​​subjected to low-pass filtering to obtain the first brightness value at the sampling position.

3. The method according to claim 1 or 2, characterized in that, Before obtaining the brightness matrix of the sampling positions in the image to be processed, the method further includes: Using preset blur convolution parameters, a 2x2 pixel block in the image to be processed is filtered to obtain a blurred brightness value; The blurred brightness value of each pixel block is stored in multiple sub-pixel channels of a pixel to obtain a compressed and stored brightness value; The step of obtaining the brightness matrix of the sampling position includes: obtaining the brightness matrix of the sampling position based on the compressed and stored brightness values.

4. The method according to claim 3, characterized in that, Before obtaining the color information and brightness matrix of the sampling position in the image to be processed, the method further includes: The gaze point range is determined based on the texture size, gaze point coordinates, and gaze point region ratio of the image to be processed. The step of obtaining the color information and brightness matrix of the sampling position in the image to be processed includes: when the sampling position is located within the range of the gaze point, obtaining the color information and brightness matrix of the sampling position in the image to be processed.

5. The method according to claim 1, characterized in that, The step of performing high-pass filtering on the second neighboring pixels at the sampling location to obtain the second brightness value at the sampling location includes: Determine the extreme brightness values ​​of the second neighboring pixels at the sampling position in the four directions of top, bottom, left, and right; The gain of the high-pass filter kernel function is adjusted based on the aforementioned brightness extrema; By using a high-pass filter with adjusted gain, the second neighboring pixels at the sampling location are subjected to high-pass filtering to obtain the second brightness value at the sampling location.

6. The method according to claim 1, characterized in that, The difference between the calculated blending result and the original brightness of the pixel at the sampling location includes: If the difference between the blending result and the original brightness at the sampling location exceeds the maximum value of a threshold range, then the maximum value of the threshold range is used as the difference; and / or If the difference between the mixing result and the original brightness at the sampling location is less than the minimum value of the threshold range, then the minimum value of the threshold range is taken as the difference.

7. The method according to claim 1, characterized in that, The method further includes: The image to be processed is processed based on the difference and a preset transition gradient mask to obtain the processed image.

8. An image enhancement device, characterized in that, include: The acquisition module is used to acquire color information and a brightness matrix of the sampling position in the image to be processed; wherein, the brightness matrix is ​​obtained based on the first neighboring pixels of the sampling position; The filtering module is used to perform low-pass filtering on the brightness matrix to obtain a first brightness value at the sampling position; and to perform high-pass filtering on the second neighboring pixels at the sampling position to obtain a second brightness value at the sampling position. The calculation and processing module is used to perform a mixing process on the first brightness value and the second brightness value based on a preset sharpness coefficient, and to calculate the difference between the mixing result and the original brightness at the sampling position; An enhanced display module is used to perform a reduction processing on the color information of the sampling position based on the difference, so as to obtain the enhanced color of the sampling position.

9. The apparatus according to claim 8, characterized in that, The filtering module is used for: The first derivative value of the optomechanical pre-distortion curve of the pixel to be rendered is determined based on the distortion radius data of the pixel to be rendered position; The spatial scale scaling factor of the low-pass filter kernel function is determined based on the first-order derivative value. Based on the spatial scale scaling factor of the low-pass filter kernel function, the brightness matrix is ​​subjected to low-pass filtering to obtain the first brightness value at the sampling position.

10. The apparatus according to claim 8 or 9, characterized in that, The device further includes a pre-filtering and compression processing module, used for: Using preset blur convolution parameters, a 2x2 pixel block in the image to be processed is filtered to obtain a blurred brightness value; The blurred brightness value of each pixel block is stored in multiple sub-pixel channels of a pixel to obtain a compressed and stored brightness value; The acquisition module is used to acquire the brightness matrix of the sampling position based on the compressed and stored brightness values.

11. The apparatus according to claim 10, characterized in that, The pre-filtering compression processing module is also used to determine the gaze point range based on the texture size, gaze point coordinates, and gaze point region ratio of the image to be processed. The acquisition module is used to acquire color information and brightness matrix of the sampling position in the image to be processed when the sampling position is within the range of the gaze point.

12. The apparatus according to claim 8, characterized in that, The filtering module is used for: Determine the extreme brightness values ​​of the second neighboring pixels at the sampling position in the four directions of top, bottom, left, and right; The gain of the high-pass filter kernel function is adjusted based on the aforementioned brightness extrema; By using a high-pass filter with adjusted gain, the second neighboring pixels at the sampling location are subjected to high-pass filtering to obtain the second brightness value at the sampling location.

13. The apparatus according to claim 8, characterized in that, The computation processing module is used for: If the difference between the mixing result and the original brightness at the sampling location exceeds the maximum value of the threshold range, then the maximum value of the threshold range is taken as the difference. and / or If the difference between the mixing result and the original brightness at the sampling location is less than the minimum value of the threshold range, then the minimum value of the threshold range is taken as the difference.

14. The apparatus according to claim 8, characterized in that, The enhanced display module is also used for: The image to be processed is processed based on the difference and a preset transition gradient mask to obtain the processed image.

15. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a program or instructions that can run on the processor, the program or instructions being executed by the processor to implement the steps of the method as described in any one of claims 1 to 7.

16. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the method as described in any one of claims 1 to 7.

Images