Image enhancement processing method, device, equipment and storage medium

Abstract

The present disclosure provides an image enhancement processing method, device, equipment and storage medium, which relates to the field of artificial intelligence technology, and in particular to the field of cloud computing technology. The specific implementation scheme is: in response to the read input image, the color space corresponding to the input image and all color pixels of at least one color component in the color space are obtained; the sharpened pixel values ​​corresponding to the pixel positions of all color pixels are determined; based on the sharpened pixel values ​​corresponding to the pixel positions of all color pixels, the smoothed sharpened pixel values ​​corresponding to all pixel positions are determined; according to the smoothed sharpened pixel values ​​corresponding to all pixel positions, the image to be encoded corresponding to the input image is obtained.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, in particular to the field of cloud computing technology, and specifically to an image enhancement processing method, device, equipment and storage medium. Background Art

[0002] At present, there is a consensus on the superiority of the technical framework for pre-processing and enhancing the medium and low quality encoded content before encoding. Not only can it effectively improve the subjective experience of the encoded video, but also because the pre-processing enhancement module is located outside the encoding loop, it is non-invasive to the encoder and can be effectively adapted to multiple encoding standards.

[0003] Sharpening is a commonly used video enhancement method to improve the subjective quality of video. The existing sharpening methods used in encoders are to obtain an estimate of the optimal coding performance index through multiple encoding, obtain the corresponding optimal sharpening strength, and then use the optimal sharpening strength for sharpening processing.

[0004] However, research on the human visual system has revealed that the human eye is insensitive to high-frequency information, which means that the contribution of each pixel of a frame of coded image source to the final subjective experience is not evenly distributed. Although the traditional sharpening algorithm is adaptive, that is, the pixel changes less in the flat texture area and the pixel changes more in the complex texture area. However, this model does not take into account the fact that human visual characteristics do not pay attention to high-frequency information. This is because in traditional image processing, subjective experience is the only influencing factor, and whether it is suitable for further encoding by the encoder is not considered. Summary of the invention

[0005] The present disclosure provides an image enhancement processing method, device, equipment and storage medium.

[0006] According to a first aspect of the present disclosure, there is provided an image enhancement processing method, the method comprising:

[0007] In response to the read input image, acquiring a color space corresponding to the input image and all color pixels of at least one color component in the color space;

[0008] Determine the sharpened pixel values ​​corresponding to the pixel positions of all the color pixels;

[0009] Determine the smoothed sharpened pixel values ​​corresponding to all the pixel positions based on the sharpened pixel values ​​corresponding to all the pixel positions of the color pixels;

[0010] According to the smoothed and sharpened pixel values ​​corresponding to all the pixel positions, an image to be encoded corresponding to the input image is obtained.

[0011] Further, determining the sharpened pixel values ​​corresponding to the pixel positions of all the color pixels respectively includes:

[0012] Determine an original pixel value at a pixel position of each of the color pixels, and a Gaussian mean of all original pixel values ​​in a neighborhood of each of the color pixels;

[0013] The sharpened pixel value corresponding to the pixel position of each color pixel is determined according to the original pixel value at the pixel position of each color pixel and the Gaussian mean of all original pixel values ​​in the neighborhood of each color pixel.

[0014] Further, determining the sharpened pixel value corresponding to the pixel position of each color pixel according to the original pixel value at the pixel position of each color pixel and the Gaussian mean of all original pixel values ​​in the neighborhood of each color pixel includes:

[0015] Determine the difference between the original pixel value at the pixel position of each color pixel and the Gaussian mean of all original pixel values ​​in the neighborhood of each color pixel;

[0016] The sharpened pixel value corresponding to the pixel position of each color pixel is determined according to the product of the difference and a predetermined scaling factor.

[0017] Further, based on the sharpened pixel values ​​corresponding to the pixel positions of all the color pixels, determining the smoothed sharpened pixel values ​​corresponding to all the pixel positions, comprises:

[0018] Obtaining a sharpened pixel plane of at least one of the color components based on the sharpened pixel values ​​corresponding to the pixel positions of all the color pixels;

[0019] According to all the sharpened pixel values ​​on the sharpened pixel plane of at least one of the color components, smoothed sharpened pixel values ​​corresponding to all pixel positions on the sharpened pixel plane are determined.

[0020] Further, determining the smoothed sharpened pixel values ​​corresponding to all pixel positions on the sharpened pixel plane according to all the sharpened pixel values ​​on the sharpened pixel plane of at least one of the color components, comprises:

[0021] According to the Gaussian mean of all the sharpened pixel values ​​on the sharpened pixel plane of at least one of the color components, the smoothed sharpened pixel values ​​corresponding to all the pixel positions on the sharpened pixel plane are determined.

[0022] Further, according to the smoothed and sharpened pixel values ​​corresponding to all the pixel positions, an image to be encoded corresponding to the input image is obtained, including:

[0023] Obtaining pixel values ​​to be encoded corresponding to all the pixel positions according to the original pixel values ​​corresponding to all the pixel positions and the smoothed and sharpened pixel values ​​corresponding to all the pixel positions;

[0024] Based on the pixel values ​​to be encoded corresponding to all the pixel positions, obtaining a pixel plane to be encoded of at least one of the color components;

[0025] Based on the to-be-encoded pixel plane of at least one of the color components, the to-be-encoded image corresponding to the input image is obtained.

[0026] Further, according to the original pixel values ​​corresponding to all the pixel positions and the smoothed and sharpened pixel values ​​corresponding to all the pixel positions, the pixel values ​​to be encoded corresponding to all the pixel positions are obtained, including:

[0027] Obtaining the bit depth of the input image and the valid value range corresponding to the bit depth;

[0028] Determine the sum of the original pixel value and the smoothed and sharpened pixel value corresponding to each of the pixel positions;

[0029] If the sum value corresponding to any one of all the pixel positions exceeds the valid value range, the sum value exceeding the valid value range is truncated to within the valid value range to obtain the pixel values ​​to be encoded corresponding to all the pixel positions.

[0030] Furthermore, obtaining a color space corresponding to the input image includes:

[0031] Obtaining an initial color space of the input image;

[0032] If the initial color space is a non-target color space, converting the initial color space to obtain a color space corresponding to the input image;

[0033] If the initial color space is the target color space, the target color space is used as the color space corresponding to the input image.

[0034] According to a second aspect of the present disclosure, there is provided an image enhancement processing device, the device comprising:

[0035] an acquisition unit, configured to acquire, in response to a read input image, a color space corresponding to the input image and all color pixels of at least one color component in the color space;

[0036] A first determining unit, configured to determine sharpened pixel values ​​corresponding to pixel positions of all the color pixels;

[0037] A second determining unit, configured to determine the smoothed sharpened pixel values ​​corresponding to all the pixel positions respectively based on the sharpened pixel values ​​corresponding to the pixel positions of all the color pixels respectively;

[0038] The processing unit is used to obtain an image to be encoded corresponding to the input image according to the smoothed and sharpened pixel values ​​corresponding to all the pixel positions.

[0039] Furthermore, the first determining unit includes:

[0040] A first determining subunit is used to determine an original pixel value at a pixel position of each of the color pixels, and a Gaussian mean of all original pixel values ​​in a neighborhood of each of the color pixels;

[0041] The second determination subunit is used to determine the sharpened pixel value corresponding to the pixel position of each color pixel based on the original pixel value at the pixel position of each color pixel and the Gaussian mean of all original pixel values ​​in the neighborhood of each color pixel.

[0042] Furthermore, the second determining subunit includes:

[0043] A first determination submodule, used to determine the difference between the original pixel value at the pixel position of each color pixel and the Gaussian mean of all original pixel values ​​in the neighborhood of each color pixel;

[0044] The second determination submodule is used to determine the sharpened pixel value corresponding to the pixel position of each color pixel according to the product of the difference and the pre-determined scaling factor.

[0045] Furthermore, the second determining unit includes:

[0046] A first generating subunit, configured to obtain a sharpened pixel plane of at least one of the color components based on the sharpened pixel values ​​corresponding to the pixel positions of all the color pixels;

[0047] The third determining subunit is used to determine the smoothed sharpened pixel values ​​corresponding to all pixel positions on the sharpened pixel plane according to all the sharpened pixel values ​​on the sharpened pixel plane of at least one of the color components.

[0048] Furthermore, the third determining subunit includes:

[0049] The third determination submodule is used to determine the smoothed sharpened pixel values ​​corresponding to all pixel positions on the sharpened pixel plane according to the Gaussian mean of all the sharpened pixel values ​​on the sharpened pixel plane of at least one of the color components.

[0050] Furthermore, the processing unit comprises:

[0051] A calculation subunit, used for obtaining the pixel values ​​to be encoded corresponding to all the pixel positions respectively according to the original pixel values ​​corresponding to all the pixel positions respectively and the smoothed and sharpened pixel values ​​corresponding to all the pixel positions respectively;

[0052] A second generating subunit, configured to obtain a pixel plane to be encoded of at least one of the color components based on the pixel values ​​to be encoded corresponding to all the pixel positions;

[0053] The processing subunit is used to obtain the image to be encoded corresponding to the input image based on the pixel plane to be encoded of at least one color component.

[0054] Furthermore, the computing subunit includes:

[0055] An acquisition submodule, used to acquire the bit depth of the input image and a valid value range corresponding to the bit depth;

[0056] A calculation submodule, used to determine the sum of the original pixel value and the smoothed and sharpened pixel value corresponding to each of the pixel positions;

[0057] The truncation processing submodule is used to truncate the sum value exceeding the valid value range to within the valid value range if the sum value corresponding to any pixel position among all the pixel positions exceeds the valid value range, so as to obtain the pixel values ​​to be encoded corresponding to all the pixel positions respectively.

[0058] Furthermore, the acquisition unit includes:

[0059] A first acquisition subunit, used to acquire an initial color space of the input image;

[0060] A second acquisition subunit is used for converting the initial color space to obtain a color space corresponding to the input image if the initial color space is a non-target color space;

[0061] The third acquisition subunit is configured to use the target color space as the color space corresponding to the input image if the initial color space is a target color space.

[0062] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0063] at least one processor; and

[0064] a memory communicatively connected to the at least one processor; wherein,

[0065] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform any one of the methods described above.

[0066] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute any one of the methods.

[0067] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising: a computer program, wherein the computer program is stored in a readable storage medium, at least one processor of an electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program so that the electronic device executes the method described in the first aspect.

[0068] According to the technology disclosed in the present invention, first, in response to the read input image, the color space corresponding to the above input image and all color pixels of at least one color component in the above color space are obtained. Secondly, by sharpening the original pixel value of each color pixel and determining the sharpened pixel values ​​corresponding to the pixel positions of all color pixels, the details and contrast of the input image can be improved to make the image clearer. Afterwards, in order to eliminate the noise and artifacts that may be generated during the sharpening process and make the image smoother, the present invention determines the smoothed sharpened pixel values ​​corresponding to all pixel positions based on the sharpened pixel values ​​corresponding to the pixel positions of all color pixels. The smoothed sharpened pixel value is the sharpened pixel value after smoothing, which is usually lower in amplitude than the sharpened pixel value, and can reduce noise and artifacts in the image. Finally, according to the smoothed sharpened pixel values ​​corresponding to all the above pixel positions, the image to be encoded corresponding to the above input image is obtained. The pre-coding image enhancement method provided by the present invention can reduce the high complexity of calculation, eliminate the noise and artifacts that may be generated during the sharpening process, suppress the parts that are not conducive to coding and are not easily perceived by the human eye, make the image smoother, thereby achieving an improvement in the subjective experience while reducing the damage to the spatial redundancy of the coding source.

[0069] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0071] Figure 1 is a flow chart of an image enhancement processing method provided according to an embodiment of the present disclosure;

[0072] Figure 2 is a flow chart of an optional image enhancement processing method that can implement the embodiment of the present disclosure;

[0073] Figure 3 is a flowchart of an optional image enhancement processing method provided according to an embodiment of the present disclosure;

[0074] Figure 4 is a flowchart of an optional image enhancement processing method provided according to an embodiment of the present disclosure;

[0075] Figure 5 is a flowchart of an optional image enhancement processing method provided according to an embodiment of the present disclosure;

[0076] Figure 6 is a flowchart of an optional image enhancement processing method provided according to an embodiment of the present disclosure;

[0077] Figure 7 is a schematic diagram of a framework of an optional image enhancement processing device provided according to an embodiment of the present disclosure;

[0078] Figure 8 The block diagram is a block diagram of an electronic device used to implement an image enhancement processing method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0079] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0080] First, the terms involved in this application are explained:

[0081] Visual Multimethod Assessment Fusion (VMAF) is a subjective video quality evaluation system that introduces a deep learning mechanism to give videos a quality score that is more in line with human vision.

[0082] At present, the industry has reached a consensus on the superiority of the technical framework for pre-processing and enhancing the medium and low quality encoded content before encoding. This framework can not only effectively improve the subjective experience of the encoded video, but also because the pre-processing enhancement module is located outside the encoding loop, it is non-invasive to the encoder and can be effectively adapted to multiple encoding standards.

[0083] Sharpening is a commonly used video enhancement method to improve the subjective quality of videos. The existing sharpening methods used in encoders are to obtain an estimate of the optimal coding performance index through multiple encoding, obtain the corresponding optimal sharpening strength, and then use the optimal sharpening strength for sharpening processing. Existing methods generally adopt a given search space and brute force search in the search space; or first traverse a smaller search space, then model the joint optimization objectives of sharpening strength and bit rate-distortion, and then solve the model. This type of method is directly related to the final optimization goal, but the high complexity brought by the traversal of the search space is difficult to accept in real applications, especially in low-latency encoding scenarios.

[0084] In addition, research on the human visual system has revealed that the human eye is insensitive to high-frequency information. This means that the contribution of each pixel of a frame of coded image source to the final subjective experience is not evenly distributed. Although the traditional sharpening algorithm is adaptive, that is, the pixel changes are small in flat texture areas and large in complex texture areas. However, this model does not take into account the fact that human visual characteristics do not pay attention to high-frequency information. This is because in traditional image processing, subjective experience is the only influencing factor, and whether it is suitable for further encoding by the encoder is not considered.

[0085] In order to solve the above problems, the present disclosure provides an image enhancement processing method, device, equipment and storage medium, which are applied to the field of artificial intelligence technology, especially to the field of cloud computing technology. The present disclosure further smoothes the sharpened pixel values ​​corresponding to the pixel positions of all color pixels to reduce the noise of the sharpened pixel values. In principle, this characteristic of the human visual system is utilized to suppress the parts that the human eye is not sensitive to and retain the parts that the human eye is sensitive to, thereby achieving an improvement in the subjective experience while reducing the damage to the spatial redundancy of the coding source. Due to the introduction of this prior information, multiple encodings introduced by the a posteriori method can be avoided, and the high complexity of the calculation can be reduced, so as to achieve efficient coding source enhancement in low-latency coding scenarios and gain advantages over competing products.

[0086] Figure 1 is a flowchart of an image enhancement processing method provided according to an embodiment of the present disclosure, such as Figure 1 As shown, the present disclosure provides an image enhancement processing method, comprising the following method steps:

[0087] S101, in response to a read input image, obtaining a color space corresponding to the input image and all color pixels of at least one color component in the color space;

[0088] S102, determining the sharpened pixel values ​​corresponding to the pixel positions of all the above color pixels;

[0089] S103, determining smoothed sharpened pixel values ​​corresponding to all the pixel positions based on the sharpened pixel values ​​corresponding to the pixel positions of all the color pixels;

[0090] S104, obtaining an image to be encoded corresponding to the input image according to the smoothed and sharpened pixel values ​​corresponding to all the pixel positions.

[0091] Optionally, in the example disclosed herein, for example, an input image input from an external network or hard disk or other device is a three-dimensional data including width, height, and number of color channels, and a certain color component is a two-dimensional data plane including width and height.

[0092] Optionally, a color space is a mathematical model used to describe color. Common color spaces include RGB, YCbCr, HSV, etc. A color component is a dimension in a color space that is used to represent a certain attribute of a color, such as R (red component), G (green component), and B (blue component) in the RGB color space. A color pixel is the smallest unit in an image. Each color pixel has one or more color component values ​​that represent the color of the pixel. For example, each color pixel in the RGB color space has three color component values ​​that represent the intensity of red, green, and blue, respectively.

[0093] In one example, for example, if the initial color space of the input image is RGB and the target color space is YCbCr, then it is necessary to perform a mathematical transformation on the RGB value of each color pixel of the input image to obtain the corresponding YCbCr value. Then, at least one color component, such as the brightness component, is selected as needed to obtain the pixel values ​​of all pixel positions of the component. In this way, by converting and selecting the color space of the input image, the color information of the input image is extracted, providing a basis for subsequent image enhancement processing.

[0094] It is understood that sharpening is an image processing technique that enhances the details and contrast of an image and improves the clarity of the image by increasing the light and dark differences of the edges and contours in the image. In the disclosed example, the sharpened pixel value is obtained by sharpening the color pixel value to highlight the edges and contours in the image. By sharpening the pixel value of each color pixel, the details and contrast of the image are improved to make the image clearer.

[0095] Afterwards, since noise refers to random or irregular brightness or color changes in an image, it will affect the quality and visual effects of the image. Artifacts refer to unreal or unnatural phenomena in an image, such as aliasing, ringing, halos, etc., which will affect the authenticity and aesthetics of the image. In order to eliminate noise and artifacts that may be generated during the sharpening process and make the image smoother, the embodiment of the present disclosure also determines the smoothed sharpened pixel values ​​corresponding to all pixel positions based on the sharpened pixel values ​​corresponding to the pixel positions of all color pixels.

[0096] In the disclosed example, smoothing is an image processing to improve the smoothness of an image by reducing noise and artifacts in the image. The smoothed sharpened pixel value is the sharpened pixel value after smoothing, and is usually lower in amplitude than the sharpened pixel value to retain spatial redundancy in the image.

[0097] Optionally, the above-mentioned image to be encoded is an image to be encoded, and encoding is to reduce the amount of image data or change the image representation by compressing or converting the image. Usually, the encoded image will take up less storage space or bandwidth than the input image, or have characteristics that are more suitable for specific purposes. Afterwards, the image to be encoded after the image enhancement processing is encoded by the encoder. The image enhancement method before encoding provided by the present disclosure can eliminate noise and artifacts that may be generated during the sharpening process, suppress the parts that are not conducive to encoding and are not easily perceived by the human eye, and make the image smoother.

[0098] Based on the aforementioned human visual characteristics and coding framework features, the present disclosure implements a smoothing sharpening algorithm for subjective experience and coding performance, while avoiding the high complexity brought about by multiple encodings and VMAF index calculations, so that it can be applied in actual coding, especially in delay-sensitive application scenarios. The present disclosure adopts prior knowledge based on the human visual system and source coding, and in principle directly obtains the part of the enhanced information that is beneficial to improving the subjective experience and does not affect the spatial redundancy of pixels. The present disclosure focuses on suppressing the parts that are not conducive to coding and are not easily perceived by the human eye in the sharpening algorithm.

[0099] In one example, Figure 2 is a flowchart of an optional image enhancement processing method provided according to an embodiment of the present disclosure, such as Figure 2 As shown, determining the sharpened pixel values ​​corresponding to the pixel positions of all the above color pixels respectively includes:

[0100] S201, determining an original pixel value at a pixel position of each of the color pixels, and a Gaussian mean of all original pixel values ​​in a neighborhood of each of the color pixels;

[0101] S202, determining a sharpened pixel value corresponding to the pixel position of each of the color pixels according to the original pixel value at the pixel position of each of the color pixels and the Gaussian mean of all original pixel values ​​in the neighborhood of each of the color pixels.

[0102] Optionally, the original pixel value refers to the pixel value of each color pixel in the input image, which reflects the color intensity of the pixel. The neighborhood refers to other pixels within a certain range around a pixel, which can usually be represented by a rectangular window. For example, the neighborhood size can be a 3×3 rectangular window.

[0103] The optional Gaussian mean removes noise and details by taking a weighted average of the pixel values ​​in the neighborhood of the pixel to get the average value of the pixel. The weight of the weighted average is determined according to the Gaussian function, which is a commonly used probability distribution function that is shaped like a bell curve with a peak and two symmetrical tails. The characteristic of the Gaussian function is that the closer the pixel is to the peak, the greater its weight, and the farther the pixel is from the peak, the smaller its weight. This ensures that the pixel values ​​in the neighborhood are not processed averagely, but are appropriately adjusted according to their distance from the center pixel.

[0104] Using the above public example, the pixel value of each color pixel of the input image is read to obtain the original pixel value, and then the Gaussian mean of all pixel values ​​in the neighborhood of each pixel is calculated according to the preset neighborhood size and Gaussian function parameters. In this way, by performing Gaussian smoothing on the original pixel value of each color pixel, noise and details in the image are eliminated, providing a smooth basis for subsequent sharpening processing.

[0105] In order to enhance the details and contrast of the image and make the image clearer. Optionally, in one example, Figure 3 is a flowchart of an optional image enhancement processing method provided according to an embodiment of the present disclosure, such as Figure 3 As shown, according to the original pixel value at the pixel position of each color pixel and the Gaussian mean of all original pixel values ​​in the neighborhood of each color pixel, the sharpened pixel value corresponding to the pixel position of each color pixel is determined, including:

[0106] S301, determining a difference between an original pixel value at a pixel position of each color pixel and a Gaussian mean of all original pixel values ​​in a neighborhood of each color pixel;

[0107] S302: Determine a sharpened pixel value corresponding to the pixel position of each of the color pixels according to the product of the difference and a predetermined scaling factor.

[0108] Optionally, the difference between the original pixel value and the Gaussian mean at the pixel position of each color pixel reflects the details and contrast of the pixel at that pixel position. By multiplying the difference by a scaling factor, the amplitude of the difference can be magnified or reduced, thereby enhancing or weakening the sharpening effect of the pixel.

[0109] Optionally, the above scaling factor is an adjustable parameter, which determines the intensity of sharpening, and is usually between 0 and 2. The larger the value, the more obvious the sharpening effect, and the closer it is to 0, the weaker the sharpening effect.

[0110] In the example disclosed herein, a mathematical operation is performed on the difference between the original pixel value at the pixel position of each color pixel and the Gaussian mean of all original pixel values ​​in the neighborhood of each color pixel to obtain a sharpened pixel value corresponding to the pixel position of each color pixel.

[0111] Through the above example, by sharpening the pixel value of each color pixel, the details and contrast of the image can be improved to make the image clearer.

[0112] In an optional example, Figure 4 is a flowchart of an optional image enhancement processing method provided according to an embodiment of the present disclosure, such as Figure 4 As shown, based on the sharpened pixel values ​​corresponding to the pixel positions of all the color pixels, the smoothed sharpened pixel values ​​corresponding to all the pixel positions are determined, including:

[0113] S401, obtaining a sharpened pixel plane of at least one of the color components based on the sharpened pixel values ​​corresponding to the pixel positions of all the color pixels;

[0114] S402: Determine, based on all the sharpened pixel values ​​on the sharpened pixel plane of at least one of the color components, smoothed sharpened pixel values ​​corresponding to all pixel positions on the sharpened pixel plane.

[0115] Optionally, the sharpening pixel plane is a two-dimensional matrix, each element of which represents a sharpening pixel value, and its position corresponds one-to-one to the color pixel position in the input image.

[0116] An optional example is to store or transmit the sharpened pixel value of each color component, and then arrange the sharpened pixel value of each color component into a sharpened pixel plane in a certain order and format according to the color space and bit depth of the input image. For example, if the color space of the input image is YCbCr and the bit depth is 8 bits, then the sharpened pixel value of each color pixel is a signed integer between -255 and 255, and each sharpened pixel plane is a two-dimensional matrix composed of sharpened pixel values ​​at different pixel positions, indicating the sharpened pixel value intensity of the corresponding color component.

[0117] It should be noted that in the example disclosed herein, the above-mentioned smoothed sharpened pixel values ​​are sharpened pixel values ​​after smoothing, and are usually lower in amplitude than the sharpened pixel values ​​to preserve the details and contrast in the image.

[0118] After that, all sharpened pixel values ​​on the sharpened pixel plane are Gaussian smoothed to obtain the smoothed sharpened pixel values ​​corresponding to all pixel positions on the plane; in this way, the noise and artifacts that may be generated during the sharpening process can be eliminated, making the image smoother. Gaussian smoothing is a smoothing algorithm that eliminates noise and artifacts by weighted averaging the pixel values ​​in the neighborhood of a pixel to obtain the average value of the pixel.

[0119] In another optional example, determining the smoothed sharpened pixel values ​​corresponding to all pixel positions on the sharpened pixel plane according to all the sharpened pixel values ​​on the sharpened pixel plane of at least one of the color components includes:

[0120] According to the Gaussian mean of all the above-mentioned sharpened pixel values ​​on the sharpened pixel plane of at least one of the above-mentioned color components, the smoothed sharpened pixel values ​​corresponding to all the pixel positions on the above-mentioned sharpened pixel plane are determined.

[0121] Through the example disclosed in the present invention, mathematical operations are performed on each sharpened pixel value on the sharpened pixel plane and the Gaussian mean of all sharpened pixel values ​​in the neighborhood of each sharpened pixel value to obtain a smoothed sharpened pixel value corresponding to each sharpened pixel value.

[0122] Optionally, in this example, the calculation method of the Gaussian mean is the same as the aforementioned Gaussian mean, except that the object is changed from the original pixel value to the sharpened pixel value. The mathematical operation method is also the same as the aforementioned mathematical operation, except that the object is changed from the original pixel value and the Gaussian mean to the sharpened pixel value and the Gaussian mean.

[0123] As an alternative example, Figure 5 is a flowchart of an optional image enhancement processing method provided according to an embodiment of the present disclosure, such as Figure 5 As shown, according to the smoothed and sharpened pixel values ​​corresponding to all the above pixel positions, an image to be encoded corresponding to the above input image is obtained, including:

[0124] S501, obtaining pixel values ​​to be encoded corresponding to all the pixel positions respectively according to the original pixel values ​​corresponding to all the pixel positions respectively and the smoothed and sharpened pixel values ​​corresponding to all the pixel positions respectively;

[0125] S502, obtaining a pixel plane to be encoded of at least one of the color components based on the pixel values ​​to be encoded corresponding to all the pixel positions;

[0126] S503, obtaining the image to be encoded corresponding to the input image based on the pixel plane to be encoded of at least one of the color components.

[0127] In the example disclosed herein, a mathematical operation is performed on the original pixel values ​​and the smoothed and sharpened pixel values ​​corresponding to all pixel positions to obtain the pixel value to be encoded corresponding to each pixel position. There are many mathematical operation methods, for example, addition operation, difference operation, multiplication operation, ratio operation, etc. The specific method depends on the characteristics and requirements of the image.

[0128] For example, in the example disclosed herein, the original pixel values ​​and the smoothed and sharpened pixel values ​​corresponding to all pixel positions are added together, so that the original pixel values ​​and the smoothed and sharpened pixel values ​​can be fused together. Then, a pixel plane to be encoded is constructed based on the pixel values ​​to be encoded at each pixel position obtained by the addition. The plane contains all the color information and enhancement information of the image.

[0129] Optionally, the pixel plane to be encoded refers to a two-dimensional matrix composed of pixel values ​​to be encoded of a certain color component, each element represents a pixel value to be encoded, and its position corresponds one-to-one to the pixel position in the input image.

[0130] In order to separate the pixel values ​​to be encoded according to different color components, different pixel planes to be encoded are obtained, so as to perform different encoding processing on different color components. For example, in an optional example, each pixel value to be encoded on the pixel plane to be encoded is decomposed into pixel values ​​to be encoded of different color components according to the color space and bit depth of the input image, and then arranged into different pixel planes to be encoded in a certain order and format. For example, if the color space of the input image is RGB and the bit depth is 8 bits, then each pixel value to be encoded is a tuple composed of three pixel values ​​to be encoded, representing the intensities of red, green, and blue, respectively, and each pixel plane to be encoded is a matrix composed of one pixel value to be encoded, representing the planes of red, green, and blue, respectively.

[0131] Using the above example, the pixel values ​​to be encoded are separated according to different color components to obtain different pixel planes to be encoded, and then a pixel plane to be encoded is formed according to the pixel values ​​to be encoded at each pixel position, and then the plane is encoded to obtain an image to be encoded corresponding to the input image. In this way, by encoding the pixel plane to be encoded, the amount of image data can be reduced or the format of the image can be changed to facilitate the storage or transmission of the image.

[0132] In one example, the above step S501, according to the original pixel values ​​corresponding to all the above pixel positions and the smoothed and sharpened pixel values ​​corresponding to all the above pixel positions, obtains the pixel values ​​to be encoded corresponding to all the above pixel positions, including:

[0133] S5011, obtaining the bit depth of the input image and the valid value range corresponding to the bit depth;

[0134] S5012, determining the sum of the original pixel value and the smoothed and sharpened pixel value corresponding to each of the pixel positions;

[0135] S5013, if the sum value corresponding to any one of all the above pixel positions exceeds the above valid value range, the sum value exceeding the above valid value range is truncated to the above valid value range to obtain the pixel values ​​to be encoded corresponding to all the above pixel positions.

[0136] In the disclosed example, the bit depth of the input image and the valid value range corresponding to the bit depth can be obtained by reading the metadata of the input image (metadata refers to additional information of the image, usually including the format, size, resolution, color space, etc. of the image). In this way, the value range of the pixel value to be encoded can be determined to avoid image distortion or error caused by exceeding the range allowed by the bit depth.

[0137] Optionally, in the example disclosed herein, bit depth refers to the logarithmic value of the number of colors that each pixel can represent, usually expressed in the number of bits, such as 8 bits, 16 bits, 24 bits, etc.

[0138] Optionally, in the example disclosed herein, the valid value range refers to the maximum and minimum values ​​of the color intensity that each pixel can represent, and is usually represented by numbers, such as 0 to 255, 0 to 65535, etc.

[0139] Optionally, the above-mentioned original pixel value refers to the color intensity value of each pixel position in the input image, the smoothed and sharpened pixel value refers to the color intensity value of the pixel position after sharpening and smoothing, and the sum value is the addition result of the original pixel value and the smoothed and sharpened pixel value, which is usually higher or lower than the original pixel value or the smoothed and sharpened pixel value to highlight the edges and contours in the image while maintaining the color saturation and brightness of the image.

[0140] In the disclosed example, in order to fuse the original pixel value and the smoothed and sharpened pixel value, a comprehensive pixel value is obtained to retain the color information and enhanced information of the image.

[0141] Optionally, in the examples disclosed herein, bit depth refers to the logarithmic value of the number of colors that each pixel can represent, usually expressed in bits, such as 8 bits, 16 bits, 24 bits, etc. The valid value range refers to the maximum and minimum values ​​of the color intensity that each pixel can represent, usually expressed in numbers, such as 0 to 255, 0 to 65535, etc.

[0142] Furthermore, in the example disclosed herein, in order to ensure the validity of the sum value and avoid image distortion or error due to exceeding the valid value range, the sum values ​​corresponding to all pixel positions are judged. If the sum value corresponding to the pixel position exceeds the valid value range allowed by the bit depth of the input image, the sum value corresponding to the pixel position is truncated to the maximum or minimum value of the valid value range, otherwise the sum value corresponding to the pixel position is kept unchanged. Then, according to the pixel values ​​to be encoded of each color component, a pixel plane to be encoded is formed, and then the plane is encoded to obtain an image to be encoded corresponding to the input image.

[0143] For example, if the bit depth of the input image is 8 bits, the valid value range is 0 to 255. If the sum value corresponding to a pixel position is 300, the sum value corresponding to the pixel position is truncated to 255. If the sum value corresponding to a pixel position is -10, the sum value corresponding to the pixel position is truncated to 0. If the sum value corresponding to a pixel position is 100, the sum value corresponding to the pixel position is kept at 100.

[0144] Through the above examples of the present disclosure, pixel values ​​that exceed the valid value range are forcibly assigned to be equal to the maximum or minimum value of the valid value range, thereby ensuring the validity of the sum value and avoiding image distortion or errors due to exceeding the valid value range.

[0145] There is also an optional example of obtaining the color space corresponding to the above input image, including:

[0146] S1010, obtaining the initial color space of the input image;

[0147] S1011, if the initial color space is a non-target color space, converting the initial color space to obtain a color space corresponding to the input image;

[0148] S1012: If the initial color space is a target color space, use the target color space as the color space corresponding to the input image.

[0149] Optionally, the initial color space refers to the original color space of the input image, which is a mathematical model for describing color. Common color spaces include RGB, YCbCr, HSV, etc.

[0150] An optional implementation method is to obtain the initial color space of the input image by reading the metadata of the input image. The metadata refers to additional information of the image, which usually includes information such as the format, size, resolution, and color space of the image.

[0151] Optionally, the target color space refers to the final color space of the image enhancement processing, which is a mathematical model for describing color and is usually different from the initial color space. For example, the RGB color space can be converted to the YCbCr color space, or the YCbCr color space can be converted to the RGB color space, etc.

[0152] Optionally, the color space conversion is to change the color space of the image by mathematically transforming the color value of each pixel in the image.

[0153] In an example that can be implemented, it is determined whether the initial color space of the input image is the same as the target color space. If they are the same, the target color space is directly used as the color space corresponding to the input image. If they are different, the color space conversion process is performed. For example, the color value of each pixel of the input image is mathematically transformed according to the conversion formula between the initial color space and the target color space to obtain a new color value of each pixel, and then a new color space image is constructed according to the new color value as the color space corresponding to the input image.

[0154] In another example, if the initial color space of the input image is already the target color space, then there is no need to convert it. Directly using the target color space as the color space corresponding to the input image can avoid unnecessary conversion processing and save time and resources.

[0155] In an optional embodiment, Figure 6 is a flowchart of an optional image enhancement processing method provided according to an embodiment of the present disclosure, such as Figure 6 As shown, an optional image enhancement processing method includes the following implementation steps:

[0156] S601, in response to the read input image, obtaining a color space corresponding to the input image.

[0157] In the above step S601, the initial color space of the input image is first obtained; if the initial color space is a non-target color space, the initial color space is converted to obtain a color space corresponding to the input image; if the initial color space is a target color space, the target color space is used as the color space corresponding to the input image.

[0158] S602: Determine all color pixels of at least one color component in the color space.

[0159] S603, determining the original pixel value at the pixel position of each of the color pixels and the Gaussian mean of all pixel values ​​in the neighborhood of each of the color pixels.

[0160] S604: Determine a sharpened pixel value corresponding to each of the above pixel positions according to the original pixel value at each of the above pixel positions and the Gaussian mean of all pixel values ​​in the neighborhood of each of the above color pixels.

[0161] In the above step S604, the difference between the original pixel value at the pixel position of each color pixel and the Gaussian mean is first determined, and then the sharpened pixel value corresponding to the pixel position of each color pixel is determined based on the product of the above difference and a predetermined scaling factor (an adjustable parameter that determines the intensity of sharpening, usually with a value between 0 and 2).

[0162] S605: Obtain a sharpened pixel plane of at least one of the color components based on the sharpened pixel values ​​corresponding to all the pixel positions.

[0163] S606: Determine the smoothed sharpened pixel values ​​corresponding to all the pixel positions on the sharpened pixel plane according to the Gaussian mean of all the sharpened pixel values ​​on the sharpened pixel plane of at least one of the color components.

[0164] In the above step S6060, mathematical operations are performed on each sharpened pixel value on the sharpened pixel plane and the Gaussian mean of all sharpened pixel values ​​in the neighborhood of each sharpened pixel value to obtain a smoothed sharpened pixel value corresponding to each sharpened pixel value.

[0165] S607: Obtain the pixel values ​​to be encoded corresponding to all the above pixel positions according to the original pixel values ​​corresponding to all the above pixel positions and the smoothed and sharpened pixel values ​​corresponding to all the above pixel positions.

[0166] S608: Obtain a pixel plane to be encoded of at least one of the color components based on the pixel values ​​to be encoded corresponding to all the pixel positions.

[0167] S609: Obtain an image to be encoded corresponding to the input image according to the pixel plane to be encoded of at least one of the color components.

[0168] The main differences between the present disclosure and the existing video enhancement methods are as follows:

[0169] (1) Existing sharpening methods focus on searching and solving the optimal sharpening strength. They use the corresponding sharpening strength to sharpen in a given search space, and then calculate the corresponding optimization target index to obtain the optimal sharpening strength for video content processing. This requires multiple encoding and quality index calculations and has high complexity.

[0170] The present disclosure focuses on extracting the parts of the traditional sharpening algorithm that effectively improve the subjective experience of the video and suppressing the parts that are not conducive to encoding and are not easily perceived by the human eye, which is an optimization of the sharpening algorithm itself.

[0171] (2) The present invention is different from the traditional a posteriori enhancement processing framework. It is based on the characteristics of the human visual system and the prior information of the coding framework characteristics. It does not require multiple encodings and can avoid the delay caused by multiple encodings. It is suitable for delay-sensitive application scenarios.

[0172] (3) The present invention is based on the characteristics of the human visual system, takes source coding theory as a starting point, and is deeply integrated with subjective experience, and is suitable for coding tasks that jointly optimize bit rate-distortion performance.

[0173] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0174] According to an embodiment of the present disclosure, Figure 7 is a schematic diagram of a framework of an image enhancement processing device provided according to an embodiment of the present disclosure, such as Figure 7 As shown, the present disclosure further provides an image enhancement processing device, the image enhancement processing device 700 comprising:

[0175] The acquisition unit 701 is used to acquire, in response to the read input image, a color space corresponding to the input image and all color pixels of at least one color component in the color space;

[0176] A first determining unit 702 is used to determine the sharpened pixel values ​​corresponding to the pixel positions of all the above color pixels;

[0177] A second determining unit 703 is used to determine the smoothed sharpened pixel values ​​corresponding to all the pixel positions based on the sharpened pixel values ​​corresponding to the pixel positions of all the color pixels;

[0178] The processing unit 704 is configured to obtain an image to be encoded corresponding to the input image according to the smoothed and sharpened pixel values ​​corresponding to all the pixel positions.

[0179] According to one or more optional embodiments of the present disclosure, the first determining unit includes:

[0180] A first determination subunit is used to determine an original pixel value at a pixel position of each of the color pixels, and a Gaussian mean of all original pixel values ​​in a neighborhood of each of the color pixels;

[0181] The second determination subunit is used to determine the sharpened pixel value corresponding to the pixel position of each of the above-mentioned color pixels based on the original pixel value at the pixel position of each of the above-mentioned color pixels and the Gaussian mean of all original pixel values ​​in the neighborhood of each of the above-mentioned color pixels.

[0182] According to one or more optional embodiments of the present disclosure, the second determining subunit includes:

[0183] A first determination submodule is used to determine the difference between the original pixel value at the pixel position of each color pixel and the Gaussian mean of all original pixel values ​​in the neighborhood of each color pixel;

[0184] The second determination submodule is used to determine the sharpened pixel value corresponding to the pixel position of each of the color pixels according to the product of the difference and the predetermined scaling factor.

[0185] According to one or more optional embodiments of the present disclosure, the second determining unit includes:

[0186] A first generating subunit, configured to obtain a sharpened pixel plane of at least one of the color components based on the sharpened pixel values ​​corresponding to the pixel positions of all the color pixels;

[0187] The third determining subunit is used to determine the smoothed sharpened pixel values ​​corresponding to all pixel positions on the sharpened pixel plane according to all the sharpened pixel values ​​on the sharpened pixel plane of at least one of the color components.

[0188] According to one or more optional embodiments of the present disclosure, the third determining subunit includes:

[0189] The third determination submodule is used to determine the smoothed sharpened pixel values ​​corresponding to all pixel positions on the sharpened pixel plane according to the Gaussian mean of all the sharpened pixel values ​​on the sharpened pixel plane of at least one of the color components.

[0190] According to one or more optional embodiments of the present disclosure, the processing unit includes:

[0191] A calculation subunit, used for obtaining the pixel values ​​to be encoded corresponding to all the above pixel positions respectively according to the original pixel values ​​corresponding to all the above pixel positions respectively and the smoothed and sharpened pixel values ​​corresponding to all the above pixel positions respectively;

[0192] A second generating subunit is used to obtain a pixel plane to be encoded of at least one of the color components based on the pixel values ​​to be encoded corresponding to all the pixel positions;

[0193] The processing subunit is used to obtain the above-mentioned image to be encoded corresponding to the above-mentioned input image based on the pixel plane to be encoded of at least one of the above-mentioned color components.

[0194] According to one or more optional embodiments of the present disclosure, the above-mentioned computing subunit includes:

[0195] An acquisition submodule, used to acquire the bit depth of the input image and the valid value range corresponding to the bit depth;

[0196] A calculation submodule, used to determine the sum of the original pixel value and the smoothed and sharpened pixel value corresponding to each of the pixel positions;

[0197] The truncation processing submodule is used to truncate the sum value exceeding the valid value range to the valid value range if the sum value corresponding to any one of the above pixel positions exceeds the above valid value range, so as to obtain the pixel values ​​to be encoded corresponding to all the above pixel positions.

[0198] According to one or more optional embodiments of the present disclosure, the acquisition unit includes:

[0199] A first acquisition subunit is used to acquire the initial color space of the input image;

[0200] A second acquisition subunit is used for converting the initial color space to obtain a color space corresponding to the input image if the initial color space is a non-target color space;

[0201] The third acquisition subunit is configured to use the target color space as the color space corresponding to the input image if the initial color space is the target color space.

[0202] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0203] According to an embodiment of the present disclosure, the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute any one of the above methods.

[0204] According to an embodiment of the present disclosure, the present disclosure provides a computer program product, which includes: a computer program, the computer program is stored in a readable storage medium, at least one processor of an electronic device can read the computer program from the readable storage medium, and at least one processor executes the computer program so that the electronic device executes the solution provided by any of the above embodiments.

[0205] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0206] like Figure 8 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0207] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0208] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as image enhancement processing methods. For example, in some embodiments, the image enhancement processing method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the image enhancement processing method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the image enhancement processing method in any other appropriate manner (e.g., by means of firmware).

[0209] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0210] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0211] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0212] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0213] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0214] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0215] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0216] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. An image enhancement processing method, the method comprising: In response to the read input image, acquiring a color space corresponding to the input image and all color pixels of at least one color component in the color space; Determine the sharpened pixel values ​​corresponding to the pixel positions of all the color pixels based on the difference between the original pixel values ​​corresponding to the pixel positions of all the color pixels and the Gaussian mean of all the original pixel values ​​in the neighborhood of each of the color pixels, and the product of a predetermined scaling factor; Obtaining a sharpened pixel plane of at least one of the color components based on the sharpened pixel values ​​corresponding to the pixel positions of all the color pixels; Determine, according to the Gaussian mean of all the sharpened pixel values ​​on the sharpened pixel plane of at least one of the color components, the smoothed sharpened pixel values ​​corresponding to all the pixel positions on the sharpened pixel plane; The pixel value to be encoded corresponding to each pixel position is obtained according to the original pixel values ​​and the smoothed and sharpened pixel values ​​corresponding to all the pixel positions, and the pixel plane to be encoded of at least one color component is obtained based on the pixel values ​​to be encoded corresponding to all the pixel positions, so as to obtain an image to be encoded corresponding to the input image.

2. The method according to claim 1, wherein: Before determining the sharpened pixel values ​​corresponding to the pixel positions of all the color pixels, the method further includes: The original pixel value at the pixel position of each of the color pixels among all the color pixels and the Gaussian mean of all the original pixel values ​​in the neighborhood of each of the color pixels are determined.

3. The method according to claim 1, wherein: According to the original pixel values ​​corresponding to all the pixel positions and the smoothed and sharpened pixel values ​​corresponding to all the pixel positions, the pixel values ​​to be encoded corresponding to all the pixel positions are obtained, including: Obtaining the bit depth of the input image and the valid value range corresponding to the bit depth; Determine the sum of the original pixel value and the smoothed and sharpened pixel value corresponding to each of the pixel positions; If the sum value corresponding to any one of all the pixel positions exceeds the valid value range, the sum value exceeding the valid value range is truncated to within the valid value range to obtain the pixel values ​​to be encoded corresponding to all the pixel positions.

4. The method according to any one of claims 1 to 3, wherein: Obtaining the color space corresponding to the input image, including: Obtaining an initial color space of the input image; If the initial color space is a non-target color space, converting the initial color space to obtain a color space corresponding to the input image; If the initial color space is the target color space, the target color space is used as the color space corresponding to the input image.

5. An image enhancement processing device, the device comprising: an acquisition unit, configured to acquire, in response to a read input image, a color space corresponding to the input image and all color pixels of at least one color component in the color space; A first determining unit, configured to determine sharpened pixel values ​​corresponding to pixel positions of all the color pixels; A second determining unit, configured to determine the smoothed sharpened pixel values ​​corresponding to all the pixel positions respectively based on the sharpened pixel values ​​corresponding to the pixel positions of all the color pixels respectively; A processing unit, configured to obtain an image to be encoded corresponding to the input image according to the smoothed and sharpened pixel values ​​corresponding to all the pixel positions; The first determining unit includes: a second determining subunit, and the second determining subunit includes: A first determination submodule, used to determine the difference between the original pixel value at the pixel position of each color pixel and the Gaussian mean of all original pixel values ​​in the neighborhood of each color pixel; A second determination submodule, configured to determine a sharpened pixel value corresponding to a pixel position of each color pixel according to a product of the difference and a predetermined scaling factor; The second determining unit includes: A first generating subunit, configured to obtain a sharpened pixel plane of at least one of the color components based on the sharpened pixel values ​​corresponding to the pixel positions of all the color pixels; A third determining subunit, wherein the third determining subunit comprises: A third determination submodule, configured to determine the smoothed sharpened pixel values ​​corresponding to all pixel positions on the sharpened pixel plane according to the Gaussian mean of all the sharpened pixel values ​​on the sharpened pixel plane of at least one of the color components; The processing unit comprises: A calculation subunit, configured to obtain a pixel value to be encoded corresponding to each pixel position among all pixel positions according to the original pixel values ​​corresponding to all the pixel positions and the smoothed and sharpened pixel values; A second generating subunit, configured to obtain a pixel plane to be encoded of at least one of the color components based on the pixel values ​​to be encoded corresponding to all the pixel positions; The processing subunit is used to obtain an image to be encoded corresponding to the input image based on a pixel plane to be encoded of at least one of the color components.

6. The device according to claim 5, wherein: The first determining unit further includes: The first determination subunit is used to determine the original pixel value at the pixel position of each of the color pixels among all the color pixels, and the Gaussian mean of all the original pixel values ​​in the neighborhood of each of the color pixels.

7. The device according to claim 5, wherein: The computing subunit comprises: An acquisition submodule, used to acquire the bit depth of the input image and a valid value range corresponding to the bit depth; A calculation submodule, used to determine the sum of the original pixel value and the smoothed and sharpened pixel value corresponding to each of the pixel positions; The truncation processing submodule is used to truncate the sum value exceeding the valid value range to within the valid value range if the sum value corresponding to any pixel position among all the pixel positions exceeds the valid value range, so as to obtain the pixel values ​​to be encoded corresponding to all the pixel positions respectively.

8. The device according to any one of claims 5 to 7, wherein: The acquisition unit comprises: A first acquisition subunit, used to acquire an initial color space of the input image; A second acquisition subunit is used for converting the initial color space to obtain a color space corresponding to the input image if the initial color space is a non-target color space; The third acquisition subunit is configured to use the target color space as the color space corresponding to the input image if the initial color space is a target color space.

9. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 4.

10. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-4.

11. A computer program product, comprising a computer program, which implements the steps of the method according to any one of claims 1 to 4 when executed by a processor.

Images