Image enhancement method, electronic device and computer readable storage medium

Abstract

The present application discloses an image enhancement method, an electronic device, and a computer-readable storage medium, the method comprising: extracting an initial low-frequency image from an acquired initial image; determining a first texture region and a second texture region in the initial low-frequency image according to attribute information of the initial low-frequency image, wherein the object edge size in the first texture region is larger than the object edge size in the second texture region; performing denoising on the first texture region in the initial image according to a first denoising weight corresponding to the first texture region, and performing denoising on the second texture region in the initial image according to a second denoising weight corresponding to the second texture region, to obtain a target low-frequency image; determining a target image based on the initial image and the target low-frequency image. The above scheme uses different denoising weights for denoising different texture regions, which can meet the needs of different application scenarios, effectively remove noise in the initial image, and improve the image enhancement effect.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image enhancement method, electronic equipment and a computer-readable storage medium. Background Art

[0002] With the development of camera technology, taking images has gradually become the best choice to record a moment. However, due to the process and environment of the image sensor, the image may have excessive noise, which affects the visual experience. To solve this problem, image enhancement can be used to reduce image noise and increase image clarity.

[0003] Traditional image enhancement methods mainly use filters to globally filter the original image to remove small textures in the image to obtain an enhanced image. This method easily leads to incomplete removal of texture information in low-frequency images and poor retention of edge information, resulting in problems such as halos and excessive noise in the enhanced image. Summary of the invention

[0004] The main technical problem solved by the present application is to provide an image enhancement method, an electronic device and a computer-readable storage medium, which can improve the effect of image enhancement.

[0005] In order to solve the above technical problems, a technical solution adopted in the present application is: providing an image enhancement method, the method comprising: extracting an initial low-frequency image from an acquired initial image, wherein the pixels of the initial image correspond one-to-one to the pixels of the initial low-frequency image; determining a first texture area and a second texture area in the initial low-frequency image according to attribute information of the initial low-frequency image, wherein the object edge size in the first texture area is larger than the object edge size in the second texture area; performing denoising on the first texture area in the initial image according to a first denoising weight corresponding to the first texture area, and performing denoising on the second texture area in the initial image according to a second denoising weight corresponding to the second texture area, to obtain a target low-frequency image; and determining a target image based on the initial image and the target low-frequency image.

[0006] To solve the above technical problem, another technical solution adopted in the present application is: to provide an electronic device, including a memory and a processor, wherein the memory stores program instructions, and the processor retrieves the program instructions from the memory to execute the above image enhancement method.

[0007] In order to solve the above technical problem, another technical solution adopted by the present application is: providing a computer-readable storage medium including program data stored therein, wherein the program data is used to implement the above image enhancement method when executed by a processor.

[0008] The present application proposes an image enhancement method, which includes: extracting an initial low-frequency image from an acquired initial image; determining a first texture region and a second texture region in the initial low-frequency image according to attribute information of the initial low-frequency image, wherein the object edge size in the first texture region is larger than the object edge size in the second texture region; performing denoising on the first texture region in the initial image according to a first denoising weight corresponding to the first texture region, and performing denoising on the second texture region in the initial image according to a second denoising weight corresponding to the second texture region, to obtain a target low-frequency image; determining a target image based on the initial image and the target low-frequency image. The above scheme first extracts texture regions with different object edge sizes from the initial image, thereby improving the fidelity of the edges of each object, and then uses different denoising weights to perform denoising on different texture regions, which can meet the needs of different application scenarios, effectively remove noise in the initial image, and improve the image enhancement effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work, among which:

[0010] Figure 1 is a flowchart of an exemplary embodiment of an image enhancement method shown in the present application;

[0011] Figure 2 yes Figure 1 A schematic flow chart of an exemplary embodiment of step S110 in the image enhancement method is shown;

[0012] Figure 3 is a schematic diagram of a broken line between a pixel value and a noise reduction weight shown in the present application;

[0013] Figure 4 is a schematic diagram of a stretching curve shown in the present application;

[0014] Figure 5 It is a specific flow chart of an exemplary embodiment of the image enhancement method shown in the present application;

[0015] Figure 6 is a schematic structural diagram of an exemplary embodiment of an image enhancement device shown in the present application;

[0016] Figure 7 It is a structural schematic diagram of an embodiment of an electronic device provided by the present application;

[0017] Figure 8It is a structural schematic diagram of an embodiment of a computer-readable storage medium provided by the present application. DETAILED DESCRIPTION

[0018] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It will be appreciated that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for ease of description, only some but not all structures related to the present application are shown in the drawings. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the art without making creative work are within the scope of protection of the present application.

[0019] First of all, it should be noted that in order to enhance the quality of images, the current main acquisition methods can be roughly divided into statistical-based image enhancement algorithms, layered-based image enhancement methods and learning-based image enhancement algorithms.

[0020] The statistical image enhancement algorithms mainly include global histogram enhancement method, local histogram enhancement method, CLAHE (Contrast Limited Adaptive Histogram Equalization) and other classic algorithms. The layered image enhancement methods mainly include Retinex (retina cortex) method, which uses filtering or equation optimization to extract high frequencies for enhancement. The learning-based image enhancement algorithms mainly include sparse coding method and deep learning method.

[0021] However, the above method has the following problems: first, the texture information in the low-frequency signal is not removed thoroughly enough, resulting in insufficient texture enhancement; second, the edge information in the low-frequency signal is not well preserved, resulting in a halo problem in the enhanced image. The halo problem will make the face of a person in a bright background appear dark; third, the low-frequency signal is deformed, resulting in an oil painting effect in the enhanced image.

[0022] Based on this, the present application proposes an image enhancement method, an electronic device and a computer-readable storage medium, which use different noise reduction weights to perform noise reduction on different texture areas in the initial image, and can effectively remove small textures and retain large textures. Figure 1 , Figure 1 It is a flowchart of an exemplary embodiment of the image enhancement method shown in the present application.

[0023] The execution subject of the image enhancement method may be a terminal device or a server or other processing device, wherein the terminal device may be a user equipment (UE), a computer, a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. The execution subject of the image enhancement method may also be an image enhancement device. In some possible implementations, the image enhancement method may be implemented by a processor calling a computer-readable instruction stored in a memory.

[0024] Specifically, the image enhancement method of this embodiment includes the following steps:

[0025] S110: extracting an initial low-frequency image from the acquired initial image, wherein the pixels of the initial image correspond to the pixels of the initial low-frequency image one by one.

[0026] The initial image refers to an image that needs to be enhanced. For example, the original image acquired by an image acquisition device or the like can be used as the initial image, or the image that has been pre-processed can be used as the initial image. The pre-processing includes image cropping, rotation, and the like.

[0027] The initial low-frequency image is an image obtained by extracting a low-frequency signal from the initial image. In some embodiments, the initial image can be filtered to obtain the initial low-frequency image. The filtering process is actually to filter each pixel in the initial image to obtain the filtered pixel, and the filtered pixel constitutes the initial low-frequency image. Therefore, the pixel of the initial image corresponds to the pixel of the initial low-frequency image one by one. In order to better distinguish the texture area with a small object edge size and the texture area with a large object edge size in the initial low-frequency image, a filtering template can be determined according to the image information of the initial image; then, the initial image is filtered according to the filtering template to obtain the initial low-frequency image. In other embodiments, the initial image can also be converted from the spatial domain to the frequency domain by Fourier transform, and the low-frequency information is extracted in the frequency domain to obtain the initial low-frequency image.

[0028] S120: determining a first texture region and a second texture region in the initial low-frequency image according to attribute information of the initial low-frequency image, wherein an object edge size in the first texture region is greater than an object edge size in the second texture region.

[0029] The attribute information may be various parameters and data describing the initial low-frequency image, including, for example, image size, contrast, brightness, pixel value, etc. of the initial low-frequency image.

[0030] The initial low-frequency image includes an area where the pixel value changes slowly and an area where the pixel value changes greatly. The first texture area and the second texture area belong to areas where the pixel value changes differently in the initial low-frequency image. Exemplarily, the first texture area and the second texture area in the initial low-frequency image can be determined according to the attribute information of the initial low-frequency image. Among them, the object edge size of the first texture area and the second texture area are different. In some application scenarios, the object edge size in the first texture area is set to be larger than the object edge size in the second texture area. Of course, in other application scenarios, the object edge size in the first texture area can also be set to be smaller than the object edge size in the second texture area. In some embodiments, the object edge size can be determined by the pixel value of each pixel point in the initial low-frequency image, and the first texture area and the second texture area are determined according to the pixel value of each pixel point in the initial low-frequency image. In other embodiments, each object in the initial image and the edge size corresponding to each object can also be identified by a pre-trained network model, and the first texture area and the second texture area in the initial low-frequency image are determined according to the object edge size of each object.

[0031] The object edge size may be the size of the contour of each object in the initial image. Exemplarily, the object edge size may be determined by the total area enclosed by the corresponding pixel points of the edges of each object in the initial image. Among them, an object with a larger object edge size may be composed of objects with a smaller object edge size. As an example, if the initial image includes a tree and leaves on the tree, the edge size of the leaves is smaller, and its texture may belong to the second texture area, and the edge size of the tree is larger, and its edge contour may belong to the first texture area. This is just an example, and it needs to be determined based on the proportion of the object edge size of each object in the initial image in the initial image. If the edge size of the leaves occupies a larger proportion, the pixel points to which the edge contour of the leaves belongs may also be determined as the pixel points in the first texture area.

[0032] S130: Performing denoising on the first texture region in the initial image according to a first denoising weight corresponding to the first texture region, and performing denoising on the second texture region in the initial image according to a second denoising weight corresponding to the second texture region, to obtain a target low-frequency image.

[0033] The pixels of the initial image and the initial low-frequency image correspond one to one, so the texture area of ​​the initial image and the texture area of ​​the initial low-frequency image also correspond to each other. After determining the first texture area and the second texture area of ​​the initial low-frequency image, the first texture area and the second texture area in the initial image can be determined by the correspondence between the pixels.

[0034] The denoising weight is used to measure the degree of denoising. Generally, the larger the denoising weight, the higher the denoising degree, and vice versa. The first denoising weight is used to perform denoising on the first texture area. The second denoising weight is used to perform denoising on the second texture area. The first denoising weight and the second denoising weight can be set to denoising weights of different sizes. By performing denoising on different texture areas with denoising weights of different sizes, different effects can be achieved. As an example, if the first denoising weight is greater than the second denoising weight, the denoising degree of the first texture area is greater than the denoising degree of the second texture area, which can make the first texture area smoother and retain the second texture area, thereby smoothing large edges and highlighting small details. This method can be used when it is necessary to highlight details. As another example, if the first denoising weight is less than the second denoising weight, the denoising degree of the first texture area is less than the denoising degree of the second texture area, which can remove the second texture area and retain the first texture area, thereby removing small textures and retaining large edges. This method can be used when it is necessary to blur details and highlight large contours. In other embodiments, the second texture area in the initial image also includes a first sub-texture area and a second sub-texture area, and the corresponding second denoising weight may also include a first sub-denoising weight and a second sub-denoising weight, and the object edge size of the first sub-texture area is larger than the object edge size of the second sub-texture area.

[0035] The target low-frequency image is a low-frequency image obtained after performing noise reduction processing on each texture area in the initial image. Exemplarily, the initial image can be filtered to obtain the target low-frequency image.

[0036] The image enhancement device first extracts an initial low-frequency image from an initial image, determines a first texture area and a second texture area in the initial low-frequency image, and distinguishes texture areas with different object edge sizes; determines corresponding denoising weights according to the determined texture areas as denoising references for the initial image, sets the denoising weight of the first texture area to the first denoising weight, and sets the denoising weight of the second texture area to the second denoising weight; thereby, different denoising weights are used to perform denoising on different texture areas, thereby meeting the needs of different application scenarios.

[0037] S140: Determine a target image based on the initial image and the target low-frequency image.

[0038] The target image is an image after image enhancement. In some embodiments, the target high-frequency image can be determined based on the initial image and the target low-frequency image, and then the target high-frequency image and the target low-frequency image are fused to obtain the target image. In other embodiments, the initial image and the target low-frequency image can be fused to obtain the target image. In addition, the image can be sharpened, contrasted, and brightness adjusted to obtain the target image.

[0039] It can be seen that the image enhancement method of the embodiment of the present application extracts an initial low-frequency image from the acquired initial image; determines the first texture area and the second texture area in the initial low-frequency image according to the attribute information of the initial low-frequency image, and the object edge size in the first texture area is larger than the object edge size in the second texture area; performs noise reduction processing on the first texture area in the initial image according to the first noise reduction weight corresponding to the first texture area, and performs noise reduction processing on the second texture area in the initial image according to the second noise reduction weight corresponding to the second texture area, to obtain a target low-frequency image; determines the target image based on the initial image and the target low-frequency image. The above scheme uses different noise reduction weights for different texture areas to perform noise reduction processing, which can meet the needs of different application scenarios, effectively remove noise in the initial image, and improve the image enhancement effect.

[0040] Based on the above embodiments, the embodiments of the present application adopt Figure 2 The flowchart details how to extract the initial low-frequency image from the initial image. Figure 2 , Figure 2 yes Figure 1 The flowchart of an exemplary embodiment of step S110 in the image enhancement method is shown. Specifically, the process of extracting the initial low-frequency image from the acquired initial image in step S110 specifically includes the following steps:

[0041] S210: determining a target filter template size from a preset correspondence table according to a resolution of the initial image, wherein the preset correspondence table includes a correspondence between a preset filter template size and a preset resolution.

[0042] Resolution refers to the number of pixels in the width and height of the original image. For example, a 3000x2000 resolution image means it is 3000 pixels wide and 2000 pixels high.

[0043] The preset correspondence table includes the correspondence between the preset filter template size and the preset resolution. The larger the filter template size, the more image information in the initial image is listed as the second texture area. The number of pixels in images of different resolutions is different. The higher the resolution, the larger the filter template size is required to better distinguish the first texture area and the second texture area in the image. Exemplarily, the preset correspondence table can be the filter template size used by images of various resolutions obtained by R&D personnel in advance, which can make the filtered image better distinguish the first texture area and the second texture area, that is, distinguish the large edges and small scales, small gradients and weak textures in the image. As an example, for an initial image with a resolution of 4K (3840 x 2160), a filter template with a filter template size of 11 x 11 can be used to filter the initial image, which can better distinguish large edges from small-scale objects, edges with small gradients, and weak textures.

[0044] The target filter template size is the filter template size corresponding to the resolution of the initial image. The filter template of the target filter template size has a characteristic attribute for extracting the first texture area, and the object edge size of the first texture area can be set to be larger than the preset edge size.

[0045] The image enhancement device first obtains the resolution of the initial image, then traverses the preset correspondence table according to the resolution of the initial image, determines the preset resolution that matches the resolution of the initial image, and determines the preset filter template size corresponding to the preset resolution as the filter template size of the initial image.

[0046] In other embodiments, when determining the target filter template size, the object edge size of each object in the initial image may also be considered. Exemplarily, the object edge size of each object in the initial image is detected; the target filter template size is determined according to the object edge size of each object, so that the target filter template size can extract the object edge whose object edge size is larger than the preset edge size, and blur the object whose object edge size is smaller than or equal to the preset edge size.

[0047] S220: filtering the initial image with a filter template of a target filter template size to obtain an initial low-frequency image of the initial image.

[0048] After obtaining the target filter template size of the initial image, the image enhancement device performs filtering processing on the initial image with the filter template of the size to obtain an initial low-frequency image of the initial image. Thus, the first texture area and the second texture area in the initial image can be distinguished more effectively by simply selecting the template size. Exemplarily, a larger target filter template size can be selected to distinguish large edges and small edges in the initial image from the perspective of scale.

[0049] As an example, after the image enhancement device determines the target filter template size, it can use 4 filter templates of the target filter template size. The values ​​in the filter template can be set to 1 and -1, or other values ​​can be used. The values ​​are not limited here. The 4 filter templates are used to detect the vertical edge, horizontal edge, 45-degree edge, and 135-degree edge in the initial image. The initial image is filtered using these 4 filter templates to obtain 4 low-frequency images, which are named , , , , then the final initial low-frequency image can be obtained using the following formula:

[0050]

[0051] in, represents the initial low-frequency image, Indicates taking the absolute value, Indicates taking the maximum value.

[0052] As another example, after the image enhancement device determines the target filter template size, it can use a filter template of the target filter template size to filter the initial image to obtain an initial low-frequency image. The filter template can be any one of the vertical edge, horizontal edge, 45-degree edge, and 135-degree edge of the initial image. For example, if the target filter template size is 11×11, the filter template used to detect the vertical edge is as follows:

[0053]

[0054] By filtering the initial image using this filtering template, pixel changes in the initial image in the vertical direction can be detected.

[0055] In other embodiments, the image enhancement device may also filter the initial image using algorithms such as sobel (Sobel Operator), Laplacian (Laplacian of Gaussian (LoG)), and Prewitt (Prewitt Operator) to obtain an initial low-frequency image. The initial low-frequency image extracted by these algorithms will extract the edges of small-scale objects, edges with small gradients, and weak textures. Compared with the initial low-frequency image of the initial image extracted by the target filter template size, which only extracts textures with object edge sizes greater than the preset edge size, these algorithms extract all texture information, which may result in incomplete removal of small textures (second texture areas). Therefore, the preferred solution of this embodiment is to determine the target filter template size according to the resolution, and the determined target filter template size can extract the first texture area in the initial image, the object edge size of the first texture area is greater than the preset edge size, and the first texture area is distinguished from the second texture area.

[0056] After extracting the initial low-frequency image from the initial image, the first texture region and the second texture region in the initial low-frequency image are determined according to the attribute information of the initial low-frequency image. The attribute information includes the pixel value of each pixel in the initial low-frequency image. In response to the pixel value of the pixel in the initial low-frequency image being greater than the first preset pixel value, the corresponding pixel is determined as the pixel in the first texture region; in response to the pixel value of the pixel in the initial low-frequency image being less than or equal to the first preset pixel value, the corresponding pixel is determined as the pixel in the second texture region.

[0057] In the initial low-frequency image, the pixel value of each pixel point can represent the pixel value difference between the corresponding pixel point and the adjacent pixel point in the initial image. The larger the pixel value of the pixel point in the initial low-frequency image, the larger the pixel value change gradient in the initial image, and the more likely it is to belong to the first texture area; and the smaller the pixel value, the smaller the pixel value change gradient in the initial image, and the more likely it is to belong to the second texture area. Therefore, the first texture area and the second texture area in the initial image can be determined according to the pixel value of each pixel point in the initial low-frequency image. The pixel point with a pixel value greater than the first preset pixel value is determined as the pixel point of the first texture area, and the pixel point with a pixel value less than or equal to the first preset pixel value is determined as the pixel point of the second texture area.

[0058] In some other embodiments, the second texture area can also be divided into a first sub-texture area and a second sub-texture area, and the pixel points in the initial low-frequency image whose pixel values ​​are less than or equal to the second preset pixel value are determined as the pixel points in the second sub-texture area, and the pixel points in the initial low-frequency image whose pixel values ​​are greater than the second preset pixel value and less than or equal to the first preset pixel value are determined as the pixel points in the first sub-texture area. The first preset pixel value can be set to 0.8, and the second preset pixel value can be set to 0.1.

[0059] After determining different texture areas in the initial image, corresponding denoising weights are determined according to the different texture areas, and then the corresponding texture areas are subjected to denoising with the denoising weights. Specifically, a first preset weight and a second preset weight are obtained, wherein the first preset weight is less than the second preset weight; the first preset weight is set as the first denoising weight of the first texture area; the second preset weight is set as the second denoising weight of the second texture area; the first texture area in the initial image is subjected to denoising with the first denoising weight, and the second texture area in the initial image is subjected to denoising with the second denoising weight, to obtain a denoised initial image; the denoised initial image is used as the target low-frequency image.

[0060] The first preset weight is set to the first noise reduction weight of the first texture area, and the second preset weight is set to the second noise reduction weight of the second texture area. In order to retain the first texture area and remove the second texture area, the first preset weight can be set to be greater than the second preset weight. In other embodiments, in order to retain the second texture area and remove the first texture area, the first preset weight can also be set to be less than the second preset weight.

[0061] In some other embodiments, the second texture region may be further divided into a first sub-texture region and a second sub-texture region. The denoising weights of the first texture region, the first sub-texture region and the second sub-texture region may refer to Figure 3 ,in, Figure 3 The horizontal axis is the pixel value, the vertical axis is the denoising weight, Gthr1 represents the second preset pixel value, which can be set to 0.1, and Gthr2 represents the first preset pixel value, which can be set to 0.8; when the pixel value of a pixel point in the initial low-frequency image is less than or equal to the second preset pixel value, it belongs to the second sub-texture area in the initial image, and its corresponding denoising weight can be set to the second sub-denoising weight, that is, Figure 3 Gra2 on the middle vertical axis; when the pixel value of a pixel point in the initial low-frequency image is greater than the second preset pixel value and less than or equal to the second preset pixel value, it belongs to the first sub-texture area in the initial image, and the denoising weight of each pixel point can be determined according to the pixel value of each pixel point in the first sub-texture area, that is, Figure 3The oblique line connecting the two points (Gthr1, Gra1) and (Gthr2, Gra2) in the initial low-frequency image; when the pixel value of the pixel point in the initial low-frequency image is greater than the first preset pixel value, it belongs to the first texture area in the initial image, and its corresponding denoising weight can be set to the first denoising weight. Exemplarily, it can be expressed as:

[0062]

[0063] in, represents the denoising weight, represents the second sub-denoising weight, represents the first sub-denoising weight, represents the first denoising weight, Represents the pixel value of the pixel in the initial low-frequency image, represents a second preset pixel value, The formula indicates that the denoising weight is determined according to the pixel value of each pixel point in different initial low-frequency images.

[0064] In other embodiments, Gaussian filtering is performed on the initial image to obtain a Gaussian image; a first denoising weight of the first texture area is determined based on the pixel value of each pixel in the first texture area and the pixel value of the corresponding pixel in the Gaussian image; a second denoising weight of the second texture area is determined based on the pixel value of each pixel in the second texture area and the pixel value of the corresponding pixel in the Gaussian image; the first texture area is denoised with the first denoising weight, and the second texture area is denoised with the second denoising weight to obtain a target low-frequency image, wherein the first denoising weight is less than the second denoising weight.

[0065] Based on the above embodiment, this embodiment can also determine the noise reduction weight of each pixel in the initial image in combination with the Gaussian image. Specifically, the image enhancement device first performs Gaussian filtering on the initial image to obtain a Gaussian image. Exemplarily, the filtering template of the Gaussian filter can be:

[0066]

[0067] In other embodiments, the Gaussian filtering process may also use a two-dimensional Gaussian distribution formula to generate Gaussian weights, which satisfies the following formula:

[0068]

[0069] in, represents Gaussian weight, represents the position coordinate of the pixel relative to the center pixel, a represents the deviation distance relative to the horizontal axis, and b represents the deviation distance relative to the vertical axis. represents pi, Represents standard deviation.

[0070] The image enhancement device determines the denoising weight of the initial image by the pixel value of each pixel in the initial low-frequency image, and the specific method can refer to the above method. In this embodiment, the first preset weight is set as the first initial denoising weight of the first texture area, and the second preset weight is set as the second initial denoising weight of the second texture area. After determining the initial denoising weight of each pixel in the initial image, the denoising processing algorithm is guided by the initial denoising weight of each pixel in the initial image to perform denoising on the initial image. Therefore, the initial low-frequency image can also be called a guide map in this embodiment, and the guide map coefficient (initial denoising weight) of each pixel is determined based on the guide map. The denoising processing algorithm can be any denoising algorithm based on weight fusion, such as bilateral filtering, non-local mean, wavelet, BM3D (Block-Matching and 3D filtering, image denoising algorithm based on block matching and three-dimensional filtering), etc.

[0071] Then, the first denoising weight of the first texture area and the second denoising weight of the second texture area are determined in combination with the pixel values ​​of the corresponding pixels in the Gaussian image. Taking the denoising process of the initial image by the bilateral filtering algorithm as an example, the image enhancement device obtains the pixel difference between each pixel and the neighboring pixel in the Gaussian image; calculates the ratio between the square of the pixel difference and the initial weight corresponding to the pixel to obtain the ratio corresponding to each pixel in the first texture area; and determines the exponential function of the inverse of the ratio as the first denoising weight of the first texture area. The calculation method of the second denoising weight is the same as the calculation method of the first denoising weight.

[0072] Among them, the pixel difference between each pixel in the Gaussian image and the neighboring pixel, the neighboring pixel refers to a rectangular area centered on the pixel, such as 3×3, 5×5, 9×9, etc., and a 9×9 rectangular area can be selected. The larger the area, the better the low-frequency effect. Exemplarily, the calculation of the denoising weight of each pixel in the initial image satisfies the following formula:

[0073]

[0074] in, represents the denoising weight, represents the pixel value of pixel p in the Gaussian image, Represents the pixel value of the neighborhood pixel q of the pixel p in the Gaussian image, represents the initial denoising weight of the neighborhood pixel point q, if the neighborhood pixel point p is located in the first texture area, it is the first initial denoising weight, if the neighborhood pixel point p is located in the second texture area, it is the second initial denoising weight. In some other embodiments, the second texture area also includes a first sub-texture area and a second sub-texture area, if the neighborhood pixel point q is located in the first sub-texture area, it is the first sub-initial denoising weight, if the neighborhood pixel point q is located in the second sub-texture area, it is the second sub-initial denoising weight.

[0075] Then, the initial image is subjected to denoising using the denoising weights of each pixel in the initial image, and the denoising formula satisfies the following formula:

[0076]

[0077] in, represents the t+1th filtering result of pixel p in the initial image, represents the neighborhood pixel point centered on pixel point p, represents the pixel value of pixel p in the Gaussian image, Represents the pixel value of the neighborhood pixel q of the pixel p in the Gaussian image, represents the initial denoising weight of the neighborhood pixel q, Represents the filtering result of pixel point q for the tth time. Compared with the rolling guided filtering formula, the weight of the rolling guided filtering formula is based on the last bilateral filtering result, which will retain the last error, while the weight of the formula of this embodiment is based on the Gaussian image, which will not retain the filtering error, and can effectively prevent the signal distortion, water ripples and oil painting feeling of the target low-frequency image; further, the weight of the formula of this embodiment is also guided by the initial denoising weight, which is determined by the pixel value of each pixel in the initial low-frequency image, and the initial low-frequency image is filtered according to the target filtering template size determined by the resolution of the initial image. Therefore, the initial denoising weight is used for guidance, and the first texture area and the second texture area are filtered with different intensities to effectively remove the small texture in the initial image and retain the large edge.

[0078] Furthermore, when performing denoising on the initial image, an iterative initialization method can be used, and the specific process is: performing a first denoising on the first texture area in the initial image according to the first denoising weight corresponding to the first texture area, and performing a first denoising on the second texture area in the initial image according to the second denoising weight corresponding to the second texture area, to obtain the initial image after the first denoising; in response to the initial image after the first denoising does not meet the preset requirements, performing a second denoising on the first texture area in the initial image after the first denoising according to the first denoising weight corresponding to the first texture area, and performing a second denoising on the second texture area in the initial image after the first denoising according to the second denoising weight corresponding to the second texture area, to obtain the initial image after the second denoising; in response to the initial image after the second denoising meets the preset requirements, the initial image after the second denoising is determined as the target low-frequency image. In this way, the target low-frequency image that meets the preset requirements can be obtained by multiple iterative denoising methods.

[0079] Finally, after obtaining the target low-frequency image, the target image is determined based on the initial image and the target low-frequency image. In some embodiments, the target high-frequency image may be determined first, and then the target image may be determined by the target low-frequency image and the target high-frequency image. Specifically, the initial high-frequency image may be determined based on the initial image and the target low-frequency image; the initial high-frequency image may be enhanced to obtain the target high-frequency image; and the target high-frequency image may be fused with the target low-frequency image to obtain the target image.

[0080] The initial high-frequency image may be determined by the initial image and the target low-frequency image. For example, the initial high-frequency image may be determined by the pixel difference between each pixel point of the initial image and the target low-frequency image. The mapping relationship between the low-frequency image and the high-frequency image may also be learned from a large amount of training data by a machine learning model or a deep learning model, thereby outputting the initial high-frequency image of the initial low-frequency image. For example, the formula for determining the initial high-frequency image by pixel interpolation is as follows:

[0081]

[0082] in, represents the initial high-frequency image, represents the initial image, Represents the target low-frequency image.

[0083] Then, the initial high-frequency image is enhanced. In some embodiments, the enhanced pixel value corresponding to the pixel value of each pixel point in the initial high-frequency image can be determined from a preset mapping table, and the preset mapping table includes a correspondence between a preset pixel value and a preset enhanced pixel value. Generally, in the correspondence, the preset enhanced pixel value is greater than the preset pixel value. The preset mapping table can be determined by a table stretching curve, such as Figure 4 As shown, the horizontal axis on the coordinate axis represents the preset pixel value, and the vertical axis represents the preset enhanced pixel value. The preset mapping table is determined according to the coordinates of each point in the table stretching curve. Among them, the table stretching curve can be adjusted as needed. For the initial high-frequency image, the pixel value of each pixel point in the initial high-frequency image is matched with the preset pixel value in the preset mapping table to obtain the matched preset pixel value; the preset enhanced pixel value corresponding to the matched preset pixel value is determined as the enhanced pixel value of the pixel point, thereby outputting the enhanced pixel values ​​of all pixel points in the initial high-frequency image, and combining them to obtain the target high-frequency image.

[0084] Among them, the adjustment formula of the table stretching curve is as follows:

[0085]

[0086] in, represents the enhanced pixel value of the target high-frequency image, represents the pixel value of the initial high-frequency image, Indicates the number of bits of the original image, such as 8bit, 10bit, 12bit, 14bit or 16bit; Used to control the enhancement strength. The smaller the value, the greater the enhancement strength.

[0087] In some other embodiments, the initial high-frequency image can be enhanced by combining a step function, and the combination method can be to add or multiply the value output by the step function and the value output by the enhancement formula to obtain a target high-frequency function. The step function is a piecewise constant function that represents the change in pixel values ​​in different intervals. Exemplarily, the formula of the step function satisfies the following formula:

[0088]

[0089] in, Represents the pixel value. When the pixel value is greater than 0, the step value is 1. When the pixel value is equal to 0, the step value is 0. When the pixel value is less than 0, the step value is -1.

[0090] The image enhancement device determines the enhanced pixel value of each pixel in the initial high-frequency image through a table stretching curve, determines the step value of each pixel in the initial high-frequency image through a step function, multiplies the enhanced pixel value of each pixel in the initial high-frequency image and the step value of the corresponding pixel to obtain the pixel value of each pixel in the target high-frequency image, thereby obtaining the target high-frequency image.

[0091] After obtaining the target high-frequency image, the target high-frequency image and the target low-frequency image are fused, and the fusion process can be performed by adding or multiplying pixel values. For example, the fusion formula can be as follows:

[0092]

[0093] in, represents the target image, Indicates the table stretch curve, represents the initial high-frequency image, represents a step function, Represents the target low-frequency image.

[0094] In other embodiments, the target low-frequency image is obtained by multiple iterations of the initial image, and each iteration result is obtained; the corresponding iterative initial high-frequency image is determined based on each iteration result; each iterative initial high-frequency image is enhanced to obtain the corresponding iterative target high-frequency image; each iterative target high-frequency image is fused with the target low-frequency image to obtain the target image. As an example, if the iteration is performed three times, the initial image after the first denoising process, the initial image after the second denoising process, and the initial image after the third denoising process (target low-frequency image) are obtained; the first initial high-frequency image is obtained based on the initial image and the initial image after the first denoising process, the second initial high-frequency image is obtained based on the initial image after the first denoising process and the initial image after the second denoising process, and the third initial high-frequency image is obtained based on the initial image after the second denoising process and the initial image after the third denoising process; the first initial high-frequency image, the second initial high-frequency image, and the third initial high-frequency image are enhanced respectively to obtain the first target high-frequency image, the second target high-frequency image, and the third target high-frequency image; the first target high-frequency image, the second target high-frequency image, the third target high-frequency image, and the target low-frequency image are fused to obtain the target image.

[0095] Exemplarily, the target image calculation formula of this embodiment is as follows:

[0096]

[0097] in, represents the target image, , and There are three different table stretching curves, which are used to control the enhancement intensity of different frequency bands; represents the initial image, represents the initial image after the first denoising process, represents the initial image after the second denoising process, Represents the initial image after the third denoising process, that is, the target low-frequency image.

[0098] It can be seen that the image enhancement method of the embodiment of the present application determines the initial high-frequency image based on the initial image and the target low-frequency image; performs enhancement processing on the initial high-frequency image to obtain the target high-frequency image; and performs fusion processing on the target high-frequency image and the target low-frequency image to obtain the target image. The target image is obtained by fusing the high-frequency image and the low-frequency image, which can enhance details, reduce noise, and improve resolution while retaining the overall structure of the image.

[0099] In order to elaborate on the image enhancement method of the present application, Figure 5 The flowchart shown further illustrates it, and the details are as follows:

[0100] First, an initial image is acquired, a target filter template size is determined according to the resolution of the initial image, and the initial image is filtered using the filter template of the target filter template size to obtain an initial low-frequency image (guiding map) of the initial image; then, an initial denoising weight (guiding map coefficient) of the corresponding pixel is determined according to the pixel value of each pixel in the initial low-frequency image. Specifically, if the pixel value of the pixel in the initial low-frequency image is less than or equal to the second preset pixel value, the denoising weight of the corresponding pixel is determined as the second sub-initial denoising weight; if the pixel value is greater than the second preset pixel value and less than or equal to the first preset pixel value, the denoising weight of the corresponding pixel is determined as the first sub-initial denoising weight; if the pixel value is greater than the first preset pixel value, the denoising weight of the corresponding pixel is determined as the first initial denoising weight, wherein the first initial denoising weight is less than the first sub-initial denoising weight, and the first sub-initial denoising weight is less than the second sub-initial denoising weight.

[0101] Then, the initial image is subjected to Gaussian filtering to obtain a Gaussian image. The denoising weight of each pixel is determined based on the pixel value of each pixel in the Gaussian image and the initial denoising weight of the corresponding pixel. The corresponding pixel is iteratively filtered using the denoising weight of each pixel to obtain a target low-frequency image that meets the preset requirements.

[0102] Finally, the initial high-frequency image is determined by the initial image and the target low-frequency image; the initial high-frequency image is enhanced to obtain the target high-frequency image; the target high-frequency image and the target low-frequency image are fused to obtain the target image. In this way, image enhancement can be completed with a simple filter, which is very easy to chip.

[0103] See also Figure 6 , Figure 6 6 is a schematic diagram of an exemplary embodiment of an image enhancement device shown in the present application. The image enhancement device 600 includes an extraction module 610, a region determination module 620, a noise reduction module 630 and an image determination module 640. The extraction module 610 is used to extract an initial low-frequency image from the acquired initial image, and the pixels of the initial image correspond to the pixels of the initial low-frequency image one by one; the region determination module 620 is used to determine the first texture region and the second texture region in the initial low-frequency image according to the attribute information of the initial low-frequency image, and the object edge size in the first texture region is larger than the object edge size in the second texture region; the noise reduction module 630 is used to perform noise reduction processing on the first texture region in the initial image according to the first noise reduction weight corresponding to the first texture region, and perform noise reduction processing on the second texture region in the initial image according to the second noise reduction weight corresponding to the second texture region, so as to obtain a target low-frequency image; the image determination module 640 is used to determine the target image based on the initial image and the target low-frequency image.

[0104] In the above scheme, the image enhancement device extracts an initial low-frequency image from the acquired initial image; determines the first texture area and the second texture area in the initial low-frequency image according to the attribute information of the initial low-frequency image, and the object edge size in the first texture area is larger than the object edge size in the second texture area; performs noise reduction processing on the first texture area in the initial image according to the first noise reduction weight corresponding to the first texture area, and performs noise reduction processing on the second texture area in the initial image according to the second noise reduction weight corresponding to the second texture area, to obtain a target low-frequency image; determines the target image based on the initial image and the target low-frequency image. In the above scheme, different noise reduction weights are used for different texture areas for noise reduction processing, which can meet the needs of different application scenarios, effectively remove noise in the initial image, and improve the image enhancement effect.

[0105] The functions of each module can be found in the embodiment of the image enhancement method and will not be described in detail here.

[0106] To implement the image enhancement method of the above embodiment, the present application proposes another electronic device, for details, please refer to Figure 7 , Figure 7 It is a structural schematic diagram of an embodiment of an electronic device provided by the present application.

[0107] The electronic device 700 includes a memory 710 and a processor 720 , wherein the memory 710 and the processor 720 are coupled.

[0108] The memory 710 is used to store program data, and the processor 720 is used to execute the program data to implement the image enhancement method of the above embodiment.

[0109] In this embodiment, the processor 720 may also be referred to as a CPU (Central Processing Unit). The processor 720 may be an integrated circuit chip having signal processing capabilities. The processor 720 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. A general-purpose processor may be a microprocessor or the processor 720 may also be any conventional processor, etc.

[0110] The present application also provides a computer-readable storage medium, such as Figure 8 As shown, the computer-readable storage medium 800 is used to store program data 810. When the program data 810 is executed by a processor, it is used to implement the image enhancement method in the method embodiment of the present application.

[0111] The method involved in the embodiment of the image enhancement method of the present application, when implemented in the form of a software functional unit and sold or used as an independent product, can be stored in a device, such as a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or part of the contribution to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0112] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. An image enhancement method, characterized in that: The image enhancement method comprises: Extracting an initial low-frequency image from the acquired initial image, wherein the pixels of the initial image correspond to the pixels of the initial low-frequency image one by one; determining a first texture region and a second texture region in the initial low-frequency image according to the attribute information of the initial low-frequency image, wherein the object edge size in the first texture region is larger than the object edge size in the second texture region; Performing denoising processing on a first texture region in the initial image according to a first denoising weight corresponding to the first texture region, and performing denoising processing on a second texture region in the initial image according to a second denoising weight corresponding to the second texture region, to obtain a target low-frequency image; determining a target image based on the initial image and the target low-frequency image; The step of extracting the initial low-frequency image from the acquired initial image comprises: Determining a target filter template size from a preset correspondence table according to the resolution of the initial image, wherein the preset correspondence table includes a correspondence between a preset filter template size and a preset resolution; Then, the target filter template size is determined according to the object edge size of each object in the initial image, and the initial image is filtered using the filter template of the target filter template size, so that the filter template of the target filter template size can extract the object edges of the object edge size in the initial image that is larger than the preset edge size, and blur the objects whose edge size is smaller than or equal to the preset edge size, so as to obtain an initial low-frequency image of the initial image.

2. The image enhancement method according to claim 1, characterized in that: The attribute information includes a pixel value of each pixel in the initial low-frequency image, and the step of determining a first texture area and a second texture area in the initial low-frequency image according to the attribute information of the initial low-frequency image includes: In response to a pixel value of a pixel point in the initial low-frequency image being greater than a first preset pixel value, determining the corresponding pixel point as a pixel point in the first texture area; In response to a pixel value of a pixel point in the initial low-frequency image being less than or equal to the first preset pixel value, the corresponding pixel point is determined as a pixel point in the second texture area.

3. The image enhancement method according to claim 1, characterized in that: The step of performing denoising on the first texture region in the initial image according to the first denoising weight corresponding to the first texture region, and performing denoising on the second texture region in the initial image according to the second denoising weight corresponding to the second texture region, to obtain a target low-frequency image, comprises: Obtain a first preset weight and a second preset weight, wherein the first preset weight is less than the second preset weight; Setting the first preset weight as a first denoising weight of the first texture area; Setting the second preset weight as a second denoising weight for the second texture area; Performing denoising processing on a first texture region in the initial image with the first denoising weight, and performing denoising processing on a second texture region in the initial image with the second denoising weight, to obtain a denoised initial image; The denoised initial image is used as the target low-frequency image.

4. The image enhancement method according to claim 1, characterized in that: The step of performing denoising on the first texture region in the initial image according to the first denoising weight corresponding to the first texture region, and performing denoising on the second texture region in the initial image according to the second denoising weight corresponding to the second texture region, to obtain a target low-frequency image, comprises: Performing Gaussian filtering on the initial image to obtain a Gaussian image; Determining a first denoising weight for the first texture area according to a pixel value of each pixel point in the first texture area and a pixel value of a corresponding pixel point in the Gaussian image; Determining a second denoising weight for the second texture area according to a pixel value of each pixel in the second texture area and a pixel value of a corresponding pixel in the Gaussian image; The first texture region is subjected to denoising processing with the first denoising weight, and the second texture region is subjected to denoising processing with the second denoising weight, to obtain the target low-frequency image, wherein the first denoising weight is less than the second denoising weight.

5. The image enhancement method according to claim 4, characterized in that: The step of determining a first denoising weight of the first texture area according to the pixel value of each pixel point in the first texture area and the pixel value of the corresponding pixel point in the Gaussian image comprises: Obtaining the pixel difference between each pixel point and the neighboring pixel points in the Gaussian image; Calculating a ratio between the square of the pixel difference and the initial weight corresponding to the pixel point to obtain a ratio corresponding to each pixel point in the first texture area; An exponential function of the inverse of the ratio is determined as a first denoising weight of the first texture region.

6. The image enhancement method according to claim 1, characterized in that: The step of performing denoising on the first texture region in the initial image according to the first denoising weight corresponding to the first texture region, and performing denoising on the second texture region in the initial image according to the second denoising weight corresponding to the second texture region, to obtain a target low-frequency image, comprises: Performing a first denoising process on a first texture region in the initial image according to a first denoising weight corresponding to the first texture region, and performing a first denoising process on a second texture region in the initial image according to a second denoising weight corresponding to the second texture region, to obtain an initial image after the first denoising process; In response to the initial image after the first denoising process not meeting the preset requirement, performing a second denoising process on a first texture region in the initial image after the first denoising process according to a first denoising weight corresponding to the first texture region, and performing a second denoising process on a second texture region in the initial image after the first denoising process according to a second denoising weight corresponding to the second texture region, to obtain an initial image after the second denoising process; In response to the initial image after the second noise reduction processing meeting a preset requirement, the initial image after the second noise reduction processing is determined as the target low-frequency image.

7. The image enhancement method according to claim 1, characterized in that: The step of determining the target image based on the initial image and the target low-frequency image comprises: determining an initial high-frequency image based on the initial image and the target low-frequency image; Performing enhancement processing on the initial high-frequency image to obtain a target high-frequency image; The target high-frequency image and the target low-frequency image are fused to obtain the target image.

8. An electronic device, characterized in that: include: A memory and a processor, wherein the memory stores program instructions, and the processor retrieves the program instructions from the memory to execute the method according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that: include: Program data is stored, and when the program data is executed by a processor, it is used to implement the method according to any one of claims 1 to 7.

Images